mixed methods

Just Add Water: Lessons Learned from Mixing Data Science and Design Research Methods to Improve Customer Service

OVETTA SAMPSON IDEO Chicago and DePaul University Case Study—This case study provides an inside look at what occurs when methods from the data science and ethnographic fields are mixed to solve perennial customer service problems within the call center and cruise industries. The paper details this particular blend of ethnographic practitioners with a data scientist resulted in changes to design approaches, debunking myths about qualitative and quantitative research methods being at odds and altering team member perspectives about the value of both. The project also led to the creation of innovative blended design research and data science methods to discover and leverage the right customer data to the benefit of both the customer and the call center agents who serve them. This paper offers insight into the untold value design teams can unlock when data scientists and ethnographers work together to solve a problem. The result was a design solution that gives a top-performing company an edge to grow even better by leveraging the millions...

Designing for Interactions with Automated Vehicles: Ethnography at the Boundary of Quantitative-Data-Driven Disciplines

MARKUS ROTHMÜLLER School of Architecture, Design and Planning, Aalborg University Copenhagen, Denmark and Shift Insights & Innovation Consulting PERNILLE HOLM RASMUSSEN School of Architecture, Design and Planning, Aalborg University Copenhagen, Denmark SIGNE ALEXANDRA VENDELBO-LARSEN School of Architecture, Design and Planning, Aalborg University Copenhagen, Denmark Case Study—This case study presents ethnographic work in the midst of two fields of technological innovation: automated vehicles (AV) and virtual reality (VR). It showcases the work of three MSc. Techno-Anthropology students and their collaboration with the EU H2020 project ‘interACT’, sharing the goal to develop external human-machine interfaces (e-HMI) for AVs to cooperate with human road users in urban traffic in the future. The authors reflect on their collaboration with human factor researchers, data scientists, engineers, experimental researchers, VR-developers and HMI-designers, and on experienced challenges between the paradigms of qualitative and quantitative...

Gaming Evidence: Power, Storytelling and the “Colonial Moment” in a Chicago Systems Change Project

NATHAN HEINTZ Systems Change Consultant Case Study—In 2016 The Chicago Community Trust (“The Trust”), a local Chicago foundation, partnered with Roller Strategies (“Roller”), an international professional services firm, to deploy an innovative mixed-methods approach to community-driven social change on the South Side of Chicago. This partnership convened a diverse group of stakeholders representing a microcosm of the social system, and launched a project with the aim of developing resilient livelihoods for youth aged 18-26 in three specific South Side neighborhoods. Roller designed and facilitated a process through which the stakeholder group scoped, launched, piloted and prototyped community-driven initiatives. While innovative and successful by some metrics, the project had its challenges. The convening institutions and their staff were often perceived as “outsiders” and “experts” without intimate local knowledge of the social challenges they were attempting to address. This dynamic played out in complex power...

ReHumanizing Hospital Satisfaction Data: Text Analysis, the Lifeworld, and Contesting Stakeholders’ Beliefs in Evidence

JULIA WIGNALL Seattle Children's Hospital DWIGHT BARRY Seattle Children's Hospital Case Study—Declining clinician engagement, increasing rates of burnout, and stagnant patient and family experience scores have led hospital leadership at Seattle Children's Hospital to submit requests to a data scientist and an anthropologist to identify key themes of survey comments and provide recommendations to improve experience and satisfaction. This study explored ways of understanding satisfaction as well as analytic approaches to textual data, and found that various modes of evidence, while seemingly ideal to leaders, are hard pressed to meet their expectations. Examining satisfaction survey comments via text mining, content analysis, and ethnographic investigation uncovered several specific challenges to stakeholder requests for actionable insights. Despite its hype, text mining struggled to identify actionable themes, accurate sentiment, or group distinctions that are readily identified by both content analysis and end users, while more...

Below the Surface of the Data Lake: An Ethnographic Case Study on the Detrimental Effect of Big Data Path Dependency at a Theme Park

JACOB WACHMANN ReD Associates ANDREAS JUNI ReD Associates DAVE BAIOCCHI ReD Associates WILLIAM WELSER IV ReD Associates Case Study—This case-study details how a team of anthropologists and a team of data scientists sought to help a Middle Eastern theme park make use of their big data platform to measure ‘the good customer experience’. Ethnographic research within the theme park revealed that visitors yearned to bond with the other members of their group, as they rarely got the chance during their busy everyday lives back home. However, trying to build a measurement of how the theme park delivered on bonding – through the development of a ‘bonding index’ – turned out to be unfeasible, because the big data platform focused on capturing operational data. The decision to focus on operational data had unintentionally created a path dependency that made the big data setup unfit for answering some of the theme park’s most fundamental questions. This is a problem ReD Associates has observed across clients and to solve it this...

Revitalising Openness at Mozilla: A Mixed Method Research Approach

RINA TAMBO JENSEN Mozilla Case Study—This is a case about how Mozilla, the open source browser company, set out to reconnect with ‘collaborating in the open’ to regain its competitive advantage. This case describes how a multi-disciplinary research team used ethnographic, market, and data analysis to articulate and clarify the problem, and build a strategy towards revitalizing Openness at Mozilla. It will aim to prove that the subsequent change achieved could only have been accomplished by a mixed method research approach. And importantly show, how the team used data to prove the distribution of findings, coupled with ethnography to shine light on the why and how of those findings. The case study will do this by discussing the key insights and how these fueled recommendation and subsequent change in the organisation. The project presented many problems: from convincing stakeholders of the need to fully explore the problem, to connecting widely different research methods and gleaning insights that built strongly on all strands...

Data Science and Ethnography: What’s Our Common Ground, and Why Does It Matter?

by TYE RATTENBURY (Salesforce) & DAWN NAFUS (Intel) As EPIC2018 program co-chairs, we developed the conference theme Evidence to explore how evidence is created, used, and abused. We’ll consider the core types of evidence ethnographers make and use through participant observation, cultural analysis, filmmaking, interviewing, digital and mobile techniques, and other essential methods, as well as new approaches in interdisciplinary and cross-functional teams.1 We’ve also made a special invitation to data scientists to join us in Honolulu to advance the intersection of computational and ethnographic approaches. Why? One of us is a data scientist (Tye) and the other an ethnographer (Dawn), both working in industry. We regularly see data science and ethnography conceptualized as polar ends of a research spectrum—one as a crunching of colossal data sets, the other as a slow simmer of experiential immersion. Unfortunately, we also see occasional professional stereotyping. A naïve view of “crunching” can make it seem...

A Mixed Method Approach for Identifying Emerging Fields and Building Collaborative Teams: Leveraging Network Ethnography to Design Experimental Interventions

THERESE KENNELLY OKRAKU Microsoft VALERIO LEONE SCIABOLAZZA University of Florida RAFFAELE VACCA University of Florida CHRISTOPHER MCCARTY University of Florida Rapid innovation in science and technology has led to the development of new fields that transcend traditional disciplinary boundaries. Previous studies have retroactively examined the emergence of these fields. This paper outlines a mixed method approach for using network ethnography to identify emerging fields as they develop, track their evolution over time, and increase collaboration on these topics. This approach allowed us to simultaneously analyze organizational trends and gain an understanding of why these patterns occurred. Collecting ethnographic data throughout the course of the study enabled us to iteratively improve the fit of our models. It also helped us design an experimental method for creating new teams in these fields and test the effectiveness of this intervention. Initially, organizational leaders were wary of using a network intervention...

The Domestication of Data: Why Embracing Digital Data Means Embracing Bigger Questions

DAWN NAFUS Intel Corporation The EPIC community has been wrestling with ways to integrate quantitative and qualitative methods in light of the increasing role that digital data plays in business practices. Some focus on methodological issues (digital data as method), while others point to the consumer value in data products (data as thing in the world). This paper argues that “digital data as method” and “digital data as thing in the world” are becoming increasingly intertwined. We are not merely witnessing ethnographers’ haulting embrace of digital data, but a wider process of the domestication of data, in which we, alongside the people we study, are participants. The domestication of data involves everyday situations in which ordinary people develop their own sense-making methods—methods remarkably similar to ethnographic knowledge production. In this way, the domestication process tightens the connection between data as thing in the world and data as method. I argue that seeing the interconnection gives us the...

Unleashing the Power of an Analytics Organization: Why a Large Financial Institution Used Ethnography to Transform Analytics

by CHRISTINE BIRTEL (Senior Vice President), JASON PAJTAS (Vice President) and MICHELLE GREEN (Vice President), Customer Experience Insights, Wells Fargo Large organizations have been on a quest to harness the power of big data to inform and drive all kinds of decisions—from finance and fraud prevention to product development and marketing. Organizations from the U.S. government, to retailers, to major financial institutions are appointing Chief Data Officers (CDOs) to evangelize the power of big data. The path of creating a data-savvy organization, however, has complicated turns and roadblocks. Starting with the technological foundation for analytics, there exists a dizzying array of platform and software solutions all claiming to be best-in-class. When success drives an organization beyond this foundation, evolving an existing culture and data infrastructure around things like cloud, big data, machine intelligence and block chain can be an overwhelming challenge that seems insurmountable. Beyond infrastructure challenges,...

Ethnography for Smart Service Systems in Product Design

by WILLIAM O. BEEMAN, Department of Anthropology, University of Minnesota In today’s rapidly changing, highly competitive world, product design requires swift translation of human needs and desires into technical specifications for the development of devices and services that meet those needs (Salvador et al. 2013). This calls for a complex integration of qualitative and quantitative data. But despite some notable successes, product design failures are today both extensive and expensive (Anonymous 2015), consuming enormous amounts of time and human labor. Any improvement to the process of product design would be of great public benefit. Many “smart systems” approaches to the product design process address the problems inherent in this process with limited success. In fact, existing “smart systems” are not very smart. There is an extensive literature available to product designers and engineers addressing lapses in strategies for the successful integration of qualitative and quantitative factors in the design process. It...

Big Data or ‘Big Ethnographic Data’? Positioning Big Data within the Ethnographic Space

JOHN CURRAN This paper offers a cultural analysis of the different narratives that currently frame the concept of Big Data. With specific attention to how the ethnographic community has approached Big Data, I will make the point that the ethnographic community needs to rethink what its offer is within the business world. Instead of trying to position ethnography as a discipline that provides deep insights to human behaviour (which we often call ‘the why’), while Big Data offers broad accounts based on large data collection, I make the case that both approaches should be seen as being positioned within an ethnographic space. This is because both ethnography and Big Data are interested in human behaviour and the cultural field and both are interested in generating insights. We should therefore situate Big Data and ethnography as a relationship that exists in a new epistemological field, a field that is both interpretative and data driven. This field I call 'Big Ethnographic Data'....

The Conceit of Oracles

TRICIA WANG Good morning, I am really excited to be here for my first EPIC conference. There are just so many amazing people in the audience as I look at you guys, and so many of you guys I've been following on blogs and Twitter and especially Natalie Hanson’s anthrodesign listserv. I can’t wait to talk to you guys all afterwards. Just as a reminder, I don’t know if Simon already said it, but if you’re tweeting or instragramming—use the conference hashtag EPIC 2013. If throughout the talk you have any questions, or if anything resonates with you, this is my Twitter and Instagram handle.For over twelve centuries in Ancient Greece in consulting oracles, a person who could predict the future was a part of everyday Hellenistic life. People—poor, wealthy, slave and free—asked oracles for them to answer important life questions such as should I get married, or will I come back from war alive, or questions related to business matters. Should I invest in this voyage? There were questions related to political affairs like should...

Understanding Mediated Practices: Combining Ethnographic Methods with Blog Data to Develop Insights

JONATHAN BEAN and ZEYNEP ARSEL While theories of practice have been influential in the social sciences, these frameworks have seen limited application in ethnographic and applied inquiry, perhaps because few methods for carrying out practice theoretical research have been elaborated. We address this opportunity and provide an account of a multi-method inquiry on domestic practice. First, we explain methods for integrating data from blogs with ethnographic methods and how this data can be used to develop theory. Second, we share our experience as interdisciplinary researchers using ethnographic and quantitative data to connect work at the boundaries of social practice theory and theories of consumption. Finally, we share our insights on why industry should aim to better understand existing and emergent consumer practices....

From Street to Satellite: Mixing Methods to Understand Mobile Money Users

ERIN B. TAYLOR and HEATHER A. HORST How do users incorporate mobile money into their existing practices and adapt it to their needs? The answers can be surprising. Simultaneously a commodity, a store of value, and a social good, mobile money combines a large array of applications within the one platform. This is why mobile money has been touted for its potential for socioeconomic development, as a profitable commercial enterprise, and even as a tool for strengthening governance. The fact that customers rarely use it for just one purpose can also make it difficult to untangle customers’ motives and behaviors. In this paper we compare our own research with other studies to demonstrate how deploying a full suite of ethnographic methods (qualitative and quantitative) can provide significant insights into users. We present three key insights relating to time, trust, and traces / trajectories, and make suggestions for the future of mobile money research....