International Finance Corporation
Case Study—Trust motivates people’s uptake and use of digital financial services (DFS). Understanding the socio-cultural determinants of DFS trust are needed to scale financial access and drive financial inclusion. These are core components of international development strategies, such as the Sustainable Development Goals (SDGs) or Universal Financial Access (UFA2020). The IFC-Mastercard Foundation Partnership for Financial Inclusion (the Partnership) conducted ethnographic research to understand factors that impact people’s attitudes and perceptions of DFS. Nine months of field work each in Cameroon, DRC, Senegal and Zambia were conducted, in collaboration with local research institutes’ Anthropology departments and the African Studies Center at the University of Leiden. The results of the ethnographic research produced a framework for understanding drivers and barriers to people growing trust in digital financial services.
This paper analyzes the...
LARRY S. MCGRATH
Design Science Consulting, Inc.
Using eye tracking in ethnographic research poses numerous theoretical and practical challenges. How might devices originally intended to record individuals' vision of two-dimensional planes be useful in interpersonal contexts with dynamic visual interfaces? What would the technology reveal about collegial environments in which different levels of knowledge and expertise come together and inform decision-making processes? Why would pupil movement show us anything that conventional ethnographic methods could not? In this paper, I argue that these challenges are not intractable. When tailored to specific questions about perception, action, and collaboration, eye trackers can reveal behaviors that elude ethnographers' gaze. In so doing, the devices enrich the observational and interview-based methods already employed in ethnographic studies of workplace dynamics.
Hospitals are a fruitful context in which to test the value of eye-tracking evidence. Healthcare professionals look, interpret,...
imec-SMIT, Vrije Universiteit Brussel
imec-SMIT, Vrije Universiteit Brussel
This paper aims to contribute to the debate on the integration of ethnography and data science by providing a concrete research tool to deploy this integration. We start from our own experiences with user research in a data-rich environment, the smart city, and work towards a research tool that leverages ethnographic praxis with data science opportunities. We discuss the different key components of the system, how they work together and how they allow for human sensemaking....
MILLIE P. ARORA
As algorithms play an increasingly important role in the lives of people and corporations, finding more effective, ethical, and empathetic ways of developing them has become an industry imperative. Ethnography, and the contextual understanding derived from it, has the potential to fundamentally change the way that data science is done. Reciprocally, engaging with data science can help ethnographers focus their efforts, build stronger and more precise insights, and ultimately have greater impact once their work is incorporated into the algorithms that increasingly power our society. In practice, building contextually-informed algorithms requires collaboration between human science and data science teams who are willing to extend their frame of reference beyond their core skill areas. This paper aims to first address the features of ethnography and data science that make collaboration between the two more valuable than the sum of their...
ELIZABETH A. KELLEY
ILLUME Advising, LLC
AMANDA E. DWELLEY
ILLUME Advising, LLC
Case Study—This case draws on work in the energy efficiency industry where many utilities rely on data-driven insights and decision-making to encourage consumers to adopt energy-saving products and behaviors. In this highly regulated industry, utility staff must show value through big data, and studies often rely exclusively on quantitative data analytics to create behavioral models to explain or predict behavior. However, purely data-driven research often fails to answer questions about why customers behave a certain way, and what product or program managers and marketers can do about it. In this case study, the team from ILLUME Advising LLC (ILLUME), a research consultancy in the clean energy industry, illustrates how their cross-functional team paired qualitative and quantitative research on residential home energy use. The case study draws on an exploratory market and segmentation study for an electric utility interested in engaging customers...
Seattle Children's Hospital
Seattle Children's Hospital
Case Study—Declining clinician engagement, increasing rates of burnout, and stagnant patient and family experience scores have led hospital leadership at Seattle Children's Hospital to submit requests to a data scientist and an anthropologist to identify key themes of survey comments and provide recommendations to improve experience and satisfaction. This study explored ways of understanding satisfaction as well as analytic approaches to textual data, and found that various modes of evidence, while seemingly ideal to leaders, are hard pressed to meet their expectations. Examining satisfaction survey comments via text mining, content analysis, and ethnographic investigation uncovered several specific challenges to stakeholder requests for actionable insights. Despite its hype, text mining struggled to identify actionable themes, accurate sentiment, or group distinctions that are readily identified by both content analysis and end users, while more...
Amazon Prime Video
Amazon Prime Video
Case Study—The Amazon Prime Video User Experience (UX) Research team endeavored to balance qualitative and quantitative insights and translate them into the currency that drives the business, specifically customer engagement, to improve decision-making. Researchers conducted foundational qualitative research to uncover what matters most to Prime Video customers, translated resulting insights into a set of durable, measurable customer outcomes, and developed a global, longitudinal online survey program that validated the importance and perception of these outcomes at scale. Researchers then systematically linked customers’ attitudinal survey results to their usage patterns and overall satisfaction with the service. The resulting data showed how investing in improving a customer outcome is likely to increase service engagement, thus closing the loop between insights and business metrics for the first time. Prime Video executive leadership has not only embraced...
Hitachi, Ltd. Research & Development Group
Independent UX Research & Design Consultant
In international business ethnography, clients and subjects don’t share the same background. Without an understanding of the underlying factors affecting the subject’s behaviors, data can lead to false home-market based assumptions about cause and effect. Where do we as researchers look to detect meaningful findings in international contexts? Drawing on our decades of international fieldwork, we describe how focusing on culture or cultural differences to interpret differences in workflows and attitudes can sometimes hamper accurate interpretation of observations. We describe through case studies how instead, identifying foundation factors shaping behaviors and mindsets such as market forces, government policy, labour markets, and financial schemas can be the key to insight in international contexts.
Keywords: Ethnography, International, Japan, fieldwork, workflow, products and systems, user research, UX,...
Program of Applied Anthropology, Oregon State University
Program of Geography, Oregon State University
Program of Mechanical Engineering and Program of Applied Anthropology, Oregon State University
For its volume, velocity, and variety (the 3 Vs), big data has been ever more widely used for decision-making and knowledge discovery in various sectors of contemporary society. Since recently, a major challenge increasingly recognized in big data processing is the issue of data quality, or the veracity (4th V) of big data. Without addressing this critical issue, big data-driven knowledge discoveries and decision-making can be very questionable. In this paper, we propose an innovative methodological approach, an archaeological-ethnographic approach that aims to address the challenge of big data veracity and to enhance big data interpretation. We draw upon our three recent case studies of fake or noise data in different data environments. We approach big data as but another kind of human...
Case Study—We report on a two-year project focused on the design and development of data analytics to support the cloud services division of a global IT company. While the business press proclaims the potential for enterprise analytics to transform organizations and make them ‘smarter’ and more efficient, little has been written about the actual practices involved in turning data into ‘actionable’ insights. We describe our experiences doing data analytics within a large global enterprise and reflect on the practices of acquiring and cleansing data, developing analytic tools and choosing appropriate algorithms, aligning analytics with the demands of the work and constraints on organizational actors, and embedding new analytic tools within the enterprise. The project we report on was initiated by three researchers; a mathematician, an operations researcher, and an anthropologist well-versed in practice-based technology design, in collaboration...
CUNY Graduate Center / Data & Society
Cloudera Fast Forward Labs
The successes of technology companies that rely on data to drive their business hints at the potential of data science and machine learning (DS/ML) to reshape the corporate world. However, despite the headway made by a few notable titans (e.g., Google, Amazon, Apple) and upstarts, the advances that are advertised around DS/ML have yet to be realized on a broader basis. The authors examine the tension between the spectacular image of DS/ML and the realities of applying the latest DS/ML techniques to solve industry problems. The authors discern two distinct ways, or modes, of thinking about DS/ML woven into current marketing and hype. One mode focuses on the spectacular capabilities of DS/ML. It expresses itself through one-off, easy-to-grasp marketable projects, such as DeepMind’s AlphaGo (Zero). The other mode focuses on DS/ML’s potential to transform industry. Hampered by an emphasis on tremendous but as of yet unrealized...
Nissan Research Center – Silicon Valley; Massachusetts Institute of Technology
Nissan Research Center – Silicon Valley
In this paper we explore the idea of a system of care through a city transit system. We argue that a systematic orientation to care is central to what makes a transit system work for people. Further, we suggest that this care orientation is recognized as such, even though it is not apparent in typical modes of systems management. Care is what knowing in this system is for. We examine how participants in the system navigate different epistemic bases of their work, focusing on how care work and information work intertwine. How is this work practiced and known? And how could we, as design researchers, use these practices to design systems of care? In service of these goals, we expand the notion of care work toward care of non-human actors as well as that of people. We focus particularly on the roles of automation and the risks automation presents for care. In a moment of increased...
Digital environments can expand the distances between people. While this is often challenging, it can also be leveraged to do great things. This PechaKucha explores how we can take this negative divide between people and flip it on its head to discover more powerful insights. By looking at a range of studies focused on sensitive subjects, we explore how technology has the power to create a safer environment for vulnerable participants. While technology is an often underutilized research tool, these technology enabled environments can lead to richer data and insights. As a result, we, as researchers, can create the space needed to share some of our most intimate stories.
Jess Shutt has dedicated her career to studying & creating technology to help democratize complex processes & systems from finance to consumer robotics. Her current work as the Lead User Researcher of Einstein at Salesforce focuses on applying these concepts to artificial intelligence.
2018 EPIC Proceedings,...
MADELEINE CLARE ELISH
Data & Society Research Institute
The wide-spread deployment of machine learning tools within healthcare is on the horizon. However, the hype around “AI” tends to divert attention toward the spectacular, and away from the more mundane and ground-level aspects of new technologies that shape technological adoption and integration. This paper examines the development of a machine learning-driven sepsis risk detection tool in a hospital Emergency Department in order to interrogate the contingent and deeply contextual ways in which AI technologies are likely be adopted in healthcare. In particular, the paper bring into focus the epistemological implications of introducing a machine learning-driven tool into a clinical setting by analyzing shifting categories of trust, evidence, and authority. The paper further explores the conditions of certainty in the disciplinary contexts of data science and ethnography, and offers a potential reframing of the work of doing data science and machine learning as “computational...
When we study human systems and organizations we have a job that requires to empathize or at the very least be compassionate towards the experiences others are having. This allows to understand their goals, problems, and how we can best make their lives better. When machines start to do things that we can't imagine how do we continue to work with them? What is necessary to create great combinations of humans and machines? What is a machine's purpose? Very simply: it is to serve human purposes. As technology continues to build facades that hide the human element we need to pull back the curtain (like the one in the Wizard of Oz) and see that the tools we build are really us reflected back. We have the choice to make tools that are good or bad for us.
Chris Butler, is the Director of AI at Philosophie and frequently speaks on the intersection of product, design, and AI. He has extensive experience from Microsoft, Waze, KAYAK, among others. Through his practice he has created...