mixed methods

Intelligences

Creating Resilient Research Findings: Using Ethnographic Methods to Combat Research Amnesia

Posted by  • 0 Comments

Kristen Guth speaking at EPIC2022. Slide reads "Organizations can learn by reflection on memories, or shared understandings and beliefs, of specific events and contexts
KRISTEN L. GUTH Reddit, Inc. DOWNLOAD PDF Product teams, including those I work with, struggle to connect the challenges observed in prior research to issues that endure in the field and market space. As a shortcut for efficiency gains, product partners rely on researchers to succinctly summarize deep insights, sometimes preferring reductive quantitative interpretations to enable a bias toward action in product development cycles. Challenges facing researchers in product development include maintaining the relevance of prior research, providing a way to make it evergreen and accessible, and building on it to deepen and expand an existing model of behavior. This case introduces the concept of Research Amnesia, which poses a threat to organizational resilience. Using core ethnographic methods, a strategic methodological approach is outlined to frameshift the value of existing research within a company to develop new insights, bring together disparate analyses and teams, and propel product partners forward by offering more questions...

Perspectives

Seeing the World at Scale and in Depth: A Journey with Big and Thick Data

Posted by  • 

by QAMAR ZAMAN, Stripe Partners When I was studying economics at university one of our professors introduced us to Jorge Luis Borges’s “On Exactitude in Science”, a one-paragraph story. It imagines an empire so enthralled by cartography that larger and larger maps of the place are built by successive generations until a map on the same scale as the empire is drawn. Following generations realise a map of such magnitude is cumbersome and “in the western deserts, tattered fragments of the map are still to be found, sheltering an occasional beast or beggar”. Our professor’s point back then was that in a world where trying to see and make sense of too much is impossible, simple models to comprehend the world (and economics was built on simple models) carry immense value. Some years on from that, combining big data and thick data promises the ability to see and understand much more. Their combination can provide maps which are vast but also allow us to make sense of the landscape and people inhabiting them. This piece shows what...

Intelligences

Futures in Things: Locating the Promise of Infrastructures in Public Libraries

Posted by  • 0 Comments

SANDJAR KOZUBEV Georgia Institute of Technology CARL DISALVO Georgia Institute of Technology Public libraries in the U.S. and around the world are rapidly changing due expanding technological and social needs of their communities. The Covid-19 pandemic has intensified the debates about the future of public spaces and public services. In this paper, we report on a qualitative study of librarians in a U.S. urban public library system. The focus of the study was to understand how the concept of “the future of library” is constructed and contested both socially and materially. Using mixed methods, including participant observation, interviews, participatory design and action research, we developed insights about the socio-political dynamics of futures in a public infrastructure. We argue that futures can be shaped not only by socio-technical imaginaries, and representations, which tend to be abstract and distant, but also by socio-material conditions in the present. Specifically, drawing on the work of infrastructure studies,...

Intelligences

On Being Well in a Time of Hell

Posted by  • 0 Comments

MIRA SHAH Spotify CHLOE EVANS Spotify CAMIE STEINHOFF Spotify PechaKucha Presentation—For the past year, people around the world have adjusted quickly to unforeseen constraints presented by the COVID-19 pandemic. Upheaval during the pandemic has resulted in a deep sense of grief leaving people in an unpredictable cycle of losing control and attempting to regain it. But, guess what? Researchers also have been experiencing the highs and lows of the pandemic and haven’t been immune to the fulcrum of loss and unexpected buoyancy. In sensing the importance of the moment, a group of researchers came together to learn how people around the world were adjusting and coping, and to anticipate how adaptations in contexts, habits, and tools may lead to enduring changes in everyday life. On Being Well in a Time of Hell is a bricolage from Brazil, Indonesia, and the United States told through diary entries, photos, and drawings that bounce from despair to moments of unexpected connection, creativity, and sometimes, believe it or...

Intelligences

Change the Category, Change the World: How Research and Great Storytelling Drove a Headline-Making World First for Supermarkets

Posted by  • 0 Comments

JENNIE LENG Independent PechaKucha Presentation—The butterfly effect – a small change that has big ripples. This is what Jennie Leng created when she persuaded NZ’s largest supermarket to change its language from “sanitary products”. Phrases like “sanitary products” and “feminine hygiene” are ubiquitous around the world, but these euphemisms have connotations of dirtiness, and perpetuate the idea that menstruation is embarrassing and shameful. Jennie used context, anecdotes and quantitative research to build a case for the supermarket to change their language – and they went for it! Countdown Supermarkets now uses the phrase “period care” in all of its digital channels and have rolled the label out in store. Because if you can have skin care and hair care, why not period care? In this PechaKucha, Jennie will touch on the types of evidence and the framing she used to influence the business, and the world-first, headline-making outcomes of this change. Jennie Leng is a New Zealander, mother-of-3 and...

Intelligences

Tutorial: Beyond “Quant-vs-Qual”—Creating Collaborative Inquiry

Posted by  • 0 Comments

ALEX HUGHES UC Berkeley JENNY LO Grammarly WILL MONGE Good Research Overview Qual and quant are so divided these days—by academic discipline, language, communities of practice, job titles. Too often, quantitative research is conflated with data science (or vice versa), and data science with optimization algorithms or simply engineering. In many organizations, being “data-driven” tends to define “data” with a narrow conception of enumeration and (mis-) conceptions about the kind of evidence that is suitable to act on. This tutorial critically examines this territory and move beyond it, empowering ethnographers to develop more interdisciplinary programs of inquiry. First the instructors review fundamentals of quantitative research and provide tools ethnographers can use to evaluate its quality and validity. Then they examine constraints and barriers to quant/qual collaboration, including time, funding, values, epistemological conflicts, organizational silos, and more. Finally, using core principles that underlie...

Intelligences

Ghost in the Machine: How Taxonomic Metadata Allows for Scaling Ethnographic Insights Into Search Algorithms

Posted by  • 0 Comments

AMANDA KRAUSS Duo Security (formerly Indeed) ALEXANDRA TEODORESCU Indeed LEORA YARDENAY Indeed This case study explores how we personalized search results by turning ethnographic insights into taxonomic metadata, which in turn allowed us to use quantitative methods to assess business impact. The first part of the case study focuses on the problem we were trying to solve – creating better search results for nurses – and using ethnographic interviews to understand how nurses approached looking for jobs. The second part of the study dives more deeply into how metadata works, and why it was the perfect partner for capturing our ethnographic findings and making them into a scalable and measurable part of the design process. The third part of the study details how we tested and scaled our designs in the live project, and why we believe others might benefit from using a similar approach. Keywords: mental models, taxonomy, business impact. Article citation: 2020 EPIC Proceedings pp 177–188, ISSN 1559-8918, https://www.epicpeople.org/epic...

Intelligences

DIYing along with DIYers: Juggling with Scales During a Home-Improvement Research

Posted by  • 0 Comments

GUILLAUME MONTAGU _unknowns As a team of researchers was asked by a French home-improvement retailer to redefine their strategy, they designed and carried out an ethnographic and quantitative research to identify new business opportunities. But no sooner had they set foot in field, they were struck not only by the richness and complexity of such ordinary activities to the point they asked themselves if these practices were even measurable? Scaling from ethnography to quantitative research was not as seamless as they expected, they had to find their way to deal with two sets of data that belong to different scales if not ontological worlds. Are these two scales really strictly separated? Can't there be a way to combine them and to make them coincide? Based on the study of DIYing practices, this case study presents an attempt to integrate ethnographic and quantitative research and the challenge of resolving the scale differences between two methodologies. From turning DIYers into numbers and vice-versa, it explores the implications...

Intelligences

Hybrid Methodology: Combining Ethnography, Cognitive Science, and Machine Learning to Inform the Development of Context-Aware Personal Computing and Assistive Technology

Posted by  • 0 Comments

MARIA CURY* ReD Associates ERYN WHITWORTH* Facebook Reality Labs *Lead co-authors The not-too-distant future may bring more ubiquitous personal computing technologies seamlessly integrated into people's lives, with the potential to augment reality and support human cognition. For such technology to be truly assistive to people, it must be context-aware. Human experience of context is complex, and so the early development of this technology benefits from a collaborative and interdisciplinary approach to research — what the authors call “hybrid methodology” — that combines (and challenges) the frameworks, approaches, and methods of machine learning, cognitive science, and anthropology. Hybrid methodology suggests new value ethnography can offer, but also new ways ethnographers should adapt their methodologies, deliverables, and ways of collaborating for impact in this space. This paper outlines a few of the data collection and analysis approaches emerging from hybrid methodology, and learnings about impact and team collaboration,...

Intelligences

Ethnographic Agency in a Data Driven World

Posted by  • 0 Comments

NADINE LEVIN Facebook This paper argues that ethnographers can gain increased agency in data-driven corporate environments by increasing their quantitative literacy: their ability to create, understand, and strategically use quantitative data to shape organizations. Drawing on the author's experience conducting strategic user research at a technology company, the paper explores how the ability to engage with quantitative data can increase ethnographers’ independence and autonomy within organizations, and can also up-level the role and value of qualitative research. The paper also explores how a deep familiarity with quantitative data can enable ethnographers to imbue quantitative data itself with new forms of agency, and can ultimately give ethnographers the tools to change institutions from within. With a greater understanding of how quantitative data is made and used, ethnographers can ensure that data is collected in representative ways, point out the limitations of existing metrics, and argue for new ways of measuring and...

Intelligences

Just Add Water: Lessons Learned from Mixing Data Science and Design Research Methods to Improve Customer Service

Posted by  • 3 Comments

OVETTA SAMPSON IDEO Chicago and DePaul University Case Study—This case study provides an inside look at what occurs when methods from the data science and ethnographic fields are mixed to solve perennial customer service problems within the call center and cruise industries. The paper details this particular blend of ethnographic practitioners with a data scientist resulted in changes to design approaches, debunking myths about qualitative and quantitative research methods being at odds and altering team member perspectives about the value of both. The project also led to the creation of innovative blended design research and data science methods to discover and leverage the right customer data to the benefit of both the customer and the call center agents who serve them. This paper offers insight into the untold value design teams can unlock when data scientists and ethnographers work together to solve a problem. The result was a design solution that gives a top-performing company an edge to grow even better by leveraging the millions...

Intelligences

Designing for Interactions with Automated Vehicles: Ethnography at the Boundary of Quantitative-Data-Driven Disciplines

Posted by  • 0 Comments

MARKUS ROTHMÜLLER School of Architecture, Design and Planning, Aalborg University Copenhagen, Denmark and Shift Insights & Innovation Consulting PERNILLE HOLM RASMUSSEN School of Architecture, Design and Planning, Aalborg University Copenhagen, Denmark SIGNE ALEXANDRA VENDELBO-LARSEN School of Architecture, Design and Planning, Aalborg University Copenhagen, Denmark Case Study—This case study presents ethnographic work in the midst of two fields of technological innovation: automated vehicles (AV) and virtual reality (VR). It showcases the work of three MSc. Techno-Anthropology students and their collaboration with the EU H2020 project ‘interACT’, sharing the goal to develop external human-machine interfaces (e-HMI) for AVs to cooperate with human road users in urban traffic in the future. The authors reflect on their collaboration with human factor researchers, data scientists, engineers, experimental researchers, VR-developers and HMI-designers, and on experienced challenges between the paradigms of qualitative...

Intelligences

Gaming Evidence: Power, Storytelling and the “Colonial Moment” in a Chicago Systems Change Project

Posted by  • 0 Comments

NATHAN HEINTZ Systems Change Consultant Case Study—In 2016 The Chicago Community Trust (“The Trust”), a local Chicago foundation, partnered with Roller Strategies (“Roller”), an international professional services firm, to deploy an innovative mixed-methods approach to community-driven social change on the South Side of Chicago. This partnership convened a diverse group of stakeholders representing a microcosm of the social system, and launched a project with the aim of developing resilient livelihoods for youth aged 18-26 in three specific South Side neighborhoods. Roller designed and facilitated a process through which the stakeholder group scoped, launched, piloted and prototyped community-driven initiatives. While innovative and successful by some metrics, the project had its challenges. The convening institutions and their staff were often perceived as “outsiders” and “experts” without intimate local knowledge of the social challenges they were attempting to address. This dynamic played out in complex power...

Intelligences

ReHumanizing Hospital Satisfaction Data: Text Analysis, the Lifeworld, and Contesting Stakeholders’ Beliefs in Evidence

Posted by  • 0 Comments

JULIA WIGNALL Seattle Children's Hospital DWIGHT BARRY Seattle Children's Hospital Case Study—Declining clinician engagement, increasing rates of burnout, and stagnant patient and family experience scores have led hospital leadership at Seattle Children's Hospital to submit requests to a data scientist and an anthropologist to identify key themes of survey comments and provide recommendations to improve experience and satisfaction. This study explored ways of understanding satisfaction as well as analytic approaches to textual data, and found that various modes of evidence, while seemingly ideal to leaders, are hard pressed to meet their expectations. Examining satisfaction survey comments via text mining, content analysis, and ethnographic investigation uncovered several specific challenges to stakeholder requests for actionable insights. Despite its hype, text mining struggled to identify actionable themes, accurate sentiment, or group distinctions that are readily identified by both content analysis and end users, while more...

Intelligences

Below the Surface of the Data Lake: An Ethnographic Case Study on the Detrimental Effect of Big Data Path Dependency at a Theme Park

Posted by  • 0 Comments

JACOB WACHMANN ReD Associates ANDREAS JUNI ReD Associates DAVE BAIOCCHI ReD Associates WILLIAM WELSER IV ReD Associates Case Study—This case-study details how a team of anthropologists and a team of data scientists sought to help a Middle Eastern theme park make use of their big data platform to measure ‘the good customer experience’. Ethnographic research within the theme park revealed that visitors yearned to bond with the other members of their group, as they rarely got the chance during their busy everyday lives back home. However, trying to build a measurement of how the theme park delivered on bonding – through the development of a ‘bonding index’ – turned out to be unfeasible, because the big data platform focused on capturing operational data. The decision to focus on operational data had unintentionally created a path dependency that made the big data setup unfit for answering some of the theme park’s most fundamental questions. This is a problem ReD Associates has observed across clients and to solve it this...