machine learning

Hybrid Methodology: Combining Ethnography, Cognitive Science, and Machine Learning to Inform the Development of Context-Aware Personal Computing and Assistive Technology

MARIA CURY* ReD Associates ERYN WHITWORTH* Facebook Reality Labs*Lead co-authors The not-too-distant future may bring more ubiquitous personal computing technologies seamlessly integrated into people's lives, with the potential to augment reality and support human cognition. For such technology to be truly assistive to people, it must be context-aware. Human experience of context is complex, and so the early development of this technology benefits from a collaborative and interdisciplinary approach to research — what the authors call “hybrid methodology” — that combines (and challenges) the frameworks, approaches, and methods of machine learning, cognitive science, and anthropology. Hybrid methodology suggests new value ethnography can offer, but also new ways ethnographers should adapt their methodologies, deliverables, and ways of collaborating for impact in this space. This paper outlines a few of the data collection and analysis approaches emerging from hybrid methodology, and learnings about impact and team collaboration,...

Supporting Real-Time Contextual Inquiry through Sensor Data

KATERINA GORKOVENKO University of Edinburgh DAN BURNETT Lancaster University DAVE MURRAY-RUST University of Edinburgh JAMES THORP Lancaster University DANIEL RICHARDS Lancaster University A key challenge in carrying out product design research is obtaining rich contextual information about use in the wild. We present a method that algorithmically mediates between participants, researchers, and objects in order to enable real-time collaborative sensemaking. It facilitates contextual inquiry, revealing behaviours and motivations that frame product use in the wild. In particular, we are interested in developing a practice of use driven design, where products become research tools that generate design insights grounded in user experiences. The value of this method was explored through the deployment of a collection of Bluetooth speakers that capture and stream live data to remote but co-present researchers about their movement and operation. Researchers monitored a visualisation of the real-time data to build up a picture...

Scale, Nuance, and New Expectations in Ethnographic Observation and Sensemaking

ALEXANDRA ZAFIROGLU Intel Corporation YEN-NING CHANG Intel Corporation Case Study—We consider new expectations for ethnographic observation and sensemaking in the next 20-25 years, as technology industry ethnographers' work unfolds in the increasing presence of the type of analytical capabilities specially trained (and self-training) machines can do ‘better’ and ‘cheaper’ than humans as they can take in, analyze and model digital data at much higher volumes and with an attention to nuance not achievable through human cognition alone. We do so by re-imagining three of our existing ethnographic research projects with the addition of very specific applications of machine learning, computer vision, and Internet of Things sensing and connectivity technologies. We draw speculative conclusions about: (1) how data in-and-of-the world that drives tech innovation will be collected and analyzed, (2) how ethnographers will approach analysis and findings, and (3) how the evidence produced by ethnographers will be evaluated and validated....

How Modes of Myth-Making Affect the Particulars of DS/ML Adoption in Industry

EMANUEL MOSS CUNY Graduate Center / Data & Society FRIEDERIKE SCHÜÜR Cloudera Fast Forward Labs The successes of technology companies that rely on data to drive their business hints at the potential of data science and machine learning (DS/ML) to reshape the corporate world. However, despite the headway made by a few notable titans (e.g., Google, Amazon, Apple) and upstarts, the advances that are advertised around DS/ML have yet to be realized on a broader basis. The authors examine the tension between the spectacular image of DS/ML and the realities of applying the latest DS/ML techniques to solve industry problems. The authors discern two distinct ways, or modes, of thinking about DS/ML woven into current marketing and hype. One mode focuses on the spectacular capabilities of DS/ML. It expresses itself through one-off, easy-to-grasp marketable projects, such as DeepMind’s AlphaGo (Zero). The other mode focuses on DS/ML’s potential to transform industry. Hampered by an emphasis on tremendous but as of yet unrealized...

The Stakes of Uncertainty: Developing and Integrating Machine Learning in Clinical Care

MADELEINE CLARE ELISH Data & Society Research Institute The wide-spread deployment of machine learning tools within healthcare is on the horizon. However, the hype around “AI” tends to divert attention toward the spectacular, and away from the more mundane and ground-level aspects of new technologies that shape technological adoption and integration. This paper examines the development of a machine learning-driven sepsis risk detection tool in a hospital Emergency Department in order to interrogate the contingent and deeply contextual ways in which AI technologies are likely be adopted in healthcare. In particular, the paper bring into focus the epistemological implications of introducing a machine learning-driven tool into a clinical setting by analyzing shifting categories of trust, evidence, and authority. The paper further explores the conditions of certainty in the disciplinary contexts of data science and ethnography, and offers a potential reframing of the work of doing data science and machine learning as “computational...

“Empathizing” with Machines

CHRIS BUTLER Philosophie PechaKucha Presentation When we study human systems and organizations we have a job that requires to empathize or at the very least be compassionate towards the experiences others are having. This allows to understand their goals, problems, and how we can best make their lives better. When machines start to do things that we can't imagine how do we continue to work with them? What is necessary to create great combinations of humans and machines? What is a machine's purpose? Very simply: it is to serve human purposes. As technology continues to build facades that hide the human element we need to pull back the curtain (like the one in the Wizard of Oz) and see that the tools we build are really us reflected back. We have the choice to make tools that are good or bad for us. Chris Butler, is the Director of AI at Philosophie and frequently speaks on the intersection of product, design, and AI. He has extensive experience from Microsoft, Waze, KAYAK, among others. Through his practice he has created...

Design & Data

Chair: JEANETTE BLOMBERG, Distinguished Researcher, IBM Almaden Research Center Panelists: MARC BÖHLEN, Professor, Department of Art, Emerging Practices, State University of New York at Buffalo TOM LEE, Director of Data Science, Fisher Center for Business Analytics, University of California Berkeley What does a data expert see when they look at a design problem? This panel immerses us in the practices of two data experts, both of whom have collaborated with ethnographers, as they navigate through design challenges in different ways. Chair Jeanette Blomberg draws the panelists and audience into conversation about synergies and challenges for interdisciplinary design collaborations. Jeanette Blomberg is Distinguished Researcher at the IBM Almaden Research Center and Adjunct Professor at Roskilde University in Denmark. She has done foundational work on ethnography in design processes over three decades, and her current research is focused on organizational analytics and the linkages between human action, digital data production, data...

Reading the Tea Leaves: Ethnographic Prediction as Evidence

CLAIRE MAIERS WillowTree, Inc. Those who work in research know that we live in a world that is strongly influenced by what Tricia Wang has called the quantification bias. More so than other forms of information, numbers have incredible formative power. In our culture, numbers are seen as trustworthy representations of reality that are strongly associated with objectivity and untainted by human bias and shortcomings. Recently, data science, big data, algorithms, and machine learning have fueled a new wave of the quantification bias. One of the central fascinations of this wave has been the promise that humans now have the power of prediction at their fingertips. In this paper, I reflect on what it means to make predictions and explore the differences in how predictions are accomplished via quantitative modeling and ethnographic observation. While this is not the first time that ethnographic work has been put in conversation and in contrast with quantified practices, most theorists have framed the role of ethnography as providing context...

Towards Multi-Dimensional Ethnography

JULIA KATHERINE HAINES Google, Inc. In this paper, I argue for the value of multi-dimensional ethnography. I explore the potential for ethnography to venture beyond sites, into different dimensions. As an example of work moving in this direction, I present a new approach, dubbed TRACES, which emphasizes the assemblages that constitute our lives, interweaving digital, embodied, and internal experiences. Various data streams and sources provide different vantage points for analysis and synthesis. I illustrate how we have used these to gain greater insight into the human lives we study, with different data sources providing different perspectives on a world, then delve into our use of tools, data sources, and methods from other traditions and other fields, which, combined, give us not only a more holistic picture, but a truer one, which refutes the false dichotomy of the digital and the real. I argue that we must continue to adapt and extend ethnography today into such spaces, and that reformulating the sites of ethnography as dimensions...