Hybrid Methodology: Combining Ethnography, Cognitive Science, and Machine Learning to Inform the Development of Context-Aware Personal Computing and Assistive Technology

NOTES

Acknowledgments – The authors would like to thank Sean Keller, Joshua Walton, Jerome Holman, James Hillis, Sean Trowbridge, Karl Ridgeway, and Hrvoje Benko for the support of this work at Facebook Reality Labs as well as Dave Baiocchi, David Kearford, Michael Marcusa, Mads Nielsen, and Gumbo for their support with this work at ReD Associates.

* Additional co-authors: 
[contrib_author name=’SEBASTIAN BARFORT’ post_id=’21117′], ReD Associates
[contrib_author name=’SÉRÉNA BOCHEREAU’ post_id=’21117′], Facebook Reality Labs
[contrib_author name=’JONATHAN BROWDER’ post_id=’21117′], Facebook Reality Labs
[contrib_author name=’TANYA R. JONKER’ post_id=’21117′], Facebook Reality Labs
[contrib_author name=’KAHYUN SOPHIE KIM’ post_id=’21117′], Facebook Reality Labs
[contrib_author name=’MIKKEL KRENCHEL’ post_id=’21117′], ReD Associates
[contrib_author name=’MORGAN RAMSEY-ELLIOT’ post_id=’21117′], ReD Associates
[contrib_author name=’FRIEDERIKE SCHÜÜR’ post_id=’21117′], Cityblock Health
[contrib_author name=’DAVID ZAX’ post_id=’21117′], ReD Associates
[contrib_author name=’JOANNA ZHANG’ post_id=’21117′], ReD Associates

2019 EPIC Proceedings, ISSN 1559-8918, https://www.epicpeople.org/epic

REFERENCES CITED

anderson, ken, Dawn Nafus, Tye Rattenbury. 2009. “Numbers Have Qualities Too: Experiences with Ethno-Mining.” Ethnographic Praxis in Industry Conference Proceedings: 123-140. https://www.epicpeople.org/numbers-have-qualities-too-experiences-with-ethno-mining/

Arora, Millie P., Mikkel Krenchel, Jacob McAuliffe, and Poornima Ramaswamy. 2019. “Contextual Analytics: Towards a Practical Integration of Human and Data Science Approaches in the Development of Algorithms.” Ethnographic Praxis in Industry Conference Proceedings: 224-244. https://www.epicpeople.org/contextual-analytics/

Benoit, Kenneth, Drew Conway, Benjamin E. Lauderdale, Michael Laver, and Slava Mikhaylov. 2016. “Crowd-sourced text analysis: Reproducible and agile production of political data.” American Political Science Review, 110 (2): 278-295.

Biehl, João. 2013. “Ethnography in the Way of Theory.” Cultural Anthropology 28 (4): 573-597.

Bornakke, Tobias and Brian L. Due. 2018. “Big–Thick Blending: A Method for Mixing Analytical Insights from Big and Thick Data Sources.” Big Data & Society January—June 2018: 1-16.

Braverman, Irwin. 2011. “To see or not to see: How visual training can improve observational skills.” Clinics in Dermatology 29 (3): 343-346.

Christensen J. C., Estepp J. R., Wilson G. F., and Russell C. A. 2012. “The Effects of Day-to-Day Variability of Physiological Data on Operator Functional State Classification.” Neuroimage 59: 57–63.

Hwan-Hee Choi, Jeroen J. G. van Merriënboer and Fred Paas. 2014. “Effects of the Physical Environment on Cognitive Load and Learning: Towards a New Model of Cognitive Load.” Educational Psychology Review 26 (2): 225-244.

Csikszentmihalyi, Mihaly. 2008. Flow: The Psychology of Optimal Experience. New York: Harper Perennial Modern Classics.

Cury, Maria and Daniel Bird. 2016. “Applying Theory to Applied Ethnography.” Ethnographic Praxis in Industry Conference Proceedings: 201-216. https://www.epicpeople.org/applying-theory-applied-ethnography/

Dijkstra, Edsger W. “The Humble Programmer.” 1972. ACM Turing Award Lecture 15 (10): 859-866.

Duranti, Alessandro and Charles Goodwin. 1992. “Rethinking Context: An Introduction.” In Rethinking Context: Language as an Interactive Phenomenon, 1-42, Vol. 11, edited by Alessandro Duranti and Charles Goodwin. Cambridge: University Press.

Engeström, Yrjö, Reijo Miettinen, and Raija-Leena Punamaki (eds.) 1999. Perspectives on Activity Theory, Cambridge: University Press.

Fraser, Kristin, Irene Ma, Elise Teteris, Heather Baxter, Bruce Wright, and Kevin McLaughlin. 2012. “Emotion, Cognitive Load and Learning Outcomes During Simulation Training.” Medical Education 46: 1055-1062.

Fridman, Lex, Daniel E. Brown, Julia Kindelsberger, Linda Angell, Bruce Mehler, and Bryan Reimer. “Human Side of Tesla Autopilot: Exploration of Functional Vigilance in Real-World Human-Machine Collaboration.” 2019.

Geertz, Clifford. 1973. The Interpretation of Cultures. New York: Basic Books.

Geertz, Clifford. 1998. “Deep Hanging Out.” New York Review of Books 22 October 1998 issue. Accessed November 1, 2019. https://www.nybooks.com/articles/1998/10/22/deep-hanging-out/

Glaser, Barney G. and Anselm L Strauss. 2017. Discovery of Grounded Theory: Strategies for Qualitative Research. New York: Routledge.

Goldhagen, Sarah Williams. 2017. Welcome to Your World: How the Built Environment Shapes Our Lives. New York: HarperCollins.

Grasseni, Cristina, ed. 2007. Skilled Visions: Between Apprenticeship & Learning. New York and Oxford: Berghahn Books.

Helton, William S. and Katharina Näswall. 2015. “Short Stress State Questionnaire: Factor Structure and State Change Assessment.” European Journal of Psychological Assessment 31: 20-30.

Hou, Carolyn and Mads Holme. 2015. “From Inspiring Change to Directing Change: How Ethnographic Praxis Can Move beyond Research.” Ethnographic Praxis in Industry Conference Proceedings: 190-203. https://www.epicpeople.org/from-inspiring-change-to-directing-change-how-ethnographic-praxis-can-move-beyond-research/

Ingold, Tim. 1993. “The Temporality of the Landscape.” World Archaeology 25 (5): 152-174.

Jonker, Tanya R., Eryn Whitworth, Kahyun Sophie Kim, Jonathan Browder, Serena Bochereau, Hrvoje Benko, Sean Keller, Sebastian Barfort, Maria Cury, Mikkel Krenchel, Morgan Ramsey-Elliot, Friederike Schuur, David Zax, Joanna Zhang. In review. “Mental Effort During At-Home Activities: Moments of High Mental Effort Can Reveal New Opportunities for Technology.” Submitted to ACM CHI Conference on Human Factors in Computing Systems 2020.

Kane, Michael J., Leslie H. Brown, Jennifer C. McVay, Paul J. Silvia, Inez Myin-Germeys, and Thomas R. Kwapil. 2007. “For Whom the Mind Wanders, and When: An Experience-Sampling Study of Working Memory and Executive Control in Daily Life. Psychological Science 18 (7): 614-621.

Kaptelinin, Victor and Bonnie A. Nardi. 2006. Acting with Technology: Activity Theory and Interaction Design. Cambridge: The MIT Press.

Khatri, Chandra, Rahul Goel, Behnam Hedayatnia, Angeliki Metanillou, Anushree Venkatesh, Raefer Gabriel, and Arindam Mandal. 2018. “Contextual Topic Modeling For Dialog Systems.” In 2018 IEEE Spoken Language Technology Workshop (SLT), 892-899.

Killingsworth, Matthew A., and Daniel T. Gilbert. 2010. “A Wandering Mind is an Unhappy Mind.” Science 330.6006: 932.

Larson, Reed, and Mihaly Csikszentmihalyi. 2014. “The Experience Sampling Method.” In Flow and the Foundations of Positive Psychology, 21-34. Dordrecht: Springer.

Lottridge, Danielle, Mark Chignell, and Aleksandra Jovicic. 2011. “Affective Interaction: Understanding, Evaluating, and Designing for Human Emotion. Reviews of Human Factors and Ergonomics 7 (1): 197-217.

Madsbjerg, Christian and Mikkel Rasmussen. 2014. The Moment of Clarity: Using the Human Sciences to Solve Your Biggest Business Problems. Boston: Harvard Business Review Press.

Matthews, Gerald, James Szalma, April Rose Panganiban, Catherine Neubauer, and Joel S. Warm. 2013. “Profiling Task Stress with the Dundee Stress State Questionnaire.” In Psychology of Stress, 49-91, edited by Leandro Cavalcanti and Sofia Azevedo. Nova Science Publishers, Inc.

Mauss, Marcel. 2009. Manual of Ethnography. Translated by N.J. Allen. New York: Berghahn Books.

Menkedick, Sarah. 2018. “Behind the Writing: On Interviewing.” Longreads, July. Accessed November 1, 2019. https://longreads.com/2018/07/20/behind-the-writing-on-interviewing/

Nardi, Bonnie A., ed. 1996. Context and Consciousness: Activity Theory and Human Computer Interaction. Cambridge: The MIT Press.

Paas, Fred G. 1992. “Training Strategies for Attaining Transfer of Problem-Solving Skill in Statistics: A Cognitive-Load Approach.” Journal of Educational Psychology 84 (4): 429-434.

Pink, Sarah. 2007. Doing Visual Ethnography. London: Sage Publications.

Pink, Sarah. 2009. Doing Sensory Ethnography. London: Sage Publications.

Redelmeier, Donald A. and Daniel Kahneman. 1996. “Patients’ Memories of Painful Medical Treatments: Real-Time and Retrospective Evaluations of Two Minimally Invasive Procedures.” Pain 66 (1): 3-8.

Reis, Harry T. and Gable, Shelly L. 2000. “Event-sampling and Other Methods for Studying Everyday Experience. In Handbook of Research Methods in Social and Personality Psychology, 190-222, edited by H. T. Reis & C. M. Judd. New York: Cambridge University Press.

Roth, Wolff-Michael. 2004. Activity Theory and Education: An Introduction. Mind, Culture, and Activity, 11 (1): 1-8.

Schmeck, Annett, Maria Opfermann, Tamara van Gog, Fred Paas, and Detlev Leutner. 2015. “Measuring Cognitive Load with Subjective Rating Scales During Problem Solving: Differences Between Immediate and Delayed Ratings.” Instructional Science 43: 93-114.

Searle, John R. 1969. Speech Acts: An Essay in the Philosophy of Language. Cambridge University Press.

Smallwood, Jonathan and Jonathan W. Schooler. 2015. “The Science of Mind Wandering: Empirically Navigating the Stream of Consciousness.” Annual Review of Psychology 66: 487-518.

Sweller, John. 1993. “Some Cognitive Processes and Their Consequences for the Organization and Presentation of Information.”Australian Journal of Psychology 45: 1–8.

Van Gog, Tamara, Femke Kirschner, Liesbeth Kester, and Fred Paas. 2012. “Timing and Frequency of Mental Effort Measurement: Evidence in Favor of Repeated Measures.” Applied Cognitive Psychology 26: 833–839.