The Automation of Qualitative Methods

Share Share Share Share Share

by SALLY A. APPLIN, PhD

Bauhaus by cdschock via flickr

Introduction

Anthropology and its methodologies cannot easily be automated. However, both design and engineering based organizations are attempting it. I argue that this is based in part on historic legacy systems, a misunderstanding of the ethnographic toolkit, and an over-reliance on the principles of Bauhaus, Six Sigma, and Science Fiction.

Quantifiying and Automating the Qualitative

After interviewing at several other engineering focused companies in their User Experience groups, I recently interviewed for a job at a renowned design firm. The design firm advertised for a "Director of Insights and Strategy," a job I’m well suited for. However, after I travelled to their offices, gave a presentation and spoke with them, it became apparent that what they really wanted was probably just a Research Manager (e.g. someone to provide a checklist of conventional methods that can be replicated, and someone who would guide others to use them as well). Prior to my arrival, the company had said that they wanted "innovation and creativity," but when confronted with the prospect of that and what it meant, they reverted in the interviews to requiring me to list traditional methods—methods which they had initially claimed weren't working, and methods which they appeared to quantify as a mark of assessing my expertise.

The repeated pattern I’ve seen in design firms, start-ups, and even some established larger companies, uses automation principles in its enactment. Initial correspondence may be with the hiring manager, but interviews are set up by an HR person or calendar bot and scheduled via email. The hiring manager delegates the interview to a junior employee, who usually has no scope of project and little experience in a range of project types, and who most often comes from an entirely different discipline. Interview questions are formulaic, relating to quantifying my methods, and streamlining particular methods—without regard for the broader methodological context of when these methods may or may not work—or why—or in what context. I express curiosity about the questions the interviewers ask, and bring up the importance of a discovery or definition phase to determine appropriate methodology. I also query how what they are proposing to research fits into a broader realm. These questions are not interesting to these interviewers, who prefer to guide the interview with role-playing scenarios, and I have been asked to "geek out" on terminology and methods. In these cases, my lines of qualitative inquiry are not identified as such and are misinterpreted as naive and uninformed, rather than the 'probe and discovery' method that I am using. Although some interviewers allotted me more time, in many situations the interviewer spoke for exactly 25 minutes and offered me 5 minutes to ask questions, with a sharp termination at the 30 minute mark, leaving little or no time for my questions to be answered.

Based on repeated exposure to this pattern, it has occurred to me that the disciplines of engineering and design are trying to find ways to codify, repeat, and, more critically, automate the process of hiring anthropologists (and others), as well as to automate their research methodology.

The other pattern that I have noticed is that corporate User Experience Research groups are frequently headed by a male with an upper management title who holds a degree in a field such as literature, arts, business, economics—but not anthropology, sociology, or psychology—and who has built up a group of much younger practitioners generally with design or other non–social science backgrounds. Occasionally, I was asked to interview with an early career cognitive scientist (a discipline mostly is concerned with an individual's brain function). Thus, as I seek my post-doctorate work 'home,' I’m becoming increasingly concerned, in a broad sense, about the role of anthropology in the design and technology industries, and where my skills will be authentically valued.

I consulted former colleague and design anthropologist Dr. Susan Squires, and asked for her take on why I was repeatedly being asked to list methods and to "geek out" on terminology in these research interviews. She observed that those who are measuring my worth by lists of methods may be confusing data collection with research, and that to them, the more methods one has or can discuss, the better and more senior they will be at what they define as "research." For designers who are not steeped in research, and lack a research degree, methods may be their only metric for data collection, so data collection is their definition for research, rather than methods merely being a part of what research encompasses.

These research managers seem to have a need to quantify methods or other attributes. For the most part, ethnography is omitted from the conversation, or described as a single method rather than as a toolkit of methods. The people in control of interviews do not understand the basics of ethnographic methodology and what anthropology does, or can do for them, instead assuming it to be a part of a discipline that can be streamlined, made 'lean,' easily quantified, and lumped in with their other methods.

Bauhaus, Six Sigma, and Science Fiction

In both design and engineering, there is a similar trend to ‘simplify' and 'reduce to basic principles,' and at times, both share a 'least common denominator' way of thinking. In design, the idea of 'simplify' has worked well visually in terms of clearing up visual space, but it has done nothing to handle complexity.

I spoke about this with Daniel Drennan El-Awar, a graphic designer and former professor in graphic design and media design. He reminded me that designers today are still influenced by Bauhaus, an experimental German post-World War I modernist movement founded on the idea of reduction and simplicity of both form and color palette, which additionally incorporated white space. Building on that notion, as a design movement from the early part of the 20th century, Bauhaus could be considered a form of automation, which was also debated at the time.

Today, we still see the Bauhaus influence in our furnishing movements, architecture, and other areas of design that rely on the notion and visual language of clean, white space, and simplicity as aspirational qualities—even though for the most part Bauhaus did not produce livable spaces that endured for communities of people. Its reductionist principles are simply unsuited to address social complexity.

Meanwhile, the idea of Six-Sigma has largely influenced User Experience and User Interaction design. Developed by an engineer at Motorola as a process improvement strategy, Six Sigma was adopted across a wide range of industries, especially business. Six-Sigma relies on two methods: DMAIC (Define, Measure, Analyze, Improve, Control) and DMADV (Define, Measure, Analyze, Design, Verify). Much of the focus of Six Sigma is on quantitative measurement, and more recently, Six-Sigma has been adopted as a way to streamline UX design, by reducing it to standardized principles, and applying these across all forms of design. Organizations wanting a fast formula for 'better' processes can easily be persuaded to include the reductionist rules and methodology of Six Sigma, which advocates rigid methods that are perceived to be a reliable formula. When designers and engineers work together in engineering based organizations, any process that provides quantified results, which support a quantitative, mathematical, and reductionist approach, seems to be preferred.

It makes logical sense that Six Sigma's process would derive from an engineering mindset, and it makes logical sense that when faced with users and their needs, engineering would look for an engineering solution to solving said 'problems' with regard to people.

Engineering, in particular as it is practiced in Silicon Valley, also has an aspirational vision described in Science Fiction, and which is built visually on the design forms founded in the 1950s and 1960s (based on Bauhaus). Although we've moved along in years, many of our design and engineering visions are inspired by a 60-year-old imagination of the future, which is often described in a context 300 years ahead of where we are now.

Where This Leaves the Ethnographer and Other Qualitative Researchers

The current path of User Experience design (and by association, User Experience Research) has been deeply co-opted by Six Sigma principles, by designers and design firms steeped in Bauhaus training, and practitioners required to function within the scope of an engineering industry that prizes mathematics and quantification as their only acceptable metric. If ethnographers and other qualitative researchers join these companies in a research capacity, we are expected to embrace three 'beliefs': 1) design 'simplicity' and Bauhaus; 2) a Six Sigma method of quantification as the only method for measuring results; and 3) an engineering vision derived from science fiction, which provides a working solution that only requires measurement of effectiveness because it's functionality has been previously demonstrated to 'work' in Science Fiction narratives.

Ethnographic praxis does not easily lend itself to any one of these disciplines or methods precisely because people are complex, take agency in their own ways, and are part of a reality that is the now: the nested and interrelated systems in which we live and exist, overlap, fragment, spiral, swirl, and intertwine with each other, or miss and do not overlap at all.

All three of these dominating 'beliefs' from design, manufacturing, and engineering lend themselves easily to forging automated processes, where rules can be put in place and enacted in projects that fulfill the vision of the aspirational future from both design and engineering. However, the outcomes from these over-focused, rigid, and brittle processes will be vulnerable, as systems become inflexible to change and human agency. We are up against a triad of methodology and beliefs that were created for things, but are now being applied to people. People are creative and prefer to take agency when we are trying to solve problems, cooperate, and engage with the world. The metrics-based 'automated process' pattern is building a world that will be easy to automate initially, but that may be—and already is in some instances—ineffective and potentially dangerous as brittleness and inflexibility increases, and human agency decreases, limiting our options and choices for problem solving and cooperation.

As EPIC practitioners, we must consider how to offer our tools, knowledge and skills to organizations and industries that value what we have to offer, and that are not yet automated, or which may be but, after many mistakes, now understand the grave nature of getting it wrong. We will do best in companies who have expanded their belief systems to include us, and our toolkits, in order to create more flexible, adaptive systems, which will preserve and encourage the entire range of human creativity.

Photo: "Bauhaus" by cdschock (CC by 2.0) via flickr.


sally applinSally A. Applin is a Senior Researcher based in Silicon Valley, examining the changing relationship between humans and algorithms, the impact of technology on culture, Maker culture, leading technologies, and the outcomes of network complexities as modeled by PolySocial Reality (PoSR). Dr. Applin earned her PhD in Anthropology at the University of Kent at Canterbury, UK, working with the Centre for Social Anthropology and Computing (CSAC). Sally is an Associate Editor of the IEEE Technology and Society Magazine, Associate Editor of the IEEE Consumer Electronics Magazine (Societal Impacts Section), a member of IoT Council (a think tank for the Internet of Things (IoT)), and a board member of the Edward H. and Rosamond B. Spicer Foundation.


Related

The Rise of the User and the Fall of People: Ethnographic Cooptation and a New Language of Globalization, Shaheen Amirebrahimi

The De-skilling of Ethnographic Labor: Signs of an Emerging Predicament, Gerald Lombardi

The Future of User Research, Apala Chavan

Ethnographers, Bearers of Bad News, Sam Ladner

  5 comments for “The Automation of Qualitative Methods

Leave a Reply