Advancing the Value of Ethnography

Flexibility and the Curatorial Eye: Why and How Well-Documented Fieldwork Sustains Value Over Time

Share:

Download PDF

Cite this article:

Ethnographic Praxis in Industry Conference Proceedings 2009, pp. 72–80. © American Anthropological Association, some rights reserved.

This paper discusses two distinguishing features of ethnographic work, program flexibility and data flexibility. These forms of flexibility deliver business benefits at two different timeframes. Within the timeframe of a given project, ethnographic practice is necessarily reflective and reflexive; ethnographic training insists researchers be flexible. This means projects are responsive to emerging results, and can be reframed in situ without significant additional investment. After the project has completed, carefully managed, stored and curated ethnographic materials can answer new questions, perhaps posed years after the project has ended. Thus, ethnographic work can generate business impact by sustaining value over time. Two cases illustrate the value-generating qualities of ethnographic work: one recently conducted investigation of drinking water in India and one conducted ten years ago looking at changes in collaborative practices as a result of the adoption of mobile technologies. Discussion of these cases illustrates the importance of flexibility and the curatorial eye for generating business value from ethnography.

INTRODUCTION

Qualitative, field-based studies are increasingly called upon within business contexts, and, over the past ten years, design-oriented ethnography has garnered much attention in the press (e.g., Ante 2006). As has been well documented in the EPIC conferences and elsewhere, qualitative work has been carried out to drive insights into products, features, services, applications, infrastructures, data flows, data representations, security and access protocols, organizational processes, and business decisions (e.g., Cefkin 2009; Christensen and Raynor 1997; Fine et al. 2008; Flynn and Lovejoy 2008; Squires and Byrne 2002). There is no question that well conducted qualitative research has delivered value by providing insights into markets that cut across traditional market segments, by presenting understandings of the everyday consumption of services and products, by painting realistic scenarios of uptake and offering views onto early adopter behaviors that are more nuanced than data that can be gathered remotely.

USE AND REUSE: FLEXIBILITY AND DATA CURATION

In this paper, we wish to further the discussion of our potential for highlighting areas of impact within organizations, as discussed by Flynn and Lovejoy (2008) who underscore the ways in which contextual analysis of our own practices can illuminate potential for new and renewed forms of strategic engagement within corporations. In furthering this discussion, we wish to focus on two aspects of our professional practice which are less often cited as offering clear business value: program flexibility and data flexibility. Flexibility in these two different timeframes—within the program and within data reanalysis after the program finishes—both contribute to the business value of ethnography. We will address each in turn.

Program Flexibility

By program flexibility, we mean the ability to re-direct the course of a project while conducting the research. In design-oriented ethnography, the research process is iterative, with the discussion evolving to take advantage of what the researcher learns during the observation or interview. For example, interview protocols may evolve through the course of a project, if necessitated by emerging understandings of the situated practices under investigation. Changes made in response to observations, often at fast pace and in situ, require a reconfiguration of the data gathering instruments—which in ethnographic work are the ethnographers themselves. The reason for this flexibility is ideological and rooted in the validity of open-ended conversations that vary individual to individual, rather than an insistence on a consistently applied instrument administered to every participant. By contrast, experimental methods and survey methods embody predetermined assertions about construct relevance and are less suited to the making of modifications while a program is running. The tight relationship between data collection and analysis better equips ethnographic methods to respond quickly to emergent directions during research.

Data Flexibility

By data flexibility, we mean the revisiting and reviewing of data from previous projects for new insights, which can lead to the derivation of previously unseen connections between areas. The very ‘messiness’ of ethnography’s systematic, but open approach to data collection and agnosticism about data formats is its strength for providing ongoing long-term value. Design ethnographers generate written notes, photographs, video and films, written and drawn sketches, and produce recreations of relevant artifacts. As a result of the richness of the collected data, datasets contain not just the answers to the questions posed, where analysis can focus on predetermined or emerging issues in relevant frames, but also contain rich contextual, “extraneous” data. Qualitative, field-based data collection, therefore, yields highly rich and nuanced data that carry contextual meta-data that enable the setting and activities to be re-experienced through different lenses. Data from other forms of empirical enquiry are often not so rich and not as facile; experimental control implies data can seldom be reviewed for new insights except within the confines of the experimental questions that were initially posed. The nature of evidence and its form, constrains what questions can be asked of it. By maintaining an open mind, field researchers often collect additional contextual data that later can yield great gains.

These are central areas of value that have to date been undersold, usually being subjugated to the ‘product’ of the field program – usually design recommendations, guidelines and prototypes. Both of these capabilities imply a particular set of practices around program management, including data gathering, anonymisation, storage, and curation. These have, to date, been taken as necessary skills for an ethnographer, but have not been identified as providing particular value in an organization, in part as a result of the focus on outcome/product rather than process. We believe it is important to highlight these two skill-sets. Firstly, qualitative researchers are trained to be reflective about the specific program details (Are we collecting the right data? Are we using the right methods to gather those data?) and also trained to be reflective at a strategic level (Why are we conducting this research? What are the implications of using the current methods?). Researchers are also trained to be critically reflective of programs that are not delivering results to questions posed. Strategic insights can result from such critical reflections, but credit may not be accorded to the field research team for driving these insights. Worse, if a program is halted or realigned during a field trip, on occasion the assumption is that the original field program was a failure – not that the orienting assumptions at strategy level were identified as being flawed, and that investment has been protected through reorientation. Secondly, while qualitative data is acknowledged to be ‘rich’, two critiques that are levied are that it is more time consuming to collect and less able to be generalized. We believe that the additional contextual detail means qualitative data can often be re-visited for new insights, and this can save time in project planning downstream. It is important to understand that investment in a field project can continue to have pay-offs much later. Data storage, organization, indexing, and reanalysis are key skills of good qualitative researchers. Indeed, an essential part of our craft is the storage and curation of field results, and well-trained field researchers are careful to incorporate data storage and indexing as part of the program planning and ongoing program management.

CASE STUDIES

To illustrate our points, in this paper we discuss two projects, one conducted recently and one conducted several years ago. The first, a study conducted in 2009, explored market-driven approaches to improving access to drinking water by the poor in India. Field research focused on finding opportunities for transportation and storage of treated water in slums and villages in three Indian states. The second study, conducted in 1999-2000, addressed changes in work collaborations resulting from the introduction of mobile technologies, in particular, cell phones and laptops.

Case Study 1: Drinking Water and Storage in India

In January 2009, an IDEO research team spent three weeks in India collaborating with Acumen Fund on the Ripple Effect, an eighteen-month project backed by the Gates Foundation to increase access to drinking water by the poor. The strong social mission of this project mandated a very lean budget, and the team was challenged to deliver maximum business value with limited financial outlay. The three-week duration of this field visit was a hard constraint determined by the budget, and the team was responsible for reporting opportunities from diverse water contexts throughout India during that time. There were significant logistical challenges to accessing our field sites in Gujarati villages, slums in Bangalore, and peri-urban villages outside Hyderabad. The team traveled approximately 4,000 km/2,485 miles around India by plane, car, and rickshaw in a program designed to balance breadth, or reporting needs from multiple contexts, and depth, or taking the time to cultivate and maintain the personal relationships essential to working in India.

Recruiting participants started with Acumen Fund’s network of social enterprises. We relied on their introductions to local water organizations and used those organizations to get access to communities. Our partner organizations live and work in the communities we visited, so we deferred to them about how to navigate community politics. Despite our intense time constraints, we spent up to one third of our time on visits to the water organizations’ regional offices and in introductory meetings with community leadership. Once in the field, getting access to appropriate participants was still a challenge since local leaders sometimes had strong opinions about who would show the community in the best light, putting intense pressure on getting the most useful responses during precious time with our participants.

Program flexibility allowed us to respond to emerging information in the field and to deliver maximum business value for Acumen Fund and Gates Foundation without a need to extend the time in the field. One example of in situ adaptation during this program was guiding the snowball sampling method to appropriate respondents. For example, if participants were embarrassed to discuss something as personal as water use in front of strangers, or over-dramatizing their difficulties, we moved on. Because of our flexibility, including the opportunistic decisions to divide into two parallel tracks or respectfully cut short awkward encounters, we were able to meet the program’s goals and interview a mix of families including those without access to treated drinking water (11 families), those who purchased treated drinking water (5 families), and those who had access to treated drinking water but chose not to purchase it (5 families).

Over the course of this fieldwork, our team made changes to the protocol that allowed our team to deliver maximum business value under intense time constraints. We knew from previous field research in the developing world (see Human Centered Design Toolkit in Web Resources), that supporting interviews with physical artifacts is effective. We went into the field with photo cards and sketches to help communicate, but made a decision to add a new activity in response to emerging needs. Because water is relatively cheap for its weight and bulk, we experimented with light-weight, high-value items that could be bundled with water by showing participants products from local kiosks such as singe shampoos, incense, and fruit that a door-to-door water peddler could also sell to make a water delivery route economically viable for the peddler while keeping water affordable for customers. This activity effectively helped the team gather more specific feedback, and was a nice complement to the photo sorting and discussion cards we used throughout the fieldwork.

A second change to the program demonstrates an agile re-framing of the design challenge while in the field. We were contracted to explore community water storage solutions including shared tanks, and our research plan dedicated some of our very limited time to that activity. However, in the field we quickly learned that shared tanks are undesirable because the origin and freshness of the water inside is questionable. We made an in situ decision to divert time away from community storage solutions towards a new activity better able to serve our ultimate objective of increasing access to treated water by the very poor. We were surprised that groups of slum families pooled their resources to order an entire tanker truck of water because our earlier research suggested it was not feasible. As soon as the team identified the mistaken assumption about the desirability of shared community storage early in the research, and learned that tankers we feasible, we were able to respond by recruiting a tanker truck for a visit and interview the driver. The flexibility to adapt the program while it was running led our team to gather meaningful insights with business value in a limited time.

Case Study 2: Working With Mobility

From 1999 until May 2000 a three-phase project was conducted to address changes in work-based communications resulting from the introduction of mobile technologies. This period of time saw an explosion of the use of mobile technologies such as cell phones, laptops, mobile email, and SMS providers, as well as the convergence of walkie-talkie functionality with cellular telephones. The project was developed to more deeply understand the implications of the introduction of these technologies into the workplace, and consisted of three parts: a diary study of the use of the RIM Blackberry 850 email/pager by two collaborating co-workers, a field-based interview study of 17 sales people from a multi-national printer company in the US, and a field-based study of 11 mobile phone users in the UK, looking at patterns of recreational and work use. In all three phases, data gathered were text, photographs, drawings, representations of regularly used artifacts, and, where possible, video recordings. In the case of the field studies rather than the diary study, field interviews were conducted and, where possible, interviewees were shadowed between several hours to a day. The study results focused primarily on people’s management of data flows in collaborations and on development of applications for graceful management of service interruptions. Key findings from the study with respect to use of mobile technologies were reported internally, and a framework for design was developed and published (Churchill and Wakeford 2001).

For the purposes of this paper we will focus on the second phase of the study of sales people. Whilst the study was predominantly about the use of mobile technologies, two themes emerged at the time and were also documented initially in the form of ‘asides’—reflecting our point in the previous section of this paper about program flexibility. The first theme was the effect of “hot-desking” that was intimately linked to the provision of laptops and cell phones to all personnel, replacing desktop computers and phones. The second related observation was the use of fliers and posters in the physical space as a means of communication between members of the sales team. Both of these issues were pursued as side issues to the primary data collection. These insights from this phase of the project, together with supporting data, were subsequently used on three occasions. First to drive proposals for the development of digital bulletin boards with a back-end social networking site that is also accessible from cell phones (Churchill and Nelson 2009). Secondly the data and frameworks were used to outline design opportunities for mobile applications for recreational use, and to drive the project ideation phase of a project conducted in Japan, comparatively addressing infrastructure, hardware, and service provision in Japan, the US and Europe; data from the earlier study were used as a primer and to drive research questions prior to the project launch. Finally, the data and frameworks were adapted in developing a program addressing the practices of remote work in homes and in Internet cafés, conducted in 2004.

In all instances, the original data were mined for potential insights, the original framework was questioned and elaborated, and new questions were generated to illuminate what factors did and did not generalize in the face of market changes. A key factor in the success of this data reuse was that data collected were systematically organized according to a scheme agreed upon by the team. Following Emerson et al.’s schema, during the original studies we made “jottings”, that is recording activities in terms of ‘what, how, where and when’ on paper; we made drawings of the space and took photographs of spaces used and of artifacts; we wrote extensive notes following time in the field in the form of multi-media documents depicting the settings and specific scenes; and we generated characterizations of the people involved and sketches of incidents, combining all of these into episodes and tales with dialogue (Emerson et al. 1995). In all instances, while generating detailed representations for the project at hand, we were also generating rich data capable of providing opportunities for re-experiencing, collecting, and storing data with the curatorial eye of a documentarian. It is this focus that distinguishes qualitative work from other forms of research—the focus on providing sufficient data to enable re-experiencing.

DISCUSSION: TAKING CARE OF BUSINESS

The topic of EPIC 2009 is how we can continue to have an impact and stay relevant in today’s economic climate. We have argued that program and data flexibility are two key areas of value that are currently undersold in business contexts. The two key ways program flexibility generates business value are by minimizing risk and creating rapid responses to changing market directions and organizational directives. The ability of ethnography to uncover latent customer needs and drive innovation is well documented as noted above, but an under-sold aspect of practice is the ability to reduce risk by iteratively evolving the research plan during a program and by reflecting back on orienting assumptions at business strategy levels during the initial phases of a field program. For example, if the second week of interviews on a project about in-home health monitoring devices reveals ambivalence about the topic and confusion about why it would be desirable, an ethnographic project can alter the discussion guide to focus more on community support mechanisms. This reduces risk by allowing program managers to spend a fixed amount on need-finding through ethnographic methods, and emerge at the end of the program with identified needs, rather than suggestions for further studies. A second benefit of program flexibility is the ability to evolve the research questions to meet changing organizational imperatives. For example, a company that has manufactured industrial equipment competing on low price may decide that future products should help the company build credibility as a service organization specializing in high-value added interactions with staff using their formerly commoditized machines. Some forms of customer research would be unable to respond to the shift and would result in answers to questions that are no longer relevant or in cancelled programs. In contrast, design-oriented ethnography can evolve the program mid-stream by maintaining the core human experience of interacting with the equipment, and identifying the most frustrating elements of interactions with the system that are prime candidates for escalation into a service. Of course, as Flynn and Lovejoy (2008) point out, internal buy-in at higher management levels must be in place; reorientations must be interpreted as having potential to drive strategic reassessment, not interpreted as failed field programs.

Post-program completion ethnographic data can be re-mined to provide insights on questions and issues that may not emerge for some time. This property of ethnographic work is data flexibility. A key part of data flexibility is a commitment to a good data management scheme and careful curation of rich, multimedia data. Also worthy of note is that ethnographic training foregrounds ethical practices around data gathering, storage, and retention. Informed consent, reflective consideration of the potential for deception and/or unintended negative consequences, and guidelines for careful data anonymisation and storage are a required part of the practice of the trained ethnographer. This is of particular significance in these times of increasing concern for people’s privacy and the focus on establishing improved data retention policies within corporations and academic institutions. In many ways, ethnographers are well positioned to advise on such data retention policies—in addition to a clear ethical stance with regard to the collection of data, part of the craft of ethnography is the recreation of experiences and “persons” from multiple, partial, multi faceted data; the addition of an ethnographic sensibility to more traditional data mining may well yield more nuanced, effective guidelines for data management.

SUMMARY

In this paper, we have presented two case studies where we have highlighted how key features of ethnographic practices and processes, in addition to specific project deliverables and products, offer sustained business value. These features are program and data flexibility. Program flexibility derives from the reflective process of framing and reframing projects as data are gathered in the field, and the skill of the ethnographer in multi-layered analysis of activity. We further discussed how a key aspect of this business value comes from management, indexing, and annotation of stored multi-media data. We see opportunities not only for better technologies of data management and curation, but also business value in more clearly supporting these features of field activity.

NOTES

The authors would like to acknowledge Sally Madsen, Maria Redin, Jocelyn Wyatt, Caroline Stanculescu, Holly Kretchmar, Arna Ionescu, Sangeeta Chowdhry, Adrien Couton, Uma Hemachandran, and Sam Sternin for their contributions.

Elizabeth Churchill is a Principal Research Scientist at Yahoo! Research, where she manages the Internet Experiences research group. Her research focuses on personal and public social technologies.

Ame Elliott is a Senior Human Factors Specialist at IDEO in Palo Alto, California. Her research focuses on developing technology strategy grounded in human needs.


REFERENCES

Cefkin, Melissa (Ed.)
2009 Ethnography and the Corporate Encounter: Reflections on Research in and of Corporations. Berghahn Books.

Christensen, Clayton E. and Raynor, Michael E.
2003 The Innovator’s Solution: Creating and Sustaining Successful Growth. Boston, Mass, US: Harvard Business School Press.

Churchill, Elizabeth and Nelson, Les
2009 From Media Spaces to Placed Media: Ubiquitous Computing, Community Network and Public Space Publishing. In S. Harrison (Ed.) Media Space: 20+ Years of Mediated Life. Springer Verlag.

Churchill, Elizabeth F. and Wakeford, Nina
2001 Framing Mobile Collaborations and Mobile Technologies. In B. Brown, N. Green, R. Harper (Eds.) Wireless World: Social and Interactional Aspects of Wireless Technology. Springer Verlag.

Flynn, Donna K. and Lovejoy, Tracey
2008 Tracing the Arc of Ethnographic Impact. In Proceedings of EPIC 2008. pp 238-250.

Emerson, Robert M., Fretz, Rachel I. and Shaw, Linda L.
1995 Writing Ethnographic Fieldnotes. The University of Chicago Press.

Fine, Gary, Morrill, Calvin and Suiamarain, Sharmi
2008 Ethnography in Organizational Settings. In Handbook of Organizational Research Methods, edited by David Buchanan and Alan Bryman. London: Sage.

Ragin, Charles C.
1994 Constructing Social Research. Pine Forge Press.

Squires, Susan and Byrne, Bryan (Eds.)
2002 Creating Breakthrough Ideas: The Collaboration of Anthropologists and Designers in the Product Development Process. Westport, CT: Development Industry, Greenwood Publishing.

Web resources

Ante, Spence E. The Science of Desire
http://www.businessweek.com/magazine/content/06_23/b3987083.htm, accessed March 12, 2009.

Human Centered Design Toolkit
http://www.ideo.com/images/HCD_COMPLETE_TOOLKIT.pdf, accessed August 15, 2009.

Share: