This paper explores a way to expand business using ethnography as an industrial service or product. First, a challenge that companies are facing and trying to deal with, which is industrialization is described. In the software industry, as computer prices go down, the requirements for software development involve accurate estimates of the cost, the time and the resources involved in the process. Due to these new market demands, software development reached a level of maturity, which required a new approach to product development. Likewise, as ethnography grows into more intricate realms, there is a need for a more robust approach to ethnography application in business to help it achieve the right maturity level of industrialized processes. In this context of complexity, case studies from Fujitsu and examples from literature were used to test the Capability Maturity Model Integration (CMMI) framework to use to evaluate the practice of ethnography in business. As a result, a brief assessment of ethnography using the CMMI framework is shown. It describes how ethno-methodology can be used in a repeatable manner and provides results within a small margin of difference in quality. Finally, a reflection on the value of experts in this new trend in ethnography is described.
Ethnography has grown popular in the business world. High-tech firms such as IBM, Yahoo!, and Google have hired anthropologists and developed ethnographic capabilities. Similarly, several government research labs, such as NASA and Sandia1, have brought anthropologists in-house to work in labs or in organizationally focused roles as ethnographers and scholars (Cefkin 2009).
In the software history, thirty years ago computers were huge and expensive; software was developed by highly educated designers and programmers to make beautifully optimized programs with deep knowledge. Now computers became cheep with high speed CPUs and a large amount of memory space. Customers ask programmers to make cheaper programs with reasonable quality. Programs became huge and complicated and customers require accurate estimates of the cost, the time and the resources. So development projects have to be managed to keep their quality reasonable.
Ethnography seems to follow the track of software development. Some years ago, customers asked ethnography as an art, but they are thinking the result of business ethnography as a product. They started to ask estimation of resources to compare between plans by research companies. Ethnography will have to be managed as same way as a software development project shortly.
Following this trend, requirements for business ethnography will keep growing. There are huge opportunities for large global companies to involve. Some customers will require really deep insight by highly educated experts, while others will be satisfied with the results by more light and abridged ethnography. In any case, ethnography efforts will have to be managed in a stricter manner to comply with business requirements.
This paper shows a trial application of the Capability Maturity Model Integration (CMMI) framework to evaluate the state of ethnography and the highlights key findings from this trial in a large software company that brought ethnography into their business context. It explores the implications of this experiment to the ethnography practice in business and discusses the implications for the future of ethnography.
OUTLINE OF CAPABILITY MATURITY MODEL INTEGRATION (CMMI)
Capability Maturity Model Integration – CMMI; is an approach for improving processes that provides organizations with the essential elements of effective processes that ultimately improve their performance (Godfrey 2008). A CMMI model may also be used as a framework for appraising the maturity of processes in the organization. The Capability Maturity Model involves five aspects: maturity levels, key process areas, goals, common features, and key practices.
The Capability Maturity Model involves the following aspects:
- Maturity Levels: a five-level process maturity continuum where the uppermost (5th) level is a notional ideal state where processes would be systematically managed by a combination of process optimization and continuous process improvement.
- Key Process Areas: a Key Process Area (KPA) identifies a cluster of related activities that, when performed collectively, achieve a set of goals considered important.
- Goals: the goals of a key process area summarize the states that must exist for that key process area to have been implemented in an effective and lasting way.
- Common Features: common features include practices that implement and institutionalize a key process area. There are five types of common features: commitment to Perform, Ability to perform, Activities performed, Measurement and Analysis, and Verifying implementation.
- Key Practices: The key practices describe the elements of infrastructure and practice that contribute most effectively to the implementation and institutionalization of the KPAs.
FIGURE 1 CMMI maturity levels
In this paper, we will focus on the maturity levels to assess some cases from our experience and from literature to evaluate ethnography’s maturity through process optimization and continuous process improvement. There are five maturity levels defined along the continuum of the CMMI:
- Level 1 – Initial (Chaotic): A characteristic of processes at this level is that they are (typically) undocumented and in a state of dynamic change, tending to be driven in an ad hoc, uncontrolled and reactive manner by users or events. This provides a chaotic or unstable environment for the processes.
- Level 2 – Repeatable: A characteristic of processes at this level is that some processes are repeatable, possibly with consistent results. Process discipline is unlikely to be rigorous, but where it exists it may help to ensure that existing processes are maintained during times of stress.
- Level 3 – Defined: A characteristic of processes at this level is that there are sets of defined and documented standard processes established and subject to some degree of improvement over time. These standard processes are in place (i.e., they are the as-is processes) and used to establish consistency of process performance across the organization.
- Level 4 – Managed: A characteristic of processes at this level is that, using process metrics, management can effectively control the as-is process (e.g., for software development). In particular, management can identify ways to adjust and adapt the process to particular projects without any measurable loss of quality or deviations from specifications. Process capability is established from this level.
- Level 5 – Optimized: A characteristic of processes at this level is that the focus is on continually improving process performance through both incremental and innovative technological changes and improvements.
At maturity level 5, processes are concerned with addressing statistical common causes of process variation and changing the process (for example, to shift the mean of the process performance) to improve process performance. This would be done at the same time as maintaining the likelihood of achieving the established quantitative process-improvement objectives.
ASSESSMENTS FROM PERSPECTIVES OF MATURITY
In this section, assessments of ethnographic research using the maturity levels approach are shown. The following evaluations depict our point of view how to make customers satisfy in ethno-methodology business through actual case studies.
Level1 (Initial): Traditional ethnographic research
Traditional ethnographies are assessed as being level 1, which is “Initial”. Sato has commented, “As it has been pointed out for a long time, it is almost impossible to create a ‘cookbook’ type of manual for fieldwork, which is because nothing is as far from the intrinsic nature of on-site investigation than strict application of some formulae or rules” (Sato 2002). The essence of ethnography is in the depiction of culture and there may be a great variety of target sites. For that reason, traditional ethnography combines different investigation methods best suited to the situations at the sites. For example, Lancaster University, which has made use of ethnography to design systems for air traffic, says that there is no well-established investigation method:
“There is no one method of ethnographic analysis. The approach we have adopted involves various practices, alerting concepts, and semi-technical terms, which we can indicate but have no space to defend in detail. The fieldworkers immersed themselves in the work by spending several months observing activities on and around the sites, talking to staff, and talking with them so that they could develop an understanding of what the controllers do.” (Hughes, 1992)
Level 2 (Repeatable): Repeatable ethnography
Spradley et al. (1979) has structured the process of ethnographic interviews and defined elements that are important in interviewing. Beyer et al. (Beyer 1997) has defined a method of gaining an understanding of the user’s context for system design and has defined procedures for understanding the context such as the artifact model and sequence model.
These techniques are intended for navigating the process of ethnography and can be seen as indicators of the procedure of ethnography that has been conducted at a personal level.
Case examples have been reported in which ethnography was used to determine specifications in the fields of system development and product development. For example, Bentley et al. (Bentley 1992) have reported on case examples where ethnography has been used in the design of air traffic control systems, in which they say that it is difficult to put successful cases to wider applications:
“We have had the advantage that the people involved from both sociology, and software engineering were willing to be flexible and willing to recognize that the other discipline’s viewpoint may be valid, irrespective of how alien it might be. In this respect, we believe our collaboration has been successful but there have been difficulties which we believe are likely to arise in other collaborative projects of this nature.”
This shows that success can only be achieved under certain conditions, which leads to the conclusion that the standard process of ethnography has not been defined.
Level 3 (Defined): A case study
In this section, activities in Fujitsu are described, and it is assessed whether Fujitsu satisfies level 3 or not.
Fujitsu is a Japanese based system integration company and that has made use of fieldwork not as a tool for research but as a tool for qualitatively analyzing customers’ work practices and problems that supplement quantitative customer analysis in the conventional marketing and product design. Ikeya et al. (Ikeya 2007), an anthropologist who conducted fieldwork training for Fujitsu, has commented: “As they accumulate their experiences, they customize fieldwork into a more systematic method in order to cope with constraints they have on the overall schedule. They have developed a quite effective way of doing team ethnography.”
In the activity called “engineering of ethnography,” Kishimoto et al. (Kishimoto 2009) standardized operation procedures and outputs to implement them at more than 50 customer sites and gave training to more than 400 employees for the purpose of conducting fieldwork in a few weeks, which is generally executed over a few months to a few years in the academic world, to business.
Kishimoto and his teams collected a series of field notes and findings made by a trained team and they counted the volume of the notes and the number of findings day by day. Based on this empirical study, the number of days for shadowing has been established. The team also developed a technique and guidelines for standardizing the quality of activities of ethnography in addition to the quantitative aspect.
It is defined that several viewpoints such as time sequence, space and human relations are necessary to obtain a large amount of qualitative information in a short time – as shown in Figure 2.
FIGURE 2. A meta-model of work context
To assist with exhaustively covering these viewpoints for interviewing, several types of worksheets are provided (Ishigaki 2009). Quantitative issues have been standardized by designing in advance which worksheet to use at a given time or situation, which also allows for the definition of the types of worksheets to use and the extent of information obtained by interviewing, like shown in Figure 3.
FIGURE 3 – Worksheet examples
In the study’s events, a trained team was charged with the negotiations for a system renewal and the front-end of requirements definition for a branch office at a financial institution. They conducted interviews and a one-day observation at a financial institution to analyze the workers’ invisible human awareness of issues and their intrinsic motivation. The findings were shared in a session with executives, general managers of each branch office, and with directors and chiefs of the department of clerical work. The results were praised higher than the team’s original expectations and they succeeded in the system development negotiation without lapsing into price competition.
It has been pointed out that fieldwork by engineers without fieldwork experience tend
s to result in analysis from an etic2 point of view such as pointed out by Obata et al.(Obata 2007): “They evaluated work practices based on their own observations and perspectives rather than describing problems from the workers’ point of view. We reminded them repeatedly about the guidelines and procedures for an ethnographic approach”. So, guidelines to help lead to emic3 results, as depicted in Figure 4, were developed
FIGURE 4 – A process in the guideline (Obata 2007)
Aim is proposed by Yagi et al. (2009) to help researchers gain an understanding right down to the inner deep level of customers. Aim is composed of a phase of asking customers about past positive experiences and a phase of discussing, based on the result of the previous phase, the source of energy for action and values of the customers. “The most distinct features of this model are the ability to extract “future vision and direction and past positive experiences,” The focus is on positive feelings—values, features, source of motivation that envelop the everyday duties of workplace users and the products and services that they use.(Yagi 2009)
FIGURE 5 – Assessment of the case
Figure 5 is a summary of an assessment of the case. As the figure shows, Fujitsu’s standard processes are in place and they are trying to establish consistency of process performance across the organization. Through this assessment case study, Fujitsu can be assessed as level 2 which is “Repeatable” and they are on the way to level 3 which is “ Defined.” To be assessed as level 3, Fujitsu needs to establish consistency of process performance across the organization and needs some degree of improvement over time. Measurement is required as level 4. Although Fujitsu’s method contains steps or concepts for improving the quality, apparently it does not extend to the assessment of quality.
IMPLICATIONS FOR FUTURE ETHNOGRAPHY
Toward Level 4: Measured
A number of researchers have pointed out qualitative issues with fieldwork executed by engineers. Obata et al. (Obata 2007) mention, in addition to the tendency toward being etic, that the description of episodes tends to be insufficient and the observation on the relation between an episode and recognition often contains a jump: “However, the descriptions of episodes were sometimes insufficient. Members of the inspection team who were outside the project teams had difficulty understanding what was going on in the projects. Professional ethnographers sometimes pointed out that the relationship of the episodes to the discussions was unclear.”
Jordan et al. (2008) point out that fieldwork conducted by engineers lacks a theoretical framework and does not go beyond stopgap solutions: “They were missing the kind of theoretical framework that would allow them to go beyond proposing “band-aid” solutions and instead consider the implications of their local observations for the company as a whole.”
This indicates that there is more than one viewpoint in assessing quality, to say the least. One is validity, or whether the validity of the analysis is sufficient as a baseline. Assessment of whether or not a proposition objectively deduced from facts is clearly distinguished from the assertion of an observer is required. The second is ethnocentric findings. The core of ethnography is to obtain the viewpoint of residents, and how much has been found from an emic point of view can be a measure of assessment. The third is business effectiveness. There is apparently a discussion of whether or not the results of analysis are able to support the deduction of essential improvement plans in business.
Rejon et al. (2009) propose a framework of metrics and index for ethnography, which are believed to provide measures of business effectiveness. “Although there is no proven formula for ethnographic value recognition, based on our projects, we have defined a method called SEED (Savings, Efficiency, Envisioning and Differentiation) that identifies several key factors for tracing how ethnography affects business and what contributions to business it makes.”
Toward Industrialized Ethnography
Fujitsu’s case is considered a successful endeavor where a large company successfully brought ethnography into the business context. They are successfully continuing to manage ethnography research teams like software development projects. These teams are required to make their output measurable by the request from their internal customers; however they are yet to have good indicators.
If organizations using ethno-methodology are assessed as higher than level 2 with CMMI maturity level, their customers will be able to expect consistent process with consistent results. The customers will be able to see how much the research has proceeded, what the current middle output is, how good the output is when comparing with their initial expectations to judge to if the project should continue or not, for example. The detailed process will be modified to adjust for the field where researchers study and there will be no same result. In software industries the situation is the same and there are no identical projects and no identical software. However CMMI is working effectively in software industries because it fits requirements of customers.
Software development is industrialized with CMMI and other lots of efforts. In the software development industry, only some special software such as encrypt systems or the core part of operating systems is developed by highly educated programmers. Most of the software is developed by programmers who have received a few months of training. In the architecture world, some customers ask for famous professionals to design their house, but most of the people are satisfied with houses designed by anonymous designers.
In this article, we explored a new wave in the world of ethnography, which is the expectation to industrialized ethnography or engineered ethnography. Such ethnography requires stability, repeatability and measurability. Though some researchers call them “dirty” – with no depth and patched; many customers will ask for such ethnography from now. It is time for each ethnographer to choose to be an expert or to be a manager of industrial ethnographic research teams.
Takanori Ugai has been a researcher at Fujitsu Laboratories for nearly 20 years, specializing in knowledge management, requirements engineering, and groupware development and use. He received his MS in computer science, and is completing his PhD from the Tokyo Institute of Technology
Kouji Aoyama is a researcher of Knowledge Research Center at Fujitsu Laboratories ltd. He majored computer science. He engages in developing methodologies to conduct ethnography in business context.
Akihiko Obata works at Fujitsu Laboratories Ltd., researching Integrated Telecommunications Networks.
1 The National Aeronautics and Space Administration (NASA) is an Executive Branch agency of the United States government, responsible for the nation’s civilian space program and aeronautics and aerospace research. Since February 2006 NASA’s self-described mission statement is to “pioneer the future in space exploration, scientific discovery and aeronautics research. Sandia is a government-owned/contractor operated facility. Sandia Corporation, a Lockheed Martin company, manages Sandia for the U.S. Department of Energy’s National Nuclear Security Administration. Since 1949, Sandia National Laboratories has developed science-based technologies that support our national security.
2 An “etic” account is a description of a behavior or belief by an observer, in terms that can be applied to other cultures; that is, an etic account is ‘culturally neutral’. From Wikipedia visited on July 23, 2010.
3 An “emic” account is a description of behavior or a belief in terms meaningful (consciously or unconsciously) to the actor; that is, an emic account comes from a person within the culture. Almost anything from within a culture can provide an emic account. From Wikipedia visited on July 23, 2010.
Bentley, R., Hughes, J.A., Randall, D., Rodden, T., Sawyer, P., Shapiro, D., and Sommerville, I.
1992 Ethnographically-informed systems design for air traffic control, CSCW1992
Beyer, G., and Holtzblatt, K.
1997 Contextual Design: Defining Customer-Centered Systems, Morgan Kaufmann
2009 Introduction: Business, Anthropology, and the Growth of Corporate Ethnography. pp1–37, in Ethnography and the corporate encounter
2008 What is CMMI ?. NASA presentation
Hughes, J.A, Randall, D., and Shapiro, D.
1992 Faltering from Ethnography to Design, CSCW1992
Ikeya, N., Vinkhuyzen, E., Whalen, J., and Yamauchi, Y.
2007 Teaching organizational ethnography. pp270–282, EPIC2007
Ishigaki,K., Sashida,N., and Yajima, A.
2007 Ethno-cognitive interview method for work practice understanding, FUJITSU SCIENTIFIC & TECHNICAL JOURNAL, 2007 October Vol.43, No.4, pp390-397
Jordan, B., and Lambert M.,
2009 Working in Corporate Jungles: Reflections on Ethnographic Praxis in Industry. pp95–136, in Ethnography and the corporate encounter
Jordan, B., and Yamauchi, Y,
2008 Beyond the University Teaching Ethnography Methods in the Corporation. Anthropology News 49:6:35
Kishimoto, K., Terasawa, M. and Hirata, S.
2009 Innovation of working style through ethnography for business and organizational monitor. FUJITSU November Vol.60, No.6, pp591-598 (in Japanese)
Obata, A., Yamada, S., Harada, H., Hirata, S., and Ito, S.
2007 Ethnographic inspection identifying project risks, EPIC2007, pp157-168
Rejon J., V
2009 Showing the value of Ethnography in Business, EPIC2009
2002 Fieldwork Techniques, Shin-yosya (In Japanese)
Spradley, J. P.,
1997 The ethnographic interview, Wadsworth Pub Co
Yagi, R., Harada, H., and Ishigakai.K.
2009 Qualitative design for visualizing user view points, FUJITSU SCIENTIFIC & TECHNICAL JOURNAL, 2009 April Vol.45, No.2, pp202-209