Advancing the Value of Ethnography

Ethnographic Inspection Identifying Project Risks


Download PDF

Cite this article:

Ethnographic Praxis in Industry Conference Proceedings 2007, pp. 158–169.


We propose a new approach for project inspection applying ethnography to identifying IT system project risks. Guideline-based inspection is generally conducted in the IT industry to reduce project risks. Guidelines are created based on analysis of failures in past projects. However, it is difficult to detect new risks that emerged according to changes of project environments. We hypothesized that an ethnographic method would have some value to detect such emerging risks by capturing insiders’ perspectives. We developed procedures and guidelines to conduct inspection by ethnographic approach for IT experts. To clarify the value of the method, we conducted project inspection by the ethnographic approach on two projects that have already received guideline-based inspection. Problems found by the ethnographic inspection were not quite new for the members observed and interviewed. However, the ethnographic inspection successfully captured tacit problems that were rooted in organizational structures and culture, and triggered discussion and review of policy and standard models.


As information technology has been rapidly progressing, troubles caused by IT systems have the possibility of causing a big loss in society. Non-technical issues are perceived as important issues in reducing risks of system development and maintenance projects (Ewusi-Mensah 1997, Keil 1998, Barret 2004). In this paper, we propose a new approach for project inspections applying ethnography to identify IT system project risks caused by non-technical factors.

Guideline-based project inspections are generally conducted in the IT industry to reduce project risks (Ewusi-Mensah 1997). We hypothesized that guideline based inspections have some limitation. It is difficult to detect new risks that emerge according to changes in project environments, since guidelines are created based on analysis of failures in past projects.

Jordan and Putz (Jordan 2004) discussed other limitations of documentary based assessment such as using checklists. Documentary assessment could produce negative side effects by generating unanticipated “work-arounds” that undermine the intent of the assessment. For example, when Quality Standard norms were introduced in an auto parts industry, all evidence of non standard practices was removed before the audit (Bueno Castellanos 2001).

We hypothesized that an ethnographic method would have some value to detect such emerging risks and hidden problems by capturing the insiders’ perspectives (emic) that are hardly detected by guideline-based inspections (etic). In this paper, we propose a project inspection method that is based on an ethnographic approach for IT experts who have no experience of fieldwork, and discuss the value of the method by comparing the results of ethnography with guideline-based inspections.


First of all, we studied problems of guideline-based inspection based on observations of two inspection teams and questionnaires received from 101 inspectors. Then we developed guidelines and procedures of ethnographic inspection for IT experts by articulating the essence of ethnography based on the analysis of the guideline-based inspection. To clarify the value of the method, we conducted project inspections using an ethnographic approach on two projects that had already received the guideline-based inspections by IT project experts and discussed the difference between the two methods.

Target projects of inspections are IT maintenance projects. The main tasks of these projects are changing and adding new functions and capabilities in application software and hardware according to clients’ requirements, daily operations on IT systems, and trouble shooting.


In guideline-based inspections, inspectors were provided a guideline that includes 300 or more checklists (e.g., Did you provide clear escalation rules for emergent cases?). They inspected documents and interviewed several project members based on the guideline as well as on their experiences, and then pointed out problems that should be fixed in the agreed upon time limit. We observed two inspection teams and received questionnaires from 101 inspectors about the difficulties of inspections. The following episodes illustrate our findings. Episodes 1 to Episode 4 are examples taken from the observations of the two inspection teams. Episode 5 is taken from questionnaires completed by 101 inspectors.

Episode1: Mismatch to the project context

A project manager argued that a problem pointed out by inspectors was mismatched to the current project situation. The inspectors argued that the project manager should clearly define roles between the project side and the client side. On the other hand, the project manager argued that the issue pointed out by the inspectors was generally reasonable, however, not realistic in their project context. They continued the discussion, but failed to reach an agreement at the observed meeting.

Inspector: “You should avoid ambiguous descriptions in the contract document, such as one party performing the main role, and the other performing the support role.”

Project manager: “It is difficult in our project situation to clearly distinguish roles with our customer. We have been working quite well with our customer for decades, and have successfully developed good team work. If you argue the issue without regard to our project context, it is not persuasive.”

Episode2: Lack of evidence for shared understanding

An inspector pointed out a communication issue among project members. The project manager under inspection could not understand the problem because the inspector could not provide the details of the problem.

Inspector: “We did not get the frequency, but there are some communication errors from team A to team B that fully depend on oral communication and e-mail.”

Project manager: “I could not figure out what was happening. Communication errors are important problems; however, it is of no use if you can not give us more detailed information.”

Episode3: Difficulty of relaxing wariness

As their general policy, the inspection teams tried not to be auditors but helpers. However, a project manager and a leader received the suggestions from the inspection teams as criticisms and rejected the team’s suggestions.

Inspector: “Some of members of your project have been in your project for many years. There are possibilities that certain skills belong to individuals and are not shared with others.”

Project manager: “I could not figure out what kinds of skills are not shared. Specific individual skills are needed to some extent. However, we rotate members inside our project, assuming there is a risk that some members will leave the team. It is basic of project management. There would be no problem even if someone passed away.”

Inspector: “We raised this issue as a suggestion from a third party’s point of view. I hope you will regard our suggestions as positive feedback.”

Episode4: Questions based on past experience of failures

Typical questions of the inspectors we observed are questions based on past experience of failures. Most of them are questions that confirm how they take actions for some risk factors.

“What actions do you plan for transferring skills of experienced members who are going to retire?”

“What is being done for variation of work load?”

“How frequently do you report to your client about the work progress?”

Episode5: Difficulty of uncovering real problems

The inspectors were concerned that they failed to put project members at ease and, therefore, could not uncover the real problems the members faced. The result of the questionnaire from 101 inspectors showed that 25% of inspectors described the difficulties of uncovering real problems.

“I always face difficulty in making project members feel at ease enough to speak about real problems.”

“Project members would not tell us about their real problems. I don’t think they like to be told of problems by inspectors since that will produce re-work.”

“Before inspections, some projects create documents that are rarely used in their projects.”

On the other hand, some of the questionnaire data showed that there are episodes that inspectors sometimes ignored problems, considering the burdens of project members.

“Considering the burdens of project members, I sometimes do not dare to point out problems that may be hard to tackle.”

“The leader of my inspection team told me, you should not describe problems that are hard to solve.”


The inspection teams tried not to be auditors but instead to be helpers as their policy. However, guideline-based inspection does not always fit the project context, and sometimes they failed to obtain shared understanding of problems because they lacked evidence, or caused wariness among project members. Questions raised by inspectors tend to be closed questions confirming how they take actions for some risk factors that are invoked by past experience. We also found evidence in the questionnaire responses that guideline-based inspection produces the side effect of generating unanticipated “work-arounds” as Jordan and Putz pointed out (Jordan 2004). Moreover, considering the burdens of project members, inspectors sometimes did not dare to point out problems that were hard to tackle.

Guideline-based inspection is a good tool for checking whether projects follow basic standard rules or not. However, the evidence showed that guideline-based inspection is not always effective in uncovering the real problems that projects face.


We hypothesize that an ethnographic approach will be effective in addressing the issues of guideline-based inspection that we illustrated in the previous section. First, an ethnographic method is suitable for pointing out problems in a way that is suitable for the project contexts. Understanding detailed contexts of the projects is helpful to avoid conflicts of problem descriptions that do not match the project context, as shown in episode 1. Second, detailed information about problems like that in fieldnotes might help to create shared understanding among inspectors and project members in such situation where evidence is needed, as shown in episode 2. Detailed descriptions, observations and interviews will be a common ground to discuss problems and might lead to constructive discussions by which different stakeholders and experts can contribute to solving the problems using their own experience and knowledge. Consequently, the problems pointed out by inspectors will be persuasive to project members and reduce the wariness of project members. Moreover, ethnographic interviews that encourage informants to speak in the same way they would talk to others in their cultural scene (Spradley 1979) might help to figure out tacit problems from the insiders’ point of view that are rarely uncovered by such closed-ended questions primarily based on past experience, as shown in episode 4. These features of the ethnographic method might help to externalize the tacit problems of projects and help to treat difficult problems as organizational issues rather than project matters.


We developed guidelines for ethnographic inspection for the IT experts by articulating the essence of ethnography. The following is an overview of our guidelines.

Describe problems from the workers’ perspective – To describe problems from the workers’ perspective, we developed the following guidelines:

  1. Do not attribute causes of problems to workers’ mistakes.
    You should look for causes of problems in the workers’ environment, such as tools, organizational structures, policy, or pressure from outsiders. Try not to judge a problem in such a way that makes the workers feel their performance is insufficient to accomplish their tasks. Instead, try to describe the workers’ difficulties in accomplishing their goal or fitting their work to the standard processes.
  2. Do not use evaluative questions. Get episodes of everyday work practice.
    You should encourage informants to tell you about their everyday work practices. Ask probing questions to elicit more detail about the episodes. When your informants talk about their problems, you should not evaluate them using standard models.

Describe episodes in detail – To encourage description of problems that would be like fieldnotes, we developed the following guidelines:

  1. Describe problem episodes in detail to create a common ground.
    Do not describe problems in abstract terms. Describe concrete episodes that occurred in the field. It allows many stakeholders to discuss problems from a common ground.
  2. Do not introduce outsiders’ standards uncritically.
    Problems pointed out by inspectors tend to introduce outsiders’ points of view uncritically. These points sometimes cause conflicts between inspection teams and project members, whose perspectives may be different. Describe a problem by dividing it into three parts: 1) the name of the problem, 2) the episodes that illustrate the problem, and 3) discussions about the problem. Refer to descriptions of the episode in which you do not to introduce outsiders’ point of view.

Analyze patterns and structures – It is important for the ethnographic method to focus on what people take for granted and rarely discuss (Jordan 2006). We developed guidelines for helping inspectors discover important patterns and structures from everyday work practice:

  1. Using heuristics for detecting patterns.
    Jordan and Dalal argued that ethnographic study requires years of theoretically grounded training and practical experience. The special skill of trained ethnographers is the ability to look for patterns (Jordan 2006). To help IT experts who have no experience in fieldwork, we provided heuristics for detecting patterns. When inspectors analyze filed memos, they consult the heuristics to find interesting patterns. We provide heuristics written in short messages that are easy to remember. We developed these heuristics based on papers by Hughes (1997) and Martin (2004). For example, these papers described heuristics such as “working division of labor” and “distributed coordination” to find interesting patterns.
  2. Analyzing problems as systems.
    We provided a process for analysis by integrating several episodes to determine the tacit structural problems that were deeply rooted in organizational structure and culture. After describing several episodes of problems, inspectors created causal loop diagrams (Senge 1990) discussing the casual relationships among episodes.


Ethnographic inspection is conducted in the following six steps. It takes approximately three weeks from step 1 to step 3 for a few days’ observation and a few members’ interviews. The time needed for steps 4, 5, and 6 depend on the difficulty of the problems.

Step 1: Setting up the fieldwork theme – Interview the manager of the target project, and then discuss the theme and target tasks and workers. Examples of themes are; “Why does the same kind of mistake frequently occur?”, “Why do they have difficulty following the standard process?”

Step 2: Interview and observation – Get agreement on conducting ethnographic inspection with informants. Interview and observe informants whose work is related to the theme, and collect episodes that illustrate the problems.

Step 3: Co-analysis with informants – Show the collected episodes to the inspection team and discuss them with both inspectors and informants. Confirm the validity of the episodes and try to get a deeper understanding of the problems.

Step 4: Analyzing patterns and structures – Based on the collected episodes from multiple observations and informants, analyze the patterns and structures of the problems.

Step 5: Co-design with stakeholders – Feedback the problems based on episodes from interviews and observations, and create shared understanding of problems among stakeholders. Then discuss a vision for resolution of the problem and actions to solve the problem. Participants in these discussions are not limited to project members, but also include executive managers, clients, and experts from other projects.

Step 6: Following up actions – Observe how actions are executed, and what the effects are of the actions. Continue these steps repeatedly if new important issues emerge in the process.


We conducted inspections using this ethnographic approach on two projects that had already received guideline-based inspections by IT project experts. We discussed the difference between the two methods. The guideline-based inspections successfully pointed out several new problems that project members did not realize before the inspections. Problems pointed out were mainly about documents and processes that did not fit the standard models. On the other hand, each of the problems uncovered through ethnographic inspection were not quite new to the project members who were observed and interviewed, because the ethnographic inspection found problems from the insiders’ point of view. However, the ethnographic inspection successfully captured tacit problems that were rooted in organizational structures and cultures, and triggered discussion about and review of policy and standard processes.

Case 1: Organizational structures and culture behind human errors

One of the projects that we inspected had approximately 50 project members. Several executive managers perceived that this project produced more system troubles than other projects, and wanted to know the underlying problems behind the recurring troubles. On the other hand, the project manager perceived that human errors were unavoidable to some extent. He was unsure how he could reduce human errors.

Inspectors using guideline-based inspection pointed out problems in several documents. Descriptions of work distribution with their client in the contract document were too ambiguous, and lacked description of the escalation process and rules. They also suggested that the project manager should get agreement with the client and make the project members aware of the new rules. In contrast , the inspectors judged that the project provided documents properly for quality management that were in alignment with the standard model. We interviewed one of the inspectors about the recurring troubles. He suspected that the recent replacement of the project manager might be a cause of the problem. He suspected that the relationships with customers and sub-contractors might not be going well.

We conducted fieldwork to determine the underlying causes of the recurring troubles. We observed several meetings such as progress reporting and the issues management meeting, and took notes during the meetings. We interviewed several members of projects sharing with them descriptions of episodes that had occurred in the meetings.

We obtained several episodes from meetings that indicated a leader of the project had just tried to publicize the standard model rules, but that he did not make sure that his members rigidly follow the rules. Here is a excerpt of a conversation that occurred in a progress meeting:

Project manager: “What did you do for this issue?”

Leader: “I just told them again and again that they should follow the standard rules.”

We interviewed several leaders by sharing episodes to understand the stories behind them. Then we found a leader who perceived that the project team members were overloaded:

Leader: “We have many things to do. We don’t have enough time to pay attention to each incident. I asked for too much work from my members.”

Moreover, we got episodes illustrating that work was increasing unexpectedly due to the retirement of experienced members from the information system division of their customers:

“There have been a number of experienced customers in the information system division who got involved in building the current system from the scratch. Those experienced customers are gradually retiring. Recently, two experienced customers retired and were replaced by young people. “

“When I asked a customer about their work process, they directed me to investigate the work ourselves. I think they did not know their work flow.”

“Customers of different divisions tend to impose additional work on each other. They don’t like to get additional work. They only know their related tasks. Currently no one knows the whole system and can not coordinate such conflicts. There is no way, except that we are taking on this role. ”.

Team members are willing to do such additional hidden work because of their cooperate culture:

“Even if we defined detailed work distribution with customers in the contract, maybe we can’t say no to our customers. This may be our corporate culture.”

From analysis of these observation and interviews, underlying problems of recurring human errors emerged. There are organizational and cultural issues such as retirement of experienced customers in the information system division that is producing unexpected hidden work, and there is willingness to do overloaded work for their customers. The executive manager thought similar problems might also be emerging in other projects, and began discussion with members of his department to overcome these problems.

Case 2: Problems behind multi-layered business

The second project that we inspected had approximately 300 project members in the development phase, and the project downsized to 18 members in the maintenance phase after years of development. This project is part of a multiple systems integration project. Three system integrators were participating in the project. One of these system integrators was a first-tier company, and others, including the project we inspected, were second-tier companies.

The problems pointed out by inspectors using guideline-based inspection were similar to Case 1. They pointed out that descriptions of the distribution of work with customers in the contract document were too ambiguous, and there was a lack of documents that described the escalation process and rules. In addition, they suggested adding a cross-review process to the library management process. The project manager appreciated the suggestions of the inspectors.

We interviewed the project manager to discuss the purpose of the fieldwork. He was not concerned with system problems. His primary concern was how they were to handover the knowledge of the system during the transition from the development phase to the maintenance phase. We focused on meetings that were related to handover of system knowledge.

We uncovered episodes that the inspectors who followed guideline-based inspection had not pointed out. The project team had difficulty in handing over system operations work to the first-tier company. The responsibility of the first-tier company was not clearly defined, and they had to do additional work for their real customers. Here is a sample of comments that illustrate the episodes:

“Mr. Suzuki who participated in the meeting (a meeting for handover) is a newcomer. He has not been here before. We did handover the process with Mr. Sato who accompanied the three operators. But they were missing. “

We reported these episodes to the executive manager as well as to the project manager. They recognized the problem well. However, the discussion based on these episodes created reflection. They said they should clearly define work distribution and responsibility for the maintenance phase in the system proposal phase. They appreciated the results of our inspection:

“It is important to describe such tacit problems that occur in everyday work to discuss standard process. It might be effective to create new guidelines for inspections.”


Ethnographic inspection as a tool for organizational learning

We have described the situation in which guideline-based inspections do not take into account the project context, and sometimes these inspections failed to obtain shared understanding of problems, and caused wariness among project members. Questions raised by inspectors tended to be closed-ended and simply reconfirmed, actions invoked in response to some risk factors encountered in past experience. Moreover, the primary focus of the inspectors in our study was correcting work practices to create a better fit to standard models. Our result suggested that guideline-based inspection is not appropriate for pointing out emerging and hidden issues that trigger reviewing company policy and standard models.

On the other hand, ethnographic inspection successfully uncovered organizational issues that were not pointed out by guideline-based inspection, such as retirement of experienced customers in the information system division that produced unexpected hidden work, and the ambiguous scope of responsibility in a multi-layered business. The executive manager perceived these problems not as a project matter, but as general issues that should be addressed by the organization. Our results suggest that ethnographic inspections could be a tool for reviewing organizational policy and standard models.

We obtained some evidence that ethnographic inspection could capture not only risk factors, but also risk-mitigating best practices devised in the field. Externalizing and sharing tacit knowledge is an important practice for organizational learning (Nonaka 1995). Ethnographic inspection might also useful to create a shared vision, another important factor for organizational learning (Senge 1990). Executive managers and project managers might have personal visions that never get translated into shared visions and, there, cannot be of value in guiding and encouraging project members. Policy and standard models that are reviewed based on episodes taken from many projects can reflect these many contexts and might strongly support project members for overcoming difficult issues. We will continue to explore ethnographic inspection as an organizational learning tool.

Cooperation and diffusion

It is important to develop tools for IT experts that they can easily and effectively use to conduct ethnographic inspection and penetrate work practices organizational wide. Four IT experts who work as inspectors participated to our ethnographic inspection following the guidelines and the procedures that we illustrated in this paper. At the start of the project, participants were accustomed to established practices for evaluating project work, and tended to violate the guidelines we provided. They evaluated the work practices based on their own observations and perspectives rather than describing problems from the workers’ points of view. We reminded them repeatedly about the guidelines and procedures for an ethnographic approach, and they gradually understood the value of approach. By the conclusion of the project, the guidelines for ethnographic interviewing seemed very useful to them. Some inspectors said they enjoyed doing ethnographic interviews because they could get various interesting and informative episodes from project members.

However, the descriptions of episodes were sometimes insufficient. The members of the inspection team who were outside the project teams had difficulty understanding what was going on in the projects. Professional ethnographers sometimes pointed out that the relationship of the episodes to the discussions was unclear. It took many hours to review episodes before people outside the work were able to easily understand the project contexts. Analysis also required many hours. We totaled the time we needed for each step. We found that the analysis step was the most time consuming task. Moreover, some inspectors were concerned that there would be a risk that they would not get interesting episodes within the time allotted for inspection, while they could achieve their goal of checking all guidelines in a guideline-based inspection within established time limits. In the future, we need to develop support tools for IT experts to conduct ethnographic inspection more effectively if we are to diffuse this practice company-wide.

Acknowledgments –We thank Jack Whalen, Erik Vinkhuyzen, Nozomi Ikeya, and Yutaka Yamauchi for basic training of ethnography and useful comments on our research. We also thank Julia Gluesing and anonymous reviewers for useful comments on the draft version of our paper.

Akihiko Obata is a senior researcher at Fujitsu Laboratories LTD. His current research focuses on knowledge management and organizational learning based on ethnography. His research interest also includes CSCW, and human interface.

Shigeru Yamada is a researcher at Fujitsu Laboratories LTD. His research area is from Organizational Development and Self-organized Meeting, Combination of Coaching and Ethnography, Usability Engineering and User Interface to Product Design.

Hiroaki Harada is a senior research fellow at Fujitsu Laboratories LTD. His research focuses on knowledge management and organizational learning based on ethnography.

Sadayo Hirata is a system engineer at Fujitsu Limited. Her research focuses on knowledge management and organizational learning based on ethnography.

Seisuke Ito is a system engineer at Fujitsu Limited. His research focuses on knowledge management and organizational learning based on ethnography.


Barret, R., Kandogan, E., Maglio, P., Haber, E., Takayama, L., Prabaker, M.
2004 Field studies of computer system administrators: analysis of system management tools and practices, Proceedings of the 2004 ACM conference on Computer supported cooperative work, November 06-10, ACM.

Bueno Castellanos, Carmen
2001 “Globalization” of Global Quality. Practicing Anthropology 23:14-17

Ewusi-Mensah, K.
1997 Critical Issues in Abandoned Information Systems Development Projects. Communications of ACM, Vol.40, No.9, 74-80, ACM.

Hughes, J., O’brien, J., Rodden, J., Rouncefield, M., and Blythin, S.
1997 Designing with Ethnography: A Presentation Framework for Design. Proceedings of DIS’97: Designing Interactive Systems: Processes, Practices, Methods,& Techniques 147-158

Keil, M., Cule, E., P., Lyytinen, K., and Schmidt, C., R.
1998 A framework for identifying software project risks. Communications of ACM, Vol.41, No.11, 76-83, ACM.

Martin, D., and Sommerville, I.
2004 Pattern of Cooperative Interaction: Linking Ethnomethodology and design, ACM Trans. on Computer-Human Interaction (TOCHI), 11 (1), 58-89.

Nonaka, I., and Takeuchi, H.
1995 The Knowledge Creating Company, Oxford University Press, Oxford.

Jordan, B., and Putz, P.
2004 Assessment as practice: notes on measures, test and targets. Human Organization Vol.63, No.3, 346-358.

Jordan, B., and Dalal, B.
2005Persuasive Encounters: Ethnography in the Corporation, Field Methods, Vol.18, No.4, November 1-24

Senge, P.
1990 The Fifth Discipline: The Art and Practice of the Learning Organization, Currency

Spradley, J.
1979 The ethnographic interview. Wadsworth Group/ Thomson Learning.