From Experience Models to Immersion Tools: Transferring Ethnographic Knowledge In An Agile World

Share Share Share Share Share

by JOHN PAYNE, Moment

Introduction

Like many design consultancies, Moment uses a variety of research methods to help us develop a contextual understanding of our clients’ customers. We do this to discover and adapt new business opportunities to prospects’ wants, needs and desires. The value to the business is that their products and services better fit their audience, increasing adoption and use. Tangible results from this work range from incremental product enhancements to disruptive innovations that provide significant competitive advantage.

Design ethnography is how we approach “fuzzy front end” projects—those that require us to define the problem before formulating a solution. Through ethnography, our field team achieves a robust understanding of the situation, but then faces the challenge of transferring the richness of these learnings into the narrow frame of new product development methodology. This make-or-break moment of transfer is when design ethnography truly delivers—or doesn’t.

Speaking for the design community in his 2011 opening keynote for EPIC, Hugh Dubberly talked about this issue. “This is where we go wrong,” he said. “Formal research and ad hoc observations often fail to transfer. It’s not that the observations and research are not valid, they may be. Or that there are no useful insights, there probably are. But rather, designers and product managers and engineers can’t connect the research insights and the products.” [1]

If we can’t transfer ethnographic knowledge in a useful way for our audiences—product owners, executives, marketers, developers—have we really created the business value we claim?

Models of Experience

As far back as the mid-90s, pioneers, like Rick Robinson and others at eLab, tackled this problem. Their innovation was to design new and more accessible kinds of output. Instead of the findings presentation or highlight reel more typical of traditional market research, they created “frameworks” of experiences that went beyond simple description. These frameworks identified and communicated relevant patterns of human behavior to better guide product development.

In a 2000 lecture, Robinson used Clifford Geertz’s classic essay, "Deep Play: Notes on the Balinese Cockfight" to explain the appeal of frameworks. Put simply, Deep Play describes social factors that drive people to engage passionately with competitive sport. As Robinson says, “This idea sits above the particularities of the events described and lets different audiences make sense of them.” Deep play “…becomes a lens through which not only can he show you what’s important about the Balinese cockfight, but through which his colleagues can address things like riots in Scottish soccer matches. [2]

Refining this idea and adding visualization techniques led to an approach called Experience Modeling, which was further developed at Sapient. They defined this as, “the practice of breaking down an experience and visually communicating its key elements.” [3] By adding visualization to the mix, Experience Models expose the underlying structure of an experience to make what’s important about it more immediately accessible to the people who need to know.
experience_model

The experience model above, an example cited by Jeanette Blomberg in a 2002 HCI text, illustrates findings from research into individual investor behavior and attitudes. It represents, among other things, a concept of five “financial development zones,” shown on the X axis, and illustrates how an individual’s perceptions of “real,” “play” and “foundational” investment types affected their investing behavior across these zones. [4]

An experience model like this can crystallize ethnographic knowledge and make it accessible and actionable by designers and business people. It serves as a kind of boundary object that can be used for a variety of purposes, from generating new ideas to prioritizing existing initiatives or even aligning marketing and organizational strategy with user experience.

A Maturing Practice

In the past decade, the differentiating value of providing a good “user experience” (UX) has been widely recognized. UX has grown into a thriving community of practice focused on the understanding and design of technological products and services. This community has capitalized on the value of modeling to transfer research findings more effectively from one set of stakeholders to the next. Central to the UX process are a multitude of models—personas, user journeys, mental models, experience maps—some of which are now standard practice across organizations as varied as startups, non-profits and global conglomerates.

But the fast-paced technology development cycle of most UX projects often requires a just-in-time approach to research and an iterative approach to design. Agile is the hot new thing and Big Design Up Front (#BDUF) is persona non grata. The resulting quick-and-dirty approach to research has robbed UX models of some of the analytical rigor of their experience model predecessors, sometimes leading to beautifully rendered visualizations of dubious interpretive value. Despite the best intentions of the UX community, it’s the rare UX model that helps you reconsider an entire type of experience the way “deep play” does.

From Experience Models to Immersion Tools

Despite the accelerating business environment, Moment has been hesitant to abandon the analytical rigor of design ethnography and experience modeling for lighter, descriptive approach to modeling that the UX community has adopted. Instead, we’ve created a more flexible approach to ethnographic work for the agile world we are now living in. In order to enable our models to work more effectively in an iterative product development cycle, Moment has adapted our practices of design ethnography and experience modeling in a few key ways.

From monolithic to modular
We have shifted from monolithic single studies to a series of ongoing small-scale investigations that build on one another over time. Our output has shifted too, from comprehensive all-in-one deliverables like experience models, to emergent, modular ones that can be developed (and consumed) a piece at a time. We combine and recombine these components and use them in a variety of ways to address questions and opportunities as they arise.

From ad hoc to defined process
As Rachel Jones wrote in her 2006 paper for EPIC, “It is not apparent that there is a clear process, or set of activities, or working practices, other than collaboration and dialogue between ethnographers and designers to facilitate the transition of understanding into design.” [5] Recognizing that transferring our findings is critical, we’ve worked to create process around it, designing not just the models, but repeatable immersion activities that can be used to socialize them among the team.

From informing to imbedding
We’ve adopted a practice of co-authorship with our clients, designing our models not just to “inform” them, but also to function as tools that can “imbed” ethnographic knowledge in the synthesis we produce together. We’ve found that turning our models into immersion tools enables our clients to think WITH our findings rather than ABOUT them, making the ideas more instrumental to our collaborative process.

We have built on the foundation of experience modeling, but taken our models a step beyond visual representations. We call them Immersion Tools. They are designed to make ethnographic understanding available to the team DURING the design process, not only BEFORE it; to render complex social phenomena more accessible by breaking it down into consumable pieces; to adapt to different but related business problems and iteration through subsequent “releases” as the products they inform evolve. Here are two examples:

Card games for innovation
johnpayne2
For one client, Moment conducted research to analyze personal and social factors that influence where and how people get lunch in the city. While interpreting our observations we noticed significant day-to-day variation among users. To represent these shifting characteristics we created a modular persona model and a “card game” to socialize them with the team. Mixing and matching cards based on real world patterns of behavior, we brainstormed business opportunities that could address the combinations. With our client, “we generated and inspired dozens of actionable product ideas. And following the workshop, our team had a common language to use with our clients to discuss their target audience’s needs and behaviors.” [6]

 

Sticker fun for evaluation
jonpayne3
For another client, Moment synthesized our research on financial behavior into a series of design principles to drive new concept development. To activate those principles, our team developed a set of stickers representing the positive and negative sides of those values and a critique format that made use of them. Stickers were distributed to participants, who would use them to indicate how a concept did or did not support a principle as they evaluated product ideas. This ensured each team member was using the agreed user-centered rationale for product feedback.

Future directions

In practice, we’ve seen that deliverables produced by one group for the intention of “informing” the next often fail to transfer. At Moment, we believe that the best way to become fluent with an idea isn’t to be informed about it, but to try to use it. We embrace the notion of learning by doing and we try to bring that experience to our clients and partners as well. We’ve seen significant success in connecting our research insights directly to the products and services we design by using immersion tools. We still use a variety of methods to capture data, analyze and communicate our findings, but we think we’re on to something fundamental here, and we're excited to see where we can take it.

We welcome your input, stories and ideas on our concept of Immersion Tools in the comments below.

References

[1] DUBBERLY, H. “Why Modeling Is Crucial To Designing & Design Research.” Ethnographic Praxis in Industry Conference 2011. St Julien Hotel Conference Center, Boulder. 19 Sept. 2011. Lecture. http://epiconference.com/2011/program/keynotes/opening

[2] ROBINSON, R. E “Things To Think With.” Doors of Perception 6 Conference. RAI Convention Centre, Amsterdam, Netherlands. 11 Nov. 2000. Lecture. http://museum.doorsofperception.com/doors6/doors6/transcripts/robinson.html

[3] MORRIS, M., and LUND, A. M. (2001). Experience Models: How Are They Made And What Do They Offer? Loop: AIGA Journal of Interaction Design Education. http://loop1.aiga.org/common/modules/display/dsp_ContentTemplate01b.cfm?ContentID=49&CreateTemplate=0&NavType=SiblingContent

[4] BLOMBERG, J., BURRELL, M., and GUEST, G. (2002), “An Ethnographic Approach to Design.” In The Human-computer Interaction Handbook : Fundamentals, Evolving Technologies, and Emerging Applications, edited by Julie A. Jacko, 964-984.

[5] JONES, R. (2006), Experience Models: Where Ethnography and Design Meet. Ethnographic Praxis in Industry Conference Proceedings, 2006: 82–93 http://onlinelibrary.wiley.com/doi/10.1111/j.1559-8918.2006.tb00038.x/abstract

[6] CURTIS, J. A. and PAYNE, J. (2012), Drivers, Behaviors, and Scenarios: A Card Game for Concept Development. Ethnographic Praxis in Industry Conference Proceedings, 2012: 379–380. http://onlinelibrary.wiley.com/doi/10.1111/j.1559-8918.2012.00059.x/abstract

 


 

About the author:

John Payne is a principal at Moment, a digital product design firm he co-founded in 2002. His current work explores the positive contribution that design can make to healthcare. Educated at Auburn University and Institute of Design at IIT, John has lectured and taught courses in design methodology at Pratt, Parsons, and NYU.

 

  8 comments for “From Experience Models to Immersion Tools: Transferring Ethnographic Knowledge In An Agile World

  1. August 13, 2014 at 11:19 pm

    Thank you John for situating Moment’s work in a long history of work that predated your own organization; for making visible the thinking and innovations that both anchored Moment and gave you a starting point that you have clearly moved with and beyond. It’s so interesting to me that the “just in time approach” of today’s UX wants to enable easy transfer and while it does lend itself to being easily understood, transfer is about more than immediate gratification. We could all find better ways of getting our organizations to understand this very fundamental point.

  2. August 14, 2014 at 12:07 am

    As researchers, we spend a great deal of energy focused on identifying and designing the methodology to answer a given research question. Less time is typically spent on how to best transfer the knowledge and empathy we have gained to others. If our stakeholders do not internalize the findings and in a way they can act upon them, what is the point in doing the research in the first place? I love the approach of learning by doing! Design Workshops or Sprints following research are excellent but the activities you do in the Workshops/Sprints must enable the attendees to leverage the knowledge gained. I’d love to hear more detail about the methodologies Moment uses!

  3. August 14, 2014 at 2:30 pm

    Thanks Maria. As a design firm working in the digital space, it’s a challenge to keep an ethnographic perspective in our work. It would be much easier to adopt more common UX approaches, but having experienced the transformational power of well executed design ethnography first hand, I know we can produce more successful outcomes by sticking to our (analytically rigorous) guns and evolving our approach the moment of transfer that you and others began with experience modeling.

    Our thinking has some deep roots, but it’s worth pointing out that it’s also inspired by the contemporary thoughts coming out of the EPIC community. If I had included every reference that had inspired us, the list would be quite a bit longer. It would include related pieces like your own “Valuable to Values: How ‘User Research’ Ought to Change” and “Flux: Changing Social Values and their Values to Business.” Hugh, Shelley, and Rick’s “The Analysis-Synthesis Bridge Model” and Ken’s “Models in Motion: Ethnography Moves from Complicatedness to Complex Systems” have also played a part in helping us arrive here.

    And, thank you Kathy for seconding the critical importance of the often neglected moment of transfer. We’ve found quite an improvement in uptake and longevity of our synthesis by adopting the experiential learning approach I describe here. I’d be happy to share more… anyone need a book chapter? 🙂

  4. August 14, 2014 at 3:20 pm

    Nice one Mr. Payne!
    I enjoyed it simply as a round up of the “greatest hits” from 20 years of innovation & design research, but even more…as a timely reminder that our discipline IS cumulative, and the way we “progress” is not usually through the sudden breakthroughs that always get hyped, but, in Newton’s apt phrase, by “standing on the shoulders of giants” AND the methods they bequeath us…

    • August 22, 2014 at 4:14 pm

      Thanks, Stokes. The view from up here is grand! We definitely have a lot to be thankful for.

  5. August 17, 2014 at 10:42 am

    Thanks John extremely timely as I find myself neck deep in a large agile UX environment 🙂

    I wonder what your experience of educating agile teams “beyond usability testing” has been? The conflation of user research with usability testing is of course an issue I find myself contending daily. The scrum teams and the project over all are very “user friendly” so I am assuming that the issue is more of awareness and experience than anything else. Any tips and tricks you’d care to share? Is the best approach “experiential learning” as you say, how do you tackle the issue of opening up “epistemelogical awareness” in teams where hypo-thetico deductive models (and consequently deductive reasoning and quant) reign supreme?

    • August 22, 2014 at 4:13 pm

      Hi Cat, Thanks for the note.

      To be honest, we’ve had mixed success in educating agile teams about the research of anything beyond usability testing. Some teams are receptive, some are not. There are a variety of reasons some teams may not be receptive to non-evaluative research. Some of the more obvious ones include: a) they are designing for people like themselves, b) it runs counter to the “cowboy” culture the build-it-first mentality, c) like other groups, they simply think they understand their user base well enough already, or d) all of the above (?!?)

      No matter the reason, I think sometimes the problem rises above lack of awareness and can’t be tackled through education, but should be approached with more change management tactics. The doctrine of agile is strong, and shifting the values of it’s adherents can be as much of a challenge as any attempt to change a cultural belief system.

      That said, we’ve found some success with teams that are in the early stages of product development, have an explicit goal of user adoption, or are particularly interested in innovation as a means of differentiation. In these situations, our little-by-little approach to fieldwork and our hands-on working style during synthesis have worked to make generative research a bit more friendly to the agile way of working.

  6. September 9, 2014 at 10:49 am

    […] In the fast-paced world of agile website development, holding on to data and findings creates blockages in progress that can knock on, in unforeseen ways, to other areas of work. For those of us in the user research world this is a big focus of thinking. […]

Leave a Reply