Advancing the Value of Ethnography

Evolution of User Experience Research


Ten years is an eternity in the tech world. But the speed of change makes the classic “decade of reflection” even more valuable for assessing which changes really count and why. We had a chance to reflect systematically on the last decade of user experience research for the second edition of our book “Understanding Your Users,” which was first published in 2005. Since then, user research has become more widespread and more sophisticated. It has also responded to challenges of faster development cycles while simultaneously contributing more, not just to product development, but to the bigger picture of strategic innovation.>

So among the noise of trendy terms and fashionable phrases we found five key trends that really do matter:

1. Usability to User Experience

One of the biggest changes is the shift from a focus on “usability” to “user experience.” “Usability” is an attribute of a system or user interface (UI). It refers to the effectiveness, efficiency, and satisfaction with which users can achieve tasks when using a product. “User experience,” on the other hand, is human-centered and is the co-creation of an interaction between a person or persons and an artifact. User experience includes usability and all aspects a user encounters when dealing with a product or service from the branding to the motion and audio design to customer support. It is about how the product or service makes the user feel and encompasses both user behavior and the social context of the user-artifact relationship. You can see this change reflected in job titles (e.g., from “Usability Engineer” in the 1990’s and early 2000’s to “User Experience Researcher” today) and professional organizations (e.g., the “Usability Professionals Association” changed their name to “User Experience Professionals Association” in 2012). The result of the change in focus means that a User Experience Researcher (UER) must conduct research that goes beyond the UI and think about all of the elements influencing a user’s experience.

2. Lightweight, iterative, or agile approaches

A second change in the profession is the popularity of lightweight, iterative, project-based approaches development processes such as agile. Agile is a rapid development process used by a cross-functional team that ideally works together closely and meets daily. While agile development does include continuous customer or stakeholder involvement, the competing requirement to iterate quickly in small operable chunks means it can be difficult to incorporate user research into the process. This makes conducting early user research to understand user requirements critical because there may be less time in later cycles to conduct longer-term research. Regardless of the development process a company uses, it is imperative to take a cumulative approach to user research, constantly building upon previous work and triangulating across methodologies to create a holistic understanding your users, domain, industry, etc.

3. Innovation at speed

A third development is the increasing variety of technology and contexts where research is needed. For example, products that were considered science fiction ten years ago such as wearables (e.g., smart watches, Google Glass), self-driving cars, and increasingly complex consumer medical devices (e.g., contact lens that measure blood glucose in tears) are becoming reality. In addition, with the explosion of mobile apps and devices means that work is not just being done at a desk or even at one particular place. Our research is less often conducted inside a usability lab and more often done in context. This means we must understand not just whether or not users can find the feature they want and whether it meets a real user need but also how people and things around those the device(s) feel and respond.

4. Vulnerable user types and research ethics

The importance of research ethics isn’t a new development; however, our ability to do research with individuals we have never considered or experienced before has become easier than ever. When you conduct research with vulnerable populations (e.g., children, illiterate, homeless, sick, aging), you must be aware of other factors that could mean your typical research protocol is coercive or not transparent enough about the risks to the participant and obligations of the researcher. For example, if a participant has low or no literacy, s/he cannot read the non-disclosure agreement (NDA) or informed consent form and must take your word when you provide a verbal protocol. The incentive amount you might offer participants for studies in your lab in Silicon Valley can be greater than the monthly income of a participant in a field study in another country and could be coercive if they are uncomfortable inviting you into their home but they desperately need that money. When you provide a prototype of your product for a child to use in his/her home, all of their family members will likely interact with it and therefore, become unknowing participants in your study, even if they did not sign an NDA or provide informed consent. These are issues that increasingly researchers must think about, and take personal responsibility for. Especially since most companies do not have formal Institutional Review Boards (IRBs) to review and approve research, it is all too rare for these issues to be discussed in an open and neutral environment. The push to collect user data faster and faster often negatively impacts this type of deep review.  

5. Remote Research

A fifth development is the dramatically increased availability of tools that enable remote user research. These range from tools created specifically to make the job of a user researcher easier (e.g., Mechanical Turk for crowdsourcing data analysis, Paco for experience sampling methodology, for fast, large scale usability evaluations), to general purpose communication tools like Skype, Citrix GoToMeeting, or Google Hangouts that now make conducting interviews across the country as convenient as conducting them in the user researcher’s home city. Tools that analyze user sentiment on social networks allow you to hear from users in the public sphere and understand your perceived brand.

We’d love to hear from you the changes you have observed in our field in the last 10 years, for better or worse. What changes have you noticed? Which do you feel have made the most impact? What has made doing UX research easier? Harder?



Kathy Baxter, Salesforce

Kathy is a Principal User Researcher at Salesforce in San Francisco. Previously, she worked at Google for over 10 years as a Staff User Experience Researcher & UX Infrastructure Manager. Prior to 2005, she worked as a Sr. UER at eBay and Oracle. Kathy is active in the UX community, volunteering on the EPIC and CHI conference committees, as well as teaching courses and mentoring young girls in STEAM careers. She received her MS in Engineering Psychology and a BS degree in Applied Psychology from the Georgia Institute of Technology. The second edition of the book she coauthored, Understanding your Users, was published in May 2015 & was the #1 New Release in HCI & Software & Product Design on Amazon the first several weeks it was on sale.

Catherine Courage, DocuSign

Catherine’s passion is transforming corporate culture by making customer-focus a driver of innovation and change. She leads the DocuSign Customer Experience team where her group’s mission is to create world-class products and services for customers, partners and employees. Catherine co-authored Understanding Your Users and is an active writer and speaker on creativity, innovation and design. She has been featured in Harvard Business Review, The Wall Street Journal, Fast Company, Huffington Post, and TEDx.  She has twice been selected by the Silicon Valley Business Journal – in 2011 as one of Silicon Valley’s tech leaders, and in 2013 as one of Silicon Valley’s 100 Most Influential Women. Also in 2013, Catherine made Forbes list of “Top 10 Rising Stars at The World’s Most Innovative Companies.” In 2014, the National Diversity Council named her one of the Top 50 Most Powerful Women in Technology., @ccourage

Kelly Caine, Clemson University

Kelly is a researcher and professor working at the intersection of people and technology. She directs the Humans and Technology Lab at Clemson University where she and her students advocate for users and create easy to use, useful technology that meets people’s needs. She co-authored a Understanding your Users, has published dozens of peer-reviewed papers and is regularly cited by media such as the AP, Washington Post, NPR, and New York Times, making her a sought-after speaker, thinker and writer on understanding people and their relationship to technology. @kellycaine