Advancing the Value of Ethnography

Human-Centered Research in Policymaking


Article 5 in the series Data, Design and Civics: Ethnographic Perspectives

Mural, Internet Week NYC

“[Aims] such as unslumming slums, catalyzing diversity, nurturing lively streets, are unrecognized today as objectives of city planning. Therefore, planners and the agencies of action that carry out plans possess neither strategies nor tactics for carrying out such aims. …although city planning lacks tactics for building cities that can work like cities, it does possess plenty of tactics. They are aimed at carrying out strategic lunacies. Unfortunately, they are effective.” (Jane Jacobs, The Death and Life of Great American Cities, 1961, 321)

Growing density, climate change, economic instability, migration, the increasing penetration of information and communication technologies: these urban trends are pushing traditional city management approaches to their limit. It’s no surprise that the ‘smart city’ and related technology-oriented approaches are a leading innovation model among forward-thinking municipal policymakers. The open question is whether the benefits of technology can be delivered in ways that truly serve and empower city dwellers, on their own terms.

A recent partnership with the New York City Mayor’s Office of Digital Strategy provides some hope: A team including the Public Policy Lab worked with the Chief Digital Officer and her staff to use ethnographic research to develop the New York City Digital Playbook, a policy and operational guide to help City staff develop new and improved digital services for New Yorkers. As we’ll describe below, the project shows how ethnographic and design methods are emerging as best practices in public policymaking for smart cities.

But What Is a ‘Smart City’?

The term ‘smart city’ is as complex and contested as it has been popular. There are multiple definitions, but most promote the idea that investment in information systems drives efficiencies by standardizing and scaling platforms for citywide service infrastructure, such as high-speed networks, data accessibility, and cloud computing. The benefits of a ‘smarter’ city are typically described in economically rationalized terms – quicker service delivery, lower costs, energy efficiency – although more nuanced and holistic accounts also reference civic outcomes, such as greater government transparency, citizen engagement, and social equity.

As essential as technological investment and development certainly is, smart city policy approaches are too often characterized by the same functionalist, techno-scientific paternalism that defined the urban planning innovations of the 1950s and 1960s and continue to taint our cities with social and architectural repercussions. Governments’ 21st-century dependency on private-sector partnerships and infatuation with so-called data-driven approaches compound the danger of such an approach today.

For example, smart city projects often require significant financial investment and complex technological implementation – more than government alone can typically provide. Public entities therefore enter into partnerships with private-sector providers, such as business-to-business (B2B) enterprise system vendors whose role becomes to implement B2B-to-consumer systems at a city scale. To succeed beyond a minimal level of technical feasibility, however, these initiatives must also support specific and highly variable public needs in urban contexts. These include the needs of residents who are rarely customers for enterprise solutions – poor, disconnected, burdened, or otherwise outside the scope of the imagined (usually narrowly) ‘user’. Failure to respond to these needs has resulted in large, expensive systems that suffer low adoption, reinforce existing inequalities, and even put some of the city’s most vulnerable dwellers at risk.

These failures are rooted in a fundamental misunderstanding: Cities are about people, not just technological platforms. The impact is human, not just technical and fiscal. And smart city initiatives too rarely prioritize the question of just who these people actually are—from the diverse residents of cities to the people designing and implementing technological platforms.

Digitally wired policymakers – often fluent English speakers with advanced degrees, upper-quintile incomes, and multiple broadband-enabled devices within arms reach literally 24 hours a day (a profile that also describes the authors, not coincidentally) – authorize ‘smart’ solutions that may not reflect the priorities or preferences of a vastly more diverse user base of city residents. Policies and programs that privilege digital channels or non-interpersonal interactions can disadvantage residents who may need or prefer analog tools or human-to-human engagement, whether for reasons of personal ability, time burden, financial resource, cognitive load, cultural background, or emotional wellbeing.

Challenges of the ‘Smart’ Approach

These are the types of issues that come to light via the investigative lenses of disciplines such as sociology, political science, anthropology, and design. Yet Big Data is frequently positioned as the lifeblood of the smart city, with data scientists interpreting statistical evidence to inform policy decisions and service improvements. Although access to large pools of environmental, system, and user data is an incredibly valuable resource, quantitative analysis alone offers a limited view of ‘what’ is happening, and insufficient to address ‘how’ and ‘why’. Social and cultural complexities must be understood to marry the ‘hard’ infrastructural investment with the ‘soft’ infrastructure we also need to make our cities more equitable and resilient (Fullilove 2005, Klinenberg 2012).

A host of related issues about ‘smart city’ initiatives therefore come into question. For example, the normalization of data gathering at such a scale raises a series of concerns relating to citizens’ data rights: In the tension between privacy and surveillance, is it possible – or advisable – for an individual to ‘opt out’? Would opting out result in limited access to city services? And how might resident recusal impact data sets? Private sector partners are increasingly empowered stakeholders in the civic realm, with their own specific goals and agendas: What level of scrutiny should they be held to? How can city managers ensure that existing digital divisions aren’t further exacerbated, so that all city dwellers have equitable access to technological resources and related opportunities?

Human-Centered Approaches to Urban Innovation

An ethnographically informed, design-led approach to policymaking and service delivery is necessary for crafting truly smart cities – cities where innovation can be implemented at scale while responding to the context of individual residents’ lives. Ethnography offers both a systematic approach to understanding the complex social life of a city that helps policymakers frame the “problems” they are solving for, and a central method for surfacing and interpreting the complexity and diversity of lived, urban experience. When coupled with design, ethnographic insights take on a pragmatism that becomes the foundation for a design process characterized by participatory design, iteration, and testing. A service design approach recognizes that services are ‘living’ – they are both defined by and influence the dynamic environments of which they are a part and will need to continually adapt and evolve.

Public Policy Lab fellows Hefen Wong and Sarah Lidgus work on synthesizing research insights.

This approach could ensure that a common system implementation issue – such as technical interoperability – would gain more holistic operational considerations. What if system integration was mapping along social networks, as well as information architecture? Behavioral insights then inform design adaptations and identify opportunities for systemic innovation. This means understanding types of users and how they may be different from one another, discovering how to prioritize and design for specific scenarios, while also surfacing ‘surprises’ that only ethnographic methods such as observation and interviewing can reveal. These are the local hacks and micro-innovations that may very well hold the key to significant new smart city policy or design decisions.

A Case Study: The NYC Digital Playbook

Both city and federal officials are increasingly interested in using ethnographic research to inform policy. In our work on the Digital Playbook for New York City, we interviewed nearly fifty City leaders and civic technology experts, and together with our project partners met with dozens of New Yorkers in their homes and workplaces to learn about their experiences accessing City services. We visited community activists in East New York, church leaders on Staten Island, and veterans in Manhattan. We sought out residents living below or near the poverty line – who make up nearly 50% of the city’s population – from single moms in the Bronx and Harlem to undocumented immigrants in Brooklyn and Queens. We specifically focused on speaking with New Yorkers who were experiencing a range of difficult challenges accessing or using the City’s digital tools, from New Yorkers with visual impairments to non-native English speakers with limited literacy to elderly New Yorkers and those with mental-health issues who are not Internet users. These face-to-face engagements allowed the team to record a cross-section of New Yorkers’ experiences with digital services – data that we then catalogued and analyzed to identify key aspects of New Yorkers’ digital lives.

In addition to generating valuable human insights, research with end-users both generates new ideas for more effective policies and programs and also creates a sense of urgency and value around the work. As part of the Digital Playbook project, staff from the Office of Digital Strategy accompanied researchers on multiple visits with member of the public, exposing the team to ethnographic practices. Policymakers also participated in several research synthesis sessions and design pin-ups, where we sorted through our collected insights and looked for patterns and priorities. We held a similar analysis session with extra-governmental civic technology leaders. The collaborative nature of this process is especially valuable in complex policy environments, as it demands participation of all key stakeholders. The research process becomes a method for driving greater understanding and empathy and accountability between diverse participants.

In the case of the playbook, the research effort led to development of digital policy based in human-centered values. The team’s first step was to identify a set of six core principles – grounded in ethnographic research – that will now drive the City’s digital policy and service delivery. Principles like “Listen and Respond” and “Reach People Where They Are” represent a responsive, bottom-up approach to digital innovation that’s a significant shift from top-down imposition of ‘smart’ technologies that may not actually address resident aspirations and concerns.

Only once the core principles were defined did the team begin to generate the twelve specific digital strategies that make up the heart of the playbook. These strategies were also repeatedly stress-tested against our research findings, leading to policy guidance like “Structure services around New Yorkers’ needs, rather than by agency needs” or “Engage New Yorkers in the creation and testing of services” that, while seemingly obvious, actually represent a sea change in how most government policy and services are developed. Other strategies specifically address issues of public/private partnership and the public’s privacy and security needs.

Feedback from city leaders on the collaborative development process has been encouraging. As one policymaker told us, the widely sourced and co-designed playbook provides support and authorization for applying a human-centered approach to the City’s digital service delivery. New Yorkers’ needs are dynamic and will continue to evolve, so we’re encouraged that the city expresses an ongoing commitment to research and design.

Expanding Ethnography in Policymaking

Our work is part of a broad human-centered shift in policymaking. However, design and ethnographic work is happening primarily in the arena of operational policy – the strategic direction and day-to-day practices that inform how members of the public interact with government. Less enlightened are the “upstream” phases of policymaking that constrain operational policy. These are legislative policy, laws enacted by elected legislative bodies, and regulatory policy, the arena in which civil servants interpret how a given piece of legislation should be acted upon by operational components of government.

How can policymaking progress to a more complete adoption of ethnographic and human-centered approaches? To increase the lasting impact of our work in this arena, it is helpful to understand four stages of adoption.

In stage 1, before adoption of human-centered approaches, government policymakers typically carry out no engagement with members of the public or front-line service providers during the development of a new or improved service or policy. At best, they hold community forums to inform the public of their plans or run focus groups or testing sessions to gauge the public acceptability of a forthcoming policy or service improvement. They do not carry out collaborative or iterative development with end-users, particularly their own front-line (often unionized) staff. This anti-collaborative strategy may be due to a lack of confidence in the utility of engaging with end-users, a fear of negative public or press attention, a lack of awareness about how to carry out useful research or design with end-users, or insufficient time or budget for substantive collaboration.

In stage 2, government policymakers begin to embrace a human-centered model. They typically start with co-design of individual material artifacts, digital tools, or physical spaces – static products of design. Stage 2 often leads quickly to stage 3, where government applies human-centered approaches to more intangible products, such as service interactions with the public and internal employee interactions and systems. It’s often in Stage 3 that, as policymakers begin to consider how to alter sequences of experiences over time, they also come to understand that research into users’ feelings and behaviors is integral to successful design outcomes.

Finally, in stage 4, policymakers apply human-centered approaches to their own decision-making and leadership practices. Our City partners on the Digital Playbook project were certainly at this point when they sought our engagement on the project. They and other policymakers at this stage ask how the lived experiences, preferences, and aspirations of members of the public and of service providers – and their own personal and professional experiences – can helpfully inform government policy and operations.

Our hope is that the Digital Playbook project – as well as similar ethnographically informed work the Public Policy Lab is carrying out with the U.S. Department of Veterans’ Affairs and other agencies, and that many other organizations are conducting with public service providers – will help generate a fundamental change in how government develops programs and policy. We hope to see government transformation driven by multi-disciplinary collaboration, increased adoption and efficacy in using ethnographic and design-led approaches, and a broad commitment to ongoing research and iteration to ensure that public services reflect the public’s preferences and aspirations.

chelsea maudlinChelsea Mauldin is a social scientist and designer with a focus on government innovation. She directs the Public Policy Lab, a New York City nonprofit that designs better public services for low-income and at-risk Americans. She is a graduate of the University of California at Berkeley and the London School of Economics.

natalia radywylNatalia Radywyl, PhD, is a social researcher working at the intersection of ethnography, environment and service design. With clients ranging from government government and Fortune 500 companies to non-profits and the community sector, her work reflects a commitment to using social research and design as means of improving urban resilience. She is currently Senior Design Researcher at Fjord in New York City, and a Public Policy Lab Fellow.


Introduction to Data, Design & Civics: Ethnographic Perspectives by Carl DiSalvo

Innovation Teams, Mundane Innovation, and the Public Good, Andrew Richard Schrock

Ethnography of Civic Participation: The Difficulty of Showing Up Even when You Are There, Thomas Lodato

Ethnographies of Future Infrastructures, Laura Forlano

Service Designing the City, Natalia Radywyl (free article, please sign in or sign up)



Chelsea Mauldin, Public Policy Lab