Advancing the Value of Ethnography

When Customer Insights Meet Business Constraints: Building a Go-to-Market Strategy for a Smart Home Offering in a Regulated Space


Download PDF

Cite this article:

2018 Ethnographic Praxis in Industry Conference Proceedings, pp. 405–426, ISSN 1559-8918,

Case Study—This case draws on work in the energy efficiency industry where many utilities rely on data-driven insights and decision-making to encourage consumers to adopt energy-saving products and behaviors. In this highly regulated industry, utility staff must show value through big data, and studies often rely exclusively on quantitative data analytics to create behavioral models to explain or predict behavior. However, purely data-driven research often fails to answer questions about why customers behave a certain way, and what product or program managers and marketers can do about it. In this case study, the team from ILLUME Advising LLC (ILLUME), a research consultancy in the clean energy industry, illustrates how their cross-functional team paired qualitative and quantitative research on residential home energy use. The case study draws on an exploratory market and segmentation study for an electric utility interested in engaging customers through a smart home product offering. The team used a mixed-methods, hybrid research approach, cycling between quantitaive and qualitative methods, and refining the project concept and hypotheses before each stage of research. This approach set the stage for market and consumer insights that showed opportunities beyond the clients’ original concept.


This case study explores how an energy efficiency research team used a hybrid approach that included qualitative and quantitative research practices to refine the product offering and go-to-market strategy for a large public utility interested in entering the smart home market. Operating in a regulated environment, the utility had a preliminary concept for a product offering, and the research team’s brief was to conduct exploratory research for a concept that was already encumbered by numerous constraints. The team set the stage for a choice-based segmentation study to refine the product offering by socializing a series of market and consumer insights, including customer data mining, competitive market assessment, and in-home ethnography around energy and the smart home. In-home ethnography provided preliminary customer insights around the utility’s energy app and customers’ motivations and interest in managing different aspects of their home. This shaped a large-scale quantitative segmentation survey by ensuring that the concepts and language matched consumer cognition and expression.

This case study highlights the value of layered and iterative research, particularly in new markets and stakeholder environments constrained by pre-existing business processes, capabilities, or regulatory requirements. It also explores how qualitatively-grounded and experimentally rigorous quantitative research (in this case, choice-based segmentation) can be used to shape a product offering in addition to its more common use of defining the audience.

This case study presents:

  1. The stakeholder and regulatory landscape, including unique considerations for developing a consumer product offering within a regulatory framework where products and services are required to demonstrate energy savings
  2. An overview of the research, highlighting the layering of qualitative and quantitative methods that not only strengthened the research but socialized findings gradually, with each phase setting expectations for what the next might uncover or confirm
  3. A discussion of the strategic recommendations and business outcomes, including trade-offs between addressing customer needs and meeting regulatory requirements in the utility context


Stakeholder Landscape

The authors represent the ILLUME Advising team; ILLUME is a research consultancy in the energy industry. Our researchers work with electric and gas utilities to enhance their energy efficiency offerings, with the goal of reducing energy usage and demand to meet regulatory goals. Figure 1 introduces the other key characters in this story.

In Spring 2017, the team embarked on a marketing contract with a public utility partner to support their development of Home Energy Management (HEM) offerings in the smart home space. For this engagement, the research team partnered with (a) a marketing firm to develop communications and engagement materials for a “home energy information” app-based product, and (b) two specialized research partners to design a complex pricing and marketing segmentation experiment to inform the structure and positioning of a nascent smart home offering.

The client is a large public utility providing electric and gas service. Like many public utilities, they offer commercial and residential programs to help customers save energy and meet state regulator (top box of Figure 1) mandates to reduce the energy use of their customer base by a certain percentage each year. All utility energy efficiency programs are funded by ratepayers through a per-unit surcharge leveraged on energy bills. As such, utilities are required to demonstrate that any programs or offerings using ratepayer funding deliver energy savings (i.e., reduce consumption) cost-effectively.

The utility partnered with a vendor to introduce an energy analytics app that provides customers with trends and tips on their home energy use. Customers with the app can install hardware – an energy hub – that enables real-time updates to the app. This hardware can also serve as a smart home hub by facilitating communication with other devices, illustrated in Figure 2.


Figure 1. Stakeholder Relationships


Figure 2. Concept for Smart Home with Embedded Home Energy Management

Initial Offering Concept

Seeing the growth in the smart home space, and the entry of mass-market players like Samsung, Amazon, and Apple, the utility was interested in testing their own offering. In addition to the energy analytics app, the utility offers rebates on smart thermostats and smart LED lighting. Knowing that energy analytics, thermostats, smart plugs, and lighting may not compel their customers to engage with the app (and reduce energy consumption), the utility wanted to develop a smart home offering that could pair with other, non-energy-saving devices.1 By offering smart locks, sensors, cameras, and smoke detectors in addition to energy efficient products and services, they believed they could increase customer engagement and adoption. While the ultimate goal of the pilot, from a regulatory perspective, was energy savings, the utility was also interested in customer engagement.2

The initial offering could be tested as a pilot under looser regulatory and cost-effectiveness requirements, but, ultimately, it would need to deliver cost-effective energy savings before scaling. When the researchers joined the project, the utility energy efficiency group had positioned the smart home pilot as a HEM pilot, with the vision of using broader customer interest in the smart home to generate engagement with HEM (i.e., energy analytics and appliance management). The utility brought in the ILLUME team as they were transitioning to a new version of the energy information app to conduct the pilot.

In addition to the initial concept of using a broader range of smart home devices to encourage engagement with HEM, the software, energy hub hardware, and compatible devices were in development. The energy analytics software vendor had development-phase software that connected their app to a limited set of smart home devices. Nearly all the compatible devices were white label smart home products and the software was not compatible with better-known, mass-retailer brands. These compatible devices offered more control and reliability for the vendor and utility at a much lower cost. Additionally, the software was not yet compatible with a few categories of smart home security devices. While the utility and vendor were open to a long-term strategy that might reveal a different ideal product set, in the short-term, the offering was shaped by these capabilities.

The client had a concept for product pricing at the outset of the research engagement. They preferred a subscription-based service with monthly fees, largely driven by billing capabilities and pre-existing assumptions about customer preferences.

The Role of Customer Research

The utility was considering a range of product configurations (i.e., bundles of devices and features) and pricing strategies when the research team joined the project. These considerations were based on (a) prior customer insights from related energy efficiency offerings (e.g., the energy analytics app), (b) then-current capabilities of the utility and vendor such as online shopping and fulfillment functionality, and (c) business and regulatory objectives. Like many consumer product offerings, the model needed to meet customer expectations and, at minimum, cover its costs. Unique to the utility environment, the offering also needed to meet regulatory requirements for delivering cost-effective energy savings.

Ultimately, the utility wanted to know whether the entry into the smart home space could generate enough engagement in HEM to increase energy savings and recover costs.3

The client needed the ILLUME team to quantify and characterize their customers’ interest in the smart home and their willingness to purchase those devices from a utility rather than other brands. The client also wanted to test what pricing structure might work best in the market, including options for recurring monthly fees, upfront payment, or a combination.

The utility requested market research that included a competitive market assessment and customer receptivity to, and willingness to pay for, potential offerings. This differed from standard product development research because the utility was enhancing a regulated offering – an app with real-time energy analytics – and needed to demonstrate energy savings.4 The client was interested in market research for multiple reasons, including to:

  • Validate their initial concept for a smart home offering
  • Estimate potential market share of the offering (to inform a business model)
  • Determine the appropriate pricing model and price level (to optimize customer acceptance while covering costs)
  • Inform messaging and positioning of their offering (based on customer needs and language)

Findings from this research would be used to shape the product offering and go-to-market strategy within the business constraints listed above. In other words, even if the utility could not develop or market the ideal product for their customers, they needed to understand the available market for what they could offer.


ILLUME developed a staged approach for assessing market potential and determining the go-to-market strategy. This approach see-sawed between qualitative and quantitative research, using findings from each phase to socialize potential refinements to the product concept, and develop hypotheses to test in the subsequent phase. In this case, as in Anderson, Faulkner, Kleinman, and Sherman (2017), an iterative research process allowed the research team to socialize findings with the client team over several months.

Figure 3 illustrates how these research efforts came together. The approach was iterative and built on previous research by the ILLUME team and the utility. The team used qualitative ethnographic research in the discovery phase to (a) uncover and socialize customer needs and personas, (b) set expectations for what might emerge from quantitative research (e.g., validating or invalidating the initial concept), and (c) uncover customer language, preferences, and decision-making processes to strengthen survey research. An initial competitive assessment helped set expectations for what quantitative survey research might reveal in terms of customer preferences. After discovery-phase research and prior to survey research, the team presented customer and market insights to set the stage for the quantitative survey.

The following sections detail each stage, including a sampling of customer insights. Like many research findings, these results are specific to the geography, time frame, research context, and client context, and may not be generalizable to other utility service territories or to smart home customers in general.


Figure 3. Research Process Flow


Competitive Market Assessment

At the time of this study (mid-2017), the smart home market was crowded with small and large product and service providers, including offerings from consumer products manufacturers; telecommunications and home security providers; online and brick-and-mortar retailers; and energy management start-ups. Business and pricing models varied widely, from pay-per-device packages with no recurring fees to subscription packages. Few consumer research or market studies were available to illuminate which of these offerings were most attractive to customers and where – or why – market shares were growing.

As part of the discovery-phase research, the team conducted a competitive review of smart home offerings. A key finding was equipment-based pricing: Most smart home products and services are sold as equipment with a one-time cost, and any associated apps or services are provided at no additional cost. Based on this research, only a subset of providers – primarily security and telecommunications companies – offered subscription-based models and these received mixed consumer reviews.

The prevalence of lighting and home security products in competitors’ starter kits and smart home messaging suggested their importance as entry-points into the market. Marketing positioned smart lighting as convenient, easy, and fun, and home monitoring as a tool to keep your family and home safe. In the utility regulatory context, where energy savings are critical, the offering cannot focus on security, even if customers are most interested in these products. Monitoring and security devices do not deliver energy savings – hence, based on the competitive assessment, the research team and stakeholders began to see that a utility-sponsored smart home offering might need to find a different niche than mass market retailers were filling.

A secondary role of the market assessment was to inform the latent class discrete choice (LCDC) experiment embedded in the quantitative survey. As discussed below, the LCDC presented hypothetical smart home shopping decisions to customers, and asked them to make trade-offs between brands, devices, and services. As such, the LCDC experiment needed to represent the market as realistically as possible, including leading brands and common devices.

Customer Data Mining

Next, the research team performed quantitative analysis of consumer behavior around the energy savings app, and customer characteristics of segments identified as having (a) higher engagement, or (b) higher energy savings through the app. The client team and their external evaluators had previously identified several target segments from among the Mosaic® lifestyle segments, which cluster customers based on sociodemographic characteristics. The team leveraged rich customer data from the utility to characterize the target segments in terms of demographics, past utility program participation, communication and engagement with the utility, and the frequency and recency of app use.


The team built on the learnings from the market assessment and customer data mining with ethnographic interviews. The client team wanted to learn more about the people using the app and their experiences to feed into messaging campaigns. They also wanted to ensure that, as they brought a smart home package to market, it met customers’ needs and desires.

To learn more about how the app fit into people’s lives and their concerns, the team conducted 4 phone interviews and 15 ethnographic, in-home interviews. Interviews were conducted across the utility service territory, followed a semi-structured guide, and were audio recorded. During these interviews, the researchers observed people using the app on their phone or tablet while explaining their experience and how the app fit within other household concerns or priorities.

The sampling approach focused on customers pre-identified to belong to specific Mosaic segments that the client and their external evaluators had hypothesized were good targets for their app based on past program participation data.5 The team attempted to recruit these segments, but in practice, the team found that these demographic segments were not always a good predictor of behavior or use of the app and observed some discrepancies between the segment demographics and actual participant demographics. These discrepancies led the team to abandon the Mosaic segments as an organizing structure for presenting research. The following section presents some key findings through the story of one of the interview participants.

At home with Melissa:

Two researchers, Liz and Allison, sat at the high-top table in Melissa’s dining room. It was summertime, and Melissa’s three kids, ranging in age from 2-12, wandered in and out of the room – the two-year-old joining them for much of the interview. As she discussed her approach to household maintenance, Melissa explained that she had had trouble paying her bills in the past, but now kept things in order using a binder system for recurring bills. When a utility bill arrived, she would divide it by four and would send a check out each week. Melissa’s affect around these three-ring binders was joyful. When the team asked her about changes or repairs she was thinking of making to the home, she immediately retrieved another binder. This one was for her kitchen renovation and contained her mood-board-style assortment of images of dream kitchens, along with print-offs from websites with specific items and their costs. Her current kitchen was original to the house and had a heavy cast-iron sink and a linoleum floor that was peeling at the edges. She explained that she had chosen everything for her new kitchen already and was little-by-little making the necessary purchases, using the binder to keep track. She had most of the floor tiles already in the basement, along with the new cabinets and countertops. She was waiting on the sink – once she was able to buy it – after enlisting some help to carry out the old sink, she was going to complete the renovations.

When the conversation turned to her experience with the utility’s energy savings app, her face lit up with delight – she had been using the app for several months and had recently installed the hub showing her usage in real time. When they moved into the house, she explained, the first bill was $400 or $500 – well beyond what she could afford. She made it her goal to bring the bill down. Maintaining a close watch on the bill was particularly important to her because she had once had her power shut off for being in arrears. She never wanted to be in that situation again, which is why she had started paying her bills in weekly installments. She also had taken steps, such as unplugging devices and appliances when not in use, to lower her bill. She explained that every autumn she would set aside a day to seal the windows with plastic and to weather-strip the doors for the winter; she showed us on her calendar when she planned to do that this year.

As concerned as she was about bringing down her bill, she was more concerned about her family’s safety. Although she knew that unplugging devices and chargers when not in use could lower her bill, she did not like her children touching the plugs for fear of an electric shock. Instead, after her kids were in bed, she would go around the house unplugging any of their devices to stop electricity “vampires” from driving up the bill. Concern for her children’s safety motivated other changes as well: she and her husband removed a ceiling fan over their son’s bed that she worried would fall in the night.

In discussing the app, she liked that she could see how much energy they were using at any given time, but she wanted to see more details about their use. She was also interested in sensors or other features that would help her understand what was going on at home when she was at work.

Key Themes from Ethnography

The team’s interaction with Melissa provided several key insights around the app and the potential market for smart home products. The “thicker” data that emerged from the ethnography added nuance and individual stories to the data projections based on Mosaic segments or utility data analytics (see Wang 2013). For example, traditional propensity models would likely have excluded Melissa from a target group given her preference for paper bills and paying by check instead of online. As the team learned, this analog system provided Melissa with control over her bills and a real-time knowledge of what she owed. Melissa supplemented her binders with information from tracking applications such as the utility energy app. Often considerations of self-tracking behaviors, such as considerations of the Quantified Self movement, presume biosensors and other technical apparatuses (see for example, Nafus 2016). Melissa was interested in closely monitoring her household expenditures and usage; she did so through a mix of technologically-enabled tracking apps and her paper binders.

A general theme that emerged during the ethnography was around the value the app provided its users in bill control and visibility. This value of control was not uniform. For some people, it was linked to a confidence around their ability to pay; for others, it had more to do with managing their home remotely (for example, someone who traveled for work enjoyed the ability to ensure his home was using minimal electricity while he was away).

A desire for control and a granular knowledge of what was happening in the home was similarly important in the context of what kinds of features and smart home devices people had installed or were hoping to install. The team found that security – both financial and physical –was a key interest and driver. Many of the participants were looking to install smart door locks, the video-camera doorbell, or other security cameras, along with smart smoke detectors or carbon monoxide (CO) sensors. These products enabled greater control and awareness, suggesting a substantial market for a range of security features, including sensors, smart locks, and cameras. Importantly, the motivation to install these security features was much more often about understanding what was happening in the home than protection of the home. Where safety was a concern, it was more often expressed in terms of family safety (did my children come home from school?) than in terms of fear of crime or intrusion.

The team also spoke with several individuals who had already installed smart home features. While these customers saw the value in the app, several noted that the current functionality and user experience was not on par with larger brand names. These individuals had also already installed many of the features that might be offered in the smart home pilot – from smart thermostats, to lights, to open-close sensors on windows and doors. Therefore, the team hypothesized that these early adopters were not an ideal market.

Ethnography: Implications and Next Steps

The ILLUME team shared with the client that their customers were looking for home security products alongside other smart home products. Although security features had emerged as an area of customer interest in the comparative market research, the utility initially saw this as a supply rather than demand issue. Just because other companies were offering these products did not necessarily mean their customers were likely to purchase them.

After the team presented the qualitative research findings, highlighting the ways in which security and control were so often linked, the client team agreed to consider security features in the next research phase. Some of these features were not compatible with the current version of the hub, and security devices do not have an obvious energy-saving application. However, the team agreed it was important to test them in the LCDC survey to get an accurate image of what smart home features people are interested in purchasing.


The team conducted an LCDC survey with the objective of (a) segmenting the utility’s customers based on their smart home purchase preferences, and (b) understanding overall preferences for products, features, and pricing models. LCDC is a type of stated preference trade-off analysis that combines elements of latent class analysis and discrete choice analysis into a single framework. ILLUME worked with two research partners to design the experiment and analyze results. Members of this combined research team conducted an LCDC experiment for LED bulb preferences in California when LEDs were an emerging technology. The methodology was based on that study and the report, which contains a detailed methodology (Opinion Dynamics and StatsWizard 2012).

Survey Approach

ILLUME deployed the LCDC experiment to utility customers as part of a smart home survey containing attitudinal, behavioral, and demographic questions, in addition to the experiment. The team used results from 1,047 completed surveys for segmentation, segment profiling, and marketing insights.

The survey presented each customer with nine hypothetical shopping scenarios (stores) where they selected between smart home packages with different attributes, shown in Table 1. The packages contained different brands and quantities of smart home devices, including home security, monitoring, lighting, and thermostats, offered at different pricing levels with different support options.

Table 1. Attributes Tested in Embedded LCDC Experiment

Brands Six total brands representing different product/service providers
  • Thermostat: Smart thermostat, learning thermostat, or no thermostat
  • Lighting: Dimmable smart bulbs, white smart bulbs, or no bulbs
  • Smart plugs (or none)
  • Smart light switches
  • Sensors: Open/close sensors, motion sensors, smart door locks, a “full package”, or no sensors
  • Cameras: Smart camera, doorbell camera, or no camera
  • Other: Smart smoke/CO sensor, water leak sensor, or none
  • Home energy information: Basic home energy feedback or advanced feedback (real-time, with tips, targets, and alerts; appliance “health” monitoring), or none
  • Voice control (via Home Assistant)
  • Care plan
  • Pay upfront or spread costs over 12 months (total price determined based on market prices, plus a +/- 10% discount or premium for experimental purposes)
  • Monthly service fees ranging from $0-$19.99 (in increments)

The goal of discrete choice experiments is to represent the complexity of consumer decision-making (Boomer 2014). Given the many emerging products and features in the smart home space, the team chose an LCDC experiment to uncover customer preferences by allowing them to make trade-offs between different hypothetical product sets. In contrast with “direct” stated preference questions, this method presents customers with “all-in-one” product and feature bundles where they may like one element of one package, and another element of another package, but must choose one or none based on what features matter most.7

Each respondent saw nine experimentally-designed stores and selected one of four smart home packages or “None of These”. The team tested two “blocks” of stores in two sub-groups of the sample to ensure that the full range of trade-offs was tested. Figure 4 shows just one example of a store from the survey. Before customers began the experiment, they read instructions about the shopping scenarios and saw an example store. The team elected to represent the brands and attributes in text rather than images to minimize visual emphasis on any one feature and allow respondents to use or develop their own mental models of each feature (Hurtubia et al 2015).

Building on Qualitative Findings

The findings of the qualitative research – including nuances around how customers described their home and expressed interest in smart home features – informed the survey design. This approach required alignment with customer language to produce valid results. The in-home interviews were critical to understand how consumers perceive their homes, how they think about the smart and connected home market, and what features they are interested in. Given customers’ interest in security features, the team sought to represent a range of security and monitoring devices, even if they were not compatible with the client’s offering. In real-life scenarios, customers might be weighing options with these devices. The team also spoke with several people who mentioned they used color-changing smart bulbs in their home for ambience, for fun, to simulate morning or night, and to signal different things to their children (e.g., reading time). From these lighting stories, the team hypothesized that lights may be an important entry point into the smart home, and color-changing bulbs (not then-compatible with the client’s offering) may have experiential value beyond controllable standard white bulbs.


Figure 4. LCDC Survey Store Example

The qualitative interviews and market assessment also provided guidance about what language to use to describe technical features. For example, to represent the smart home packages’ compatibility with Home Assistants, the team used the term “Voice Control”, as the term “home assistant” did not seem commonly-used. The interviews and market assessment also showed that HEM and home feedback are not typical components of smart home offerings, and that the term “Home Energy Management” does not resonate with customers. Energy management is typically a stand-alone product offered by specialized companies or utilities. Through interviews, the team learned that many customers did not have pre-existing concepts of what “home energy information” or “home energy management” was; therefore, the experiment provided extra explanations of this feature.

Analytic Results

Analysis of the Stated Preference Survey provided two results: (a) A rank-ordered set of feature/pricing preferences among all customers and by segment, showing the biggest drivers, and (b) smart home customer segmentation for the utility, organizing customers into five distinct groups with different product/feature preferences, affinity for the utility, and likelihood to purchase.8 The segmentation was a means to understand overall feature/pricing preferences, as well as characterize the utility’s best target.

Overall Results and Preferences

The ILLUME team used the LCDC survey to uncover smart home and HEM preferences specific to the product concepts the utility was considering. The first analysis step was to identify customer segments based on their preferences; overall preferences were derived after segmentation by calculating weighted average preferences across segments. Figure 5 shows the results across all segments. Overall, brand was the leading driver of decisions. Those who preferred to purchase through the utility perceived it to be reliable, accessible and convenient, because it was a local company that had maintained many long-term relationships.

Smart thermostats were not a strong driver of package selection, nor was home energy analytics, one of the key differentiators of the clients’ offering. While smart lights were popular, and are energy-efficient LEDs, the quantities offered in smart home kits are typically not large enough to drive energy savings. Furthermore, emerging research on whether controlling lighting through a smart home platform actually saves energy is mixed (Efficiency Vermont 2016). The relatively low rankings of potentially energy-saving devices highlights the challenge of developing an energy-saving smart home offering and finding the niche of customers most likely to use HEM features. Can the utility attract enough customers with an offering differentiated by HEM, or should they offer something more similar to competitors and hope that a sufficiently large slice of customers begin using the energy-saving features?

In the survey, the team included open-ended questions asking customers what they see as the benefits or drawbacks of purchasing smart home packages through their utility. Responses revealed that the client has a strong reputation for service and reliability, which could be a driver for choosing the utility for a smart home package – despite their lack of a track record or reputation in the space:

  • Reliability and trust: “I’ve been with [UTILITY] for years and it’s a trusted and known Company”; “Reliable company that can come out and provide service if needed”
  • Expectations: “I would expect the package to work more seamlessly from [UTILITY].”; “Local company for faster service, billable via my energy bill.”


Figure 5. Relative Preferences for Features and Devices from the LCDC Experiment. The importance levels speak to whether differences in a feature or device between packages drove overall preferences for that package. If a feature or device appears as “unimportant” in this rank-ordering, it does not mean it is unimportant in general. Customers may take the feature for granted (e.g., voice control) such that they were not concerned about differences. Additional data points and customer insights are needed to understand why an item ranked lower. Note that the exact order of customer preferences has been obscured to protect client confidentiality.

In return for placing their trust in the utility, customers expect to be able to call easily for help. The value of strong customer service is meaningful in the smart home space, as the earlier ethnographic research revealed several reliability and usability issues and complaints with the utility’s then-current app and hardware solution.

Negative customer comments about purchasing through the utility also highlighted skepticism about the utility’s ability to produce a viable offering in this space: “Does [UTILITY] have sufficient experience and expertise?”; “To me UTILITY does not have a track record or earned reputation as a retail service like Amazon.” This highlights the challenges of offering a product suite through a utility, where the individual products may not carry the brand reputation of market leaders.

Finally, the experiment showed a preference for bundles (e.g., a starter kit) rather than a la carte offerings, particularly if offered by a trusted brand. The research team’s interpretation was that smart home technology is relatively new and perceived as complex. Therefore, customers are looking for sets that they can be confident work well together and use brand as a shortcut for quality. This finding also aligns with decision-making theory around cognitive burden and decision fatigue – the value of curating consumer choices to facilitate making any decision and improving satisfaction with decisions (See Iyengar and Lepper 2001). This finding is particularly important for marketing and communications considering the segment findings that highlighted that the “mid-market” customers may be the best target. These customers are eager to make their home a smart home but may not be as comfortable with compatibility and set up.

Segmentation Results

Throughout both the qualitative and quantitative research, the ILLUME team worked with the Mosaic segments the utility client already purchased. In the qualitative research, the Mosaic segments did not map easily or reliably onto observed or reported behaviors or preferences. As Flynn, Lovejoy, Seigel, and Dray (2009) and Cuciurean-Zapan (2014) have suggested, when customer segmentations are divorced from behavioral inputs or become overarching, their usefulness is frequently curtailed (even when their application may not be). Given this limitation, the team recognized the opportunity to use layered research to not only understand customer preferences, but to identify a potential market on the basis of behaviors alongside stated preferences. The team used relative preferences from the LCDC experiment (e.g., more interest in security vs. energy management features; importance of brand vs. pricing) and covariates (other survey questions) to develop five cluster-based customer segments.

A valuable output of the segmentation research was confirmation that the utility’s core target for their HEM offering was not early adopters. The ethnography team interviewed a few of these tech-adept customers. They had invested considerable effort into installing and networking an array of emerging smart home products and apps, often overcoming compatibility and connectivity issues. The profiles of this highly-engaged segment presented a tempting target for the client. At that time, the team discussed some of the potential drawbacks, including the segment’s already-formed brand preferences and expectations, particularly for name-brand devices that were not compatible with the client’s offering. The survey segmentation identified these “technophiles”, who seem attractive due to their smart home purchase propensity and acceptance of service fees. However, they are also price-sensitive, have high expectations for products, compatibility, and service (prefer buying bundles), and perceive the utility as an unknown in this product space, suggesting that many may choose other providers.

Instead, two mid-market segments emerged as promising targets for the utility’s offering, shown in Figure 6. Target Segment B: with moderate income, middle age range, and kids at home, and who were already buying smart home equipment at a solid rate, and Target Segment A: older, interested and engaged in smart home, in particular, easier-to-understand security products like smart locks and smoke detectors, and a high comparative level of comfort with service fees. This was unique among the segments, making them an attractive target for the utility’s offering, which may include a service fee.

The quantitative data allowed the team to characterize these segments not just in terms of their relative preference, but in their distance between each other (i.e., degree of difference), and relative size. The relative importance of brand, price, and smart home features allowed the team to position the five segments in terms of their likelihood to adopt, and their relative preference for the utility as the service provider. The team selected these two dimensions to summarize the segments as the question of whether they would purchase from their utility is an overarching consideration given the growing strength of mass-market players.


Figure 6. Example Segment Positioning: Likelihood to Adopt Smart Home vs. Relative Preference for Utility

Bringing Segments to Life

The research team examined each segment’s preferences for the smart home devices, features, pricing, and service options, and developed profiles of their relative preferences. While these segments were defined by the relative importance of smart home features, they could be described by the contextual data gathered in the survey. For example, their attitudes toward the utility, self-report of existing smart home equipment, early adopter profile, and demographics. Further, the team used relational and behavioral customer data from the utility, such as past participation in energy efficiency programs, online account access, and online billing preferences. Using these three layers of data – experimentally-determined preferences; self-reported attitudes, behaviors, and demographics; and utility relational data – the team created a rich profile of the five segments, illustrated in Figure 7.

In Figure 7, the customer characteristics and their smart home and device preference provide a starting point for the product development and marketing team to refine their positioning. For example, highlighting capabilities rather than brand differentiation may be more relevant to a segment that typically prefers packages, is not tech-savvy, and trusts the utility. Emphasis on “why buy from us” may be less important than showing how this technology might fit into their family life. As is frequently the case in more quantitative segmentation, decontextualized demographic details do not always provide adequate understanding of individual concerns or drivers. Moreover, while knowing that individuals in this segment are most interested in smoke/CO2 sensors and security packages might allow marketers to create their own story of a segment members’ life and concerns, that story may not align with actual individual’s experiences in that region, market, or segment.


Figure 7. Data-driven Segment Findings – Target Segment B Example (from survey and segmentation analysis only)

To make sure product and marketing staff interpreting findings could envision a living, breathing customer, the team coupled the generalized, data-driven profiles with examples from ethnography. For instance, in the team’s conversations with Melissa, she described her concerns around her children and electrical outlets, a safety dimension not addressed by the smart home devices tested. Her interest in smart home devices to monitor the house, like open/close sensors, was not focused on neighborhood safety but her children’s physical safety when interacting with their home. Integrating details such as these into the data-driven segment findings allowed for a more nuanced and robust segment depiction.

Recognizing that safety and security may manifest in multiple ways, the team used individual stories to disrupt assumptions about a uniform interest in security, and to inspire consideration of other angles for positioning the same set of products. For instance, highlighting family security within the home in contrast to a neighborhood or crime-oriented approach to security.8

Estimating the Market Opportunity

Based on the two target segments, the team developed business models for the utility offering. The models leveraged a market simulator tool that calculated likely market shares of a utility-sponsored smart home package given a specific set of devices, features, and pricing vs. competitors – i.e., entering specific configurations comprised of individual elements tested in the LCDC. The team used the market simulator tool to estimate relative market share of realistic utility packages compared with other providers’ most heavily-promoted starter kits. Because the LCDC experiment only represented a sample of competitors, and the survey respondents likely had a higher affinity for the utility than the general market by virtue of completing a somewhat onerous utility survey, the analysts developed adjustments for the utility’s likely market share using secondary data. Based on this analysis, and assumptions about the product offering and pricing, the team concluded that the utility had set ambitious first-year participation goals. The utility plans to re-visit the market sizing model soon after the start of the pilot, once enrollment begins.


The team presented several strategic findings with the goal of providing concrete feedback to shape the product offering and go-to-market strategy and quantifying the market size and opportunity for the utility. These included:

  1. Mid-market target: The team suggested a focus on mid-market adopters with higher utility affinity as the base market, rather than the most affluent early adopters. This finding emerged out of the ethnographic research, where the team had observed that many of these more tech-adept individuals had already installed smart home features offered by competitors. This finding was validated in the LCDC where the tech-savvy segment was found to be highly likely to adopt smart home features but unlikely to do so using a utility-branded package. Mid-market adopters in the target segments are identifiable through supervised machine learning (e.g., random forest) that leverage LCDC data, and the research team developed propensity scores to use in recruitment campaigns.
  2. Security and lighting as entry-points: The team recommended using security features and messaging as an entry-point with inspirational and capabilities-driven illustrations that evoke feelings of family and home security rather than a products-first message. Similarly, the team recommended that the offering contain and promote smart lighting as a gateway to the smart home, focusing on curating the home’s atmosphere.
  3. Product bundling: The team suggested that the product offering include an option to purchase products and features in bundles in addition to a la carte to lighten the decision-making burden. This was particularly helpful to reach less tech-savvy customers and align with competitors’ starter kits. Central to this objective is developing education and marketing that focuses on what you can do with a smart home rather than individual products.

While this paper was drafted, the pilot was nearing launch. The research team worked with other stakeholder groups including the product vendor to fine-tune the pilot structure. In parallel, the marketing team drew on the research findings to develop educational and recruitment emails, and online and in-app content on smart home capabilities. The team’s research – both quantitative and qualitative – shaped the marketing messages, highlighting what products can do in the home, specifically for security and lighting. The team was previously focused on messaging around smart thermostats (a favorite of energy efficiency programs), and the team decided to de-emphasize thermostats based on (a) overall attribute importance in the LCDC, (b) the ethnographic research showing lighting and security as potential entry points, and (c) relatively low penetration of thermostats compared with other smart home devices. Still, the offering must deliver energy savings, and the team is counting on HEM analytics and messaging, additional thermostat savings from home automation, and LED lighting upgrades to generate savings.

Some findings from customer research cannot be implemented in the short-term. These include offering devices and features as bundles, which the current online shopping application does not support, and an upfront pricing structure rather than a fee-based model. As mentioned in the Initial Offering Concept section, institutional precedent for monthly fee-based pricing established by IT and billing processes as well as the pricing structure in place for the energy analytics hub makes it difficult for the utility to offer upfront pricing in the short-term.

These trade-offs between customer needs and preferences, business constraints, and regulatory objectives highlight the challenges of conducting product development research in a regulated environment. In this case, some of the devices and features customers value most do not save energy (a regulatory requirement). Therefore, the utility is faced with the decision of whether to develop an energy-focused niche offering that may not attract many customers or an offering with more mass market appeal and an embedded HEM offering and hope customer engagement and education encourages HEM adoption and use to save energy within the general population.


This case study tells two stories. The first is about conducting iterative and adaptive research that layers qualitative, quantitative, and secondary research. An iterative and mixed-methods approach was needed in part because of the product space (an emerging market), and because of the stakeholder and regulatory context (the timing of the study within the product/concept development timeline, and the regulatory requirement to save energy).

The second story focuses on the stakeholder and regulatory context: the challenge of creating a product offering in a constrained regulatory context, layered onto a stakeholder environment in which product and service concepts had become entrenched due to a combination of (a) assumptions about customers and their needs from prior research, and (b) assumptions about organizational capabilities that might limit innovation. The two stories are intertwined – the research approach was developed to respond to the interests, assumptions, and constraints of the stakeholder team within the energy efficiency regulatory context. Through an iterative approach, the research team was able to share challenging insights gradually, creating multiple opportunities for communication.

The research toggled between identifying customer interests and motivations from an exploratory perspective, and testing concepts or assumptions constrained by the utility or vendor’s existing capabilities, and regulatory guidelines. When the research suggested that the products and features valued by most customers were not well-aligned with saving energy, the team had to find pathways to a viable market strategy. The team used customer insights to adapt to these constraints, such as (a) finding pockets of customers who may be more interested in a utility-sponsored offering, (b) recommending bundles where energy-saving features can be combined with products that have more customer appeal, like home security sensors, monitoring, and color-changing light bulbs (despite the lack of energy savings), and (c) identifying ways to message monthly service fees that align with customers’ brand perceptions of reliable and available utility service.

Amanda Dwelley is Director of Quantitative Research at ILLUME. She leads data mining, predictive modeling, segmentation, and evaluation efforts for energy efficiency programs, including behavior-based and Smart Home programs. She champions mixed methods to quantify “what happened?”, understand customer engagement and help improve customer experiences.

Elizabeth Kelley is Director of Qualitative Research at ILLUME. As a sociocultural and linguistic anthropologist (Ph.D. University of California, Berkeley), she leads ethnographic and qualitative research studies that explore how people (in their homes or businesses) think about, use, and rely on energy.


Acknowledgments – The authors would like to thank the ILLUME team (Anne Dougherty, Jason Turner, Jon Koliner, Sharon Talbott, Allison Musvosvi, Shefije Miftari, Jes Rivas, Robert Jamie, and Erin LaVoie), as well as our research partners Jennifer Mitchell-Jackson and Mary Sutter (of Grounded Research LLC), and George Boomer (StatWizards LLC), for their instrumental contributions to the research process and the development of this paper. The team would also like to thank all the individuals who participated in the research, particularly those who briefly shared their homes with us.

1. “Smart” thermostats are those that are WiFi-enabled and programmable, allowing people to change the temperature of their home remotely through their phone. “Learning” thermostats are similar, with the added feature of automatically adjusting to the homes’ occupancy norms and comfort preferences without resident input. Smart plugs allow users to turn on or off any appliance plugged into a given outlet.

2. Many of the research team’s utility clients are trying to increase brand relevance and customer engagement. This effort is intended to keep lines of communication open for new and emerging business models and maintain strong reputations within their communities to help with regulatory cases and discussion. For the past few decades, many public utilities have come to use their refined energy efficiency offerings as a customer communications and engagement platform. However, not all customers engage with utility-sponsored energy efficiency offerings in their current form. With more consumer products focusing on energy (e.g., smart thermostats and lighting), and non-utility energy service companies expanding offerings into the utility space (e.g., solar and electric vehicles), some utilities are looking for more avenues of customer engagement. The smart home is one of many market opportunities that utilities are eyeing to strengthen their relationship with customers. this!

3. Aside from fulfilling state requirements, there are independent business rationales for utilities to support efficiency programs that result in energy savings as this lightens the demand on the electric grid and may enable them to delay building new plants or other costly infrastructure. Similarly, many utilities are exploring new revenue models with the recognition that alternative energy sources (e.g., renewables), non-utility service providers, and smart/connected products are disrupting the relationship between utilities and their customers.

4. A regulated offering refers to a program with energy savings goals that is regulated by the state (through a Public Utilities Commission) and where energy savings are evaluated by a third-party evaluation team. In the early phases of this research, the ILLUME team also worked with another external partner, the third-party evaluator, who had been conducting research on customer satisfaction with the app, usage behavior, and subsequent energy savings.

5. As an energy-efficiency program, the app had been evaluated by a third-party company. Their analysis used the Mosaic segments to understand patterns in usage and associated savings. The segments with the greatest savings were identified as “target” segments for the research that the ILLUME team conducted.

6. In the emerging smart home market, crowded with large and small, and well-known and lesser-known providers, a fully representative experiment would have 20+ brands or providers. To derive meaningful results from the experiment, the team chose to limit the number of brands to a representative set that included one well-known player from different entry points of the market. This allowed the experiment to test the product and pricing features, at varying levels, to identify the ideal service offering and pricing. One limitation this presents in analysis is potentially inflating each brand’s market share, since not all brands were represented. However, the team felt this trade-off was warranted to get a better read on general consumer preferences and price sensitivity.

7. The team determined segments by an expectation-maximization (EM) algorithm that calculates the probability of membership in each segment. The methodology was developed by Jay Magidson and Jeroen Vermunt (2004). A detailed description of the approach, as applied to a similar LED lighting study, can be found in Opinion Dynamics (2012). The analyst establishes the number of segments to mode and starts with random guesses for the segment (latent class) membership probabilities for each respondent. For each latent class, the analyst develops a conditional logit model and obtains a maximum-likelihood estimate of the segment-level parameters. As the preference data from the experiment (and covariates) is added to the logit models, the software computes new segment-membership probabilities, and customers are re-arranged into more similar segments based on these probabilities. The modeling follows an iterative process of re-estimating the logit models until the improvement in likelihood is marginal (maximum likelihood).

8. In previous research efforts, The ILLUME team has conducted segmentation research where the data-driven segments are derived and defined prior to qualitative research, then segment members are identified in the customer database, and recruited for interviews. This approach can be effective in filling in specific insights and nuance that the team needs for the project. However, waiting until after the quantitative research to conduct qualitative research has trade-offs, in particular, losing the opportunity to use qualitative research to inform the survey development.


Anderson, Faulkner, Kleinman, Sherman.
2017     Creating a Creators’ Market: How Ethnography Gave Intel a New Perspective on Content Creators. 2017 Ethnography in Praxis Conference Proceedings. 425-443

Boomer, George.
2014     “Building Choice Models that Forecast Well.” StatWizards LLC.

Cuciurean-Zapan, Marta
2014     Consulting Against Culture: A Politicized Approach to Segmentation. 2014 Ethnography in Praxis Conference Proceedings. P. 133-146

Flynn, Lovejoy, Seigel, and Dray
2009     Name that Segment: Questioning the Unquestioned Authority of Numbers. 2009 Ethnography in Praxis Conference Proceedings. P. 81-91.

Hurtubia, Ricardo, Guevara, Angelo, and Donoso, Pedro.
2015     “Using Images to Measure Qualitative Attributes of Public Spaces through SP Surveys.” Transportation Research Procedia 11 (December 29, 2015): 460-74. doi:

Iyengar, Sheena & Lepper, Mark.
2001.     When Choice is Demotivating: Can One Desire Too Much of a Good Thing?. Journal of personality and social psychology. 79. 995-1006. 10.1037/0022-3514.79.6.995.

Magidson, Jay & Jeroen Vermunt.
2004     “Latent Class Models.” Statistical Innovations LLC.

Nafus, Dawn (ed)
2016     Quantified: Biosensing Technologies in Everyday Life. Cambridge, MA and London, England: The MIT Press.

Opinion Dynamics and StatWizards LLC
2012     The Southern California Edison (SCE) Advanced Light Emitting Diode (LED) Ambient Lighting Program Customer Preference and Market Pricing Trial. Prepared for Southern California Edison, December 2012.

Wang, Tricia
2016     “Why Big Data Needs Thick Data.” Medium website. January 20. Accessed [October 01, 2018].