How to connect experience measurement to business performance
By: Nathan Shetterley
Tags: Design Process, Data, Metrics, Design, Product Management

At Fjord & Accenture, we design services that extend far beyond the screen and the app. We design end-to-end processes, workflows, and experiences. We don’t want our services to “just work” — we want them to delight, seamlessly.

Why? Because we believe that the entire user experience is important to the success of the product and the company. But belief is not good enough. We need proof, for ourselves and for our clients.

That’s where data comes in.

Typically, UX KPIs (User Experience Key Performance Indicators) focus on explicit and economic measures of success, such as NPS (net promoter score) or conversion rate. However, these are quite limited in scope. To capture a much fuller picture of how users feel and experience a service or product, we must include metrics that are also implicit and experiential — known as Key Experience Indicators (KEIs).

This leads us to think in terms of two axes: (1) economic to experiential and (2) explicit to implicit. This simple change in perspective greatly expands the notion of success, especially for service design.

As an example, let’s say we’re a hotel interested in creating more loyal guests to drive better profitability. A KPI might be the total lifetime value of any given guest, a measure that is both explicit and economic. Looking at occupancy rates implies profitability but depending on rates may or may not actually drive profit. A KEI that is implicit would be how often a customer uses free services at the hotel like the gym or business centers. Or a KEI that is explicit could be a spot survey in the hotel app that asks guests how they like their experience. While more implicit and experience-centric, these measures arguably provide more insight into the user experience than mere total lifetime value.

Of course, coming up with KEIs is not always easy — and for this, we draw on work by Google, which has spearheaded research on KEIs. In particular, an article by Rodden recommends that teams go through the process of first defining Goals, then Signals, and finally KEIs/Metrics, leveraging the HEART (Happiness, Engagement, Adoption, Retention, and Task Success) framework. However, Google tends to be product centered rather than experience centered. Tomer Sharon further expounds on this in a series of Medium articles about KEIs, with examples and use cases. Easier said (or read) than done, we have built on this preexisting work to create a customized method to work through with our clients. Accenture’s The Love Index was the first tool in the market to create an analytic framework for measuring experience at key moments that matter and connecting the KEIs to KPIs like revenue or size of basket through statistical modeling.

If the user experience is truly important, then KPIs and KEIs must work together to expand the notion of success from solely explicit and economic terms to implicit and experiential ones as well. This is how data and design will drive the success of a feature, product, service, and ultimately, company.

Thank you to Krista Schnell for researching and co-authoring this with me, and to Nandini Nayak, and Matt Hochman for their contributions, debates, and ideas.