Avoiding the Pitfalls of KPI design.
By: Guillermo Esquivel
Tags: Data Analytics, Statistics, Kpi, Metrics, Business Intelligence

As simple as possible, as complex as necessary.

Simple metrics tend to be better KPIs, a rule of thumb is that any business metric should be able to be recreated with a cheap calculator. Not all your stakeholders will understand your methodology in-depth, and if your metrics will be the common language they better be easy to pronounce.

Start simple, with total volumes, averages, percentage of a total, medians or percentiles if you’re feeling brave, but if your metric looks more like a model, go back to the basics.

If your metrics need a degree of normalization, you can transform them to percentages, but avoid additional coefficients, or weighting methods.

Simple yet effective metric:

New clients — Total number of clients without purchase history that completed an order in the last 30 days.

Model-like metric:

% Absorbed clients — Total number of clients that completed a purchase from the competitor in the last 60 days and completed an internal order in the last 15 days over total number of new clients.

Model-like metrics can have a lot of thought behind, and they can be very insightful. But if they are not easy to explain or recreate, they will be rarely trusted and hence, used. Metrics and KPIs are here to help us inform our decisions, and if they add more doubt than clarity the purpose is defeated.

Find the dimensionality.

It’s tempting to try to create a KPIs that solves for everything in a single view — but you will end up with a convoluted metric that is hard to explain and read.

It’s better to have a group of metrics that tell the full story, in other words, KPIs that are mutually exclusive and collectively exhaustive*. Each of them should cover different angles of the same dimension, and all of them read together should cover the entire scope to observe; the project, the operation, or the company.

So find your dimension to observe and adhere to it; financial health, quality, growth, efficiency, landscape, people, etc. There are a myriad of metrics for each of these dimensions, for example, for financial health you could be looking at gross bookings, revenue, Net income, OpEx or CapEx just to mention a few. Picking the right ones is the art.

*- MECE, (I got you my McKinsey people)

Tie back to performance.

You will want to understand what you aim to change with your project or initiative to measure the change. Try to answer the question;

If I change X, which Ys I think will change?

Speculative? Absolutely — but that’s how hypotheses work, we want to prove such change.

Note from a concerned analyst: Do not design a project around a metric — The operation should drive the metrics, not the other way around. There’s a subtle but crucial difference between:

“I will do this to improve our quality metric”

and

“I will do this to improve our quality, and it will reflect on our quality metric.”

Note aside, Imagine for a second you are the growth manager of the Bay Area based soon-to-IPO, Sourdough bread delivery company, Yeastly.

Your dimension to work with is clear — Growth, and you have a set of metrics to tell your story.

Now clearly lay out the purpose of your project or initiative to quantify.

This is, identify the end point of what you’re trying to achieve.

Let’s do a couple of scenarios, where we want to improve the different sub-dimensions of growth, depending on our goal, we will have to choose our KPIs.

Table 1. Our designers told us bright red is “yeasty”, we don’t know what they mean with that.

Understand the cause-effect relationship. The simple exercise of questioning yourself:

“I do X to improve the Z dimension, which W metrics are likely to react?”

Those are your KPIs — The metrics you hypothesize will move after launching your initiative.

Be ready to double-click.

A good KPI will always enable deep dives and insights generation. Business lingo aside, this means that you should be able to make different cuts of your KPI that will enrich the conversation and support the larger trends. If you see that Yeastly’s Oder Volume is growing you should be able to easily break down the KPI to the market level to understand where the growth is coming from.

When a metric is too complex, these cross-sections are harder to do and slow down our decision making. If your KPIs are good, explaining trends will become an easy task.

Wrapping up.

I know this only covers the very surface of the art of choosing and designing metrics — I promise I’ll write more in-depth about this. But for now, just remember that; simplicity is better, follow a business objective, base the selection on the cause-effect, and be ready to double-click.