Product Metrics That Matter
By: Cláudia Delgado
Tags: Metrics, Data Analysis, Product Management, Analytics, Product Design

The power of metrics entered Product Management to change it forever.

Before product metrics, we were shooting blindfolded. We just understood the result of our shots at the end of the game or even the championship. Feedback and learning took too long.

We were relying on business metrics. Consider revenue, for example. That’s a lagging success indicator. It might take months for a decrease in user engagement to be reflected in a decrease in revenue. However, we could only be reactive about the lost revenue — that was the only visible indicator.

Since then, connected devices have enabled an unprecedented volume of data, and there were significant improvements in analytics tools. As a result, product metrics quickly got established. Now, we can understand user behavior at a grand scale and be proactive about a decline in engagement months sooner. We know the results of our shots right away, and we can adapt immediately.

Product metrics allow measuring product progress and creating alignment in an outcome-oriented way. They developed in us an appetite for experimenting and being data-informed/inspired. If we don’t use product metrics, we’re at a disadvantage. We’ll understand less about users, be prone to opinions, take unnecessary risks, work harder instead of smarter, and have less understanding of our work’s impact.

There’s no going back.

What to Measure

As now it’s so easy to know what our users are doing by tracking events (actions taken in our product) and properties (details attached to either users or events), our first intuition will be to measure all the things. So we’ll set dashboards all over the place with flashing numbers. I’ve done it, and I felt all important by being capable of dealing with such data density.

However, our job isn’t to measure things. Our job is to change the product for the better — to create value for the customers in a viable way for the business. Any product metrics dashboard is just an internal tool to help us do that. We have to make it as easy to navigate and valuable as possible, for us and for anyone we’re collaborating with. If we report on everything, everything will become noise.

That’s why we need to define key metrics — KPIs (key performance indicators) or KEIs (key experience indicators), as I’ve seen labeled as well. They’re a smaller data set that is good at helping us make decisions and take action, not just vanity numbers.

There are many product frameworks available to help us think about the right key things to track. I’ve summed up my favorites:

☠️ Pirate Metrics or, as a pirate would say, AARRR. This framework suggests that we go through users’ life-cycle through a product and track the success of the different phases.

❤️ HEART. This framework was created to address also how users interact with the product and how satisfied are they.

⭐️ North Star. This framework is a bit more radical and claims that we should distill all the product noise to the metric that matters most: the leading indicator that best captures the relationship between the core value delivered to users and the long-term business results. Still, this framework also includes 3 to 5 complementary inputs that most directly affect the North Star metric.

The key things to track will always depend on the product. Nonetheless, we may (and should) use insights from these three frameworks or any other we find appropriate.

A Closer Look

To have an example to look into, I’ve put together some standard metrics for a generic product. I went through the areas mentioned by ☠️ Pirate Metrics and ❤️ HEART and placed its key things to track in a tree-like architecture from ⭐️ North Star. Some metrics will be more visible than others, and that’s good. We’ll keep our analytics dashboard with the most impactful things first, yet we’ll still be able to deep-dive when needed.