How to focus on the right metrics and avoid vanity metrics as a PM?
By: Tianran Li
Tags: Value Time Matrix, Product Owner, Metrics, Product Manager, Vanity Metrics

How to focus on the right metrics and avoid vanity metrics as a PM?

One of the most challenging tasks for a PM is finding the metrics that really matter to your product and avoiding vanity metrics. In this article, I will introduce a framework for PMs to understand better what they are building and set up more rational metrics for their products or functionalities. Tianran LI Follow Aug 6 · 6 min read

Photo by Luke Chesser on Unsplash

1 Do You Really Know What You Are Building?

Every PM knows what they are building. Every PM knows what their products are for. Every PM knows what problems they are trying to solve. (Hopefully:))

So, why do I ask this question?

Because PMs normally focus too much on their own products, they forget to stand high to have a bird-view of general products. Have you ever asked yourself a question as a PM that whether there will be a framework that might work for most of the products?

Here is the framework and philosophy; I call it the Value-Time-Matrix framework.

Firstly, where is the user value from? Is the value directly from your product mostly, or the value is from your product connecting others? For example, Google, as a search tool, the value to users is mostly from Google itself by its powerful searching engine. However, for most B2B, C2C, B2C platforms, the value is not from the product itself, but from connecting: connecting sellers and buyers(Amazon), connecting restaurants, delivery driver and users (Wolt), connecting merchants and employee users(Epassi).

Secondly, how do you expect your users to spend their time on your product? Do you expect to help your users save their time? Or, do you expect them to spend more time on your product? For example, TikTok definitely hopes their users can stay 24/7 in their App. However, if you are Amazon, you wouldn’t hope your users spend too much time during the check-out process because the longer they stay, the higher chance they will leave without finishing the transaction. But, of course, you do hope they browse your websites as long as possible.

Therefore, we can draw a matrix like this↓

1 Value-Time-Matrix

Each quadrant presents a typical product. For example, if your product can deliver value directly from itself, you hope to save your users time using your product. Then it can be called a tool. Likewise, we can summarize four typical product categories, and these four types can cover most of the products or functionalities.

Content. Value from the product; User’d better spend time.

Tool. Value from the product; User’d better save time.

Platform. Value from connecting; User’d better save time.

Community. Value from connecting; User’d better spend time.

Here are some examples of each category.

2 Examples of each product category

2 What Metrics You Should Focus On

By using the Value-Time-Matrix, we know there are four typical product categories.

So, how do we leverage them?

Here is the approach. If you only choose one metric for each of these categories, what you will choose? My answer is

Quantity and Quality for your Content.

Efficiency for your Tool.

Conversion Rate for your Platform.

Activeness for your Community.

3 Four typical products and their key metrics

You can have different answers on the most important metrics. However, in this way, it leads you to a direction that you know is correct. Furthermore, you can extend those metrics into something you can really focus on, rather than some vanity metrics that cannot guide your next move. For example, if you are building a Tool. Then what you should focus on can be times of usage (how many users use the tool), frequency (how often do they use? once per year or once per day?), success rate (is it easy enough for them?), time spent (less time means more efficiency). All of these derived metrics come from the ultimate metric you should measure: Efficiency. All you need to do to avoid vanity metrics is to plan, test and iterate how you measure the efficiency. The same principle goes to all the rest three categories. Let’s take another example of Medium. Some meaningful metrics would be the number of articles and how good these articles are by monitoring the views, time spent reading, and even the number of claps. No matter what they select, all of these metrics should eventually go to the same ultimate metric, which is the quantity and quality of content on their platform.

4 Product category and metrics

3 Three Common Mistakes

Firstly, you should either use this framework to see a product in general or use it for specific functionality because it can have multiple categories. No doubt, every company wants their users to spend as much time as possible on their Apps. However, when you look into specific functionality or process, it should have a clear category where it belongs to. For example, the payment process, in general, should be regarded as a tool, and you should expect your users spend as little time as possible to make sure the whole process is smooth enough. However, Aliexpress and Amazon would certainly hope their users can browse as long as possible on their websites, whether they want to buy something or not until they go to the checkout stage.

Secondly, the category of a product or functionality may change in different product lifecycles. For example, in an early stage, Meituan, originally a food delivery company in China, they are a tool. However, when they grow and expand, their product category may gradually transit to Content or even community, not entirely, but at least partially. The main reason is that they need to really let their users browse their App so that there is a higher chance for them to buy something other than only ordering food delivery. Eventually, they are in a fierce fight of users’ time against all the other Chinese Internet Giants.

Thirdly, setting up metrics that can tell you what to do next. GMV sometimes is a convenient and useful metric for Platforms. However, simply looking into GMV won’t tell you anything. You can blame Covid-19 for a less GMV, while you can also explain a higher GMV as a result of the recovery of Covid. However, it can only show that you cannot control or affect your GMV because you cannot control the Covid and the pandemic. A more actionable method would be tearing down GMV by the MECE principle and analysing what metrics influence mostly, or what factors you can influence mostly which can eventually affect GMV.