CXL Institute: Growth Marketing Mini Degree Review #8️⃣
By: Evelina Califano
Tags: Ab Testing, Research, Growth Hacking

In this week’s review of the CXL Growth Marketing mini degree, we will discuss which KPIs to look out for when running experiments as well as how to properly run research before running any A/B test, in order to get more quality, higher winning % of A/B experiment.

Let’s go for steps. Let us dive first, into what are those KPIs to choose according to your company’s objectives, and then following up on what is the 6v model and why it is widely used before running tests.

Is Key Performance Indicators, also known as KPI, the right word?

When we talk about KPIs in the A/B testing world we talk about goal metrics. When running an experiment you will need to choose a specific KPI, and as well you will create something that is called an overall evaluation criterion, OEC, especially if your company is medium or large-sized.

Clicks? Transactions? Potential Lifetime value?

According to Sitespec.com, the best strategy and most universal rule of thumb for determining good KPIs are to think back and look at the whole journey a customer would take with your feature. Credits to Stephen Philips.

A business probably has more than one key performance indicator, although your A/B test also does have a key performance indicator.

The term we use in A/B testing to refer to these KPIs is the goal metric. You’re trying to change behavior to create more transactions, more leads, more clicks, or what to offer.

Practically, that’s a goal metric you’re optimizing for.



NB. Clicks are the least important KPI/ goal metric for Tom Wesseling.

To have people click on a button is not that hard. Make it bigger, make it more physical. Just tell them there’s something for free if they click on a button, and then more people will click on the button and you’ll have a significant uplift. But still, uplifting clicks doesn’t mean you have a significant uplift in behavior or even transactions. Not the best metrics if you are optimizing for money.



So goal metric is the one we’re looking for, below a pyramid of the most important goal metrics/KPI’s according to our instructor:

The KPIs pyramid of goal metrics in order from the least important to the most important, bottom to top. Credits to Tom Wesseling.

Above there is the pyramid of possible goal metrics that you will optimize for at least important clicks because it is pretty easy to shift-clicking behaviors on the website. For instance, if you want to increase the clicks on a particular button you could change the design of the font, make it bigger or better located within your website layout. Another common change is to offer something for free to stimulate clicking on that button.

I.E. you can still use clicks to understand and study your audience’s, although according to Wesseling, that is not the best goal metric you want if you want to optimize to increase revenues.

As per behavior set as goal metric, this is particularly interesting if you have a too low number to run A/B experiments on. Nonetheless, behavior can still be pretty valuable as a research methodology in A/B testing.

Regarding transactions, these should be included as goal metric if your goal is to optimise and grow your business. It could be indicted as leads within B2B industry, as regarding B2C is called transactions. I.E. if you want to increase transactions you can lower the price. (You should have at least 1000 transactions per month)

Revenue per User: this is a higher metric than transactions but harder to implement an A/B experiment on.

Potential Lifetime Value: this is what Wesseling calls a golden metric. you want to make changes on your website, run with A/B testing if a variation is generating more lifetime value than the control and more potential lifetime value.

You are looking out for indicators that provide you with information about lifetime value.

In general whenever you use clicks, behavior these are used for smaller companies that do not have enough visitors to designate experiments, whilst transactions, revenue per user, and potential lifetime value metrics are the most effective KPIs to grow and scale your business.

🛎️ In the beginning, you should focus only on 0/1 outcome, binary outcome, and then once you grow you can go ahead and use online available calculators that are able to compare metrics on a different level.

Goal Metric: Is The Goal Metric The Same For Every Team?

Of course, depending on your organization you will have different types of goal metrics to analyze. Although Wesseling suggests that to avoid teams that nullifying each other's efforts’, it is important to have the same goal metric for everyone in the company. And this is what we call OEC, namely the overall evaluation criterion.

If you do not yet have the “numbers” in terms of users, transactions and such, you will be in your team struggling on how to optimize those metrics.

When your company is growing, you will need to figure out the OEC that matters to every employee.

An OEC should be: