Is it really this easy to validate what the customers want?
Tags: Development, Leadership, MVP, Agile, Lean Startup

Is it really this easy to validate what the customers want? Tay Sep 3·4 min read

I’m currently reading The Lean Startup by Eric Ries. Eric defines a methodology to get a desired product to customers’ hands faster: build true minimum viable product (MVP), avoid vanity metrics — use real data (see below), understand when to pivot or persevere, and…. well, I haven’t finished reading the book yet!

As I write this, I wonder if it might be too soon for me to publish my early thoughts — what if I read something later on that changes my view and contradicts what I’ve already written? Or in contrast, the reason as to why I’m posting this today: what if I carry on reading and realise I am even more enthused about my thoughts?

The subject in the book which is fascinating me the most is the validated learning — most of us aspiring agile evangelists understand what MVP means, but do we really know how to apply it?

The meaning behind validated learning is that you focus on figuring the right thing to build, the thing that customers want and will pay for.

Are we building what the customers really want, or are we building what we think they want?

A lot of the direction in the book is to make sure you are speaking to customers, using actionable metrics and A/B testing.

Vanity Metrics vs Actionable Metrics

Vanity Metrics: Using metrics which look good, but don’t tell the true story.

Using stats like site hits to determine if the feature you are using is successful is wrong. It may look good that you have 250,000 hits in January, but what if February only gets 200,000 hits? Does this mean all the features you released in January were successful, but the releases in February weren’t? How can you possibly make decisions without taking educated guesses? You need metrics which tell a story and are actionable.

The graph below breaks the site visits for IMVU (an online avatar instant messaging platform) into cohorts.

Graphic from Eric Ries’ The Lean Startup — A graph which represents seven months of work.

You can see by August, more customers were having multiple conversations vs customers back in February. This graph isn’t looking at the sum of website users, it’s looking at what percentage of users are people who are actually using your platform — this is where you’re going to make better decisions. The more people using your product, the more you’ll grow.

Minimum Viable Product

Eric uses examples in his book to help explain how this works in the real-world — but, as Eric admits — there isn’t a one-stop-shop solution for anyone. The road to validation differs between all products. With complex product ideas (i.e. which might involve AI/ML), building the technology behind it might be months of work, so how do you build an MVP which validates your idea will be successful without spending months of upfront development?

You might record a video of yourself talking about it and showing mock-ups and ask users to pre-register, if many people pre-register you know you have a demographic and months of work would be worth the effort.

Or, you might start with an offering which is inefficient and un-scalable — having humans replicate the back-end as much as possible so customers think they’re talking to AI (Wizard of Oz testing)

Wow, so this is what building MVP is, it’s not always building the feature?

But what would be right for my business-2-business (b2b) environment?

We have an established customer-base, we’re constantly growing and gaining large customers and we’re pretty feature rich — but how could we validate that we were building the right features, the features that the majority of users want? Could we give users a voice… make them feel part of the build process, feel invested, feel like we actually listen to them? Could this… be that simple?

Users using our system are so important for retention — if users are using the system the company will be getting their money’s worth, they will renew their contract.

We’re pretty good at finding out what our customers want, making sure we build what is valuable — but often the conversations of what we should build is being held with the 2% of the people in the organisation who use the system (the people who procured our software), telling us what their employees want —but are we not making the same mistake of making educated guesses ourselves?

Sure — we do need to build the functionality the customers want to help manage their organisation, this is important for us to stay ahead of the competition, so we need listen to these people as they are as important as our users, but what if we gave the users a voice too?

I want to create a specific team, which soul purpose would be ask users to vote on what improvements and/or functionality mattered most for them — before the team built it. Then, use accountability metrics to validate the usage of delivered features and tune each feature until we see vast improvement.

If we ask users what they want before we build it, do we have a better chance of not pivoting and binning the feature?

The team would need to define metrics to help decide when the right time was to pivot, persevere or to mark it as a success and switch focus on the next feature which most of the users wanted to see or to see improvement on. The main goal of course: higher user-engagement.

Summary

Read “The Lean Startup”. The co-author of Mosaic (one of the first web browsers), Marc Andreessen, perfectly summed The Lean Startup up for me.

“Eric Ries has created a science where previously there was only art.”

I can’t wait to share my learnings — let’s hope the idea gets some legs.