How Not To Gather Product Feedback
By: George Fisher
Tags: Nps, Product, Apps, Product Management, Feedback

How Not To Gather Product Feedback

Some companies just want to feel liked. Photo by Adam Jang on Unsplash

Why do we gather product feedback? Is it to give us something to do? Is it to make us feel liked? Is it to impress our managers? It seems like some teams go out of their way for these things, but aren’t interested in gathering any proper user insight.

The Culprit

I saw this form the other day. Is this really feedback?

One of my most commonly used apps presented me with this feedback form the other day (I didn’t catch a screenshot, hence the mockup). It’s a simple modal requesting feedback. You see them all the time.

But this one is slightly different.

This isn’t a link to the App Store where I can tell the world about how happy I am with the app. It’s just for internal feedback, with two options: I’m either happy with the app or I’m not.

Simple enough. The team is probably collecting data on customer satisfaction to measure the impact of changes to the app, that have either already happened or will happen.

But look closely at the options, the colours and the layout. What’s really going on here?

Product Bias

I’m a product manager (PM). I interview customers everyday; I spend my time digging into what users are telling me to get to the root of what they really mean.

To get to that root, PMs are trained to remove bias by asking open questions and avoiding leading ones. How do you feel about this workflow? What option is your preference and why? That sort of thing.

Using questionnaires or surveys is a great way to gather quantitative data, which can be used to help make important decisions (it must be important if the time is being taken to conduct a survey).

If a question is leading users in a particular direction, or some other tactics are being used to bias an answer, then what’s the point? You’re going to base an important product decision on some ropey, biased evidence. You should’ve just made a decision before investing the time in the survey.

How and when a user interacts with your survey, and their mindset is critical in removing bias.

The User’s Mindset

Now back to the mockup. The user is in the middle of something when they see that page; they have a train of thought that they are trying to maintain. Anyone who uses the internet in 2020 knows that a modal or popup that they didn’t expect is almost certainly irrelevant. Users will instinctively be looking for the fastest way to close it so they can get back to what they were doing.

The first instinct is to look to the top right of the modal for a cross icon. No luck. Is there a cancel button at the bottom? No. Can the user tap on the background next to the modal to close it? Again, no.

The user has exhausted their escape options and now needs to consider the buttons presented to them. In this example, the bright blue one is the obvious choice. But it says “Yes”, which could risk taking the user to a different page. And the other option is “No, give feedback”, which sounds a lot like hard work.

The user has now exhausted every route available to them except the most laborious one: actually reading the question and answering it.

“Are you enjoying this app?”

Phew. They can safely click “Yes” and get back to what they were doing.

At no point in the user’s thought process did they think about the answer to that question. Their selection was simply the easiest route out.

Even a user who reads the question and absorbs it will click on “Yes”. It’s bright blue, it’s easier to read and the question is leading the user to click on it.

So what is the team going to learn from all of these “Yes” responses?

The Results

“99% of users are enjoying the app!”

Which we know means nothing, and hopefully the team who conducted the survey does too. Or if they’re using this enjoyment metric as a baseline to compare against in the future, the results are going to be random and unreliable.

A Better Approach to Feedback

An alternative, unbiased approach to gathering user feedback.

If the team is really interested in quantitative feedback then the bias needs to be removed from the survey. Or maybe even find another way to gather this data without interrupting users.

But if you need to run a survey then net promotor score (NPS) is a robust and easy approach.

Yes, users are going to be looking for the fastest way out but the options are being presented in an unbiased layout, without a leading question. Users instinctively know that clicking on “1” is going to get them back to what they were doing as quickly as clicking “5”, so they’re more likely to consider the options presented to them.

If you’ve spent any time reading about product online then you’ll have encountered stories of NPS’s weaknesses. But it is a generally an easily repeatable survey that can be used through the life-cycle of a product to measure changes, build a better product and ultimately improve the bottom line.