Why do you keep f***king up your user testing?
By: Ana Popa
Tags: User Experience Design, Ux Strategy, User Testing, UX, UX Research

Illustration — Alexandru Scuta

The first thing you have to do before any user test is to assume that you know nothing. Having that mindset will help you to not assume that you know all the answers. Instead it will encourage you to keep asking the users “Why?”

“You know nothing, Jon Snow!” ― Ygritte, Game of Thrones

Don’t search for words that approve your biases/thinking

It is easier to understand information that you already agree with. This is one of the biggest traps during user testing. It is incredibly easy to pay attention to data that confirm your ideas which leads to disregarding contradictory data. During a test you should consider all feedback the same, don’t sort it yet. Things can seem unimportant in the beginning but may be a part of a bigger pattern later on.

The main reason of testing is to find what to improve in your product. A user test that didn’t result in a list of improvements is useless and the test should be corrected.

Be aware, keep an objective thinking, be open to new! Iterate on the tests!

Listen and observe the user — body language is way more honest than words

Reading body language is an indispensable tool during the test. People tend to say different things than they act on. Often, they say what they think you want to hear, from the fear of not looking stupid.

Also, a natural experience is unnoticeable for the user when using a product. You need to observe:

where the user takes time before acting

how much confidence do they have when making a choice

what information are they skipping out on

when they get confused

how comfortable are they feeling about exploring the product

Don’t teach the user “How to…” let them explore

A product should speak by itself, and if it doesn’t there’s a bigger problem. Let the user explore the product without instructions, encourage them to ask questions, to be curious, and to own the journey. There you can learn what is the natural flow/way for the user to explore the product, and importantly, if you covered all user cases. This is an opportunity to find something that you haven’t thought of.

Don’t influence the user’s answer with the question

One of the most basic mistakes during user testing is to have poorly written questions. There are a lot of strategies on how to ask a question in order to receive an honest answer.

Do not include the answer you are looking for in the question. Avoid :

“Do you like …”

…” “How hard is to…”

is to…” “How intuitive is…”

Use instead :

“How do you feel about…”

about…” “Describe step by step…”

“Why did you choose to…”

Avoid asking YES or NO questions. These questions can only approve or disapprove what you think. It doesn’t give you any new or unexpected information.

Never ask for the solution, ask for the need they have, what are their pain points.

“If I had asked people what they wanted, they would have said faster horses.” ― Henry Ford

Always ask “Why?”

“Why?” is the key question during a test. The answers/result do not have any value if you don’t understand why the user chose a particular answer. It is not the answer that is important, but all the factors that influenced that answer. The more times you ask “why?”, the more chances you have to understand the real pain points. Also, comprehend how relevant are they for your product and how can you solve them.

Ask same question more than once, different words — same meaning

There are more reasons why you should ask the same question multiple times. This can ensure:

The user is being honest

The question was understood (if you get different answers, then you have the chance to find out why)

The user didn’t change their mind during the test

It is good to have duplicate questions at the start and end of questionnaire.

Negative feedback is also a result

Negative feedback is more important than positive feedback. It is a way to learn what’s not working well in your product. Assuming no product is perfect, there is always room for improvement! The design process includes lots of testing and iterations. Negative feedback takes out poor solutions and leaves space for the better ones. There are always many bad solutions before a great one appears. The faster the bad ones are cut, the closer you get to the great one.

Receiving negative feedback and understanding it, is the key to a better product.

Last question should be an open question

Always give users the opportunity to say things that you haven’t asked. By the end the of the test, people are generally comfortable enough with you and the product, and they are ready to suggest ideas, to change answers to previous questions, and to talk more.

Don’t give instant feedback to the user

Instant feedback is the easiest way to screw up user testing. By approving or disapproving an answer, you encourage or discourage the user. In this way you influence the answers you’re receiving. During user testing there are no right or wrong answers, there is only data and answers that could create more questions! Stay neutral, interact with the user, and let them know that they are doing a great job!