Testing 123… Testing 123: The Elements of User Testing
By: Jonathan Poor
Tags: User Research, Agile Methodology, Agile, User Experience, Product Management

Hello again, in my first blog post I provided you the ingredients for creating a successful product roadmap (check it out if you haven’t done so already!). One of the elements I highlighted was user testing. Since this is an essential part of Product Management, I thought it would be beneficial to write a more in-depth article on the subject. By the end of this post, you should feel confident enough to start doing your own user tests.

User testing is key to agile product development — once you put in place a framework, you will be able to conduct 2–3 user tests per week. This may sound like a lot, but the return on your time invested is worth it. The insights you will gain from directly talking to your users are worth their weight in gold so don’t underestimate these simple techniques.

Test from within your target audience

I can’t stress enough how important it is to only test users from your target audience. If you interview people who are not from your target audience, the data will be irrelevant. This is a classic case of “garbage in garbage out.” Most companies should have access to their user base, which allows them to directly contact users with a simple form for them to express interest. Another way to recruit users is by posting anonymously on Craigslist. Be sure to include a questionnaire for users to fill out so you can verify they are from your target audience. One thing to highlight — you will need to provide an incentive for users to participate. No one will volunteer out of their own goodwill. An Amazon gift card or something similar is perfect. One final recommendation is to use calendly (or any other scheduling tool) to coordinate the interview time slots. This will prevent multiple back and forths between you and the users and will help streamline the process.

Create your user testing documents

Before you bring people in for testing, you will need to create your interviewing documentation. This will allow for consistency across tests and help you quickly identify trends. You should start each test with a simple introduction. The introduction should explain the purpose of the test and explain to users that if they can’t accomplish a task it isn’t their fault. Tell users it is the product’s fault, and this is the exact type of feedback you are looking for. Let them know that there are no wrong answers and not to feel “stupid” if they can’t accomplish something. From my interviewing experience, a key question to ask is: “What is the purpose of this product?”. This should be the first question you ask a user when they arrive at your landing page since it provides valuable insights. By asking this question, you will know if users can clearly understand the problem you are trying to solve. From here forward, you can ask an assortment of questions to understand the feasibility and desirability of your product. Here is a resource for you to use when thinking of questions. A good rule of thumb is to ask open-ended non-leading questions, so you don’t create bias in your users’ responses.

Testing day is here

Woo! It’s time to conduct the actual user tests. First, you need to make sure users feel comfortable when they arrive, so be sure to greet them with a smile on your face. Also, offer them something to drink (water, coffee, tea). Remember that you are the host, and it is your job to make the interviewer feel comfortable, so they provide you with the best data. Once you start the test, read the introduction paragraph, and emphasize the fact that there are no wrong answers and that not accomplishing a task is the exact feedback you need. It is recommended to have another team member with you while interviewing to take notes. That way you can facilitate the interview and keep the user relaxed. As a facilitator, it’s also important to not help users if they encounter difficulties. If they ask you a question such as “Where will this button take me?” you should reply with a non-leading response: “Where do you think this button will take you?.” This is called parroting the user. In general, you should be as passive as possible while keeping the user relaxed.

Look at your user’s body language and listen to verbal cues

How do you know if users are having a positive experience with your product? Their body language is a phenomenal telltale sign! Are they laughing and smiling? Are they groaning and loudly sighing? Or are they things like “Honestly? Again?” and “I just did that”. Negative body language and verbal cues indicate the user is having a negative experience with your product. Identify this quickly and react.

Following up after the test

After concluding the test be sure to follow up with your users. Thank them for their time and let them know they have made a real impact on your product. If you haven’t given them their gift cards, now is a great time to do so. Lastly, you can ask them if they are willing to help again in the future.

A note on virtual user testing. With all the technology we have today, it is possible to conduct virtual user tests. Just be sure to follow your script, record the sessions (with the user’s consent), pay attention to body language (yes — even virtually!), and have a note taker with you while conducting the test. If you follow these steps you can collect solid data.

That’s it from me for now, but if you would like to connect to talk about user testing or product strategy shoot me a message on LinkedIn or email me at jonathan.e.poor@gmail.com. Best of luck on your Product Management journey!