Great User Research (for Non-Researchers) — Part 3
By: Steve Portigal
Tags: Planning Research, User Research, Acting On Research, Doing Research Well

Great User Research (for Non-Researchers) — Part 3

Summary

The demand for User Research exceeds the supply of researchers. When, ultimately, our goal is “Learning from customers’’ and our user research community is a helping profession, we should be finding ways to empower everyone to do well. These three connected essays aim to help overcome territorial and quality concerns within our community. In this final essay, I’ll provide some of the essentials about acting on user research. The previous articles are about planning research and doing research.

Working With Research Data

Some people seem to think that once you talk to people, you write up the key takeaways from each person and throw it into a deck and you’re done. But that is insufficient to get the full value out of the research. We talk about “insights”, but insights are not something that you simply gather and tabulate. They are the outcomes of a process! If you want to get to something beyond the obvious, beyond just exactly the things that people told you they wanted out of your product, you have to go through this process.

It’s a combination of analysis and synthesis.

Analysis is breaking down larger pieces of information into smaller ones, like pulling key anecdotes out of an interview.

Synthesis is combining smaller pieces into something new. Anecdotes become themes and frameworks.

It can be very time consuming but the more you put in, the more you get out. Some people plan for 2–3 hours of analysis and synthesis for every hour they spend doing research.

Jon Kolko has written a step-by-step recipe for analysis and synthesis (http://www.themoderniststudio.com/2019/01/07/the-mechanics-of-strategic-storytelling/). It’s very detailed about the mechanics and it is a great resource for the operations of the process.

However, it doesn’t tell us too much about the creative activity of trying to pull meaning from what people say and what they don’t say.

A short cut for doing the analysis is to look closely at the meaning of words. People tell us things in research and we might assume that they are using the word the same way we are, especially if it’s a technical term or a word that we use internally. ”Reporting” can be two things. A “solution” might not mean what you think it means. Look closely at what they said and what you assumed the phrase meant initially, but from the larger context of the conversation identifying how they understand that term. Sometimes when you feel surprised or confused by someone’s description of their behavior, that’s a cue to look more closely. These may be things you clarify in the field, like an example I was just reading where people explained the computers “talked to” the printers but in fact that was just local terminology for a complicated upload and transfer process. If you don’t clarify terminology in the field, you might consider this in your analysis process.

When you hear the phrase “power trip”, you might have a certain interpretation in mind, but when you see this sign (below), you are likely to have a different interpretation.

Another important way to make sense of the research data is to go back to where you started. If you recall, at the outset of Planning Research, understanding what stakeholders wanted is a necessary early step. What are their assumptions about what the problem, need and solution is? Look now at what the research is saying — does this correspond to what stakeholders believed and understood?

When Research Is Rejected

For researchers, a familiar response when sharing findings is a dismissive “oh we already knew that.” You may hear this pushback when you share what you discovered. But you have to ask your audience really, did you? Something being familiar, or feeling true, isn’t the same as knowing it. Recognition is not recall.

Beyond that, confirmatory findings are valuable! There is a plethora of reasons why research results get rejected, and one is the discomfort that comes from having to change your mind. At best, it’s inconvenient. A client asked us to look at the ways that people were using a certain popular personal technology product. Before we began the research the client team had already identified several categories of usage they planned to build for. However, the research revealed important and underserved behaviors that were not on their list. When we reported to the team that these were the activities that they should be supporting, they got very quiet, telling us that they had already assigned teams and considerable resources to developing solutions around the categories they started out with. It seemed like we were at an impasse. But this was because of the way we presented these research findings. We failed to tell them that, yes, we learned about their categories and there were other areas that were more fruitful. When we said it that way, they were able to let go of the plans they had already made and focus on where the research pointed them.

Organizations still need to be better in how they gain insights into what people want from products and services so they are better able to adapt and respond as demand, needs, and preferences change. As a user researcher, I encourage you to do research! Greater visibility and general acceptance of the importance of research is good for the whole research community, not to mention the people we are building with, and for.

If you found this article helpful, please see the linked pieces on planning research and doing research.