A Content-First Approach to UX Research
By: Casey Capachi
Tags: Content Strategy, UX Research, Ux Writing, Research, Instagram

A guide to conducting content strategy research for your products.

Every word counts. But how do you choose the right ones for your product?

Content strategy research might be the answer.

Content strategy at Facebook consists of equal parts UX writing, product strategy and design thinking. When I say I write “content” as a content strategist at Instagram, I mean coming up with the words that you see in the app’s user interface — including elements like buttons, titles, error messages and feature names.

Over the years, I’ve used a variety of methods to evaluate word choices in the consumer products I’ve worked on like Facebook Live, Instagram Direct and Messenger Rooms. Before I finalize any content, I always base my recommendations on user experience research insights.

Finding out what people want from a product — before it’s built — has helped me write the most relevant content upfront and saved me from delivering content that doesn’t resonate with people.

Whether you’re working alone or on a team, my hope is that content strategy research can help you write the best content for your product, and go on to inform your product strategy.

What is Content Strategy Research?

Content strategy research at Facebook evaluates the word choices in product interfaces and their ability to help people feel empowered to make decisions.

When people think of UX research and testing designs, they often focus on the visuals. However, the words that guide people through an experience are just as important and deserve to be studied in-depth.

Content strategy research can give you better insight into your product’s value proposition, help you find the right information hierarchy, and determine what the voice and tone should be. Content-led research methods can be used to not only validate content choices at the end of the design process, but also at the beginning to help you decide what to build in the first place.

Content research can even pay off further down the road when it’s time to market your product. You’ll be able to draw from the value propositions you identified in early content strategy research sessions later on when you promote your product publicly.

Content Strategy Research Methods

Whether you’re working alone or with a UX research team (like I’ve been fortunate enough to), you can use the following methods to conduct a study focused on content strategy. All you need is content.

All of these methods can be done in-person or remotely over a video conference call by sharing your screen and asking participants to walk you through how they’d fill in the research materials.

Five-Second Test

After showing participants a short piece of content for five seconds, ask them to describe what they remember. Use their descriptions to help inform what you should make more memorable. In a world where people are used to rushing off to the next thing, five seconds should be long enough for a design or short piece of content to get its point across.

When to use it: When you want to know if people’s first impression(s) of your content reflects what you’re trying to convey.

What you’ll need: For remote studies, a design or short piece of content that fits on one slide or screen. For in-person studies, one piece of paper with your content on it.

How to run a five-second test:

Tell participants that they’ll be given five seconds to look at your content before you ask them questions about what they just saw.

💡 Tip: Depending on what you’d like to learn from participants, you can either tell them what to look out for or leave it open if you want to see what stands out to them naturally. After the five seconds are up, ask them for their first impression(s) of what they just saw:

What were the main things that jumped out at you?

Who do you think this was written for?

How would you describe this to a friend?

Talk Bubbles

Ask participants to have an imaginary conversation with your product. This method helps you write with the user’s voice in mind and uncover how your product can help fulfill people’s needs. It can also reduce any pressure participants might feel when asking questions themselves.

When to use it: To see how people talk about your product in their own voice.

What you’ll need: For remote studies, an image of speech bubbles shown on a screen. For in-person studies, a piece of paper with two to ten blank speech bubbles, indicating a two-way conversation that will take place between a person and the product.

How to run a talk bubbles exercise:

Tell participants to imagine a conversation they would have with your product. Then ask them to either have the conversation out loud (for remote studies) or write it down inside the talk bubbles on a piece of paper (for in-person studies). Your instructions might sound like this: “If you were to have a conversation with a phone, think of what you would ask it, and how you would complete the task at hand. You might start by saying to the phone: ‘Hi, I need to talk to someone.’ And the phone might respond back: ‘Great, I have all of these options for how you can reach someone. Which one would you like to try?’ And then go from there until you’re able to complete the task.” At the end of the imaginary conversation, take note of what the participant asked for and how they asked for it. You’ll walk away with a list of their needs and you’ll have it in their tone of voice.

💡 Tip: This can also help you map out a user journey of what to design using your target audience’s own words.

Highlighter Test

Sometimes the easiest way to discover what stands out to someone when they’re reading is to ask them to highlight it. Having several highlighted mocks from participants can show you which words are resonating the most with people.

When to use it: When you want to know which pieces of content sound the most appealing to people. This method is most helpful when your content is at least one sentence long.

What you’ll need: For remote studies, a slide or screen showing your content on it, which you’ll highlight in front of participants based on how they tell you they would highlight it themselves. You can also share an editable document with them that they can highlight themselves. For in-person studies, give participants three different-colored highlighters and a piece of paper displaying your content.

How to run a highlighter test:

Ask participants to read all of the text first. It can sometimes be helpful to show a range of content options written with a different tone or value proposition, so you can get a better sense of what’s resonating the most. Using one of the highlighters, ask them to mark or tell you how they’d mark the parts that sound the most appealing to them. Using a different-colored highlighter, ask them to mark or tell you how they’d mark the parts that sound the least appealing to them or that they have concerns or questions about. Using another different-colored highlighter, ask them to mark or tell you how they’d circle the keywords that stood out to them the most.

Card-Sorting Exercise

When you need to know people’s preferences, a desirability ranking exercise can be the most efficient way to see if certain options are rising to the top or sinking to the bottom.

When to use it: When you have a range of content options and want to know which one (e.g. product name or description) is the clearest.

What you’ll need: For remote studies, a deck with slides for each content option, which you’ll rearrange according to how the participant ranks them. For in-person studies, index cards with different content options on each card that participants can rank and rearrange themselves.

How to run a card-sorting exercise:

Show all of the cards in a random order. Remember to change the order of cards for each participant and make sure the same card isn’t always presented first or last. Give participants a time limit (e.g. two minutes) to rank the cards based on favorability, with their favorite option appearing at the top and their least favorite option appearing at the bottom.

💡 Tip: Remember to take a photo or screenshot of every participant’s card-sort so you can easily compare them side-by-side at the end. Ask participants to explain their ranking to you.

Fill-in-the-Blank Test

A fill-in-the-blank test, also known as a Cloze test, involves removing words from sentences to test people’s comprehension. A fill-in-the-blank test can look a lot like a Mad Libs game where people are asked to fill in the blanks. It can also be a helpful exercise to see what names people associate with a feature or product.

When to use it: When you want to test people’s ability to easily comprehend your content.

What you’ll need: For remote studies, a slide that contains a piece of content from which you’ve removed certain words and replaced them with blank spaces. Participants can tell you what words they would fill in. You can also share an editable document so they can fill in the blanks themselves. For in-person studies, print out your content on a piece of paper for participants to write on.

💡 Tip: Which words you remove or how many are up to you, but every fifth word is a good place to start.

How to run a fill-in-the-blank test:

Instruct participants to read your content, which can be as short as a few sentences, with blank spaces for every fifth word. Ask participants to fill in one word in each blank. Be sure to inform them that they won’t be graded on their word choice or spelling. This test measures how clearly the content is structured, not their abilities. Keep time and record how long it takes for participants to fill in the blanks.

💡 Tip: You may want to set a time limit to test their comprehension speed. Review their answers and ask them to walk you through their word choices. Score the exercise after the participant leaves. Remember to note any synonyms used, which may indicate that another word might be more easily understood. A good indicator of whether people comprehend your content is when the majority of participants fill in more than 50% of their test with the correct words.

Semantic Differential Scale

Sometimes it’s useful to see how your writing falls on a scale. The semantic differential scale is a great tool to measure people’s opinions on a piece of content. Often used in surveys to rate an experience on a scale of one to five or one to seven, you can also apply this scale to your own writing to gauge sentiment. (Semantic differential scales are similar to, but different, from Likert scales. Likert scale questions often ask how much one agrees or disagrees with a question being posed and they include a neutral option in the middle of the scale. Semantic differential scales usually use numbers and have anywhere from five to seven possible ratings.)

When to use it: When you want to measure the attitude towards a piece of content.

What you’ll need: For remote studies, a slide or screen with questions and scales marked from 1 to 5 with an antonym on each end (e.g. Unhelpful ←→ Helpful). For in-person studies, print out the questions and scales on a piece of paper.

How to run a semantic differential scale test:

Give participants a screen or sheet of paper with at least one question and lines marked from one to five for scales. one and five should always have two words with opposite meanings (e.g. easy ←→ hard). Remember to rearrange the order of questions for every participant. Ask participants to rate their attitude towards the question by picking a numeral on the scale. An example of a question could be “How easy was it to tell what this product does by reading the description?”

💡 Tip: It’s important to follow up and ask why someone chose this rating. The “why” behind these responses will be more telling than the numeric scores alone.

3. Look at where their answers fall on the scale and see which areas you need to improve to move people’s attitudes in the right direction.

Content Scorecard Exercise

When you want to see how your content performs on a scale across several different criteria, it can be helpful for participants to rate content using a content scorecard. You’ll walk away with a better understanding of why people prefer one piece of content over another.

When to use it: When you want to evaluate different pieces of content across a wide spectrum of criteria.

What you’ll need: A content scorecard, which is made by taking a spreadsheet and labeling the rows with different content you want to test and labeling the columns with different categories of criteria (e.g. clarity, desirability). You should aim to present only a few categories that represent different criteria. For remote studies, share your screen and ask them to tell you what scores they’d give. You can also share an editable document with them so they can write in their scores themselves. For in-person studies, print out the scorecard for them to write on.

How to run a content scorecard test:

Ask participants to read all of the content options first before assigning any numbers. Remember to rearrange the options for each participant’s worksheet so participants don’t see the options in the same order every time. For each category, ask participants to rate each piece of content on a scale from zero to five. For example, if the category is “excitement level,” then zero would be “not at all excited” and five would be “extremely excited.” At the end, ask them to rank their favorite options overall and explain why.

Naming Exercise

It can often be challenging to come up with a product name that everyone around the world can easily understand. Here are some tried-and-tested techniques to help you get the most unbiased opinions possible.

When to use it: When you want to test a name for your product.

What you’ll need: A visual (e.g. mock, prototype) or written description of your product and a list of your proposed names. Make sure you’re changing the order of names for every participant to avoid order bias. For remote studies, share your screen and show the product visual that way. For in-person studies, you may want to print it out on a piece of paper in case participants want to write on it.

How to run a naming exercise:

Before participants see a visual or in-depth description of your product, briefly tell them about it and ask them what they would name it. After they learn about your product, ask them if it meets their initial expectation and if they would change their proposed name after knowing more now. Share your proposed names and see how participants feel about them. Remember to rearrange the order of names for every participant. At the end, ask participants how they would describe your product to a friend.

💡 Tip: Listen carefully to the words they use — a potential name could be somewhere in there.

How to Ask Questions Related to Content Strategy

If conducting a content strategy-specific research study isn’t an option, you can still add questions about content to any type of study you’re running.

Behind every effective study is a well-written moderator guide that includes questions for the person asking them. If you’re looking to get signal on content, it’s important to craft your questions in a way that allows the participant to respond using their own words.

These are some best practices to get participants to open up about their likes and dislikes when it comes to content.

Ask people to read and think out loud

One key reason for doing any study is to look beyond metrics and click-through rates and find out why people feel the way they do about something. Take advantage of this opportunity to get participants to verbalize their thought process to you during a research session.

Examples of questions to include in your research guide:

Can you walk me through how you decided to take the next step after reading this part?

Can you please read this sentence out loud and tell me what you think it means?

What would you change about this title to make it more clear?

Listen to the words people choose

The point of any content strategy-led study is to evaluate the word choices in the product interface as much as it is about understanding participants’ preferences. Writing for your audience is about meeting them where they are, whether it’s modulating the tone to match their emotional state or explaining something more clearly.

Examples of questions to include in your research guide:

How would you describe this feature to a friend if they hadn’t seen it before?

Describe a time when you might picture using this?

What does this remind you of, if anything?

Ask open-ended questions

Ask questions that require more than a “yes” or “no” answer to gain insights into people’s thinking. Open-ended questions usually begin with words like “who,” “what,” “where,” “when,” “why,” and “how.” These five W’s (and the one H) are commonly used by journalists and many others to gather information. Avoid leading questions, which start with words like “do,” “could,” or “was,”. When in doubt, add “And why?” to the end of any question.

Examples of open-ended questions:

What, if anything, is clear or unclear?

What’s your reaction to this product?

How do you feel about this name?

Examples of leading questions:

Was that sentence clear?

Would you use this product?

Do you like that name?

Leading with a Content-First Design Approach

If you lead with a content-first approach and take the time to understand how participants see your product, you’re more likely to build a product that people will want to use.

In a world that’s full of complexity, these content strategy research methods can help you find the most effective ways to cut through ambiguity and design a people-first experience.

Finding the right words that resonate most with people can help shape your product’s value proposition and, in turn, influence your design decisions. For example, choosing a name for your product based on how people talk about it in real life could very well inform what it will ultimately look like.

By using a content-first approach, you’ll see that content not only has the power to fill in designs, it also has the power to shape it.

Thank you to…