The adventures of Captain User Research in the post launch phase
By: Jerome Benaud
Tags: User Research, UX, Business

Pain points better behave : Captain User Research is back 🙀!

User Research consists in meeting final users of a website or an application to design the product that suits their needs. In the first episode, Captain User Research helped Businessman to design an app running observations, interviews, surveys and user testings. What happens now ? What do users and the app become ?

Several months have passed since Businessman’s app has been launched. The success was huge. Businessman was thrilled, and so were the users. Yet, several months after the launch, a slight decrease was observed in the use of the app.

Businessman goes straight to his very good friend, Captain User Research, and says :

-I need your help ! The success was incredible and immediate, but we now have less users, I don’t get it !

-Don’t worry, I got the solution, answers Captain User Research.

-You already used your superpower for the build phase, what else can you do ?

-My superpower is limitless and goes beyond the first steps of the lifecycle of an app. People, habits, markets, technologies . . . everything changes ! It is mandatory to get updated.

Users evolve, so do their needs and thoughts. So what you offer has to be adjusted, constantly. The only way to take into account these evolutions, is to do user research and test the design continuously to be and stay the best in the market. The International Design Foundation made this process very clear :

Users meet pain points, interaction moments that cause negative emotions. You have to identify and solve these if you want to lead your users to happiness.

Captain User Research already used surveys, observation, interview and user testing in the previous article. All these methodologies can be used during the post launch phase. The user research and test methodologies explained in this article are specially designed for this phase.

-The first step, says Captain User Research, it is to look at what we have today and the data we are able to collect.

Without further delay, they leave Captain User Research’s secret laboratory and fly to the breathtaking Businessman’s tower.

Using the data

Captain User Research accesses to the analytics page of the application. Google Analytics, Matomo, Tag commander, or GoatCounter (which is open source) . . . No matter the tool, analytics are gold.

This is a great advantage of working on an existing website or application : you have data on its use. Analytics inform you, among others, about the visitors, their origin, the pages they visit and the time spent on each page, when they stop using the website. These data are very precious and to use the best of it you have to work with an analytics expert.

What analytics give you is information about users’ behavior. What they don’t are the reasons for this behavior. 95% of the users “bounce” from the home page and don’t navigate on the website. It means that there is a problem, but it does not tell you which one.

-There is another type of information we can look at, says Captain User Research. Customers’ feedbacks : ratings given by customers on the app stores or directly on the website, or even transcripts gathered by customer service.

The principle is to collect all the verbal data coming from the customers and proceed to a lexical analysis. It is possible to use dedicated tools, or a thematic analysis table, exactly after an interview. It is a good start to gather qualitative data and identify needs, expectations and pain points.

Another interesting methodology consists in combining analytics data with a tool like Hotjar that allows to gather others users behaviors like clics, moves and scrolls heatmaps, but also to ask questions to users and make polls on specific pages.

Using different data types is an excellent starting point. The rub is that data is not always available. Then, to receive feedbacks, Captain User Research will set up UX diaries.

-A diary ? asks Businessman, skeptical.

The UX diary

The objective of this methodology is to collect the participants thoughts and feelings toward the product in their everyday life. It consists in giving them a diary and asking them to fill it on a given period, a month at least.

The diary is set up by the designer : the information to write, the rating scales, the filling frequency. It can be physical or online, but considering the huge amount of data to analyze, you better choose the online option.

The filling of the diary depends on the participants : no more battery (or pen), hangover, Netflix . . . there are so many good reasons not to fill it. To make sure the methodology is effective, it is mandatory to onboard the participants. It means having an active follow up by, for example, reminding them daily to fill it.

Let Captain User Research take care of the diaries, and we will come back when she is done.

A/B testings

-UX diaries delivered many useful information and revealed some pain points met by the users.

-Great ! What do we do now ? replied Businessman.

-I have several optimization leads for your application, says Captain User Research to an impatient Businessman. I will set up A/B testings, to compare KPIs of several versions of a page.

On the studied website or app, some of the users will see the A version of a page, and the others will see the B version. Here are some examples of the differences between these pages :

Text (product description, menu titles . . . )

Buttons and CTA (Call To Actions)

Colours

Typography

Zoning of the page (position of the elements of the page)

After a defined period, or when a specified amount of visitors is reached, the test is stopped. Users are not always aware that an A/B test is running, consent is made with cookies.

This methodology focuses on performance : we will look at precise KPI like the bounce rate or the conversion rate obtained by each version. These are quantitative data, easily comparable in order to keep the most effective version. This methodology is often used for landing pages or product pages because effectiveness is crucial.

During the build phase, Captain User Research led user testings. To recall, this methodology starts with an user interview, followed by a presentation of the product to gather his feedback. It is a powerful methodology because the designed product faces directly its real users. The good news is that you can use this methodology in the post launch phase ! User testings and A/B testings don’t follow the same objectives. Meeting users allows to gather qualitative data on needs and expectations. A/B testings are used to gather quantitative data on performance.

So the most effective version is kept on Businessman’s app. He is delighted, the performances increase and are incredible !

-User research is fierce ! Thank you Captain !

-All the pleasure was mine. We will meet again soon Businessman, because user research and testing have to be led regularly !

Meet the users and test the product, this is the key of success. To be effective, it is necessary to do a bit all the time. The adventures of Businessman and Captain User Research are then not over yet and they will have more beautiful days together.