User research for an e-commerce start-up
By: Biswajeet Das
Tags: User Research, Design, User Experience Research, User Experience, Strategic Planning

1. Challenges

Our challenges were to understand the resellers—

a. Why the New Resellers on the Zeeto platform are dropping off in the early stages after installing the app by placing 0–3 orders.

b. How we can help them at the initial stages and also to validate some assumptions about the New Resellers which leads to them not being able to Resell well enough leading them to quit reselling.

Another challenge which we added to the list later on was to enable the partners to get involved into reselling with the Users so that the Users get a boost in their reselling specially when they are new to reselling. This we realized was a necessary thing to do based on the insights from the first Descriptive research with the resellers.

2. Aspirations

Our aspirations are —

a. To make the life of New Resellers on the app a little easier so that they don’t drop off.

b. By the end of this research we would like to build a strategy which would significantly boost New Resellers performance on the app and increase Orders Per Reseller.

3. Focus Areas

Our focus areas will be primarily on the —

a. General resellers behaviors and motivations on and off the app while reselling.

b. New Users behavior, motivation, attitude towards reselling, their pain points while reselling, Journey maps, Empathy maps, personas and their Archetypes.

c. We will also focus on Partner’s involvement areas with the New Users, their Journeys with the users, Empathy maps.

A deeper dive into these areas would give us some direction towards a solution to help the New Users in reselling and henceforth retain them on the app,

4. . Guiding Principles

a. Descriptive Research on the Resellers — Behavior Maps, Reseller Insights — Product, Design, Customer Experience, Strategies towards reselling, etc.

b. Directional research on the New User Resellers — Assumption Validations, Personas, Archetypes, Empathy Maps, Journey Maps, Usability Testing.

c. Directional research on the Partner Users — User-Partner Journey Maps, Empathy Maps, E-mail Survey.

5. Activities To Perform

The activities we will perform to reach our goals are:

a. Secondary Research about the Resellers and the company’s Business Model.

b. 1 on 1 interviews — Sampling, mobilizing the users, questionnaire building, telephonic interviews, transcribing, documenting the insights.

c. Surveys — English and Hindi

d. Data Analysis.

e. Usability Testing.

f. Presenting the insights to the — Product Team, Design Team, Customer Experience Team and Marketing & Sales Team(Aligning all the teams)

g. Iterations to the App Design, Marketing Strategies, etc.

h. Ideating Solutions — product strategy building.

6. Measurements

a. Increase in order per reseller.

b. Decrease in frequency of New Reseller drop offs.

c. Increased apps ease of use.

d. Post Product iteration reseller feedbacks.

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

So that we don’t deviate from our mission we kept this blueprint with us all the time.

To know more about UX Strategy and how to create UX Blueprints and effectively use it you guys can also go through this great article by Jim Kalbach by clicking on this link. https://articles.uie.com/ux_strategy_blueprint/ It helped me a lot.

Now since we already know the problem statement and our missions and objectives are clear we start with our work. We started off with the Descriptive research on the resellers.

Descriptive Research On The Resellers

Our user assessing criteria — App optimization skills, Customer Base expansion skills, Skill & Market intelligence, Order Placing Potential.

Aim:

To understand the Zeeto Resellers behaviors, journeys, motivations, problems, attitudes when it comes to placing orders, sharing products with customers. By this understanding our aim is to increase order per reseller and help the resellers to earn more money.

Our Approach

We planned to do a Design Sprint which would take us 7 weeks to complete.

We started by building the team — Designers, Product managers, Researchers, Data Analysts, Marketing people, Business people. We tried to get at least one member from each team to be present during the entire sprint. This was necessary to keep everyone aligned towards the same common goal. Every team will have their own perspective of things and working in a company we have to keep not of everyone’s perspectives. Most of the heavy loads were on the Product and the Design team.

In some other article I will talk about User Researchers key role of uniting teams across an organization.

a. First Week — Day 1–3:

a. Secondary Research using previous research material

b. Building the problem statement, the research approach and presenting it to various teams across Zeeto on a PPT format.

To save time we emailed the PPT’s ahead so that members from respective teams can come prepared with their feedbacks and suggestions for us to take notes of.

b. First week — Day 4–5:

Building the questions for the 1 on 1 interviews with the users and discussing them with the Design and Product Team and getting their feedbacks, cluster sampling of users and mobilizing the users for interviews which starts on next week, working on interview guidelines — do’s and don’ts and logistics.

The Sample —Number of Users : 16 female participants, Region : Urban, Semi- Urban and Rural.

We clustered our users based on their regions from where they belong to Urban, Semi-Urban and Rural so that our insights have some variety as Zeeto Resellers are from all over the country from various backgrounds. Customer Experience(CX) team gave us the list of 16 highly motivated users by looking at their reselling stats like — earnings, orders, etc. They called them up, took their permission and lined up the interviews from Monday to Thursday.

c. 2nd Week — Day 1–4:

1 on 1 Telephonic Interviews

a. During the interviews one of us focuses only on asking questions, others take notes and interrupt either at the end or in between only if it is very important and record the interviews on our phones as we keep it on loudspeaker so that everyone can keep taking some notes to discuss them later. We later on listened to the recordings and transcribed and documented them word by word, extracted insights and showed them to teams across Zeeto at the end of every day so that they also stay on track and also get to know from them if there is anything we can find out next from the users as the research proceeds.

d. 2nd Week: Day 5(Friday)

The last day of the week was completely dedicated into documenting, making Powerpoint slides. We had to decide what insights to show to each team — Product,Design, Marketing, Sales, Customer Experience. We wanted each team to get something out of this research to work on so our focus was on filtering at least some actionable insights for each team to work on towards our common goal. Although we tried to finish as much work as possible in our office hours, we ended up doing some transcribing of the recorded user interviews and other remaining work on the documents and the PPT Presentation over the weekend as we had to present it in front of the company’s CEO on Monday.

Results & Observations

d. 3rd Week: Day 1:

a. On Monday we Presented the Insights in front of all the different team members. Notes were taken down, feedbacks and suggestions shared and discussed. We had a Product Working Session where we discussed actionable insights for the product.

b. The insights were categorized into these categories — Product & technology, Product Category, Design, Customer & Reselling Experience, Reseller Behavior and Strategic approach to reselling.

e. Third Week: Day 1- 3

a. From here on the Design and the Product team started working on the actionable insights to make minor changes to the Product and Design.

b. Changes were made to the Search function, Filter function and Category Sections on the app which the users said were not very easy to use and discoverable.

c. The Design team took 2 days to make a quick prototype to do a Remote Usability Test with our users which was planned for the 4th day of the week.

d. I got involved into developing the script for the Remote Usability test, mobilizing the participants and deciding on the metrics for the usability test on whose basis we will try to understand our users — time taken to perform the tasks, success(whether user can perform the task or not), users subjective satisfaction and error rate.

e. The Sample — We did a cluster sampling of the users where we divided them according to the number of orders placed by them and time spent on the app. We divided the participants into three categories — New Users, Active Users and Pro Users.

f. Third Week: Day 4

On the fourth day of the third week of the sprint we started of with the Remote Usability Testing. We formed two teams for the test — each team had a User Researcher whose. We kept taking notes by asking the participants questions when they get stuck and also after they finish their tasks.

The tasks given to the participants were:

Task 1

To search for an item under a sub-category which comes under a category.

Aim — a. To check whether the new way in which sub categories were organized under categories were discoverable and easily understood by the users and also to check the path taken by them to accomplish the task.

b. To see whether the names given to the new sub-categories were easy to understand.

Observations:

Only the New Users were struggling to perform the tasks, they were not able to use the filter option as well as add margin while placing the order. Users used 3 types of path to reach to the product under the sub categories

a. Using the search function to directly find the product under the sub category.

b. Browsing and clicking on the category until they found the sub category to find the product using search function again.

c. Clicking on the category and browsed for the product inside it without going to the sub-categories. It took them longer to find the product in this manner.

Task 2

To use filter option for applying price range, material and color and place the order for a Saree from the Clothing Category.

Aim — a. To see if users can discover the new filter option on the app.

b. To see if they can use it easily to find the product of their choice and apply price range, material, color, etc.

Observations:

Most reseller’s were able to use the filter option to customize the products they are searching for except the New Users.

2. Resellers had to be guided towards the new Filter option. It was not discoverable. So intuitively they ended up using the old filter option which they are used to using.

“Nobody cares about the thing you’ve designed, unless you can get them past the beginning.” — Julie Zhuo, Product Design VP @ Facebook

Our Scope:

a. To make the user flow more intuitive for the New Users keeping in mind of the Next Billion Users.

b. To announce the feature during the onboarding.

c. To communicate relevant value with context — at the right time, with the right messaging when the new feature is introduced.

User Research Insights:

Some of these resellers said that they take their partners help during all the stages of reselling and others took some help initial stages when they are new to the platform.

Pro and Active Resellers were easily able to finish their tasks.

New Resellers were the ones who struggles the most to perform the tasks. They lacked app optimization skills.

Interestingly very few Pro resellers were also not able to perform the tasks very properly. When asked how they did it otherwise while placing orders they said that their husbands helped them all the time.

3rd Week : Day 5

a. On the last day of 3rd week the Insights were presented across all the teams and planning for the next research started.

b. Since we got some insights which indicated that our New Users are facing some problems in using the app and backed up by some more New User assumptions related to reselling from the Product and Marketing Team we decided to do a Directional Research to validate or invalidate these assumptions and also try and understand how the partners were helping them in their journey.

Research Scope: To find out how can we enable partners to be more involved with the New Users so that their overall reselling experience improves hence more orders placed by the resellers.

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

4th Week : Day 1–2

The first two days of the fourth week was dedicated to :

a. Building the Research Approach —

A background research about the New User Resellers — orders placed, time spent on the app, their feedbacks. Get the list of assumptions to validate. Building the Questions for the 1 on 1 telephonic interview which would be in the direction of validating those assumptions. Deciding on the User Sample — 30 female participants from all over India. We categorized the resellers into two categories:

a. O-1 Category — The resellers in this category has recently downloaded the app but not yet done more than 1 order.

b. 1–8 Category — The resellers in this category has not done more than 8 orders since last 6 months.

5. Mobilizing the users — The CX Team randomly called up the users. 15 from the 0–1 category and 15 from the 1–8 category. This time we gave the CX Team a participant screening script to make sure we have the right willing to participate participants with us. There were few who were not interested for interviewing, and since we had to know the reasons behind their demotivation we requested them to give us their time and also offered them some reward.

Directional Research On The New Resellers

Problem Statement-

New users drop off at the initial stages after placing 0–3 orders using the app.

Aim —

a. To identify motivations, behaviors, attitudes of New Resellers on an off the app in the area of placing orders and understanding reselling overall.

b. To understand the partners involvement in the New Users journey of reselling. To understand the partners motivations, behaviors, attitudes while helping or being involved with the New Reseller during reselling.(For this we did a separate Directional Research ).

c. To validate some of our assumptions for the New Users which we found out during our previous research with the resellers.