Root Cause Analysis
By: Ashwin Shrivastava
Tags: Interview, Problem Solving, Management, Product Management, Structure

One of the most commonly asked question type in product management interviews- “Find Root Cause.” Root causes test you around fundamental problem solving, structuring, and narrowing skills. Someone hearing this question for the first time won’t even recognize an RCA as a challenge itself. These are just one-liner questions that would create a false impression of it being easy, doable, easily crackable, but Devil lies in the detail. I have personally found these questions to be one of the most complex problems in a product interview.

You have to maintain a delicate balance between asking questions and inferring things from the information provided, and at the same time, your approach should always converge. Call this a “Judgmental Round”- Where you will be judged on everything you ask, say or infer.

One of RCA problems’ most common types is profitability, which is quite popular, and several books have published the framework around the same. Hence, I don’t believe there is anything to add in that type. For this article, I will attempt one RCA problem for a digital product. I will also try to breakdown the problem at every step and let’s see if we can find some general structure that can be reused effectively. So let’s dive in!

Order cancellation on Amazon is up by 10%; find the root cause?

A nice little question, seems pretty straightforward?

Step 0- Clarify

A data scientist spends most of the time cleaning and preparing data (approx 70%). This step is essential because, in most situations, data provided by the customer has bad quality or just cannot be directly fed to some ML model. Consider this step to be analogous to the data-preprocessing step in data science. You will have to define the scope correctly, narrow down the situation, clarify terms.

Moving vertically and horizontally through the problem statement: Let’s try to understand this with a infographic.

**what is cancellation refers to type of cancel— User cancel/Admin cancel.

Moving horizontally to clarify terms used in question and moving vertically w.r.t to “what” and “where” to narrow down the scope. This will help us in limiting our observation to a narrow space of limited variables.

A good question before moving to step 1 could be- “have we made any changes to the definition of cancellation, or did we change the tool which used to measure this cancellation rate.”

Step 1- Check for external factors

Make this a habit; sometimes, it’s just an earthquake causing some problem (not joking). You already have some context about the situation after step-0. Use this information to think of major external factors that could have influenced the product in any way.

An excellent way to test the involvement of external factors in a problem is to ask — “Is this an industry-specific problem or company-specific?”

Industry-specific problems can have root causes in factors beyond their line of work. (For example, a roadside vegetable vendor has a bad season because people have started buying packaged food post-COVID-19).

Let’s try to list down some common external factors to check for-

1) Any recent natural calamity/Power Shutdown (can be asked when the trend is not gradual or a single point decline in the graph.)

2) New government regulation/policy change?

3) New product entry in the market/product substitute.

4) Sentiments on social media? (E.g. Controversial tweet by company, fake news around the company, etc.)

After touching possible external factors we can move to our next step- Making a user journey.

Step 2- User Journey

Try to make an end to end user journey for the product so that you can cover every tiny point of failure where a problem can reside. Let’s try to make a User Journey for our case. We will try to do this in parts-

A. Channels: Different ways a user lands on the product web page/app.

Google search ads. Ads on blogs/websites. Organic search. Emails. Shareable link (from friends, any third party.) Push notifications. Organic app open.

You can club these under promotional and organic, paid and unpaid, or any suitable classification of your choice. Ask relevant questions about each channel. E.g., how’s the trend of cancellation been in these channels?, Are we getting cancellations on order bought from any specific medium, or is it even for all channels?

B. Key Action in app/web-page- Divide the web app segments with which a user interacts during product discovery.

Search Bar- Relevant Question (R.Q.)- Are we facing cancellations on orders discovered through the search bar?

Recommendation section- (R.Q.) — Did we make any changes in recommendation algorithm?

Product category tab- (R.Q.) — Are we facing order cancellations for a specific product category? Have we made any changes to how we define types? Any new addition to product categories?

C. Selection — (R.Q.) — Any changes made to the product listing page/product description page? Any changes in how users add items to the cart/add details/add address (is it accurate enough)?

D. Payments — Always check for the mode of payment by which the order was bought. (R.Q.)- Are we facing cancellations for a specific payment mode (C.O.D., card payment, credit payment, etc.), Is there a payment failure for canceled order?

E. Confirmation and Tracking: (R.Q.) — At what point after confirmation, orders are getting canceled? Any specific source pin-codes where orders are getting canceled? Any specific mode (Airway, Roadway, Type of warehouse) for which cancellations are up?