Data Analytics in E-Commerce Retail | Towards Data Science

▶
I recently helped Colgate-Palmolive optimize the product strategy for its online catalog on Amazon. I applied data science methods to forecast the e-commerce sales, predict the profitability of the products at a granular level, and optimize the profit of the firm. And finally, the project achieved buy-in for operationalization from the firm’s eCommerce Insights & Analytics team.
This article is an extension of the experiences and learnings gained through that project. I’ll explore the different avenues where data analytics can be employed in the e-commerce retail industry, particularly by multinational consumer goods companies like Unilever, Nestle, P&G, etc. And I’ll do that in my trademark easy-to-understand, fun-to-read style.
In the words of Claire Hennah, Global VP of eCommerce at Unilever,
“E-commerce is no longer a channel simply for the future. The future is here.”
I love numbers. So, I’ll start with some numbers.
In the 2nd Quarter of 2020, US retail e-commerce sales have shot up to $211.5 billion, up from $160.4 billion in the 1st Quarter. The lockdown caused by Covid-19 has certainly contributed to this boost in e-commerce sales. Claire has a great point,
“Many people are coming to online shopping for the first time, which is driving up penetration.”
Let’s take a look at the Q2 e-commerce sales as a percentage of the total retail sales in the US, in the last 5 years, so that you get a good feel of the rise of the e-commerce potential.
Moreover, e-commerce sales in the US are set to surpass $1 trillion to make up 18.1% of total retail sales by 2024.
Let’s take a look at the performance of the top CPG companies in 2020 so far, as compared to the last year in terms of the percentage share of e-commerce sales in total retail sales.
So, now you know the importance companies are given to boost their e-commerce sales.
Now, let’s dive into the different avenues where analytics has a high potential of helping consumer goods companies scale up e-commerce sales.
Do you remember the time you visited the general store when the friendly storekeeper or an assistant helped you throughout your shopping experience? He knew the food items you bought, the laundry detergent you purchased, or the ice-cream your kid loved. He knew your shopping budget and would make recommendations. And you did end up purchasing what he recommended quite a few times. Didn’t you?
That relationship with the customers is what the companies need.
A product recommendation engine is the solution. What is a recommendation engine?
It is simply an automated form of the ‘friendly storekeeper’. It is a tool that uses machine learning algorithms to suggest products to customers that they might be interested in. The recommendations can happen through the following channels:
Within websites
Email campaigns
Online ads
The recommendation engines are of two types:
a) Impersonalized Recommendation Engine:
This is useful for sending out recommendations via welcome emails or bulk newsletters, without using customer data. Here are some useful categories:
new products like a new chocolate ice-cream that has been recently launched
trending products like a fast-selling coffee brand to help recruit new customers
slow-moving products like a luxurious bar soap at a discounted price
b) Personalized Recommendation Engine:
This is gold! It uses customer data to recommend the best offers and promotions. It uses deep learning algorithms and Natural Language Processing (NLP) to make the customer experience as personalized as possible. The algorithms use data from the following spheres:
Purchase history
Recent activity
Online behavior
Social media browsing
These algorithms are really powerful and can greatly boost e-commerce sales by:
improving conversions rates
increasing the average purchase value of customer
reducing bounce rates & cart abandonments
Here’s what Amazon’s recommendation engine shows me currently. It looks like it knows that I love to workout.
McKinsey has reported that 35% of what customers purchase on Amazon come from product recommendations based on such algorithms.
Market Basket Analysis is a technique that identifies the strength of association between a pair/group of products that are purchased together. In simpler words, it’s based on the concept that if a customer buys a product, they are more or less likely to buy another related product. Market Basket Analysis also uses machine learning/deep learning algorithms, like product recommendation engines.
I’ll explain this with an example from the project at Colgate-Palmolive that I mentioned earlier. I bundled a children’s toothbrush (unprofitable product) with a deodorant, a dad’s product (fast-moving product), and was able to increase the total weekly profit of the combination by 8%.
Association rule learning is a machine learning method that is used for discovering associations between products. It does not consider the order of the purchase of products. Let me explain this with a demonstration.
The analysis is run on millions of baskets, but I’ll demonstrate it with 7 baskets and explain the 3 important measures used. Let’s explore the association between shampoo and conditioner (Basket 1):
Definition: No. of transactions that include both the products x and y as a percentage of the total number of transactions
Business meaning: It indicates how frequently both the products are purchased together.
The association rule {shampoo} => {conditioner} has a support of 0.71 → Both products are bought together in 71% of the transactions
Definition: Ratio of the number of transactions that include both the products x and y to the number of transactions that include the product x.
Business meaning: It indicates how often the association rule is true, i.e. how often both the products are purchased together.
The association rule {shampoo} => {conditioner} has a confidence of 0.83 → 83% of the times the customer buys shampoo, conditioner is bought as well
Definition:Lift is the ratio of confidence to expected confidence. Expected confidence is simply the probability that product y is in the basket.
Business meaning: The lift of a rule indicates how strong is the association within the rule, i.e. how better a rule is at predicting the result than just assuming the result in the first place. Higher lift values indicate stronger associations.
Lift = 1 → There’s no relation between the purchase of the products.Lift > 1 → The products are likely to be bought together. Higher the lift, the higher the chances.Lift < 1 → The products are unlikely to be bought together. They are substitutes.
The association rule {shampoo} => {conditioner} has a lift of 1.17 → The probability of buying a conditioner with the knowledge that shampoo is already present in the basket is much higher than the probability of buying a conditioner without the knowledge about the presence of shampoo.The presence of shampoo in the basket increases the probability of buying conditioner.
I know lift is a little tricky. But I hope you got it now.
And after this, Chi-squared analysis is useful to determine the statistical significance of association rules.
80% of people say that the most important aspect of shopping at an online e-commerce store is competitive pricing.
So, you know how important it is to set optimal prices and strike a balance between customer satisfaction and the firm’s profit. Now, let’s explore 3 important concepts related to price optimization:
This is the first step in price optimization, where you’ll group similar products and customers. Classification (supervised) and clustering (unsupervised) algorithms are useful for this segmentation. The best technique will depend on the business requirements and the type of data.
By the way, I had used K-mean clustering to form groups of products with similar sales trends during my stint at Colgate. It had helped me to build one predictive model per cluster instead of having many models.
And, I have touched upon RFM Customer Segmentation in another article of mine. Give it a quick read!
towardsdatascience.com
Once the segmentation is done, regression model on the various data points around sales, conversion rates, seasonality, product attributes, marketing channels, etc. helps to fix optimal prices for the products.
Let me give you a hypothetical business example from the CPG industry. The regression model built by the analytics team at Unilever for Personal Care products can help it to predict that a 5% decrease in the price of Dove deodorant can boost sales by 15%.
Do you remember the last time the price of your favorite product dropped during the big sale at Amazon, and the other e-commerce stores also dropped their prices?
This is dynamic pricing. It uses competitor data as an input into machine learning algorithms for pricing to determine the product price at different times. This helps the products to remain viable in the competitive market with frequent price fluctuations.
A/B Testing can be used to refine the pricing model, but it needs a lot of technical considerations and is a whole different topic. I’ll talk about it in detail some other time. Meanwhile, you may want to read this article I have written on the top mistakes that creep in during A/B Testing, with interesting business cases & fixes.
towardsdatascience.com
Let’s imagine Benjamin Franklin and Abraham Lincoln to be e-commerce business tycoons for a minute.
Franklin seems to have warned against underestimating the potential of demand forecasting in e-commerce retail sales,
“If you fail to plan, you are planning to fail.”
And Lincoln seems to have provided the perfect strategy for including demand forecasting in the analytics scene at your e-commerce company.
“Give me six hours to chop down a tree, and I will spend the first four sharpening the axe.”
Demand forecasting refers to the use of analytics techniques to predict the demand for a product and forecast sales. When you know the sales trends in advance, it gives you an advantage over your competitors in the following ways:
An underestimation of inventory levels can lead to inventory getting out of stock and hence cause customer dissatisfaction. An overestimation of inventory stock levels can lead to overstocking of inventory and hence cause unnecessary storage costs.
Demand forecasting helps in informed inventory planning and saves you from both selling out of popular products and warehouse space wastage due to slow-moving stock.
Since the money is not tied up in slow-moving inventory, It helps you plan the budget properly and use the cash optimally. Thus, it helps you to reduce financial risk.
Effective demand forecasting also helps in improved pricing strategy. You can charge more for the products having a high demand forecast, and vice versa. You can better plan the marketing budget, advertising investment, and discount plans.
The prior knowledge of the product demands and their fluctuations helps in planning better customer service. It’s time for a business example.
Amazon patented ‘Anticipatory Shipping’ in 2013 to use predictive modeling to ship products to you before you would even make the purchase. Amazon’s algorithms would predict your purchase, and it would send the product to your nearest warehouse. Then, Amazon would wait until the actual purchase. Amazon believed that this would drastically cut the delay from the time of purchase to the actual delivery and boost the overall company sales.
There are more analytics applications in the e-commerce retail industry but these are some of the prominent ones with a direct impact on the revenue from e-commerce sales. I hope you enjoyed reading it.
To end with Joel Anderson’s (Former CEO, Walmart) powerful message,
“You can’t just open a website and expect people to flood in. If you really want to succeed you have to create traffic.”
Note: If you want to have a look at the recent project that I did at Colgate-Palmolive, please find the codes and presentation at my GitHub account. All confidential data has been hidden/anonymized as per NDA requirements.
Feel free to have a look at other articles that I’ve written on SQL:
towardsdatascience.com
towardsdatascience.com
Feel free to provide feedback and follow me for more interesting articles on data analytics. Connect with me on LinkedIn.