Reducing product stock-outs in hypermarkets with Time Series modeling
By: Kasra Mansouri
Tags: Data Science, Anomaly Detection, Forecasting, Retail, Time Series Forecasting

Proposed approaches

As explained above, the main challenge we faced was the absence of labelled data for shelf stock-outs, which prevented us from adopting the initial ML approach we had in mind. Therefore, we considered an alternative approach to build our detection model.

Time Series prediction of hourly sales

Our first alternative approach consisted of detecting sales anomalies (unusually low sales) by predicting hourly sales of products and then comparing them to the actual ones. The idea behind that is to estimate/predict the regular sales quantity that we expect a product to have when there are no “anomalies” in the store, then compare them to its real sales and raise an alert if the difference is “huge”. Hence, by applying our model everyday at 2.p.m we would predict every product’s sales until 2.p.m and then detect anomalies by comparing each product’s real sales with our estimates.

As a starting point, we developed a simple Moving Average model. For each hour, the model would make its predictions by taking an average of the product’s sales on the same hour during the last 30 days. We would then compare this value to the actual sales of the product during that hour, if the following inequality is verified then we would raise an out of stock alert.

This approach is based on the assumption that the products’ hourly sales follow a Normal distribution and aims to raise alert on products outside of the 95% of the confidence interval.

Figure 1: Example of a normal distribution

And that is not the case at all ! In fact, we have very little Time Series signal if we looked at hourly sales in a store and that is simply because the majority of products have 0 sales for several hours during the day, thus the model would predict a close to 0 value (by taking average of the past values) for the given hour. Therefore, modeling the hourly sales is clearly not the right way to go.

Figure 2: Sales quantity per hour of a soda drink

We tried to slightly modify our approach by predicting daily sales quantity until 2.p.m but we would still have many products for which we did not have enough signal, we call them «low rotation products». This approach would have probably worked on high rotation products like Coca Cola sodas, water bottles, etc. but our solution needed to work on all the products in a hypermarket, therefore we phased out this approach.

Anomaly detection using Poisson distribution

Our second approach was to try modeling the frequency of sales (and not the quantity), i.e the elapsed time between 2 consecutive sales of a single product.

We performed statistical tests on some products’ sales data and realized that the time between their 2 consecutive sales follows an Exponential distribution which intuitively makes sense since an Exponential distribution is usually used to model the time between different occurrences of an event. The logical consequence of this is that we can model a product’s «sales rate» with a Poisson distribution. By sales rate we mean the number of times a product goes through the checkout counter during an hour, regardless of the quantity that has been sold.

Figure 3: Histogram of the distribution of the time between each sale of a specific biscuit

Figure 4: Example of an exponential distribution

Hence, our second alternative approach is to estimate the regular sales rate of each product, which is basically the lambda parameter of the Poisson distribution, and then compare that to its real daily sales until 2.p.m. This is a 2-step process :