Using Machine Learning to predict Item Demands in the future!
By: Rishabh Sharma
Tags: Regression, Deep Learning, Machine Learning, Time Series Forecasting, Python

Data Preparation

Before building the great Machine Learning models, we prepare the data to our needs, so that we get the best results. We add some more time based features like ‘Year’, ‘Month’, ‘Week’, ‘Day’, ‘Dayofweek’, ‘Dayofyear’, ’weekofyear’, ‘Is_month_end’, ‘Is_month_start’, ‘Is_quarter_end’, ‘Is_quarter_start’, ‘Is_year_end’, ‘Is_year_start’.

I added some ‘lag’ features as well as they are considered sometimes useful in Time-Series Analysis. You can find more about them here.

Modelling

Now, let’s get to the good part, I mean the best one! I have divided the modelling part into 2 subsets —

— Baseline Models

— ML Models

We will use Baseline Models to predict from the previous sales and then use them as features. Sounds strange? — it is not actually. Below used are the two most important in time-series analysis.

For all the following models, we will evaluate the model on SMAPE and RMSE.

Simple Moving Average — The First Model used is the Moving Averages Model which uses the previous n values in order to predict the next value.

This can easily be implemented with pandas’ ‘rolling’ function.

Using Previous Values — Pt=(Pt−1+Pt−2+Pt−3….Pt−n)/n

For the above, the Hyperparameter is the window-size (n) which is tuned manually and it is found that the window-size of 2 is optimal for getting the best results using Moving Averages using previous 2016 values therefore we get Pt=(Pt−1+Pt−2)/2

Exponential Weighted Moving Average — This is a more logical way towards assigning weights and at the same time also using an optimal window-size.

Specify decay in terms of halflife, α=1−exp(log(0.5)/halflife)

For the above, the Hyperparameter is the halflife which is tuned manually and it is found that the halflife of 0.3 is optimal for getting the best results using Exponentially Weighted Moving Averages using previous values.

Comparing Baseline Models

We split the data into Train and Cross-Validation as 80% — 20%. Here, we do a Time based splitting and we have a sequential data. We take 4 years of data from 2013–2016 of each store and item as Training and the 2017 data as the Cross-Validation data.

Linear Regression —

Firstly, we train a Linear Regression model without hyper-parameter tuning and normalized features and here’s the result we get:

Test SMAPE: 0.8473661033300371

Test RMSE: 1.0650487158450959

Wow! Amazed right. Even the simple model, which just fits a line, giving us terrific results with less than 1% of error. Linear Regression has owned a much more solid respect for me after this.

But remember, “Feature Importance”? (doing the “quotes” gesture!). Yeah, bummer. Linear Regression doesn’t implicitly gives us feature importance. So even though we have such a great model, sadly we will not be able to accept it. (Frown!) :(

But don’t worry, we always have some alternatives. Let the bad boys come into the play. We have our powerful Ensemble techniques. What does they say about this now?

XGBoost Regressor — Training with hyper-parameter tuning. See the results.

Test SMAPE: 1.8557876826185409

Test RMSE: 2.4428290358987748

Ouch! The “great” XGBoost isn’t making you happy either. The error is higher than Linear Regression. What about feature importance?

Feature Importance

Yes!! Things are now working as expected (at some level). So, EMA and MA are indeed the most important features. We know the results are as expected as previous values do play a vital role and now we are seeing this. Thank you XGBoost. (Happy!) :)

Random Forest Regressor — Training with hyper-parameter tuning. See the results.

Test SMAPE: 0.8751925305940013

Test RMSE: 1.2986884849038731

The trees are really doing their job well! The error is so much close to what we received with Linear Regression. This could be the one we were looking for. But wait a minute! “Feature Importance”, remember?

Feature Importance

Yep! That is it. That’s just it. Exponential Weighted Moving Average is trying to get all the credit here. Even if we remove a lot of features other than ‘ewm’, there’ll not be any significant change in the performance. I believe you can see now that how important are our baseline models as features.

C onclusion

Looking at our Business Contraints, since we do not have any low-latency requirements, even simple model like Exponentially Weighted Moving Average performs the best with the lowest Percentage Error.

The machine learning model Linear Regression performs best considering the KPI (SMAPE) with Test SMAPE = 0.84.

Random Forest Regressor comes 2nd with slightly more Test SMAPE = 0.87 with some model interpretability but only considers one feature as important.

Since we want feature importance to be reasonable and useful, we can conclude XGBoost Regressor to be our best model with Test SMAPE = 1.85 as it gives reasonable Test SMAPE as compared to other ML Models, but with a lot of interpretable feature importance.

Although, we have concluded our best model, but I really want to show what’s so called “Magic” aka Deep Learning. I am going to try a simple 1-hidden layer RNN model and see can we beat Linear Regression in terms of Error.

The reason I’m doing this is because both the techniques intuitively use Optimization and I believe RNN would perform better.

LSTM Model Summary

batch size = 256

epochs = 10

LSTM = 128 units

Dropuout = 0.5

Activation = ReLU

Optimizer = Adam

loss = Mean Squared Error

Metrics = [‘SMAPE’]

So what are the results? I’m not going to tell you straight away. Below is the table created using PrettyTable library comparing all the models we created.

I hope you get surprised! Ready?

References:

1. https://www.kaggle.com/arindamgot/eda-prophet-mlp-neural-network-forecasting

2. https://www.kaggle.com/thexyzt/keeping-it-simple-by-xyzt

3. https://www.kaggle.com/c/demand-forecasting-kernels-only/data