Rossmann Drug Store Sales Prediction:
Abstract -­­ In this paper we tried to apply machine learning algorithm into a real world problem — drug store sales forecasting. Given…
Category: Machine Learning
Author: Vikas Mishra

Rossmann Drug Store Sales Prediction:
Abstract -­­ In this paper we tried to apply machine learning algorithm into a real world problem — drug store sales forecasting. Given store information, and sales record we applied Linear Regression, Support Vector Regression (SVR) with Gaussian and Polynomial Kernels and Random Forest algorithm, and tried to predict sales for [1]-­­3 weeks Root Mean Square Percentage Error (RMSPE) is used to measure the accuracy. As it turned out, Random Forest outshined all other models and reached RMSPE of 12.3%, which is a reliable forecast that enables store managers allocate staff and stock up effectively.
1. INTRODUCTION
This problem is one of several Machine Learning problems on Kaggle1 . The aim of this problem is to forecast future sales of 1,115 Rossman drug stores located across Germany based on their historical sales data. The practical meaning of solving this problem lies in that reliable sales forecasts enables store managers to create effective staff schedules that increase productivity and motivation. What’s more, for the purpose of practicing what we learnt from the Machine Learning class, this problem saves us the trouble of collecting data, and in the meanwhile provides a perfect real case to apply supervised learning algorithms.
3. DATASET AND FEATURES
The dataset of this problem can be found online and the link is provided below. The data comes in two sets
Sales Dataset -­­ Historical sales data for 1,115 Rossman stores from 2013/1/1 to 2015/7/31. Features include store number, date, day of week, whether there’s a promotion, whether it’s a school or state
holiday and sales on that day.
[1] The link to this problem: https://www.kaggle.com/c/rossmann-­­store-­­sales
[2] The link to dataset: https://www.kaggle.com/c/rossmann-­­store-­­sales/data
1. Store Dataset -­­ Stores’ individual characteristics. Features include store type, assortment level, nearest competitor’s distance and when the competitor was opened, and whether there’s a consecutive promotion.
Throughout our trial, we’ve tried to take advantage of different subset of features. However, reducing number of features didn’t increase accuracy for this problem. So all features are used for building models
70%/30% and k-­­fold cross validations are used in this problem for training and testing. Root Mean Square Percentage Error (RMSPE) is used to measure accuracy, which is defined as

4. Modeling METHODS
There are two methods to train the data. One is to train each store separately, which means forecasting sales of a single store based on its own sales record, regardless of store attributes. The other one is to train all stores together, considering store attributes as parameters.
To train each store separately, one straightforward idea is to apply linear regression. According to the normal equation,

we can easily predict sales by

Further more, we figured that this problem can actually be kernelized. Here consider that case of applying MAP estimate for
to avoid overfitting, which results in the following primal problem

We can easily predict the sales


Gaussian Kernel And Polynomial kernel
we can see that this problem can actually be kernalized, thus we can apply the kernel trick. We tried Gaussian Kernel and Polynomial Kernel in this case, which is illustrated as following.
(a) Gaussian Kernel
(b) Polynomial Kernel (Polynomials of degree up to d)
Our next model for this project is Random Forest Regression. We tried this model because it’s fast and can accommodate categorical data. RF first picked a certain amount of data from the dataset randomly (ie. bootstrap) and then picked a certain amount of features out of the total features randomly to build decision trees. The final result for each test data is average of results obtained by all these decision trees. Decision trees usually overfit the data; however randomness will average out the high variance
5. EXPERIMENTS AND RESULTS
Linear Regression
Linear regression is used as our baseline model. 70%/30% cross validation is used here to divide the data set into
training set and test set. As it turns out, linear regression gives us a RMSPE of 52.8%.
Support Vector Regression
One thing special on the implementation of SVR is that, it need to build an m*m matrix, where m indicates the number
of training samples. Since the size of our training set is ~700,000 , it’s unrealistic to operate on the whole dataset. To take use of the abundant dataset practically, we build a SVR model for each store, and compute the mean of each store’s RMSPE as our final error rate.
Firstly, we applied Polynomial Kernel and Gaussian Kernel for a single store, Store 1. By trying different pairs of
for Gaussian Kernel, and different pairs of for Polynomial Kernel, we found that when
lambda = 140,
Sigma = 45,
Gaussian Kernel gives the best RMSPE of 13.6%, when Polynomial Kernel gives the best RMSPE of 12.8%. Two kernels are comparable in this scenario.
Gaussian Kernel
Polynomial Kernel
Secondly, using the method of finding optimal pairs of parameters discussed above, we applied Gaussian Kernel and Polynomial Kernel to all stores. As we dig deeper into the dataset, we found that accuracies vary on different time period of prediction. Below are figures of how RMSPE varies with different time period for prediction. In terms of average RMSPE, Polynomial Kernel(16.3%) beats Gaussian Kernel (26.8%) significantly. In the meantime Polynomial Kernel is also more robust than Gaussian Kernel, given that there are fewer outliers and no extreme
outliers(RMSPE>1) in the figure of Polynomial Kernel. So overall, Polynomial Kernel suits the dataset better and provides more reliable results.
Random Forest
We applied Random Forest after merging all data including all the categorical data. We used scikit-­­learn package of python for implementing the algorithm[7].
The two main parameters we tuned for RF is the number of trees and the size of the random subsets of features to consider when splitting a node. We used 5 fold cross validation to get RMSPE while varying these parameters. Two plots are shown below. From these plots, we could see that RMSPE doesn’t change too much after tree number reaches 30 and after feature number reaches 20.
RMSPE with feature number
RMSPE with tree numbers
Conclusion:

As is shown in the result, among all models, Random Forest works the best, and provides a reliable prediction of the sales. Linear regression, SVR with Gaussian/Polynomial Kernels and RF all have their own strengths and limitations. By implementing these algorithms, we’ve studies the properties of the dataset and made reasonable predictions. In the future, we wish to use the fact that sales records are consecutive in time, and see how time series affect prediction result. Also there are still many effective machine learning algorithms worth trying, so we would like to try more algorithms in the future, such as Gradient Boosting and k-­­Nearest Neighbors algorithm.