ML: Black Friday Sales Prediciton (Regression)
By: Girish Salunke
Tags: Black Friday, Xgboost, Data Science, Machine Learning, Regression

This article describes machine learning project for Black Friday Sales prediction. This is the solution to the problem statement posted by analyticsvidya.com.This project was completed as the capstone by Girish Salunke, Janhavi Zarapkar, Mohnish Nathani, Oveshahmed Patel and Shloka Singh as a part of Thadomal Shahani Engineering College (TSEC) Machine Learning Internship under the guidance of Dr. Darshan Ingle (Ph.D.). You can view the python notebook on our Github repository .

Problem Statement

A retail company “ABC Private Limited” wants to understand the customer purchase behavior (specifically, purchase amount) against various products of different categories. They have shared a purchase summary of various customers for selected high volume products from last month. The data set also contains customer demographics (age, gender, marital status, city_type, stay_in_current_city), product details (product_id and product category) and Total purchase_amount from last month. Now, they want to build a model to predict the purchase amount of customers against various products which will help them to create personalized offers for customers against different products.

Dataset

Analytics Vidya Dataset format

Note: The fields Occupation and Product_Category have been replaced by Integral values to protect confidential data using the concept of Data Masking. Here is a quick look at the dataset:

dataset.head()

Performing a few standard statistical operations on the dataset we get:

dataset.describe()

Analysis

Purchase — Target Variable

To analyze the distribution we constructed a distplot of the data:

From the Distplot it looks like the data follows Gaussian(Normal) Distribution. We then perform the univariate analysis:

For Outlier Analysis we constructed a Box Plot:

Since the Target Variable Purchase is a continous valued attribute. In order to predict it we have to use the various Regression Models of Machine Learning.

Gender

Constructing a Countplot of Gender attribute:

Age

City Category

Marital Status

Stay in Current City

Occupation

The analysis indicates that the attributes Occupation,Age and Gender will have a significant impact on the predicted values.

Data Cleaning

Data cleansing or data cleaning is the process of detecting and correcting (or removing) corrupt or inaccurate records from a record set, table, or database and refers to identifying incomplete, incorrect, inaccurate or irrelevant parts of the data and then replacing, modifying, or deleting the dirty or coarse data.

Handling Categorical Data

Step 1: One Hot Encoding

We aplied get_dummies() method from Pandas Library on attributes Gender,City_Category. The newly generated columns are M,F(replacing Gender) and A,B,C(replacing City.)

Snapshot of o/p after OneHotEncoding

Step 2: Mapping Ordered Data

This transformation is used for Age attribute since the values in Age column are classified into various class intervals. We map these class intervals to integral values. For doing so we use the built-in map() function of Python.

Mapping of Age Class intervals to integral values

Step 3: Label Encoding

We used the LabelEncoder model from sklearn.preprocessing on following attributes.

Stay in Current City years: This column contains values ‘4+’ which cannot be interpreted by the machine learning models. Hence to treat it we need to use LabelEncoder for the entire column

ProductID: This column contains long redundant String values. If fed directly to the model this attribute is assumed to have a higher weight. Hence in order to increase the accuracy of the model, we apply LabelEncoder to shorten the values. By employing this methodology we were able to reduce the Mean Squared error form 3000 to 2700.

O/P after Label Encoding

Step 4: Handling Missing Values

A large chunk of values in the Attributes Product_Category1 and Product_Category2 have missing values filled in as NA. We fill these values by 999(Any random value is okay) since the model does not accept NAN values.

NAN values replaced by 999

Step 5: Removing Outliers

Box Plot shows presence of Outliers

From the box plot we can conclude that values above 20000 are all outliers. Hence we have to remove these outliers. From the displot and statistical analysis done before we know that the Mean value of purchase is 9263 and Standard deviation(SD) is 5023. We know that we need to eliminate all value above 21000 since they are outliers(derived from Boxplot). So we calculate Z score: Z= X -Mean/S.d. Here X=21000. Therefore Z=2.33. So if we eliminate all values with Z>2.33 all outliers will be eliminated.

Eliminating outliers

Data Scaling

None of the input attributes are numeric hence we do not have to apply scaling. We can scaling to Purchase attributes however doing so had no effect on accuracy. Hence we skip this part.

Model Training

We tried 3 Regression Models for this dataset. The model that gives the highest Accuracy is the most suitable model.

Criteria for Accuracy:

For regression models we calculate accuracy by finding out

i. Mean Absolute error

ii.Mean Squared error

iii. Root mean Squared error(RMSE)

Analytics Vidya Specified that results be submitted in the following format:

Hence RMSE is the final deciding factor. For visualization we plot a graph of y_actual vs y_predicted. The red line shows how the scatter plot would have been if the accuracy was perfect(RMSE). The blue points is the actual scatter plot shows that depicts deviation from actual results.The farther the spread from the red line worse the model.

1. Multiple Linear Regression

RMSE: 4444

As we can see this model gives RMSE above 4000. Even from the graph we can see that this model provides very poor accuracy. In the cases where Purchase value is above 15000 the model fails completely.

2. Random Forest Regressor

RMSE: 2946

Random forest model gives results significantly better than Linear Regression. Also it does not fail for values above 15000. The RMSE value is approx 2900.

3.XGBoost Regressor

RMSE: 2729

XG Boost produces results similar to Random Forest however the RMSE is slightly better than Random Forest with an RMSE of 2700. Hence it is the most suitable model.

Conclusion

Out of the three models XGBoost provides the highest accuracy with an RMSE of 2729. This a pretty good accuracy if we compare it to the leaderboard on Analytics Vidya. This accuracy get us into the top 37%. With a little more effort an accuracy of 2400 can be achieved via Hyperparameter tuning.