Instacart Market Basket Analysis (Kaggle Competition)
By: Tanay Kale
Tags: Recommendation System, Binary Classification, Kaggle Competition, Machine Learning

This blog is about my first Kaggle Challenge. Hopefully, this blog will give you a good understanding of solving the classification problem.

Instacart is an American company that provides grocery delivery and pick-up service. The Company operates in the U.S and Canada. Instacart offers its services via a website and mobile app. Unlike another E-commerce website providing products directly from Seller to Customer. Instacart allows users to buy products from participating vendors. And this shopping is done by a Personal Shopper.

Outline

Problem Overview Data Source Data Overview EDA Feature Engineering Modeling Future Work Reference

Problem Overview

The main objective of this competition was to predict which previously purchased products will be in the user’s next order.

The problem is a little different from the general recommendation problem, in this case, we are not recommending new products to the user instead of that we are going to recommend the product which is previously bought by that user. It is an interesting problem because here we need to find the user’s reordered behavior so we can personalize the user’s cart.

Data Source

Instacart posted this problem and data on the Kaggle platform.

(https://www.kaggle.com/c/instacart-market-basket-analysis/data)

Data Overview

There are 6 csv files. That file contains the following information.

Orders​ : This table includes all orders, namely prior, train, and test. It

has a single primary key (​ order_id​ )

has a single primary key (​ order_id​ ) order_products_train​ : This table includes training orders. It has a

composite primary key (​ order_id and product_id​ ) and indicates

whether a product in an order is a reorder or not (through the

reordered variable).

composite primary key (​ order_id and product_id​ ) and indicates whether a product in an order is a reorder or not (through the reordered variable). order_products_prior​ : This table includes prior orders. It has a

composite primary key (​ order_id and product_id​ ) and indicates

whether a product in an order is a reorder or not (through the

reordered variable).

composite primary key (​ order_id and product_id​ ) and indicates whether a product in an order is a reorder or not (through the reordered variable). products​ : This table includes all products. It has a single primary key

(​ product_id​ )

(​ product_id​ ) aisles​ : This table includes all aisles. It has a single primary key

(​ aisle_id​ )

(​ aisle_id​ ) departments​ : This table includes all departments. It has a single

primary key (​ department_id​ )

EDA

Now let’s explore the data a little bit.

What Is The Frequency Of Order Number Amongst The Users? Generally how many orders a user place?

This plot explains, What’s the order count for each user?. There is a high peak at 4,5,6,7&8 which tells us that users mostly orders 4 to 8 products in every order.

What Day Of The Week User Placed The Order?

Now if you see most of the users prefer weekends to place orders than weekdays. This makes sense because these days most of the users are free and on other days the user's order does not have high order ratio as weekends.

In which Hour User Place Order?

This plot tells us what’s the user’s preferred time for placing an order?. There are high peaks in the morning hours and it slowly gets decreases. Between 10 am to 15 pm there is a high order ratio amongst users.

After The Last Purchase, How Many Days The User Takes To Place an Order?

After placing an order, how many days a user takes to order again?. Usually, there is a gap of 7 days after each order. If you see carefully there is a high peak on the 7th day and after that on the 14th day and then at end of the month. The user takes a week to order once again.

Feature Engineering

I have created 2 types of features

User Latest Purchase User X Product

User Latest Purchase:

In this feature, I computed the user’s latest order and checking is that product is reordered or not? If it is then I put that product into the user’s latest order. The reason behind creating these features is mostly user orders the products which are they recently ordered and if you see the frequency of placing the order by user mostly they place the order after every 7 and 14 days and at end of the month.

Users Latest Purchase

User X Product :

In this feature, I am trying to find the user behavior for each product. Here I computed the total count of order and out of all orders how many times the user reordered?

User X Product Reorder Sum and Count

I have computed very minimal features but these features tell pretty good behavior of what users want? Let me give an example, If a product is recently purchased by the user and the product order count is 10 and the reorder count is 8 and now if we look at the computer features this product was recently ordered the reorder count is high so there is a high probability that the user will reorder this product again.

Modeling

We are trying to predict will the user reorder this product again?. This makes it a binary classification problem. There are various algorithms we can use for the binary classification task.

I have used XGBoost. It is a great algorithm for this task as it provides a range of wide parameters that give fine-grained control over the model training procedure. To boost the score for the Kaggle competition this algorithm helps a lot.

The XGBoost returns the probabilistic values but this is a binary classification problem. So we need to take a threshold for classification and the threshold I used in this task is 0.2. I took the reference to this link( threshold detail) to know about the threshold so it will maximize the score.

Evaluation Metric

The Kaggle evaluation metric is the Mean F1 — Score. To calculate the Mean F1 — Score use this (https://www.kaggle.com/aikinogard/python-f1-score-function)

Here is the snippet of that code.

So, the Final Kaggle Score is 0.34016 The public score is 0.34016 The private score is 0.33875

This score puts me into the Top 300 rank which Is Top 10% of the leaderboard.

Future Work

Computing more features related to the user and a product like a user cart size, user’s average day of the order, and others.

Here I used only a Machine Learning algorithm but in the future, I would like to try out some deep learning models and I think it will improve the score a lot.

Also, I am looking to try the Apriori Algorithm to get some more features.

My (GitHub Repo) with my notebook and code.

Reference