Starbucks Capstone Challenge
By: Divya Budale
Tags: Exploratory Data Analysis, Data Analytics, Classification, Data Visualization, Data Science

source: www.eater.com

Introduction

Once every few days, Starbucks sends out an offer to users of the mobile app. An offer can be merely an advertisement for a drink or an actual offer such as a discount or BOGO (buy one get one free). Some users might not receive any offers during certain weeks. Not all users receive the same offer, and that is the challenge to solve with this data set. Also, someone using the app might make a purchase through the app without having received an offer or seen an offer.

The task is to combine transaction, demographic, and offer data to determine which demographic groups respond best to which offer type.

The data is contained in three files:

portfolio.json : containing offer ids and meta data about each offer (duration, type, etc.)

profile.json : demographic data for each customer

transcript.json : records for transactions, offers received, offers viewed, and offers completed

Here is the schema and explanation of each variable in the files:

portfolio.json

id (string) — offer id

offer_type (string) — type of offer ie BOGO, discount, informational

difficulty (int) — minimum required spend to complete an offer

reward (int) — reward given for completing an offer

duration (int) — time for offer to be open, in days

channels (list of strings)

profile.json

age (int) — age of the customer

became_member_on (int) — date when customer created an app account

gender (str) — gender of the customer (note some entries contain ‘O’ for other rather than M or F)

id (str) — customer id

income (float) — customer’s income

transcript.json

event (str) — record description (ie transaction, offer received, offer viewed, etc.)

person (str) — customer id

time (int) — time in hours since start of test. The data begins at time t=0

value — (dict of strings) — either an offer id or transaction amount depending on the record

Problem Statement

In this project, I am building a model to predict whether the customer responds to the offer and makes a transaction or not. To accomplish this, the following approaches were conducted:

data preprocessing and cleaning

exploratory data analysis (EDA)

build machine learning models

evaluate the performance

Data Preprocessing and Exploratory Data Analysis

The json files were loaded into the three data frame — portfolio, profile, and transcript

portfolio : The data frame contains metadata for 10 different offers.

There are three different types of offer — BOGO (buy one get one), Discount, and Informational.

It also contains a channel column which was split into four different columns : web, email, mobile and social

2. profile — There are 17000 customer profiles available for the analysis.

The customer gender contains 13% missing values, along with Male, Female and Other

The age group ranges from approximately 20 to 100 years but also contains an outlier for age 118 years.

On further analysis, it was found that the gender and income were missing when the age was equal to 118 indicating the customer has not provided these informations.

From the became_member_on column, age of membership, membership start day, weekday, month, and year information was extracted.

From the below, it can be seen that the minimum and maximum wage for male and female are the same but the male income starts flattening out towards high income. Also, the average female income and age is greater than than the average male income and age as seen from the dashed line

Membership has started increasing over the recent years. Also, male membership is greater than the female membership in the past couple of years.

transcript — Within the transcript dataset, there are entries for 17000 customers as well matching with our profile information.

There are 4 different types of events: offer received, offer viewed, transaction and offer completed.

The processed transcript data frame looks like below:

Even after data processing, 45% of missing data was identified in the offer id column

On investigating further, it was identified that the event ‘transaction’ will not have any offer id associated with it, reward_complete is available for only the ‘offer completed’ event and ‘amount’ is available only for the ‘transaction’ event

As already mentioned, there four events: offer received, offer viewed, transaction and offer completed, and three offer types: informational, bogo, and discount

The general flow of the event for:

informational: received > viewed > transaction (no completed event since there is no reward associated with this type)

bogo and discount : received > viewed > transaction > completed

Our first data pre-processing is identifying the offer id associated with the transaction event. Here is the general approach:

create a data frame and forward fill the offer ids for viewed, received and transaction event merge with the full_transcript data frame, cleanup the missing values, drop redundant, and rename the columns

The second step is to identify the successful offer (one that resulted in the customer making the transaction)

Here is the general approach:

identify the previous offer id and previous event for the customer create a column ‘offer effect’ which will be 1 if the previous event will be ‘offer viewed’ for the ‘transaction’ event row and the offer id for ‘transaction’ and ‘offer viewed’ should be same create a column ‘offer validity’ which will be 1, if the transaction has occurred within the duration of the offer Finally, create a column ‘offer success’ which will be 1 when both ‘offer effect’ and ‘offer validity’ will be 1 Merge the final result with profile dataframe and name it all_data

We have the target column ‘offer_success’ for building the machine learning models. The distribution of the values shows a highly unbalanced dataset which will need further processing.

From the below, it can be seen that every offer is sent to almost a similar number of customers but the success rate is widely different across each one of them.

fafdcd668e3743c1bb461111dcafc2a4 and 2298d6c36e964ae4a3e7e9706d1fb8c2, both of which are discount offers perform best with more than 85% success rate whereas 0b1e1539f2cc45b7b9fa7c272da2e1d7 is the lowest performing with only 26% success rate