Starbucks Capstone Challenge
By: John Abel
Tags: Machine Learning, Udacity, Python, Kaggle

There does appear to be some differences between successful and unsuccessful offers. Offers with a difficulty of 0 are informational offers, which we saw previously did not produce a successful offer, the graphic shows that offers with a difficulty of 20 are more likely to be unsuccessful.

Looking at duration it appears that offers with a duration of 4 days or fewer did not produce a single successful offer. Where the probability for a successful offer has a peak at 7 days.

Predictive Modelling

We will now use 4 different models to try and predict whether an offer will be successful using offer and customer characteristics. Each of these models was trained on a training dataset and evaluated on a testing dataset to avoid overfitting and to see how the models would perform on unseen data.

Logistic Regression

Logistic Regression models are great for binary classification problems. They have a few underlying assumptions — they assume the class outcomes are interdependent of each other and require no or little colinearity between independent variables — essentially the interdependent variables or features should not be highly correlated.

Here we can see that the feature ‘email’ is highly correlated with every other interdependent variable so this feature was not included in the predictive modeling using Logistic Regression.

Graphic indicating class imbalance

Looking at our data it’s clear that we have a class imbalance. This is an issue because our model could just predict that the offer will not be successful (denoted as class 0 on the graphic) and could still achieve a reasonable score for accuracy.

We can solve this by a random oversampling. Where we will randomly sample data points from successful offers with replacement until we have the same number of data points for unsuccessful offers.

Before we can train a Logistic Regression model we need to handle missing values. We had missing values present for both income and age. These missing values were imputed with the medians of their respective columns because the distributions had a slight right skew and imputing the mean would affect their distributions slightly.

Preprocessing

I have decided to use MinMaxScaler from Sklearn to normalize the features hopefully improving the accuracy of the classifier.

Initially, the model was trained using the liblinear solver and it achieved an accuracy of 76% and an f1 score of 0.77.

Confusion matrix for Logistic Regression classifier using liblinear solver

As you can see the model has performed reasonably well — it shows good sensitivity (the ability to classify a positive outcome — true positive rate) however the specificity (the ability to correctly classify a negative outcome- true negative rate) is letting the model down.

Hypertuning Logistic Regression

To try and improve the classifier I decided to use GridSearch to hyper tune the parameters, where I changed the penalty and inverse regularization strength C.

Optimized parameters for Logistic Regression

The tuned Logistic Regression model achieved an accuracy of 75.8% and an f1 score of 0.77 on the testing dataset. This means that tuning the model has not given us an increase in performance.