How I solved a multi label imbalance email problem with Logistic Regression
Tags: Logistic Regression, Python, AI, Data Science, Machine Learning

I recall three decades ago when email initially came into being. Emails started out in the offices I worked in at the time and then I eventually obtained my own personal email account, which I still use today. It did not take long for business to take the email and use it for sales and marketing purposes. This competition question I will be discussing discusses emails used for sales and marketing, and whether a person is going to be inclined to open the email and click onto the link in that email. This competition question can be found in the Analytics Vidhya hackathon page and can be found here:- https://datahack.analyticsvidhya.com/contest/lord-of-the-machines/

The problem statement for this question entitled, “Lord of the Machine”, reads as follows:-

“Email marketing is still the most successful marketing channel and the essential element of any digital marketing strategy. Marketers spend a lot of time in writing that perfect email, labouring over each word, catchy layouts on multiple devices to get them best in-industry open rates & click rates.

How can I build my campaign to increase the click-through rates of email? — a question that is often heard when marketers are creating their email marketing plans.

Can we optimize our email marketing campaigns with Data Science?

It’s time to unlock marketing potential and build some exceptional data-science products for email marketing.

Analytics Vidhya sends out marketing emailers for various events such as conferences, hackathons, etc. We have provided a sample of user-email interaction data from July 2017 to December 2017. You are required to predict the click probability of links inside a mailer for email campaigns from January 2018 to March 2018”

I have written this program using Google Colab, which is a free online Jupyter Notebook that has the major Python libraries already installed. Because the libraries I need are already installed in Google Colab, all I had to do was to import them.

After I imported the libraries I would need, I loaded and read the data files, which can be found in the Problem Statement of the competition question:-

https://datahack.analyticsvidhya.com/contest/lord-of-the-machines/

The train file did not have all of the information that was needed to solve this problem, so I merged the train file with the data file. I used exactly the same technique with the test file as well:-

Once I had devised the train and test files, I checked for any null values. In this exercise there were no null values that needed to be imputed:-

I then graphically analysed the target variables and found there was a class imbalance in both of the labels I would be required to predict on. The “is_open” label is a binary variable that indicates whether the recipient of the email opened it, so this is the first target variable that needs an answer to:-

The target variable, “is_click”, has an even greater class imbalance because the values are based on whether or not an email has been opened:-

I then decided to convert the time the emails were sent to day, month, hour and minute:-

I then began working on the model for the first target, “is_open”, and defined X1, y1 and X1_test. I created a variable, id, where test.id was stored. *I also created the variables “open_train” and “click_train”, where the target variables are stored.

I created features1, where the columns I would be using were posited.

The variables that were going to be use were defined: y1 is train.is_open, X1 and X1_test were the features:-

I then created the first pipeline that would be needed to predict on who would open the email. A numerical and a categorical transformer were created to preprocess the numerical and categorical columns. The preprocessing is to be performed by the model.

I then split X1 for training and validation:-

Once the X1 dataset had been split, I created the first pipeline, which would contain the model and perform the preprocessing.

I then predicted on the validation set, attaining an accuracy of 63.74%:-

I graphically represented the value of y1_pred, which was the prediction. As can be seen, Logistic Regression, the model I used, did not fully address the class imbalance because a higher percentage of “1” samples were recorded. I would like to say that when I attempted to use other models,such as Catboost and XGBoost, the class imbalance was negatively addressed, as none of the “1” examples were picked up. Therefore, although Logistic Regression did not give outstanding results, it did perform better than other models:-

I then predicted on the test dataset. Again, the class imbalance was not adequately addressed, but this is in all likehood due to the quality of the data not being able to afford a high level of accuracy:-

Once the prediction for the individuals who had opened the email was produced, I created a column called “is_open” and added this to the test dataset:-

Once the test dataset had been modified to include “is_open”, I created the X2, y2, and X2_test values for the second round of predictions. I accomplished this by adding “is_open” to the features variable and calling it features:-

I created the second column transformer, which is identical to the first one, except I added the newly created column that had been added to the test file to the numeric transformer.

I also split the X2 file up for training and validation using train_test_split() in the same manner that I had split X1 up:-

I then created the second pipeline, which was identical to the first pipeline, except I trained the second set of datasets into this model.

“When I predicted on the new validation set I achieved an accuracy of 91.35%, which was significantly higher than the accuracy of the first model. I believe the rationale for that is the fact that more data was injected into the model in the form of the extra column, “is_open”, that had been added to the test dataset:-

I graphically represented the prediction and found that the model did predict when the email was opened and clicked on:-

“When I predicted on the second test dataset I noted that the class imbalance had been addressed, and perhaps a little more than it should have been:-

Once the second prediction had been made, I put it into a dataframe, which was to be the submission. This submission consisted of the variable, id, which I had previously made, and also the second prediction:-

“When the submission .csv file had been created, I submitted it to Analytics Vidhya’s solution checker and achieved a rating of .5307. While this rating is not necessarily high, it needs to be noted that the highest score for this question was .70.

Obviously this algorithm could be improved by perhaps using a different model that contains class weightings:-

The code for this post can be found in its entirety in my personal GitHub account, the web address found here:- https://github.com/TracyRenee61/Misc-Predictions/blob/main/AV_Emails_TS.ipynb