Get_dummies: A Solution to Kaggle Titanic Competition Question
Tags: Logistic Regression, Artificial Intelligence, Titanic, Data Science, Machine Learning

I had been working on Kaggle’s Titanic competition question off and on for several months and had experimented with several algorithms in an effort to increase accuracy. I had been inputting numeric data into the various models I had used and they would classify a passenger on the ship with a one for having survived or a zero for having perished at sea.

The Titanic is a classifier question that uses logistic regression techniques to predict whether a passenger on the Titanic survived or perished when it hit an iceberg in the spring of 1912.

Logistic regression is a statistical model that uses a logistic function to model a binary dependent variable. The classifier function will take in all of the allocated dependent variables and will output a value of zero or one in this instance. The output of the model is a sigmoid curve that gives a definite S shape, with the top of the curve representing one and the bottom depicting zero:-

With this in mind, I endeavoured to conduct an experiment to see if I preprocessed the data in the Titanic datasets as being either one or zero, would it improve the accuracy of the model that I selected. I have to say that I was very surprised by the results I received.

The highest accuracy score I have been able to achieve, using an ensemble methodology was 78.468%. When I used the Python Pandas get_dummy function to convert the independent variables to columns of integers of ones and zeros, I was very surprised at the results.

The Pandas get_dummies function converts a categorical variable to dummy/indicator variables. I therefore decided to convert all of the data in the columns in the Titanic train and test datasets into dummy variables to monitor the resulting output. The results I achieved were very surprising indeed and I would like to share them with you. I used this methodology on several models in the sklearn library, such as Gradient Boosting Classifier, Logistic Regression, Random Forest Classifier, and finally settling on Logistic Regression CV.

The algorithm I developed to convert the data in the Titanic datasets to binary integers so they could be trained, fitted and tested is as follows:-

The problem statement for the competition question is:-

The sinking of the Titanic is one of the most infamous shipwrecks in history.

On April 15, 1912, during her maiden voyage, the widely considered “unsinkable” RMS Titanic sank after colliding with an iceberg. Unfortunately, there weren’t enough lifeboats for everyone onboard, resulting in the death of 1502 out of 2224 passengers and crew.

While there was some element of luck involved in surviving, it seems some groups of people were more likely to survive than others.

In this challenge, we ask you to build a predictive model that answers the question: “what sorts of people were more likely to survive?” using passenger data (ie name, age, gender, socio-economic class, etc).

After loading in the libraries and the Titanic datasets, I read them into the computer. I replaced any null values with KNN Imputer or mode. Once I determined that there were no null values in the train and test dataframes, I began the task of preprocessing the data, using the Pandas get_dummies function, to convert each column of data to several columns of either one or zero.

I created a new column called Title and took the title out of the name:-

I mapped each title and assigned each one a number between one and five:-

I created five columns of data from the column using get_dummies:-

I converted the passenger class column to three columns of data:-

I converted the sex column to two columns of binary data:-

I created an extra column for the age range, for which I created bins, and converted that data to several columns of binary data:-

I created an extra column called family to combine two columns and then converted this to nine columns of binary data:-

I created bins for the fare to group it into five categories. I then converted this categorical data into four rows of binary data:-

I used get_dummies to convert the embarked column to three rows of binary data:-

I then merged all of the dataframes I created into two dataframes called data_train and data_test:-

I defined the X, y and X_test variables:-

I split the training dataset up for training and testing:-

I selected a model after much trial and error:-

I used the trained model to test on the validation set and obtained an accuracy of 76.67%:-

I used the trained model to make predictions on the test set and achieved an accuracy of 76.79%.

The entire code for this example can be found on the Kaggle Jupyter notebook that I have public, and you can view it on the following link:- https://www.kaggle.com/tracyporter/titanic-one-hot-encoding-log-reg-cv?scriptVersionId=43917008