Random Forests and One-Hot Encoding Introduction | by Ali Fakhry | Towards Data Science

Kaggle’s Titanic Machine Learning Dataset––a classic open-source introduction to the realm of machine learning. While this may be a beginner project, there is still a leaderboard (it is captivating to see yourself rank up as you continue to work on your code).
To start, it is required that you download the “train.csv” and “test.csv” files directly from Kaggle. This can be downloaded here: https://www.kaggle.com/c/titanic/data. Make sure to make sure that the files have been placed in a designated spot that can be pulled later.
In this competition, a large train dataset describes various characteristics of those whose lives were cut short due to the Titanic’s fatal crash and how they related to survival. This includes categorical data such as sex, where they embarked, and their respective names. Yet, there is also numerical data such as age and fare. We have to use a one-hot encoding algorithm to make the overall program and prediction more sufficient for categorical data.
The software that I will be using in the project will be Google Colab. However, this does not really matter in this situation. We do not need a GPU. Using Jupyter Notebooks without a GPU will work just fine. The first thing to do in this project, similar to many others in computer science and machine learning, is to program the imports we need to pull.
import pandas as pd from sklearn.model_selection import train_test_split from sklearn.ensemble import RandomForestClassifier 
Pandas will be used to parse files for this task. The second import, “train_test_split,” will be used to split and separate training and testing files. The last import we need will be RandomForestClassifier, the large overarching import required to use Random Forests.
As I have said, I am using Google Colab. So, I need to “mount” the program onto Google Colab to access Google Drive. I will do this with the following code.
from google.colab import drive drive.mount('/content/gdrive')
Now, we have to take in the .csv files appropriately. This can be done with the following code.
train_df = pd.read_csv('/content/gdrive/My Drive/train.csv') test_df = pd.read_csv('/content/gdrive/My Drive/test.csv') 
Next, I am going to drop a variety of categorical information. This includes the tickets, cabins, and names. The names have been removed as they generally would not relate to a correlation between survival or not. Yet, if this was going to be an extensively in-depth analysis, some names may have been more “important.” For instance, based on last name or prefix. For the most part, however, it would be too much work for nothing. Additionally, the ticket and cabin information lacked competition, so it was best to disregard these entire datasets.
This can be completed with the following code.
train_df = train_df.drop(['Name'],  axis=1) test_df = test_df.drop(['Name'], axis=1) train_df = train_df.drop(['Cabin'],  axis=1) test_df = test_df.drop(['Cabin'], axis=1)train_df = train_df.drop(['Ticket'],  axis=1) test_df = test_df.drop(['Ticket'], axis=1)
To see how the data looks like now, it may be beneficial to print out the train data collection. To do so, type the following.
train_df
The same can be done for the test data.
test_df
Next, we should consider the “embarked” category in the dataset. Similar to the ticket and cabin dataset, values are missing. However, there are not as many significant gaps. The most common embarked value is “S,” so for the few missing values, those will just be filled in with “S.” This is not precise or perfect, but every value must be filled in.
common_value = 'S' data = [train_df, test_df] for dataset in data:   dataset['Embarked'] = dataset['Embarked'].fillna(common_value)     
Now, we have to use one-hot encoding to organize the categorical data. This will be applied to the two categorical data sets remaining, which are “sex” and “embark.” This will be applied to both the “train” and “test” datasets.
def clean_sex(train_df):   try:      return train_df[0]   except TypeError:      return "None"train_df["Sex"] = train_df.Sex.apply(clean_sex) categorical_variables = ['Sex', 'Embarked'] for variable in categorical_variables:   train_df[variable].fillna("Missing", inplace=True)    discarded = pd.get_dummies(train_df[variable],prefix = variable)           train_df= pd.concat([train_df, discarded], axis = 1)   train_df.drop([variable], axis = 1, inplace = True) 
The above was applied for the “train” data. Now, we have to do the same for the “test” dataset.
def clean_sex(test_df):try:   return test_df[0] except TypeError:   return "None"test_df["Sex"] = test_df.Sex.apply(clean_sex) categorical_variables = ['Sex', 'Embarked']for variable in categorical_variables:   test_df[variable].fillna("Missing", inplace=True)   discarded = pd.get_dummies(test_df[variable], prefix = variable)   test_df= pd.concat([test_df, discarded], axis = 1)   test_df.drop([variable], axis = 1, inplace = True) 
After organizing the categorical data, we should now consider the numerical data. For the “age” and “fare” data, there are spots in which there are missing values that are not being considered. Rather than completely ignoring these datasets, we need to find a way to implement a data value. To do so, we will take the average of “age” and “fare” and use these for values we do not have. We will have to do this for both the “train” and “test” data.
train_df["Age"].fillna (train_df.Age.mean(), inplace = True)test_df["Age"].fillna (test_df.Age.mean(), inplace = True)train_df["Fare"].fillna (train_df.Fare.mean(), inplace = True)test_df["Fare"].fillna (test_df.Fare.mean(), inplace = True)
We should now look at our data again to double-check that all the appropriate applications have been made.
This will print the “train” manipulated data.
train_df
This will print the “test” manipulated data.
test_df
Now, because age is usually represented as an integer, it would be best to convert them into integer-based descriptions. We again need to do this for both the “train” and “test” datasets.
train_df = train_df.round({'Age':0})test_df = test_df.round({'Age':0})
At this point, we have manipulated our data at a “decent level.” While not fully precise, it is at a point in which we can start to use a random forest algorithm to start formulating predictions.
So, for our local estimation, we will drop the “survived” value on the train data for a new variable called “X_train.” Then, for “Y_train,” we will include just “Survived.” Then, we will make the X_test equal to “test_df” directly. Lastly, print out the shape.
X_train = train_df.drop("Survived", axis=1) Y_train = train_df["Survived"] X_test  = test_df X_train.shape, Y_train.shape, X_test.shape
To double-check, it may be useful to print out “x_train.” Again, this can be done simply with the following command.
X_train
Next, we should initialize our random forest. We should also use a variety of parameters to better our algorithm.
random_forest = RandomForestClassifier(criterion='gini',    n_estimators=700,     min_samples_split=10,     min_samples_leaf=1,     max_features='auto',     oob_score=True,     random_state=1,    n_jobs=-1) random_forest.fit(X_train, Y_train)Y_pred = random_forest.predict(X_test) acc_random_forest = round(random_forest.score(X_train, Y_train) * 100, 2) acc_random_forest 
The “n_estimators” value represents the number of estimators used in the random forest (the number of trees in the forest). In this case, 700 was a value that worked well. “Min_samples_split” is the number of samples (minimum) required to separate and split the internal node. “Min_samples_leaf” represents the minimum number of samples required to be a leaf node. “Max_features” is used to describe the size of random subsets of features considered when splitting a node. When set to auto, there is no feature subset. The “oob_score,” when set to true, will attempt to validate the random forest model with an “Out of Bag” score. When “n_jobs” is set to -1, all the processors will be used to run.
The last segment of the code above fits the model and produces a prediction (y_pred) that works alongside the X_test data. The accuracy of the random forest will then be printed out. Locally testing this data produces an accuracy of approximately 90% (91.81%). However, this is just a local prediction based on separating the survival values from the training dataset. For an accurate estimate, we need to submit it directly to Kaggle. To do so, write the following code to get the “submission.csv” file.
submission = pd.DataFrame({    "PassengerId": test_df["PassengerId"],    "Survived": Y_pred})submission.to_csv('submission.csv', index=False)
Conclusion:
Kaggle produces an accuracy value of “0.79186” or “79.186%.” When I submitted using this prediction, I was ranked at 1573 out of 19649, which is approximately the top 8%. The difference between the local prediction and the Kaggle prediction is that the “train” data was modified as a “test hybrid” dataset, and Kaggle used the actual “test data.” Yet, in both situations, there was a level of randomization.
Overall, using random forests seemed to produce a decent result. For the most part, the predictions were accurate. Yet, data was thrown out for simplicity, which definitely harmed the overall strength of this prediction. Additionally, while random forests are suitable for low-level estimations, using other algorithms would produce more solidified and accurate results and would typically correlate with a higher training time.