Logistic Regression | Sklearn | Titanic
Tags: machinelearning, logisticregression, python, sklearn
Published: 2020-03-28 07:39:42

One of the machine learning problems is the classification problems.<br> 
___" Classification is the problem of categorizing observations(inputs) in a different set of classes(category) based on the previously available training-data"___.

The simplest classification model is the logistic regression model, and today we will attempt to predict if a person will survive on titanic or not. Here, we are going to use the titanic dataset - [source](https://web.stanford.edu/class/archive/cs/cs109/cs109.1166/problem12.html).
<br>
####Variables in Data
__Survived__ - "survived -> 1", "not survived ->0"
__Pclass__- intuition here is "first class-> 1", "business class->2", 
"economy class->3"
__Name__- it's the passanger's name
__Sex__- "male->1", "female->0"
__Age__- passenger's age
__Siblings/Spouses Aboard__- numbers of siblings/spouses of passenger on the titanic
__Parents/Children Aboard__- numbers of parents/children of passender on the titanic
__Fare__- ticket price

Here, the survived variable is what we want to predict, and the rest of the others are the features that we will use for model training.

Firstly, add some python modules to do data preprocessing, data visualization, feature selection and model training and prediction etc.
```python
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt

from sklearn.preprocessing import LabelEncoder
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.utils import resample

import seaborn as sns
```  
Now, it's time to read data. 
```python
raw_data = pd.read_csv('data/titanic.csv', header=0)
raw_data.head()
```
'_DataFrame.head()_' is used to get a simple overview of the tabular dataframe.
 
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/wqy4121klin229ng693i.PNG)

If you observe closely, '_Name_' feature is redundant and It's better to remove such idle feature from the dataset also the '_Fare_' can be rounded up.
```python
raw_data = raw_data.drop(columns = 'Name')
raw_data['Fare'] = raw_data['Fare'].round(0) 
raw_data.describe()
``` 
 ![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/p93klqmw7xl8bz3qfwvd.PNG)

Moving forward, we'll check whether the data is balanced or not because of imbalance the prediction could be biased towards the bigger quantity. 
```python
sns.countplot(x='Survived',data=raw_data)
plt.title('Not Survived vs Survived')
plt.show()
```
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/9a70h7hvsb8vd60xv39o.PNG)
It's imbalanced and we will balance it using SMOTE (Synthetic Minority Oversampling Technique). Before that, we have to handle the categorical data.
<br>

####Dummy Variables
In real life datasets, more often we dealt with the categorical and the numerical type of features at the same time. creating dummy variables on categorical data can help us reduce the complexity of the learning process. (___otherwise we will have to create different equations for different labels___)

In our dataset, '_Sex_', '_Pclass_' are the two categorical features on which we will create dummy variables(features) and also going to ignore any one of the columns as to avoid collinearity. 

Read More on [dummy variables](https://towardsdatascience.com/the-dummys-guide-to-creating-dummy-variables-f21faddb1d40) 

```python
final_data = pd.get_dummies(raw_data, columns =['Sex','Pclass'], drop_first=True)
final_data.head()
```
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/qg68qvzm17y0ngssbixh.PNG)

Feature selection is one of the important tasks to do while training your model. Here for this dataset, we will not do any feature selection as it's having 
887 examples and 7 features only.

Go to [my github](https://github.com/harshitlakhani/Algorithms---Python/tree/master/Machine%20Learning) to see the heatmap on this dataset or [RFE](https://medium.com/@aneesha/recursive-feature-elimination-with-scikit-learn-3a2cbdf23fb7) can be a fruitful option for the feature selection.  
<br>
#### SMOTE
Before the data balancing, we need to split the dataset into a training set (70%) and a testing set (30%), and we'll be applying smote on the training set only. The '_imblearn_' module provides a built-in smote function for data balancing.

Read more on [SMOTE](https://machinelearningmastery.com/smote-oversampling-for-imbalanced-classification/). 
```python
X = final_data.loc[:,final_data.columns != 'Survived']
y = final_data.loc[:,final_data.columns == 'Survived']

from imblearn.over_sampling import SMOTE

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=0)

smt = SMOTE(random_state=0)
data_X,data_y=smt.fit_sample(X_train, y_train)

sns.countplot(x='Survived',data=data_y)
plt.title('Not Survived vs Survived')
plt.show()
```
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/2gbkr1fd6q4yp6q5uqrn.PNG)
<br>
####Training
For the training, we will be using '_LogisticRegression_' method provided by sklearn module and it also helps in testing different parameters of the model as well.
```python
logreg = LogisticRegression(max_iter=500)
logreg.fit(data_X, data_y.values.ravel())

y_pred = logreg.predict(X_test)
print('Accuracy of logistic regression classifier on test set: {:.2f}'.format(logreg.score(X_test, y_test)))
```
Current model has accuracy of 78%.
<br>
####Testing
```python
#confusion matrix
from sklearn.metrics import confusion_matrix
confusion_matrix = confusion_matrix(y_test, y_pred)
print(confusion_matrix)

#different perfomance matric 
from sklearn.metrics import classification_report
print(classification_report(y_test, y_pred))

#ROC
from sklearn.metrics import roc_auc_score
from sklearn.metrics import roc_curve

logit_roc_auc = roc_auc_score(y_test, logreg.predict(X_test))
fpr, tpr, thresholds = roc_curve(y_test, logreg.predict_proba(X_test)[:,1])
plt.figure()
plt.plot(fpr, tpr, label='Logistic Regression (area = %0.2f)' % logit_roc_auc)
plt.plot([0, 1], [0, 1],'r-')
plt.xlim([0.0, 1.0])
plt.ylim([0.0, 1.0])
plt.xlabel('False Positive Rate')
plt.ylabel('True Positive Rate')
plt.title('ROC')
plt.legend(loc="lower right")
plt.show()
``` 
If you don't know what is ROC curve and things like threshold, FPR, TPR. You must    
read below given writing

{% medium https://towardsdatascience.com/receiver-operating-characteristic-curves-demystified-in-python-bd531a4364d0 %}

Do comment, if you want to discuss any of the above.

##Thank You !!