Logistic Regression: A Simplified Approach Using Python | by Surya Remanan | Towards Data Science

In Logistic Regression, we wish to model a dependent variable(Y) in terms of one or more independent variables(X). It is a method for classification. This algorithm is used for the dependent variable that is Categorical. Y is modeled using a function that gives output between 0 and 1 for all values of X. In Logistic Regression, the Sigmoid (aka Logistic) Function is used.
After we train a logistic regression model on some training data, we will evaluate the performance of the model on some test data. For this, we use the Confusion Matrix. A Confusion Matrix is a table that is often used to describe the performance of the classification model on a set of test data for which the true values are already known. Given below is a Confusion Matrix.
Here, TP stands for True Positive which are the cases in which we predicted yes and the actual value was true. TN stands for True Negative which are the cases in which we predicted no and the actual value was false.FP stands for False Positive which are the cases which we predicted yes and the actual value was False.FN stands for False Negative which are the cases which we predicted No and the actual value was true.
Confusion Matrix helps us determine how often the model prediction is correct or in other words, the accuracy of the model. By the above table, It is given by:
( TP + TN ) / Total = 100 + 50 / 165 = 0.91
This means that the model is 91% correct. The Confusion Matrix is also used to measure the error rate which is given by:
( FP + FN ) / Total = 15 /165 = 0.09
There is 9% error in the model.
In this article, we will be dealing with very simple steps in python to model the Logistic Regression.
We will observe the data, analyze it, visualize it, clean the data, build a logistic regression model, split into train and test data, make predictions and finally evaluate it. All these will be done step by step. The Data we will deal with is the ‘Titanic Data Set’ available in kaggle.com. This is a very famous dataset and often a student’s first step towards learning Machine Learning based on classification. We are trying to predict the classification: Survival or deceased
First, we will import the numpy and pandas libraries:
import numpy as npimport pandas as pd
Let us do the visualization imports:
import matplotlib.pyplot as pltimport seaborn as sns%matplotlib inline
We’ll proceed by importing the Titanic data set into a pandas dataframe. After that, we’ll be checking the head of the dataframe just to get a clear idea of all the columns in the dataframe.
train=pd.read_csv('titanic_train.csv')train.head()
Most of the data that we come across has missing data. We’ll check for missing data, also visualize them to get a better idea and remove them.
train.isnull()
Here, we find boolean values. True indicating that the value is null and False the vice versa. Since there are a lot of data, we use the seaborn library to visualize the null values. In that case, our task becomes much easier.
sns.heatmap(train.isnull())
The Age and Cabin column have null values. I have dealt with the problem of dealing with NA values in my previous blog. Please have a look at it.
It is always a good practice to play around with the data and fully exploit the visualization libraries to have fun with the data.
sns.countplot(x='Survived',data=train)
This is a count plot that shows the number of people who survived which is our target variable. Further, we can plot count plots on the basis of gender and passenger class.
sns.countplot(x='Survived',hue='Sex',data=train)
Here, we see a trend that more females survived than males.
sns.countplot(x='Survived',hue='Pclass',data=train)
From the above plot, we can infer that passengers belonging to class 3 died the most.
There are many more ways by which we can visualize data. However, I’m not discussing them here because we need to get to the step of model building.
We want to fill in the missing age data instead of just dropping the missing age data rows. One way to do this is by filling in the mean age of all the passengers (imputation). However, we can be smarter about this and check the average age by passenger class. For example:
plt.figure(figsize=(12, 7))sns.boxplot(x='Pclass',y='Age',data=train,palette='winter')
We can see the wealthier passengers in the higher classes tend to be older, which makes sense. We’ll use these average age values to impute based on Pclass for Age.
def impute_age(cols):    Age = cols[0]    Pclass = cols[1]        if pd.isnull(Age):if Pclass == 1:            return 37elif Pclass == 2:            return 29else:            return 24else:        return Age
Now apply that function!
train['Age'] = train[['Age','Pclass']].apply(impute_age,axis=1)
Now let’s check that heat map again.
sns.heatmap(train.isnull(),yticklabels=False,cbar=False)
Great! Let’s go ahead and drop the Cabin column.
train.drop('Cabin',axis=1,inplace=True)
We’ll need to convert categorical features to dummy variables using pandas! Otherwise, our machine learning algorithm won’t be able to directly take in those features as inputs.
train.info()
sex = pd.get_dummies(train['Sex'],drop_first=True)embark = pd.get_dummies(train['Embarked'],drop_first=True)
Here, we are dummying the sex and embark columns. After dummying, we will drop the rest of the columns which are not needed.
train.drop(['Sex','Embarked','Name','Ticket'],axis=1,inplace=True)
We will concatenate the new sex and embarked columns to the dataframe.
train = pd.concat([train,sex,embark],axis=1)
Now, the dataframe looks like this:
from sklearn.model_selection import train_test_splitX_train, X_test, y_train, y_test = train_test_split(train.drop('Survived',axis=1),                                                     train['Survived'], test_size=0.30,                                                     random_state=101)
from sklearn.linear_model import LogisticRegressionlogmodel = LogisticRegression()logmodel.fit(X_train,y_train)predictions = logmodel.predict(X_test)
We can check precision, recall,f1-score using classification report
from sklearn.metrics import classification_reportprint(classification_report(y_test,predictions))