Logistic Regression
By: Devaraj Essampally
Tags: Logistic Regression

It is a classification supervised algorithm. It is used to predict the discrete categorical variables, like such as yes/no, Male/Female and binary outcomes (0 to 1).

Logistic Regression categorized 3 types

Binomial LR : if only two outcomes 0 or 1, spam or harm, yes or no

Multi nominal LR: two or more categorical variables with out ordering

Ex: veg, non veg, vegan.

Ordinal LR : There are two or more categorical variables with ordering

Ex: move rating 1 to 5.

Logistic regression used to describe data and to explain the relationship b/w one dependent binary variables and one or more nominal, ordinal, interval independent variables.

Ex: predict the weather email is spam or not

Ex: predict weather the tumor is malignant or no

Logistic regression is one of the most popular Machine Learning algorithms, which comes under the Supervised Learning technique. It is used for predicting the categorical dependent variable using a given set of independent variables. Therefore the outcome must be a categorical or discrete value. It can be either Yes or No, 0 or 1, true or False, etc. but instead of giving the exact value as 0 and 1, it gives the probabilistic values which lie between 0 and 1. It much similar to the Linear Regression except that how they are used. Linear Regression is used for solving Regression problems, whereas Logistic regression is used for solving the classification problems.

In Logistic regression, instead of fitting a regression line, we fit an “S” shaped sigmoid function, this will transform the hole value b/w the 0 to 1. The curve from the logistic function indicates the likelihood of something such as whether the cells are cancerous or not, a mouse is obese or not based on its weight, etc. It is a significant machine learning algorithm because it has the ability to provide probabilities and classify new data using continuous and discrete datasets, and it can be used to classify the observations using different types of data and can easily determine the most effective variables used for the classification.

Sigmoid Function:

The sigmoid function is a mathematical function used to map the predicted values to probabilities. It maps any real value into another value within a range of 0 and 1, those values which cannot go beyond this limit, so it forms a curve like the “S” form. The S-form curve is called the Sigmoid function or the logistic function. Hear we use the concept of the threshold value, which defines the probability of either 0 or 1. Such as values above the threshold value tends to 1, and a value below the threshold values tends to 0.

Mathematics Behind the Logistic Regression:

In Linear Regression the best file line is y=mx+c, to passing the two orgin it will make the slope value is 0.

so the equation is y=wTx. In linear algebra the strait line/plane passing two d/f points, the distance between the plane we calculate the equation is a unit vector of ‘w’.

so whenever we are trying to find out the distance int he equation y=wT(transpose)x above of the plane is always positive because the slope of derivative is always positive(+v)

so case-I, case-II and case-III says, the cost function of the sum of the all the points should be maximum, based on the cost function we have to focus on best fit line.

so hear yi, xi given the parameters we need to update the ‘wi’ value. This ‘wi’ is coefficient, until maximum summation of all the points that will give best fit line and linearly classified correctly.

for example we have a outlier in data points, we will convert to small value. to prevent the outliers we apply sigmoid function(f)

so after calculate the sum of maximum values to apply the sigmoid function, it will transform the whole value between 0 to 1.

Assumptions for Logistic Regression:

The dependent variable must be categorical in nature.

The independent variable should not have multi-collinearity.

Steps in Logistic Regression using Python:

To implement the Logistic Regression using Python, we will use the same steps as we have done in previous topics of Regression. Below are the steps: