Logistic Regression Explained
By: Jaime Zornoza
Tags: Machine Learning, Startup, Artificial Intelligence, Data Science, Programming

Logistic Regression Explained

[ — Logistic Regression explained simply — ]

In this post, I will explain Logistic Regression in simple terms. It could be considered a Logistic Regression for dummies post, however, I’ve never really liked that expression.

Before we start, here you have some additional resources to skyrocket your Machine Learning career:

Lets get to it and learn it all about Logistic Regression.

Logistic Regression Explained for Beginners

In the Machine Learning world, Logistic Regression is a kind of parametric classification model, despite having the word ‘regression’ in its name.

This means that logistic regression models are models that have a certain fixed number of parameters that depend on the number of input features, and they output categorical prediction, like for example if a plant belongs to a certain species or not.

In reality, the theory behind Logistic Regression is very similar to the one from Linear Regression, so if you don’t know what Linear Regression is, take 5 minutes to read this super easy guide:

In Logistic Regression, we don’t directly fit a straight line to our data like in linear regression. Instead, we fit a S shaped curve, called Sigmoid, to our observations.

Sigmoid function fitted to some data

Let's examine this figure closely.

First of all, like we said before, Logistic Regression models are classification models; specifically binary classification models (they can only be used to distinguish between 2 different categories — like if a person is obese or not given its weight, or if a house is big or small given its size). This means that our data has two kinds of observations (Category 1 and Category 2 observations) like we can observe in the figure.

Note: This is a very simple example of Logistic Regression, in practice much harder problems can be solved using these models, using a wide range of features and not just a single one.

Secondly, as we can see, the Y-axis goes from 0 to 1. This is because the sigmoid function always takes as maximum and minimum these two values, and this fits very well our goal of classifying samples in two different categories. By computing the sigmoid function of X (that is a weighted sum of the input features, just like in Linear Regression), we get a probability (between 0 and 1 obviously) of an observation belonging to one of the two categories.

The formula for the sigmoid function is the following:

If we wanted to predict if a person was obese or not given their weight, we would first compute a weighted sum of their weight (sorry for the lexical redundancy) and then input this into the sigmoid function:

1) Calculate weighted sum of inputs

Weighted sum of the input features (feature in this case)

2) Calculate the probability of Obese

Use of the sigmoid equation for this calculation

Alright, this looks cool and all, but isn’t this meant to be a Machine Learning model? How do we train it? That is a good question. There are multiple ways to train a Logistic Regression model (fit the S shaped line to our data). We can use an iterative optimisation algorithm like Gradient Descent to calculate the parameters of the model (the weights) or we can use probabilistic methods like Maximum likelihood.

If you don’t know what any of these are, Gradient Descent was explained in the Linear Regression post, and an explanation of Maximum Likelihood for Machine Learning can be found here:

Once we have used one of these methods to train our model, we are ready to make some predictions. Let's see an example of how the process of training a Logistic Regression model and using it to make predictions would go:

First, we would collect a Dataset of patients who have and who have not been diagnosed as obese, along with their corresponding weights. After this, we would train our model, to fit our S shape line to the data and obtain the parameters of the model. After training using Maximum Likelihood, we got the following parameters:

Parameters and equation of X

3. Now, we are ready to make some predictions: imagine we got two patients; one is 120 kg and one is 60 kg. Let's see what happens when we plug these numbers into the model:

Results of using the fitted model to predict obesity given patient weight

As we can see, the first patient (60 kg) has a very low probability of being obese, however, the second one (120 kg) has a very high one.

Logistic Regression results for the previous examples.

In the previous figure, we can see the results given by the Logistic Regression model for the discussed examples. Now, given the weight of any patient, we could calculate their probability of being obese, and give our doctors a quick first round of information!

Conclusion and Other resources

Logistic regression is one of the most simple Machine Learning models. They are easy to understand, interpretable, and can give pretty good results. The goal of this post was to provide an easy way to understand logistic regression in a non-mathematical manner for people who are not Machine Learning practitioners, so if you want to go deeper, or are looking for a more profound of mathematical explanation, take a look at the following video, it explains very well everything we have mentioned in this post.

That is all, I hope you liked the post. Feel free to follow me on Twitter at @jaimezorno. Also, you can take a look at my posts on Data Science and Machine Learning here. Have a good read!

For further resources on Machine Learning and Data Science check out the following repository: How to Learn Machine Learning! For career resources (jobs, events, skill tests) go to AIgents.co — A career community for Data Scientists & Machine Learning Engineers.

Also, for more posts like this one follow me on Medium, and stay tuned!

Also, to go further into Logistic Regression and Machine Learning in general, take a look at the book described in the following article: