Logistic Regression Explained. [ — Logistic Regression explained... | by z_ai | Towards Data Science

In this post, I will explain Logistic Regression in simple terms. It could be considered a Logistic Regression for dummies post, however, I’ve never really liked that expression.
Before we start, here you have some additional resources to skyrocket your Machine Learning career:
Awesome Machine Learning Resources:- For learning resources go to How to Learn Machine Learning! - For professional resources (jobs, events, skill tests) go to AIgents.co — A career community for Data Scientists & Machine Learning Engineers.
z-ai.medium.com
Lets get to it and learn it all about Logistic Regression.
In the Machine Learning world, Logistic Regression is a kind of parametric classification model, despite having the word ‘regression’ in its name.
This means that logistic regression models are models that have a certain fixed number of parameters that depend on the number of input features, and they output categorical prediction, like for example if a plant belongs to a certain species or not.
In reality, the theory behind Logistic Regression is very similar to the one from Linear Regression, so if you don’t know what Linear Regression is, take 5 minutes to read this super easy guide:
towardsdatascience.com
In Logistic Regression, we don’t directly fit a straight line to our data like in linear regression. Instead, we fit a S shaped curve, called Sigmoid, to our observations.
Let's examine this figure closely.
First of all, like we said before, Logistic Regression models are classification models; specifically binary classification models (they can only be used to distinguish between 2 different categories — like if a person is obese or not given its weight, or if a house is big or small given its size). This means that our data has two kinds of observations (Category 1 and Category 2 observations) like we can observe in the figure.
Note: This is a very simple example of Logistic Regression, in practice much harder problems can be solved using these models, using a wide range of features and not just a single one.
Secondly, as we can see, the Y-axis goes from 0 to 1. This is because the sigmoid function always takes as maximum and minimum these two values, and this fits very well our goal of classifying samples in two different categories. By computing the sigmoid function of X (that is a weighted sum of the input features, just like in Linear Regression), we get a probability (between 0 and 1 obviously) of an observation belonging to one of the two categories.
The formula for the sigmoid function is the following:
If we wanted to predict if a person was obese or not given their weight, we would first compute a weighted sum of their weight (sorry for the lexical redundancy) and then input this into the sigmoid function:
Alright, this looks cool and all, but isn’t this meant to be a Machine Learning model? How do we train it? That is a good question. There are multiple ways to train a Logistic Regression model (fit the S shaped line to our data). We can use an iterative optimisation algorithm like Gradient Descent to calculate the parameters of the model (the weights) or we can use probabilistic methods like Maximum likelihood.
If you don’t know what any of these are, Gradient Descent was explained in the Linear Regression post, and an explanation of Maximum Likelihood for Machine Learning can be found here:
towardsdatascience.com
Once we have used one of these methods to train our model, we are ready to make some predictions. Let's see an example of how the process of training a Logistic Regression model and using it to make predictions would go:
First, we would collect a Dataset of patients who have and who have not been diagnosed as obese, along with their corresponding weights.After this, we would train our model, to fit our S shape line to the data and obtain the parameters of the model. After training using Maximum Likelihood, we got the following parameters:
First, we would collect a Dataset of patients who have and who have not been diagnosed as obese, along with their corresponding weights.
After this, we would train our model, to fit our S shape line to the data and obtain the parameters of the model. After training using Maximum Likelihood, we got the following parameters:
3. Now, we are ready to make some predictions: imagine we got two patients; one is 120 kg and one is 60 kg. Let's see what happens when we plug these numbers into the model:
As we can see, the first patient (60 kg) has a very low probability of being obese, however, the second one (120 kg) has a very high one.
In the previous figure, we can see the results given by the Logistic Regression model for the discussed examples. Now, given the weight of any patient, we could calculate their probability of being obese, and give our doctors a quick first round of information!
z-ai.medium.com
Logistic regression is one of the most simple Machine Learning models. They are easy to understand, interpretable, and can give pretty good results. The goal of this post was to provide an easy way to understand logistic regression in a non-mathematical manner for people who are not Machine Learning practitioners, so if you want to go deeper, or are looking for a more profound of mathematical explanation, take a look at the following video, it explains very well everything we have mentioned in this post.
For further resources on Machine Learning and Data Science check out the following repository: How to Learn Machine Learning! For career resources (jobs, events, skill tests) go to AIgents.co — A career community for Data Scientists & Machine Learning Engineers.
Awesome Logistic Regression!
Also, to go further into Logistic Regression and Machine Learning in general, take a look at the book described in the following article: