Linear Regression Clearly Explained (Part 1)
By: Ashish Mehta
Tags: Data Science, Linear Regression, Machine Learning, Artificial Intelligence, Algorithms

Linear Regression Clearly Explained (Part 1)

Linear regression has been around since 1911. It is one of the core pillars of the data science and machine learning domain and is widely used in the industry to date.

This is Part 1 of the Linear regression series.

The main objective of this blog is to highlight the need for linear regression and to identify the domains where linear regression is used to solve real-world problems.

What is Regression?

Regression is a supervised machine learning technique in which, the target output is also provided along with the input data.

In the case of regression algorithms, the target output or the dependent variable is continuous in nature. For example, measures like height, weight, length, price, etc. all fall under continuous values.

“Regression analysis is a form of predictive modeling technique which investigates the relationship between a dependent variable and an independent variable.”

Regression analysis involves graphing a line over a set of data points that most closely fits the overall shape of the data.

A regression shows the changes in a dependent variable on the y-axis to the changes in the independent variable on the x-axis.

Basically, regression is the task of predicting a continuous quantity.

Where do we use Regression analysis?

There are three major uses of regression analysis:

1. Determining the strength of the relationship between the independent and the dependent variable

Regression can be used to determine the strength of the effect that the independent variable has on the dependent variable.

For example, we can ask questions like:

a. What is the strength of the relationship between sales and marketing spending?

b. What is the relationship between the age and the income of a person?

2. Forecasting an effect

In this case, the regression can be used to forecast the effects or impact of changes. That is, the regression analysis helps us to understand how much the dependent variable changes with the change in one or more independent variables.

For example, you can ask questions like:

a. How much additional sale income will I get for each $1000 spent on marketing?

3. Trend forecasting

In this, the regression analysis predicts trends and future values. The regression analysis can be used to get point estimates.

For example, you can ask questions like:

a. What will be the price of the Bitcoin in the next six months?

b. What will be the stock price of Apple after one year?

What Is Linear Regression?

By definition, linear regression is a statistical way of measuring the relationship between variables.

For example, let’s assume that you are a fitness researcher and the task in your hand today is to measure the height and the weight of 10 volunteers standing in front of you.

You create a table with “Height” and “Weight” as two columns and start entering the data into that table.

Linear regression attempts to measure a correlation between the independent variable and the dependent variable.

In our example, let the weight be the dependent variable and let the height be the independent variable.

In other words, linear regression finds the relationship between the independent and the dependent variable, and based on that, it can predict the future value of the dependent variable.

So for instance, as the height increases, it is more likely that the weight will also increase. This is the relation that is established from the “Height-Weight” data after applying linear regression.

So once you apply linear regression on your “Height-Weight” data, you can then predict the weight of a person if we know his height.

Some Real-world Problems That Can Be Solved Using Linear Regression

1. Evaluating trends and sales estimates

Linear regression can be used in businesses to evaluate trends and make estimates or forecasts.

For example, if a company’s sales have increased steadily every month for the past few years, then conducting a linear analysis on the sales data with monthly sales on the y-axis and time on the x-axis will give us a line that predicts the upward trends in the sale. After creating the trend line, the company could use the slope of the line to forecast sales in future months.

2. Analyzing the impact of price changes

Linear regression can be used to analyze the effect of pricing on consumer behavior.

For example, if a company changes the price on a certain product several times, then it can record the quantity sold for each price level, and then perform a linear regression with the sold quantity as the dependent variable and price as the independent variable. This would result in a line that depicts the extent to which the customer reduced their consumption of the product as the prices were increased. So this result would help us in future pricing decisions.

3. Assessment of risk in financial services and insurance domain

The linear regression can be used to analyze the risk.

For example, a health insurance company might conduct a linear regression algorithm on its data. How will they achieve it? They can do it by plotting the number of claims per customer on the y-axis against the age of the customer on the x-axis. Using this, they might discover which customer will be making more insurance claims in the future.

The result of such an analysis might guide important business decisions.

The Selection Criteria For Linear Regression

There are tones of machine learning algorithms out there that can be used to solve a business problem. Now, the question arises, when should we choose linear regression over others?

If the problem at hand fulfills the following set of criteria, then we can use linear regression to solve that business problem.

1. Classification and Regression capabilities

The regression model predicts a continuous variable such as the sales made on a day, or predict the temperature of a city. Their reliance on a polynomial like a straight line to fit a data set possesses a real challenge when it comes to building a classification capability. Hence linear regression cannot be used to solve a classification problem. The dependent variable has to be continuous in nature.

2. Data Quality

Each missing value removes one data point that could optimize the regression. In simple linear regression, the outliers can significantly disrupt the outcome of the prediction. So if we remove the outliers, the regression model performs better.

3. Computational Complexity

The linear regression is often not computationally expensive as compared to the Decision tree or the clustering algorithm. So if we don't have a high computational power to work with, then linear regression becomes our go-to algorithm.

4. Comprehensible and Transparent

The Linear regression models are easily comprehensible and transparent in nature. They can be represented by a simple mathematical notation to anyone and can be understood very easily.

So these are some of the criteria based on which you will select the linear regression algorithm.

Conclusion

Though it is one of the earliest machine learning algorithms, linear regression is a very powerful tool that is still used to solve real-world problems.

Having a good understanding of linear regression is very important and we are trying to cover all the aspects of the algorithm through these blogs.

This article was Part 1 of Linear regression from Scratch. In the next part, we will cover the underlying working and mathematics of linear regression and how it can be used to predict future values.