How Regression Analysis Works. Different forms of regression analysis... | by Mubarak Ganiyu | Towards Data Science

Regression analysis is a machine learning algorithm that can be used to measure how closely related independent variable(s) relate with a dependent variable. An extensive use of regression analysis is building models on datasets that accurately predict the values of the dependent variable.
At the beginning of regression analysis, a dataset can be split into two groups: a training dataset and a testing dataset. The training dataset can be used to create a model to figure out the best approach to apply the line of best fit into the graph. Thus, it can be a straight line or a curve that easily fits into the graph of the independent variable(s) vs the dependent variable.
This newly created model can be used to predict the dependent variable of the testing dataset. Then, predicted values can be compared to the original dependent variable values by using different accuracy measures like R-squared, root mean square error, root mean average error, pearson correlation coefficient and others.
If the accuracy score is not accurate enough and a stronger model wants to be built, the percentage of the datasets allocated to the training and testing datasets can be changed. For instance, if the training dataset had 70 % of the dataset with the testing dataset having 30%, the training dataset can now have 80% of the dataset with the testing dataset having 20%.
Another way of obtaining a stronger model is by changing from linear regression analysis to polynomial regression analysis or from multiple linear regression analysis to multiple polynomial regression analysis.
There are different regression analysis approaches for continuous variables such as Linear Regression, Multiple Linear Regression, Polynomial Regression and Multiple Polynomial Regression.
Linear Regression works by using an independent variable to predict the values of dependent variable.
In linear regression, a line of best fit is used to obtain an equation from the training dataset which can then be used to predict the values of the testing dataset. The equation can be of the form: y = mx + b where y is the predicted value, m is the gradient of the line and b is the point at which the line strikes the y-axis. Below is a set of codes for running linear regression in python:
The code was done based on a dataset of FIFA 19 player ratings. Here, potential is being used to predict players’ overall ratings.
import numpy as np## The dataset is split into two groups below. The training dataset ## had 75% of the dataset and the testing dataset had 25%split = np.random.rand(len(df_fifa)) < 0.75train = df_fifa[split]test = df_fifa[~split]## A regression model is createdregr = linear_model.LinearRegression()## Training and fitting the modelfifa_x_train = train[['Potential']]fifa_y_train = train[['Overall']]regr.fit(fifa_x_train, fifa_y_train)## Predicting the dependent variables in the testing datasetfifa_x_test = test[['Potential']]fifa_y_test = test[['Overall']]df_fifa_pred = regr.predict(fifa_x_test)## Testing the accuracy of the linear model using R-squaredr2_score(fifa_y_test, df_fifa_pred)
Polynomial Regression can use an independent variable to predict the values of dependent variable.
In polynomial regression, a curve of best fit is used to obtain an equation from the training dataset which can then be used to predict the values of the testing dataset.
In the case of polynomial regression the equation can be of the form: y = ax^n + bx^n-1 + ...+ c where y is the predicted value, a and b are constants of the equation, n is the highest degree of the equation which determines the shape of the curve and c is the point at which x is 0. Based on the value of n, the equation could be a second degree, third degree, fourth degree or much more. Below is a set of codes for running polynomial regression in python:
The code was done based on a dataset of FIFA 19 player ratings. Here, potential is being used to predict players’ overall ratings.
import numpy as np## The dataset is split into two groups below. The training dataset ## had 75% of the dataset and the testing dataset had 25%split = np.random.rand(len(df_fifa)) < 0.75train = df_fifa[split]test = df_fifa[~split]## A regression model is createdregr = linear_model.LinearRegression()## Training ,transforming the independent variable into a polynomial ## fit and fitting the model fifa_x_train = train[['Potential']]fifa_y_train = train[['Overall']]poly = PolynomialFeatures(degree=4)fifa_x_train_poly = poly.fit_transform(fifa_x_train)regr.fit(fifa_x_train_poly, fifa_y_train)## Transforming the testing dataset and predicting the dependent ## variables in the testing datasetfifa_x_test = test[['Potential']]fifa_y_test = test[['Overall']]fifa_x_test_poly = poly.fit_transform(fifa_x_test)df_fifa_pred = regr.predict(fifa_x_test_poly)## Testing the accuracy of the polynomial model using R-squaredr2_score(fifa_y_test, df_fifa_pred)
Multiple Linear Regression uses two or more independent variables to predict the values of dependent variable.
In multiple linear regression, multiple lines of best fit is used to obtain a general equation from the training dataset which can then be used to predict the values of the testing dataset. The general equation can be of the form: y = ax+ bx2 +...+ c where y is the predicted value, a and b are the gradients of the lines connecting the independent variables to the dependent variable and c is the point at which the line strikes the y-axis. Below is a set of codes for running multiple linear regression in python:
The code was done based on a dataset of FIFA 19 player ratings. Age, potential, monetary value and release clause are being used to predict players’ overall ratings.
import numpy as np## The dataset is split into two groups below. The training dataset ## had 75% of the dataset and the testing dataset had 25%split = np.random.rand(len(df_fifa)) < 0.75train = df_fifa[split]test = df_fifa[~split]## A regression model is createdregr = linear_model.LinearRegression()## Training and fitting the modelfifa_x_train = train[['Age','Value','Potential','Release Clause']]fifa_y_train = train[['Overall']]regr.fit(fifa_x_train, fifa_y_train)## Predicting the dependent variables in the testing datasetfifa_x_test = test[['Age','Value','Potential','Release Clause']]fifa_y_test = test[['Overall']]df_fifa_pred = regr.predict(fifa_x_test)## Testing the accuracy of the linear model using R-squaredr2_score(fifa_y_test, df_fifa_pred)
Multiple Polynomial Regression is done by using an two or more independent variables to predict the values of dependent variable.
For multiple polynomial regression, multiple curves of best fit is used to obtain a general equation from the training dataset which can then be used to predict the values of the testing dataset.
In the case of multiple polynomial regression, the general equation can be of the form: y = ax^n + bx^n-1 + ...+ c where y is the predicted value, a and b are constants of the equation, n is the highest degree of the equation which determines the shape of the curves and c is the point at which x is 0. Based on the value of n, the equation could be a second degree, third degree, fourth degree or much more. Below is set of codes for running multiple polynomial regression in python:
The code was done based on a dataset of FIFA 19 player ratings. Age, potential, monetary value and release clause are being used to predict players’ overall ratings.
import numpy as np## The dataset is split into two groups below. The training dataset ## had 75% of the dataset and the testing dataset had 25%split = np.random.rand(len(df_fifa)) < 0.75train = df_fifa[split]test = df_fifa[~split]## A regression model is createdregr = linear_model.LinearRegression()## Training ,transforming the independent variable into a polynomial ## fit and fitting the model fifa_x_train = train[['Age','Value','Potential','Release Clause']]fifa_y_train = train[['Overall']]poly = PolynomialFeatures(degree=4)fifa_x_train_poly = poly.fit_transform(fifa_x_train)regr.fit(fifa_x_train_poly, fifa_y_train)## Transforming the testing dataset and predicting the dependent ## variables in the testing datasetfifa_x_test = test[['Age','Value','Potential','Release Clause']]fifa_y_test = test[['Overall']]fifa_x_test_poly = poly.fit_transform(fifa_x_test)df_fifa_pred = regr.predict(fifa_x_test_poly)## Testing the accuracy of the polynomial model using R-squaredr2_score(fifa_y_test, df_fifa_pred)
Regression Analysis is a very interesting machine learning technique that can be applied in different areas to predict numerical values such as predicting the price of a product/house, predicting the number of goals soccer players score in a season and predicting the BMI of people.