Evaluation of Linear Regression Models
By: Jagandeep Singh
Tags: Machine Learning, Data Science, Linear Regression

Linear regression models are used to show or predict the relationship between two variables or factors. The factor that is being predicted is called the dependent variable and the factors that are used to predict the value of the dependent variable are called independent variables.

Evaluating a machine learning model is as important as building it. We are creating models to perform on new and unseen data. Hence, we need to evaluate if our model is performing correctly. Evaluating a Linear Regression model is not easy because there are a lot of evaluation metrics. When to use which metric depends on the data and problem of the project.

In this post, I will go over some evaluation metrics for Regression models.

R Squared(R²)

R-squared is a goodness of fit measure for linear regression models. This indicates the percentage of the variance in the dependent that the independent variables explain collectively. R-squared measures the strength of the relationship between the model and the dependent variable. R Squared value is between 0 to 1 and a bigger value indicates a better fit between prediction and actual value. Here is the formula for R-squared and the calculation of R² with sci-kit Learn is the following:

from sklearn.metrics import r2_score

true = [3, 4.5, 5, 6, 10]

preds = [3.1, 5, 3.5, 5.9, 8]

r2_score(true, preds)

Mean Absolute Error(MAE)

Mean Absolute Error is a measure of errors between observations and predictions. It is the average magnitude of the errors in a set of predictions, without considering their directions. It is the absolute value of error between actual and predicted value. Following is the formula and way to calculate with sci-kit learn.

from sklearn.metrics import mean_absolute_error

mean_absolute_error(true, preds)

Mean Squared Error(MSE)

Mean Squared Error is the sum of the square of prediction error. Mean Squared Error is similar to Mean Absolute Error. Mean Absolute Error takes the absolute value of error but Mean Squared Error takes the square of error. MSE penalize big prediction error by square while MAE treats all the errors the same.

from sklearn.metrics import mean_squared_error

mean_squared_error(true, preds)

Root Mean Squared Error(RMSE)

Root Mean Squared Error is the square root of the mean squared error. RMSE is always non-negative and a value of 0 would indicate a perfect fit to the data. Since the errors are squared before they are averaged, the RMSE gives a relatively high weight to large errors. Following is the formula of RMSE and how to calculate RMSE in python.

from sklearn.metrics import mean_squared_error

math.sqrt(mean_squared_error(true, preds))

Mean Absolute Percentage Error(MAPE)

Mean Absolute Percentage Error measures the accuracy as a percentage and can be calculated as the average absolute percent error for each time period minus actual values divided by actual values.

import numpy as np



def mape(actual, pred):

actual, pred = np.array(actual), np.array(pred)

return np.mean(np.abs((actual - pred) / actual)) * 100 mape(true, preds)

Conclusion

R Squared indicates the percentage of the variance in the dependent that the independent variables explain collectively. In other words, it explains how good the model is. But all other measures like MAE, MSE, RMSE, and MAPE measures how far the predictions are from the actual values.

There is no optimal metric to use for all tasks. It is important to consider what question you are trying to answer when deciding what evaluation metric to use. You have to decide which evaluation metric to use depending on your data and your problem.

Gain Access to Expert View — Subscribe to DDI Intel