Interpreting the Root Mean Squared Error of a Linear Regression Model
By: Tia Plagata
Tags: Linear Regression Python, Mean Squared Error, Linear Regression, Root Mean Squared Error, Model Interpretation

The first time I ever built a Linear Regression model, I thought two things:

Wow! I built something that can actually predict housing prices!

Ok, but how good are these predictions?

I had learned to check all of the assumptions of a Linear Regression model (residuals should have a normal distribution, features are linearly correlated with the target, there’s no multi-collinearity, etc.). I learned to scale and sometimes even log-scale my features and target. I even learned about mean squared error (MSE) and root mean squared error (RMSE) to interpret the residuals of the model. The problem was, once I scaled my features and calculated the MSE, that number was scaled too! I wasn’t sure how to read it accurately.

As a refresher, Mean Squared Error is the average of the squared difference between each predicted point and the actual point. It is generally a very big number because we square each difference to get rid of negative numbers before taking the average. Therefore, we sometimes use Root Mean Squared Error to bring that number back down to scale (RMSE is the square root of MSE).

For example, if the target variable is in USD (i.e. Sale Price of a home), what does a root mean squared error of 0.3 really tell you? Is that big or small? Maybe if you scaled everything from 0 to 1 you could argue that it’s small, but what if you used a different scaling technique? Moreover, one of the most important skills as a Data Scientist is being able to connect these metrics back to real-world applications and interpret them through that lens.

Photo by Evelyn Paris via Unsplash

Let’s say I created an app that could predict your home price. As a homeowner, you might see this predicted home price and feel inclined to sell. Now, what if I predicted that your home was going to sell for $350,000. Maybe you get excited and decide to sell your home. When you sell your home, you then realized that it would actually only sell for $250,000, which is less than you paid for the home. My model’s prediction was $100,000 off. For home prices, that is a lot but how would you know that if the RMSE was 0.3?

This is why it is so important to interpret your Mean Squared Error and Root Mean Squared Error correctly. Therefore, in my first Linear Regression project, I reversed my log and normalization scaling so that my MSE and RMSE were back in USD. In this way, I was able to correctly interpret the RMSE. Below I will show you how I did this in Python.

Here are the libraries you will need (this does not include the libraries needed to actually build a linear regression model).

Log Transformation & Normalization

In this example, I am building a Linear Regression model to predict housing prices. The target feature here is housing prices, which are typically in USD (or whatever currency you’re working with). In the process of building this model, I decided to log transform price (the target).

Why? In order to make the distribution of this feature more normal, and thus easier to work with. In this example, I will take the log base 10 of all of the continuous variables.

How?

Next, I will scale the target and the other continuous variables to a standard normal scale.

Why? If we do this to the target and our other continuous features, it will allow the beta coefficients of our model to be weighted in a similar way so we can interpret which features are having a greater impact on our target. I will create a function to standard normalize so that we can apply it to our entire log data frame.

How?

Awesome! Now we have a data frame of log-transformed, standard normalized continuous data. I can concatenate this with our categorical variables and run the model based on this. Next, I will fast-forward to the interpretation stage of the model to see how to change the target feature back to USD.