All that glitters is not gold
By: Rohit Tolawat
Tags: Heteroscadesticity, Linear Regression, Quadratic Regression, Crossvalidation, Machine Learning

Let’s just admit it, we love linear regression! A magical straight line penetrating a cloud of data points is all that we desire. We spot a linear model with low root-mean-squared error (RMSE), high Adjusted R-squared and we claim to have estimated the true nature of the data. I wish things were this simple. Ever heard of the adage, all that glitters is not gold? Well, I intend to underscore the same.

All we do in this post is to predict the amount of power generated (in megawatts) based on the wind speed (in meters per second). Before we dive deep into the data, let’s wear our thinking hats for a while. How would power generated by a windmill vary based on the wind speed? Are we thinking linear, are we thinking exponential? Aren’t we alike? Hold on to your thoughts, please.

Let me randomly split the data into 2 sections, training data (70%) — where we build our model and test data(30%) — where we evaluate the model built. The dataset is accessible at the link below:

https://www.tableau.com/learn/tutorials/on-demand/trend-lines?playlist=484037

Variables of importance in our case are power output and wind speed

Summary statistics for the training data is as below:

Power generated varies from 0–30 MW, Wind speed, on the other hand, varies from 1.10–18.32 ms

Some basic visualizations on data confirm of a high correlation between the response and the predictor variable, as below:

A correlation coefficient of 0.98, just as we thought (or didn’t).

Before I predict anything, let me establish my baseline model. If I had no information on the wind speed, I would use the average power generated to be my best estimator, as below:

I know this would be a pathetic model, but hey I call this baseline for a reason!

Let’s run our favorite model after baseline — Linear regression.

That’s what I am talking about, Adjusted R-squared of 0.955

The estimated relation between the variables as predicted by the model is:

Power output = -6.5832 + 2.2507 * (wind speed)

Interpretation time: A unit increase in the wind speed in meters per second increases the power generated by 2.2507 megawatts. Makes sense, right?

Hold on, before you wonder if this is what data analysts are paid for, let me throw a word to justify our existence. Homoscadesticity! Sure, this is a daunting word but is it complicated? Nope! All it means is that the model you build should be consistent and unbiased. Simplifying it further, your model must have randomly spread out residuals across the entire range of predicted values. And how do we check that? Residuals v/s predicted (fitted) values plot. Before we do just that, a refresher on residuals:

Residual = Actual value — Predicted value

A residual > 0, therefore, indicates underprediction by the model. Residual < 0 implies that the model has overpredicted the value and a residual = 0 indicates that actual and predicted values by the model match.

A residual of zero is desired, but almost never attained.

Now that we know what residuals imply, let’s jump onto the residuals v/s fitted plot for the linear model above:

The predictions as seen in the residual v/s fitted plot is inconsistent and exhibits heteroscedasticity

This can also be seen on plotting the model on the scatter plot:

The blue line is the linear regression model built

As one would eyeball, (easier in this case, not possible in multi-dimensional planes) the model overpredicts the power generated for lower wind speeds and under predicts for higher speeds constantly. Could we improve? Yep, let’s give it a shot. Let us look at the distribution of wind speed and power output.

Power output is skewed to the right, so is wind speed. Would log transformation de-clutter something?

Log transforming the wind speed leads to a scatter plot as below:

A quadratic model on the logarithm of wind speed in is trained as below:

Significant predictors, high R-squared of 0.979. Are we overfitting?

The estimated quadratic relation between the response and the predictor variable is as below:

Power output = 7.40 –17.20 * Ln(wind speed) + 9.18 * (Ln(wind speed))²

A significantly high Adjusted R-squared does worry us of an overfitted model that might not perform well when exposed to the test data. Cross-validation to the rescue?

Cross-validating the dataset yields an RMSE between 1 to 1.4 across all the folds created randomly for testing. The model does not seem to overfit.

Let us now have a look at the residual v/s fitted plot in order to check if building the model on logarithmic terms help reduce the heteroscedasticity. The plot is as shown below:

The model does show some improvement

There does seem to be some improvement in the residual v/s fitted plot, but a trend seems to appear as the wind speed increases. Imposing the built model on the scatter is done below:

Predicted values are denoted by the blue line

The quadratic model shows minor improvement based on the residual plot. Must we measure the performance of the models using the RMSE values as we expose the same to the test environment?

From the line chart above, the linear model definitely is an improvement over the baseline. The heteroscedasticity exhibited by the linear model compelled us to estimate the relation between the variables using a quadratic logarithmic model. The quadratic model did seem to improve on the heteroscedasticity front marginally with lower RMSE values. The model, however, does not visibly perform well at higher wind speeds as the power generated plateaus.

Perhaps, there is a better model. Perhaps!

The source code for the project can be accessed using the link below:

https://github.com/rohitTheSupplyChainGuy/all-that-glitters-is-not-gold