An Easy Tool to Correctly Transform Non-Linear Data For Linear Regression | by Kevin Babitz | Towards Data Science

Two important assumptions for linear regression are that the residuals are independent from each other and that there is constant variance in these errors (also known as homoscedasticity). This basically means that when we draw our regression line, we want a relatively equal number of errors on the top and bottom of our line across the entire line.
There are many cases where variables we would like to build a model for are not linearly correlated. This leads to failures for the above assumptions when fitting linear regression models to them. We may apply transformation functions to these variables in order to make the relationship linear. The following figure is useful in helping us decide what transformation to apply to non-linear data that we are working with.
The goal of this article is to demonstrate how to use this diagram to make transformations to your data. Before we get into examples, the way you can read this diagram is:
If you see data that looks like the curve in the top left, you can try to decrease the power of x and/or increase the power of y.If you see data that looks like the curve in the top right, you can try to increase the power of x and/or increase the power of y.If you see data that looks like the curve in the bottom right, you can try to increase the power of x and/or decrease the power of y.If you see data that looks like the curve in the bottom left, you can try to decrease the power of x and/or decrease the power of y.
If you see data that looks like the curve in the top left, you can try to decrease the power of x and/or increase the power of y.
If you see data that looks like the curve in the top right, you can try to increase the power of x and/or increase the power of y.
If you see data that looks like the curve in the bottom right, you can try to increase the power of x and/or decrease the power of y.
If you see data that looks like the curve in the bottom left, you can try to decrease the power of x and/or decrease the power of y.
Interestingly, there are not a lot of resources on this rule on Google (only a few thousand search results). This is a simple and powerful framework for quickly determining a transformation to use which allows you to potentially fit a linear model on non-linear data.
For this article, we will be using data generated from the following python function:
def random_data(p=1, size=200, noise_sd = .1, q=1):    x = list(np.linspace(0.01, 1, size))    y = [(3 + 5 * x_val ** p + (random.normalvariate(0,noise_sd))) ** (1/q) for x_val in x]       return x, y
The two important variables to understand above are p and q. The variable p represents the power being applied to the x values and q represents the inverse of the power being applied to y.
We are generating size values for x between 0.01 and 1 with equal intervals between each value. We are adding 3 and scaling the y values by 5 in order to avoid negative numbers which can cause issues with the rule we are discussing in this article. Finally, we are adding a bit of noise to the data using random.normalvariate(mean, sd) in order to make the data not perfectly fit the transformations we would like to demonstrate.
Let’s get started with an example of data that would be classified under the bottom right corner of the figure above.
We see that the data does not have a great linear relationship when we attempt to fit a least-squares linear regression line to it.
The diagram tells us to try decreasing the power of y or increasing the power of x. Below we see that increasing the power of x gives us a significantly better linear relationship and fit.
It is important to remember to consider a different transformation can have the same shape. Namely, the data could be generated with an increased power of y and require transforming the y variable for a better linear fit.
In this case, it would be appropriate to decrease the power of y for a better linear fit.
In the above examples, we were able to see how to apply the Tukey and Mosteller Bulging Rule to randomly generated data (specifically conforming to the bottom right corner of the diagram).
It is important to note that we already knew what the correct transformation was because we selected the parameters for the data during the data generation process. In practice, you will have to try the different transformations in the specific quadrant of the diagram to find the best transformation. The diagram is meant to be a helpful guide in quickly determining which ones to try.
Another thing to note is that this simple rule has some limitations. Specifically, it gives a good starting point for the direction to go in when transforming the data, but not the best possible transformation. Consider the Box-Cox transformation for a more complex and accurate framework (although the Tukey and Mosteller rule does a relatively good job as an approximation and this type of precision does not necessarily yield meaningful improvements to your models). Some code in R comparing the two rules can be found here.
Another thing to note is that a lot of these transformations require positive variable values to work. Thus, the rule may not be able to be applied to data with negative values or may require additional work to ensure all values are positive.
Thank you for taking your time to read this article and I hope it helps you in building your next model.