Waiting For the Bathroom — a Linear Regression Story
By: Mike Erb
Tags: Statsmodels, Linear Regression, Python, Data Science

Photo by Sidekix Media on Unsplash

I recently completed a project involving a multivariate linear regression to predict housing prices, and guess what…people really don’t like waiting for the bathroom.

That seems pretty obvious, but how much don’t they like to wait? It turns out that they may be willing to pay tens of thousands of dollars to avoid a daily wait for the bathroom for the next 13 years¹.

I’m going to walk through how I came to the above conclusion about how patient Americans are when it comes to the bathroom.

Starting the Linear Regression

I wrote my project in Python using a Jupyter notebook, and stored my data in a Pandas dataframe object. After cleaning my data and splitting it into training and test sets, I proceeded to make a linear regression model. I chose to use Statsmodels because I liked the summary report it produces.

# create a linear regression model for the training set

# statsmodels' function to create a linear regression model doesn't

# add a constant(y-intercept) by default so you will need to call

# sm.add_constant to add one import statsmodels.api as sm X_train_int = sm.add_constant(X_train)

model = sm.OLS(y_train, X_train_int).fit()

display(model.summary())

Dealing with Multicollinearity

I ended up making several models, iteratively improving them. During the process I discovered that a lot of the data in my dataset had multicollinearity issues. I found the issues by calling pandas.DataFrame.corr(), which returns a matrix showing the correlation between all the features, and also when some of my features with a positive correlation with the target had negative coefficients in the linear regression model.

# get a correlation matrix for my training set X_train.corr()

Why is Multicollinearity Bad?

When you data has a lot of multicollinearity, you can’t trust the model. You even may get better R² values or other measures of accuracy, but you won’t be able to trust the coefficients of the independent variables or their p-values. Small changes in the independent variables may result in large changes in the coefficients of the independent variables (which aren’t very independent in this situation), and the model won’t be able to make trustworthy calculations of the p-values of those coefficients.

Mean Centering

I didn’t want to just drop the features with high multicollinearity, they were telling me about the features of the house: bathrooms, bedrooms, square footage, floors, and construction grade and I wanted to keep as many of them as I could.

Mean centering or standardization of the features is a simple way to reduce multicollinearity. I mean centered all the affected features using the scale function from sklearn.preprocessing. To maintain the ease of interpretation of my model I didn’t want to scale the data to unit standard deviation, the default setting, so I had to set the with_std parameter to False. By only subtracting the mean (mean centering) the data, I was able to still interpret the features as if I hadn’t done a transformation; by increasing independent feature a by 1 unit the mean change in the target would be the coefficient of feature a.

from sklearn.preprocessing import scale # standardize the specified columns in the dataframe and give

# them updated column names for col in cols_with_multicollinearity:

scaled_col_name = col + '_scaled'

X_train[scaled_col_name] = scale(X_train[col], with_std=False)

I was in luck, mean centering got rid of most of the multicollinearity issues, but I was was still having trouble keeping both the features for number of bedrooms and number of bathrooms in my model in a way that made sense intuitively, bathrooms had a positive coefficient and bedrooms had a negative coefficient. Adding another bedroom can’t always lower the price of the house, so something else must be going on.

Adding an Interactive Feature

The confusing behavior of the coefficients of the bathroom and bedroom features lead me to try adding an interacting feature for bathrooms and bedrooms, and voila my model was making more sense.

# add a column to the dataframe for the interacting feature X_train['bathrooms_scaled*bedrooms_scaled'] = X_train.bathrooms_scaled * X_train.bedrooms_scaled

Interpreting the Coefficients

When I was done with the modeling process, I ended up with the following coefficients, all with P-Values low enough to make them significant:

bathrooms_scaled 36,470

bedrooms_scaled -4,839

bathrooms_scaled*bedrooms_scaled 11,030

Now that I had the coefficients, how are they interpreted? I have interacting terms and those terms were mean centered. With interacting features involving two terms, there are two cases that need to be interpreted, but I will focus on the case where a bathroom is added. This could mean adding a bathroom to an existing house, or you could think of it as two almost identical houses but one has an additional bedroom.

First I will assign the coefficients so that it is easier to present the formula.

a = 36,470 # the coefficient of bathrooms_scaled

b = -4,839 # the coefficient of bedrooms_scaled

c = 11,030 # the coefficient of bathrooms_scaled*bedrooms_scaled

d = number of bedrooms

e = mean number of bedrooms in the dataset

Increasing by one bedroom would increase the mean price of the house by the result of the following formula.

a + c * (d - e)

Since the features are interacting the amount of the increase in the mean price of the house by adding one bathroom will depend upon how many bedrooms the house has. For example:

d = 3 # number of bedrooms

e = 3.373 # mean number of bedrooms 36,470 + 11,030 * (3 - 3.373) = $32,355.81

So, for a three bedroom house, the model predicts that people would pay over 32 thousand dollars to have the same house but with an additional bathroom.

How Much is Their Time Worth?

Taking the median house tenure of 13 years¹, the home buyer is willing to pay about $6.80 per day to avoid waiting for the bathroom. If there is an average of 20 minutes of waiting for the bathroom a day, totalling 94,900 minutes, the home buyer is valuing their bathroom wait time at $0.34 per minute.

Should I Just Add Bathrooms?

Returning to the predicted additional value to a house by adding a bathroom, can you just add a bunch of bathrooms to make my house more valuable? No. The linear regression model is based upon real houses and the predictions would break down the stranger your house got. Looking at real house data, once your house has more than a 1:1 ratio of bathrooms to bedrooms, the prices start to drop. If you have more than one bathroom for every bedroom, those bathrooms are probably seen as a waste of space and extra maintenance costs.

My Project

Thanks for reading this post, you can check out my entire project and its findings on github here.