Linear Regression: Moneyball — Part 2 | by Sayar Banerjee | Towards Data Science

This post is the sequel to Linear Regression: Moneyball — Part 1. If you haven’t checked that out, I would highly recommend you to do so.
By the end of the prior post, we had established a linear relationship between RD (Runs difference) and W (wins).
We had also established that there was an extremely strong positive correlation of 0.9385 between RD and W. In this post, we will continue to study relationships between the numerical variables and build a few models to predict the Oakland A’s run in 2002.
Let us have a look at the data again.
df = pd.read_csv("baseball.csv")df.head()
Now let us look at the formulas for calculating some of the numerical terms that may seem obscure to most people.
On Base Percentage:
H : Hits
B: Base on balls (Walks)
HBP: Hits by pitch
AB: At bat
SF: Sacrifice fly
Slugging Percentage:
1B, 2B, 3B: singles, doubles, triples
HR: Home Run
AB: At bat
Batting Average:
H: Hits
AB: At bat
We can see that all three of the statistics previously discussed contribute in some way to the number of runs scored by the team. I won’t go into specific details of explaining each variable involved. All you need to know is that OBP, SLG and AVG are strong indicators of runs scored by the particular team.
Let us validate these assumptions by plotting the relationship between each of these variables with RS(Runs scored):
There is a clear linear relationship between each of the variables and runs scored.
# Correlation between On-base percentage and runs scored.print(np.corrcoef(moneyball.OBP,moneyball.RS))output:[[ 1.          0.90490915] [ 0.90490915  1.        ]]# Correlation between Slugging percentage and runs scored.print(np.corrcoef(moneyball.SLG,moneyball.RS))output:[[ 1.          0.92638433] [ 0.92638433  1.        ]]# Correlation between batting average and runs scored.print(np.corrcoef(moneyball.BA,moneyball.RS))output:[[ 1.          0.83162475] [ 0.83162475  1.        ]]
As a rule of thumb, correlation above 0.7 is considered as strong positive correlation. All of our variables satisfy this rule.
Similarly, we can hypothesise that OOBP(Opponent’s OBP) and OSLG(Opponent’s SLG) have a similar relationship with RA(Runs allowed).
Before we see the relationship between these variables and RA, I noticed that there are a few missing values in the OOBP and OSLG columns. We have to remove these rows of observations in order to plot a proper relationship. In a future post I shall explain about the various ways we can ‘impute’ missing data.
moneyballnew = moneyball.dropna()
We store the reduced dataset into a new variable.
Let us now see the relationship between these variables and RA:
We shall now build regression models in order to predict runs scored, runs allowed and wins.
The scikit-learn library in python is extremely powerful in building a vast variety of machine learning models using very few lines of code. We shall utilise this library in order to build our models.
Our first model is used to predict ‘runs scored’. Our independent variables as on-base percentage, slugging percentage, batting average.
# Extracting our variables from the dataframe.x = moneyball[['OBP','SLG','BA']].valuesy = moneyball[['RS']].values# Calling our model object.RS_model = LinearRegression()# Fitting the model.RS_model.fit(x,y)# Printing model intercept and coefficients.print(RS_model.intercept_)print(RS_model.coef_)Output:[-788.45704708][[ 2917.42140821  1637.92766577  -368.96606009]]
Thus, our model is of the following form:
RS = -788.46+2917.42×(OBP)+1637.93×(SLG)-368.97×(BA)
At this point, we notice something odd. Our model has a variable with a negative coefficient a.k.a Batting Average. This is counter-intuitive as a team with a higher batting average should have more runs scored. This situation represents a phenomenon known as multi-collinearity which leads to discrepancies in multiple linear regression models.
In order to avoid multi-collinearity, we must either add an interaction variable or remove the variable causing the discrepancy. For the sake of simplicity let us remove BA from our model.
# Extracting our variables from the dataframe.x = moneyball[['OBP','SLG']].valuesy = moneyball[['RS']].values# Calling our model object.RS_model = LinearRegression()# Fitting the model.RS_model.fit(x,y)# Printing model intercept and coefficients.print(model.intercept_)print(model.coef_)Output:[-804.62706106][[ 2737.76802227  1584.90860546]]
Our refined model is:
RS = -804.63+2737.77×(OBP)+1584.91×(SLG)
Similarly, we build a model for ‘runs allowed’ using opponent’s on-base percentage and opponent’s slugging percentage as independent variables:
# Extracting our variables from the dataframe.x = moneyballnew[['OOBP','OSLG']].valuesy = moneyballnew[['RA']].values# Calling our model object.RA_model = LinearRegression()# Fitting the model.RA_model.fit(x,y)# Printing model intercept and coefficients.print(RA_model.intercept_)print(RA_model.coef_)Output:[-837.37788861][[ 2913.59948582  1514.28595842]]
Model for runs allowed:
RA = -837.38 + 2913.60 ×(OOBP) + 1514.29 ×(OSLG)
Lastly, our model for predicting wins from runs difference:
# Extracting our variables from the dataframe.x = moneyball[['RD']].valuesy = moneyball[['W']].values# Calling our model object.W_model = LinearRegression()# Fitting the model.W_model.fit(x,y)# Printing model intercept and coefficients.print(W_model.intercept_)print(W_model.coef_)Output:[ 80.88137472][[ 0.10576562]]
Model for wins:
W = 80.88 + 0.11 ×(RD)
Here are the Oakland Athletics statistics in 2001 before the playoffs.
OBP: 0.339
SLG: 0.430
OOBP: 0.307
OSLG: 0.373
Lets plug in these values into the above models to generate predictions.
# Prediction for runs scored.RS_model.predict([[0.339,0.430]])Output:array([[ 804.98699884]])# Predictions for runs allowed.RA_model.predict([[0.307,0.373]])Output:array([[ 621.92581602]])
Thus, our models predicted the following:
RS ≈ 805
RA ≈ 622
This implies that RD = 183.
Finally, we plug in RD into our wins model:
# Prediction for wins.W_model.predict([[183]])Output:array([[ 100.23648363]])
Comparing our model results with the actual outcome and De Podesta’s estimates we get:
Quite remarkable isn’t it? Our model’s predictions actually turned out to be fairly accurate as compared to reality.
We finally conclude our case study of linear regression applied to Sabermetrics. An important point to note is that during this case study I made a lot of assumptions about our regression models. In a future post, I will discuss in depth about regression assumptions and how to avoid violating them.
I hope all of you enjoyed this two-part blog post on linear regression. If you liked my content, I would greatly appreciate a clap(s) and a follow. The entire code is available here.
I look forward to questions, comments, concerns. Until next time folks. ✌🏽