Linear Regression: Moneyball — Part 1 | by Sayar Banerjee | Towards Data Science

One of the most compelling stories of sports analytics made popular by the 2011 movie Moneyball, the Oakland Athletics team of 2002 created history by winning 20 consecutive games between August 13 and September 2002.
Much of the Oakland Athletics (popularly referred to as the A’s) success in that season is attributed to their General Manager, Billy Beane and former Assistant General Manager, Paul DePodesta.
DePodesta, a graduate in Economics from Harvard University joined the A’s in 1999 and quickly started incorporating analysis of baseball statistics to value and purchase players.(Sabermetrics)
This approach came into the limelight following the historical 2002 season when Billy Beane purchased vastly undervalued players to supplant the departure of key players prior to the start of the season. Due to severe budgetary constraints of the Oakland Athletics, Billy resorted to the methods being researched by DePodesta in order to purchase underrated players to meet the California based team’s payroll.
This article attempts to recreate portions of Paul DePodesta’s analysis in order to demonstrate the power of the simplest supervised learning technique a.k.a Linear Regression.
This analysis is very similar to the case study in MIT’s Analytics Edge course on edX. However, my programming language of choice for the article is Python as opposed to R which was used in the aforementioned course.
Linear regression is a supervised learning algorithm in machine learning that had it’s origins from statistical principles. It is primarily used to model the relationship between an explanatory variable usually y, with one or more independent variables denoted by X. When there is only one independent variable used to predict y, it is known as Simple Linear Regression or Linear Regression whereas when there are multiple independent variables, it is known as Multiple Linear Regression.
The β terms called regression coefficients refer to the relationship between the x variable and the dependent variable y. Let us use an example to illustrate this.
Equation:
Price of house in $= 50000+1.35×(Size of house in sqft)+ ε
The above equation is an example of a regression function used to determine price of houses given their size in square feet.
1.35 is the value of β which denotes that for every 1 sqft increase in the size of the house , there is a $1.35 increase in the price of the house. If the value of β were -1.35 then for every 1 sqft increase in house size, there would be a $1.35 unit decrease in the price of the house.
The error term ε is used to denote the residuals or the difference between the actual values and predicted values of Y. It tells us how far off our predictions are from the actual value.
Residuals formula:
ε = y — ŷ
where y denotes the actual values and ŷ denotes the value of y predicted by our model.
I will not go into the explicit details of how the β values are calculated, how the model is optimised. Instead, lets jump right into the model building framework.
The dataset I have used is the Moneyball dataset from Kaggle which was gathered from baseball-reference.com.
The data is read into python using the pandas library.
df = pd.read_csv("baseball.csv")df.head()
We need to define a few of the terms here:
RA stands for runs allowed.
RS stand for runs scored.
OBP stands for On Base Percentage.
SLG stand for Slugging Percentage.
BA is Batting Average.
OOBP is opponent’s OBP.
OSLG is opponent’s SLG.
W stands for number of wins in that season.
A new variable RD is added as the difference between RS and RA.
Additionally, we shall only keep records with Year < 2002 in order to replicate the data prior to the 2002 season.
We will use plots in order to find insights regarding the distributions of the numerical variables.
Observe that all four features(variables) are approximately normally distributed. Both, the runs scored and runs allowed graph seems to be skewed slightly towards the left. The tails of the runs scored graph seems to be thicker than that of the runs allowed graph.
According to DePodesta’s character in the clip, the A’s required at least 99 wins in order to make the playoffs in 2002. Lets try to visualise this insight.
# Plotting scatterplotplt.figure(figsize=(10,8))plt.scatter(x=moneyball0.W,y = moneyball0.RS,c="red")plt.scatter(x=moneyball1.W,y = moneyball1.RS,c="blue")plt.xlabel("Wins")plt.ylabel("Runs Scored")plt.axvline(x = 99)
From the above plot we see that DePodesta’s estimate of 99 wins makes sense since there appears to be only 3 observations (red) who haven’t made it to the playoffs with ≥ 99 wins.
DePodesta also calculated that the A’s need to score atleast 814 runs and allow only 645 runs in order to make it to the postseason. That meant that the difference between runs scored and runs allowed needed to be approximately 169(RD). Let us look at the relationship between RD and Wins.
x = np.array(moneyball.RD)y = np.array(moneyball.W)# Deriving slope,intercept valuesslope, intercept = np.polyfit(x, y, 1)abline_values = [slope * i + intercept for i in x]#Plotting the figureplt.figure(figsize=(10,8))plt.scatter(x, y)plt.plot(x, abline_values, 'b')plt.title("Slope = %s" % (slope))plt.xlabel("Run Difference")plt.ylabel("Wins")plt.show()
We can clearly see that the relationship between the two variables is linear.
print(np.corrcoef(x,y))output:[[ 1.          0.93851498] [ 0.93851498  1.        ]]
Additionally, there is a strong correlation of 0.9385 between RD and number of wins.
Thus, our exploratory data analysis confirms that DePodesta’s estimates have been spot on so far. In the next part of the study we shall build regression models to further validate DePodesta’s analysis.
Stay tuned for Part 2.