Predicting MLB Pitch Probability Based on the Game Situation | by The Data Detective | Towards Data Science

Baseball is a wonderful sport for a number person. It was one of the first sports that come to mind regarding the use of data to make decisions at just about every level. Managers use data to make in-game decisions and General Managers use data to determine contract payouts.
Baseball is also one of the first sports to incorporate statistics into its televised content. Real-time pitch speed, pitch location, scoring probability and probability of stealing a base. These are just a few of the things that viewers can absorb while watching the national pastime. With the recent incorporation of AI into the mix, more predictive and real-time content is available.
As a fan of the game, I wanted to deep dive into an area that as of yet is not a part of this content, but will in the future: predicting what pitch will come next based on the game situation. I’m pretty excited about the initial results.
First, the data. Kaggle provides a dataset with all pitch data from the 2015–2018 seasons at https://www.kaggle.com/pschale/mlb-pitch-data-20152018 . The data comes in several .csv files including pitches, atbats, games, ejections and player names. For the purposes of our goal, only the atbats and pitches datasets were utilized.
Here are the packages loaded into the notebook
import pandas as pdimport numpy as npimport matplotlib.pyplot as pltimport seaborn as snsfrom sklearn.preprocessing import RobustScaler, LabelEncoderfrom sklearn.model_selection import train_test_split, cross_val_scorefrom sklearn.metrics import accuracy_score, f1_score, confusion_matrix, classification_report, precision_scoreimport xgboost as xgbfrom keras.utils.np_utils import to_categoricalimport randomfrom keras import models, layers, optimizers, regularizersfrom keras.layers import Embedding, GRU, Dropout, GlobalMaxPool1D, Densefrom sklearn.model_selection import GridSearchCV
After reading in the data and merging the two datasets together (on ab_id), the first obstacle was the amount of data to process. From 2015–2018, over 2.8 million pitches were recorded. For the purposes of computational expense and most relevant data, the dataset was reduced to reflect only the 2018 season.
#isolating the data that only contains the 2018 seasondata18 = data[data['year'] == '2018']
This still provides over 700k observations. Now, there are over 40 columns and many of those contain pitch speed, trajectories and location information. These will do us no good in predicting the next pitch as at that point, the pitch has already been thrown. What concerns us is all the information that could contribute to what pitch may be thrown, so we will only keep the information we need:
pitch_type — Type of pitch. This is our target
b_score — score for the batter’s team
b_count — balls in the current count
s_count — strikes in the current count
outs — number of outs (before pitch is thrown)
pitch_num — pitch number (of at-bat)
on_1b — True if there’s a runner on first, False if empty
on_2b — True if there’s a runner on second, False if empty
on_3b — True if there’s a runner on third, False if empty
batter_id — player ID of the batter. Given by MLB, player names found in player_names.csv
inning — inning number
p_score — score for the pitcher’s team
p_throws — which hand pitcher throws with. Single character, R or L
pitcher_id — player ID of the pitcher.
stand — which side batter hits on. Single character, R or L
topTrue — if it’s the top of the inning, False if it’s the bottom
#new dataframe with what is initially determined as situational + the target (pitch_type)pXs = pd.DataFrame(data18[['pitch_type', 'b_score', 's_count', 'outs', 'pitch_num', 'on_1b', 'on_2b', 'on_3b', 'batter_id', 'inning', 'p_score', 'p_throws','pitcher_id', 'stand', 'top', 'b_count']])
These are the pitch types that we will be targeting:
CH — Changeup
CU — Curveball
FC — Cutter
FF — Four-seam Fastball
FS — Splitter
FT — Two-seam Fastball
KC — Knuckle curve
KN — Knuckeball
SI — Sinker
SL — Slider
There are other types of pitches in the dataset, but represent less than .1% of the total observations (pitchouts, screwballs or the rare ephesus pitch). Those were removed from the dataset, the boolean and binary objects converted to integers and nulls removed (represents less than 1% of total data).
Taking a look at the data itself, outliers in the data represented inning counts and scores. This would be typical as extra innings do happen, but not as often as does high scoring instances. Scores and innings do relate to pitches, so removing them may have a negative effect on the results, so those outliers are left in.
Checking for multicollinearity, pitch number was found to be highly positively correlated to ball count and strike count., so it was removed from the dataset.
At this point, several test models and trials were performed, but at this point, a hypertuned XGBoostClassifier provided the best results for our data.
X_train, X_test, y_train, y_test = train_test_split(features2, target2, test_size=0.30)#create xgb instance with parametersclf = xgb.XGBClassifier(learning_rate=0.1,max_depth=10,min_child_weight=15,n_estimators=250)#fit dataclf.fit(X_train, y_train)#make predictionstraining_preds = clf.predict(X_train)val_preds = clf.predict(X_test)#run classification report and confusion matrixgbt_confusion_matrix = confusion_matrix(y_test, val_preds)print(gbt_confusion_matrix)gbt_classification_report = classification_report(y_test, val_preds)print(gbt_classification_report)
This is an outright classification, so getting 51% overall accuracy over 11 pitch types is good. Translating this to an actual game however, getting a pitch right only half the time is not optimal for live consumption. It was decided at this point that perhaps predicting the probability of each pitch would be more beneficial. Fortunately, XGBoost does have a probability option for its package just by switching the objective to multi:softprob.
le = LabelEncoder()le.fit(target3)target_num = le.transform(target3)# list(le.inverse_transform(target_num)) #If you wish to retrieve the original descriptive labels post productionX_train, X_test, y_train, y_test = train_test_split(features3, target_num, test_size=0.3, random_state=42)# use DMatrix for xgboostdtrain = xgb.DMatrix(X_train, label=y_train)dtest = xgb.DMatrix(X_test, label=y_test)fit = xgb.XGBClassifier(objective = 'multi:softprob')fit.fit(X_train,y_train)# set xgboost paramsparam = {'max_depth': 10,  # the maximum depth of each tree'learning_rate': 0.1,  # the training step for each iteration'min_child_weight': 20,'n_estimators': 300,'objective': 'multi:softprob',  # error evaluation for multiclass training'num_class': 10}  # the number of classes that exist in this datsetnum_rounds=30#------------- numpy array ------------------#training and testing - numpy matricesbst = xgb.train(param, dtrain,num_rounds)preds = bst.predict(dtest)#extracting most confident predictionsbest_preds = np.asarray([np.argmax(line) for line in preds])print ("Numpy array test precision:", precision_score(y_test, best_preds, average='macro'))
We converted our dataframe to a numpy array to calculate, so in order to visualize results, it was necessary to convert the data back to a dataframe and re-label the columns to create a visual using matplotlib.
#creating a dataframe for plotting purposesplot = pd.DataFrame(data=preds[:,:], columns=['Changeup', 'Curveball', 'Cutter', 'Fastball_4S', 'Splitter', 'Fastball_2S', 'Knucklecurve', 'Knuckleball', 'Sinker', 'Slider'])#visualization function    def pitch_pred(data):    print(X_test.iloc[data])    ax = plot.iloc[data].plot(kind='barh',title ="Pitch Prediction",     figsize=(15, 10), fontsize=20)    ax.set_xlabel("Percent Chance", fontsize=24)    ax.set_ylabel("Pitchtype", fontsize=24)    plt.show()#visualize observation 6754pitch_pred(6754)
As you can see, it makes more sense to show the possibility of what pitch may come next over an outright classification. Future work includes real-time testing after deploying the model. Also perhaps finding some additional data that could be of use such as stadium and weather. Since the data was enormous, the notebook is available on Google Colab and Github, but I cannot keep the data file in the cloud. The dataset can be downloaded from Kaggle.