Package: pull-the-pitcher
Pull the pitcherDocumentationPredicting when MLB managers in the AL will pull their starting pitchers. ThisDeep, Recurrent Survival Analysismodel is trained to predict the at-bat at which a pitcher is removed from the game, earning an F1-score of 0.97.Installation$pipinstallpull-the-pitcherHow to get the dataTheptplibrary comes with two main command-line utilites. After you installptp, these should be directly available to you at the command-line, assuming you're in the environment thatptpwas installed in.Storing data in asqlite3dbThe first command-line utility isquery_statcast, which invokespybaseball'sstatcast()function to retrieve pitch-level data from statcast. This data will then be stored in asqlite3db file. Here's an example of how you could use it.$query_statcast--start_dt2019-05-07--end_dt2019-06-09--output_typedb--output_path/tmp
Thisisalargequery,itmaytakeamomenttocompleteCompletedsub-queryfrom2019-05-07to2019-05-12
Completedsub-queryfrom2019-05-13to2019-05-18
Completedsub-queryfrom2019-05-19to2019-05-24
Completedsub-queryfrom2019-05-25to2019-05-30
Completedsub-queryfrom2019-05-31to2019-06-05
Completedsub-queryfrom2019-06-06to2019-06-09Preparing data for modelingThe next command-line utility isprep_data_for_modeling, which pulls data from the database created in the previous command, then performs feature engineering and various aggregations to yield clean, at-bat level data amenable to a machine learning model. Here's an example of how you might use it.$prep_data_for_modeling--db_path/tmp/statcast_pitches.db--train_test_split_bystart--output_path/tmp/
queryingdbat/tmp/statcast_pitches.dbnow.
Inthisdataset,thereare457totalgames.
Thereare63'openers'inthedataset.
Thereare851totaleligiblegame-pitchercombinationsinthisdataset.
Justprocessed0thstart.
Justprocessed100thstart.
Justprocessed200thstart.
Justprocessed300thstart.
Justprocessed400thstart.
Justprocessed500thstart.
Justprocessed600thstart.
Justprocessed700thstart.
Justprocessed800thstart.
Thereare91uniquepitcher's in this dataset['2019']datareadyformodelingandsavedat/tmp/.FAQShould any of the data be considered "uncensored"?The great thing about baseball data is that it is comprehensive, clean, and public! So,no, none of the data is "censored"in the survival analysis sense. We know the exact at bat at which every pitcher was removed from the game.If none of the data is uncensored, why are you using survival analysis techniques?Well, the short answer is that they perform the best. Much of survival analysis is dedicated to modeling withboth censored and uncensored data. Since none of our data is cenored, we have free reign to leverage any predictive modeling technique under the sun. Here, however, the process of predicting when a pitcher will be removed from a game fits very nicely in a time-to-event modeling framework, which survival analysis techniques are designed to handle.How does this approach compare to traditional survival analysis?Traditional survival analysisis typically framed as a regression problem, which involvesregressingthe estimated units of time until the event of interest occurs. Alternatively, the approach we employ is framed as a classification problem, and involves predicting theprobabilitythat the event of interest (pitcher removed from the game) occurs at every unit of time (at bat).While this neural network, classification-esque approach is non-traditional, it is not unheard of, as seenhereandhere.How is this Deep, Recurrent Survival Analysis model different from a traditional LSTM?Like other recurrent neural networks, our model predicts the conditional probability that a pitcher was pulled after each at-bat (conditioned on the game that has occurred up to that point). The novelty here occurs in the way that the model "estimates the survival rate through the probability chain rule, which captures the sequential dependency patterns between neighboring at-bats and back-propagates the gradient more efficiently." (quote from page 2 ofthe original paper).This notion of estimation of the survival rate through the probability chain rule is further enforced by the use of theevent timeand theevent rateloss functions. Notice that while our targets are binary,we are not using traditionallog lossto train this model.