Sports Reference API Intro. How to build awesome sports datasets | by Alex Muhr | Towards Data Science

This is the second installment in my weekly sports-themed series. Each week, I demonstrate applications of data science centered around sports.
Last week in my article Beating the Odds I demonstrated how to build a dataset on NBA teams using the Sports Reference API, then used the dataset to find teams which outperformed their regular season point differential. This week I want to go in-depth on how to use this API to build comprehensive datasets for your projects.
When looking around the web at sports data projects, I’ve seen a lot of people resort to downloading tables from disparate sources, either manually or through web scraping. These tables then usually need to be joined, a very tedious process that’s highly dependent on the quality of data. Building a large dataset in this fashion may take days or even weeks. Don’t be that person! Thankfully the amazing engineers at Sports Reference have built an API that facilitates fast and easy access to Sports Reference’s massive databases on the MLB, NBA, NFL, NHL, NCAAF, NCAAB, and the many Football (soccer) leagues across the world.
To begin using the Sports Reference API you will need to install it just like any other python package using pip install sportsreference.
The basic idea of the Sports Reference API is that modules are used to instantiate class objects which contain relevant data in the class properties. The basic modules that are common among all sports are Teams, Schedule, Boxscore, Roster, and Player. NCAAB and NCAAF also have modules for Rankings and Conferences. Football (soccer) uses a slightly different naming convention, but the general API structure is the same. On the backend, these modules create class objects by sending HTTP requests which query the relevant data from Sports Reference’s servers. There’s no need to worry about stuffing your hard drive full of 100 years of Baseball boxscore data (although I suppose you could use the API to do this if you really wanted to).
In the following sections, I’ll show off features of the NBA API. If you’re interested in another sport I encourage you to follow along anyway as the basic features and processes are essentially the same. The code is intended to be executed in a Jupyter Notebook, but could easily be adapted for another environment.
Let’s start by looking at the Teams module. The following commands create an instance of the Teams class and print out all of its methods and properties.
from sportsreference.nba.teams import Teamsteams2020 = Teams(year = '2020')print(dir(teams2020))
The only public property of the Teams class is ‘dataframes’. This property is a 30 x 47 dataframe where every NBA team in 2020 is represented by a row. Columns include things like points, free throw attempts, opponent offensive rebounds, etc. The ‘dataframes’ property is generally the easiest way to access high level team information. However, it’s also possible to use an instance of the Teams class as an iterator. For example, the code below prints out the name and points scored for every team. Every column of the Teams dataframe is also accessible in this way as a team property.
for team in teams2020:    print(team.name, 'Points Scored:', team.points)
The Schedule module, as the name suggests, is used to access information about a team’s schedule. This is very useful as the Teams module does not contain information on win/loss records. Let’s create an instance of the class for the Milwaukee Bucks’ 2019 season and use it to return a dataframe containing basic information for each game.
from sportsreference.nba.schedule import Schedulemil2019 = Schedule('MIL', year = '2019')mil2019.dataframe.head()
Note the boxscore_index column, these boxscore indices can be used with the Boxscore class to obtain much more detailed game information. Similar to the Teams module, an instance of the Schedule class can also be used as an iterator. Lastly, there is a ‘dataframe_extended’ property of the Schedule class which returns a dataframe where every row is an instance of the Boxscore class. This provides richer data on each game but takes much longer to process due to building many instances of the Boxscore class, each requiring a separate server request.
Let’s take a closer look at the Milwaukee Bucks’ 2019 Game 1 in the Eastern Conference Finals against the Toronto Raptors. From our schedule dataframe we can find that the boxscore_index for this game is ‘201905150MIL’.
from sportsreference.nba.boxscore import Boxscoregame_data = Boxscore('201905150MIL')
By calling print(dir(game_data)) we can see that the Boxscore class has many more properties than the Teams and Schedule classes. The command game_data.dataframe will compile these properties into dataframe.
Well, that’s not exactly the type of boxscore we’re used to seeing, but there’s a lot of useful data here nonetheless. In order to construct something more closely resembling a traditional boxscore we’ll need to use the Boxscore class properties ‘home_players’ and ‘away_players’. By executing game_data.home_players we see that this returns a list of BoxscorePlayer class objects. Further, we can execute print(dir(game_data.home_players[0])) to view a list of all the available methods and properties for this subclass. As the BoxscorePlayer class has a ‘dataframe’ property, a more traditional boxscore dataframe can be constructed by concatenating each individual BoxscorePlayer dataframe. The code below does just this.
home_df = game_data.home_players[0].dataframefor player in game_data.home_players[1:]:    home_df = pd.concat([home_df, player.dataframe], axis = 0)home_df['name'] = [x.name for x in game_data.home_players]home_df.set_index('name', inplace = True)home_df
By checking out the columns of this dataframe we can see that not only did we get all the traditional boxscore statistics, we also got a bunch of advanced statistics as well. This wraps up the basic use cases for the Boxscore module.
The Roster module is used to obtain information on a team’s roster of players. Let’s create an instance of the Roster class for the 2007 ‘We Believe’ Warriors.
from sportsreference.nba.roster import Rostergsw2007 = Roster('GSW', year = '2007')
As I’ve shown above, we can check the methods and properties of this class object and see that there is only one property, ‘players’. Much like the ‘home_players’ property we just looked at for the Boxscore module, the ‘players’ property is a list of Player class instances. We’ll go more in-depth on the Player module in a bit. For now, let’s simply print out the name and player id of every player on the roster using the following snippet of code.
for player in gsw2007.players:    print(player.name, ':', player.player_id)
As the Roster class creates an instance of the Player class for each player on the roster, and constructing each instance requires a request to Sports Reference’s servers, it can be quite slow to use the Roster class for a simple task like this. If we wanted to print the name of every player in the league for a given season using this method we might have to wait 5+ minutes to create all the Player class instances. As an alternative, we can call the Roster class using the keyword argument slim = True. This will create an instance of the Roster class where the ‘players’ property is simply a dictionary, where keys are the player_ids and values are names. The code below demonstrates this functionality.
gsw2007slim = Roster('GSW', year = '2007', slim = True)gsw2007slim.players
Not surprisingly, the Player module is used to acquire information on individual players. Referring to the API documentation, we can see that the Player module contains an abstract class, AbstractPlayer, that is inherited by the BoxscorePlayer and Player classes that we’ve already seen in the Boxscore and Roster modules. As such, the proper way to obtain Player data is actually through the Roster and Boxscore modules. Let’s use the player_id that we pulled for Baron Davis to get his stats.
from sportsreference.nba.roster import Playerbaron_davis = Player('davisba01')baron_davis.dataframe
This simple bit of code gives us 89 columns of data for Baron Davis’ career and every one of his seasons. In addition to the ‘dataframe’ property and all of its associated columns, the Player class also contains properties such as ‘birth_date’, ‘height’, ‘weight’, ‘nationality’, and more. Further, a single career statistic, such as points, can be queried with the command baron_davis.points. Similarly, we could query Baron Davis’ total points in the 2006–07 season using the command baron_davis('2007').points. Depending on the use case, it may be easier to access player statistics in this more direct fashion and avoid dealing with a dataframe.
To wrap up this article I’ll demonstrate how to use the Teams, Roster, and Player modules to build a massive dataset of NBA player statistics. This dataset will be used in my next article where I’ll be taking a look at player development, and eventual decline, over the course of an NBA career.
The first step is to import the relevant modules and define a function that creates a player dataframe with a few extra fields that I’ve defined manually. Most important being a player’s age on January 1st of the respective season.
# Function to get player info from Player class object.def get_player_df(player):        # helper function to get player age during each season.    def get_age(year, bd):        if year[0] == "Career":            return None        else:            year_dt = datetime(int(year[0][0:4]) + 1, 1, 1)            age_years = relativedelta(year_dt, bd).years +                         relativedelta(year_dt, bd).months/12            return age_years            # helper function to get year for each row and denote    # rows that contain career totals.    def get_year(ix):        if ix[0] == "Career":            return "Career"        elif ix[0] == "1999-00":            return "2000"        else:            return ix[0][0:2] + ix[0][-2:]        # get player df and add some extra info    player_df = player.dataframe    player_df['birth_date'] = player.birth_date    player_df['player_id'] = player.player_id    player_df['name'] = player.name    player_df['year'] = [get_year(ix) for ix in player_df.index]    player_df['id'] = [player_id + ' ' + year for player_id,                       year in zip(player_df['player_id'],                       player_df['year'])]    player_df['age'] = [get_age(year, bd) for year,                        bd in zip(player_df.index,                        player_df['birth_date'])]    player_df.set_index('id', drop = True, inplace = True)        return player_df
Next, I use the function defined above to collect player data spanning entire careers for every player to play in the NBA within the last 20 years. Data is split between two dataframes, one which aggregates data at the season level and another which aggregates at the career level. This is done by using the Teams and Roster modules to iterate through every NBA roster from the year 2000 to 2020. Note that I call the Roster class using the keyword variable slim = True. Without this setting I’d have to create a Player instance for every season a player was in the NBA, increasing the running time by roughly 10x.
# initialize a list of players that we have pulled data forplayers_collected = []season_df_init = 0career_df_init = 0season_df = 0career_df = 0# iterate through years.for year in range(2020, 1999, -1):    print('\n' + str(year))            # iterate through all teams in that year.    for team in Teams(year = str(year)).dataframes.index:        print('\n' + team + '\n')                # iterate through every player on a team roster.        for player_id in Roster(team, year = year,                         slim = True).players.keys():                        # only pull player info if that player hasn't            # been pulled already.            if player_id not in players_collected:                                player = Player(player_id)                player_info = get_player_df(player)                player_seasons = player_info[                                 player_info['year'] != "Career"]                player_career = player_info[                                player_info['year'] == "Career"]                                # create season_df if not initialized                if not season_df_init:                    season_df = player_seasons                    season_df_init = 1                                # else concatenate to season_df                else:                    season_df = pd.concat([season_df,                                   player_seasons], axis = 0)                                    if not career_df_init:                    career_df = player_career                    career_df_init = 1                                # else concatenate to career_df                else:                    career_df = pd.concat([career_df,                                   player_career], axis = 0)                                # add player to players_collected                players_collected.append(player_id)                print(player.name)
The final step is to save the resulting dataframes to your favorite file format.
season_df.to_csv('nba_player_stats_by_season.csv')career_df.to_csv('nba_player_stats_by_career.csv')
Altogether this script should compile and save your dataset within about 30 minutes. If you understand all the code in this final section then you should be more than ready to start building your own datasets using the Sports Reference API.
Be sure to look out for my article next week when I use this dataset to analyze NBA career trajectories, and also check out my article from last week Beating the Odds.
I hope you’ve enjoyed the read and take care.