How to Access the Fantasy Premier League API, Build a Dataframe, and Analyze Using Jupyter, Python, and Pandas | by David Allen | Towards Data Science

As of the writing of this tutorial, before match week 30+, I’m ranked #3,919 in the world in Fantasy Premier League Soccer (team: Yin Aubameyang), which equates to the top 0.05% in the world.
It didn’t happen by accident, and it wasn’t all luck.
It’s taken several years of playing this game to learn the patience, skill, and strategy required to succeed. From #1,181,262 in 2011/12 to #39,804 in 2018/19, and now #3,919 in 2019/20 with 8 GWs left:
Over these several years, I’ve learned 2 skills really help:
Identifying your biasesUnderstanding how to read data
Identifying your biases
Understanding how to read data
The first will help you stop making stupid mistakes. The second will assist you in uncovering value that other fantasy managers are missing.
This tutorial will help develop these 2 skills by teaching you how to access new data. And then help you look at it in new ways.
This document is far from perfect, but at the very least, it will give you a taste of what is possible with Jupyter Notebooks, Pandas, Python, and a new data source.
By the end of this tutorial, you will have the basic skills to pull down data from the Fantasy Premier League API and work with the data to squeeze out new understandings.
Let’s dive in.
Note from the author: for your next FPL API challenge, bookmark my blog here on Looping through the “Element-Summary” FPL API endpoint:
towardsdatascience.com
Getting StartedAccess the API with RequestsBuild a DataFrame from your API request responseWorking with the DataframeExport to CSV for Analysis in Google Sheets or Excel (if that’s your thing)
Getting Started
Access the API with Requests
Build a DataFrame from your API request response
Working with the Dataframe
Export to CSV for Analysis in Google Sheets or Excel (if that’s your thing)
This tutorial assumes a little familiarity with Jupyter and Python. If you are just getting started, start with my tutorial here:
medium.com
Once you complete your setup, come right back to this article!
If you already have Jupyter, Python, and Pandas installed then move on to step 2!
First, we’ll need our package imports. We really only need 3:
import requestsimport pandas as pdimport numpy as np
Note: If you get an import error, it’s probably because you haven’t added the package to your environment, or activated your development environment. Head over to your Anaconda Navigator and make sure to add the package needed to whatever environment you activate for your Jupyter Notebook work. Apply the update, and don’t forget to restart your terminal before you activate your development environment!
Next, we’ll need the API URL:
url = 'https://fantasy.premierleague.com/api/bootstrap-static/'
Use the requests package to make a GET request from the API endpoint:
r = requests.get(url)
Then, transform that request into a json object:
json = r.json()
Let’s look at the json keys, and then we’ll create our dataframe(s)
json.keys()
This returns a list of all the keys to the json:
The three keys I really care for this article are elements , element_type , and teams .
Next, we’ll create three different dataframes using these three keys, and then map some columns from the teams and element_type dataframes into our elements dataframe.
Once we build our dataframes, we’ll be able to work with the data and do things like sort, filter, map (similar to a v-lookup in Excel), and organize our data into a pivot table.
So let’s start by building our dataframes:
elements_df = pd.DataFrame(json['elements'])elements_types_df = pd.DataFrame(json['element_types'])teams_df = pd.DataFrame(json['teams'])
Preview the top 5 rows of your dataframes with the head() method. Like so:
elements_df.head()
You can also get a glimpse of all of the columns in each dataframe with the columns method:
elements_df.columns
There is an awful lot of data, so I’m going to make a copy of this dataframe, but only copy over some of the columns. I like to prepend slim_ to these smaller, lighter dataframe copies:
slim_elements_df = elements_df[['second_name','team','element_type','selected_by_percent','now_cost','minutes','transfers_in','value_season','total_points']]
Now, most of the data I’m currently interested in is viewable in a single window:
slim_elements_df.head()
Success! Let’s move on.
This section will be broken into sub-sections:
Using map()
Using astype()
Using sort_values()
Using pivot_table()
Using loc[]
Using hist()
The first thing I’m going to do is map the position name from the elements_type_df to the slim_elements_df . If you’ve ever used Excel, you’ll notice that map() is very similar to a v-lookup.
slim_elements_df['position'] = slim_elements_df.element_type.map(elements_types_df.set_index('id').singular_name)
Now that we’ve done that, take a look at the top few rows:
slim_elements_df.head()
Great, now we can easily see the position. This will come in handy when we organize value_season by position.
Next, let’s add the team name. We can do that by mapping “name” from our teams_df:
slim_elements_df['team'] = slim_elements_df.team.map(teams_df.set_index('id').name)
Voilà:
Next, let’s sort this table by value_season . I’m curious to look at the top value picks this year.
BUT FIRST, we have to create a new column called value .
Why? Because some of the values in the column value_season are string values. I know this because I’ve attempted to sort by value_season and got a strange, incorrect result. Unexplained behavior in your dataframe can often be due to conflicting types of data stored in the same column (for example: strings, integers, and float values all stored in the same column).
We need to make sure every single value is of the same type. To do this, we’ll create a new column, and use the astype() method to set all of the values to a float value (fancy-talk for numbers with decimals, example: 4.5).
slim_elements_df['value'] = slim_elements_df.value_season.astype(float)
Now we can sort our dataframe on the column value . Let’s do it:
Sort_values() allows you to, well, sort values :)
By default, Pandas will sort values in the ascending direction from low to high. But we want values sorted in the descending direction: high to low. So we need to explicitly set the ascending parameter to False like so:
slim_elements_df.sort_values('value',ascending=False).head(10)
Well, will ya look at that? Two players tied for the top spot. Pope and Lundstram. A goalkeeper and a defender. Nice.
This essentially means Pope and Lundstram have earned the most points per cost, because value is simply calculated by dividing total_points by now_cost .
value = points / cost
Did you expect that to be the case? A defender and a goalkeeper? Might this change how you think about team selection next year? Or next week?
Next, we’ll create a pivot_table on the column position , and look at value-by-position:
slim_elements_df.pivot_table(index='position',values='value',aggfunc=np.mean).reset_index()
After creating the pivot_table and assigning the pivot_table to the variable pivot , we’ll sort the pivot table descending by value:
pivot=slim_elements_df.pivot_table(index='position',values='value',aggfunc=np.mean).reset_index()pivot.sort_values('value',ascending=False)
This gives us:
Interesting. One of the two highest-value players in the game is a goalkeeper, yet the goalkeeper position contributes the lowest value on average. I think this is likely because all the goalkeepers that play 0 minutes. bring down the average of the starters.
To address this, let’s remove all the players from our dataframe that have zero minutes this season. We’ll use loc[] for the first time:
.loc lets you locate specific rows that have specific column values. It’s like filtering a spreadsheet on a specific value in a column.
For our current purposes, we want to locate all rows in our dataframe that have a value greater than zero so as to remove all values = 0. We aren’t just doing this for keepers. We need to do this for every position. We write:
slim_elements_df = slim_elements_df.loc[slim_elements_df.value > 0]
Now let’s run that pivot table back again:
pivot = slim_elements_df.pivot_table(index='position',values='value',aggfunc=np.mean).reset_index()
and then:
pivot.sort_values('value',ascending=False)
There you have it:
Now let’s do a different type of pivot table next. This time, we’ll look at which teams are providing the most value this year. The only things that need to change about this pivot_table statement are the index and the pivot_table variable. Take a look:
team_pivot = slim_elements_df.pivot_table(index='team',values='value',aggfunc=np.mean).reset_index()
Now we’ll display the team_pivot sorted, from the highest value team to the lowest:
team_pivot.sort_values('value',ascending=False)
Results are as follows:
Wolves fans don’t have to worry about their innate biases. Lucky!
Spurs fans, on the other hand, have had to be a little more conscious of their bias towards their own team to succeed in the FPL this year. How stressful.
Next, let’s look at a histogram distribution of value by position. Before we do that, we’ll create some filtered dataframes for each position. We will use .loc again to accomplish this:
fwd_df = slim_elements_df.loc[slim_elements_df.position == 'Forward']mid_df = slim_elements_df.loc[slim_elements_df.position == 'Midfielder']def_df = slim_elements_df.loc[slim_elements_df.position == 'Defender']goal_df = slim_elements_df.loc[slim_elements_df.position == 'Goalkeeper']
Then, we’ll use the .hist() method on our goal_df.value dataframe column:
goal_df.value.hist()
This gives us:
There is a decent amount of value here above 20. Let’s look at our dataframe, sorted by value to see who these characters are:
goal_df.sort_values('value',ascending=False).head(10)
Take-away? There’s a lot of value available in the goalkeeper position.
Plenty of other ways to cut this cheese, but let’s move on to histograms of the defender position:
def_df.value.hist()
Our defensive histogram points out something interesting. Notice the difference between the goalkeeper histogram and the defender histogram?
For a goalkeeper, there is a “best” choice, but we don’t really know who that is going to be starting the season. It turns out, goalkeeper selection was pretty forgiving seeing how many goalkeepers offered value between 20 and 26:
Defenders, on the other hand, had just one clear winner this year, and only a few others in the 20–25 value range that stood out from the pack. Let’s look at the top of the defender value table:
def_df.sort_values('value',ascending=False).head(10)
Lundstram was THE essential defender this year, offering a huge amount of value per cost: 26.1. Impressive!
The most important takeaway from this whole analysis is that Goal Keepers and Defenders offer more value than Midfielders and Forwards. But if you review FPL Twitter, you’ll see many FPL managers that don’t quite grasp this insight. That, my friend, is called “opportunity.”
It’s not all about value of course. It’s a team optimization challenge. You have 100 currency units to spend, and a team of high-value returners may not score enough points to challenge for a high finish in your mini-leagues or overall. A team of the highest-value returners will leave you with some money in the bank. And the game is to leverage your money for POINTS, not VALUE.
Let’s look at the top midfielders, for example:
Cantwell offers the highest value out of all midfielders, but he still has just 100 points. Meanwhile, De Bruyne has a value of just 16.8, but 178 total points...
The point? You need both value and high-returns. You’re looking for a balance; a balanced optimization of value and points.
Before we go, I wanted to show you how to export to a CSV that you can work with in Google Sheets or Excel.
It’s simple:
slim_elements_df.to_csv('~/Desktop/fpl_data.csv')
Run that cell and Boom! Done. Nice job :)
Now have fun exploring the data in Google Sheets or Excel, if that’s your thing.
There is a lot more to do here, but as with a good lunch, it’s important to not swallow the sandwich whole. We’ll be back with more after giving our brains a break, whaddya say?
Let’s give ourselves a high-five for our new insights and skills, and buckle up for an exciting end to the 19/20 FPL season!
Hungry for more? Anything you are interested in that wasn’t featured here in this tutorial? Share you request in a comment here or reach out on twitter @ deallen7