When Data Science Meet Football (Part 1): Introduction | by Fahmi Nurfikri | Towards Data Science

The utilization of data in football (or soccer) has become very important to develop player skills or match analysis. Today, we can discuss the world of football in greater depth and detail than in the previous period. For those of you who are starting to explore data analytics in football, there are public data sets provided by third parties such as Wyscout and Opta.
The dataset contains football match events (e.g., Players and matches statistics). We can explore the data as an effort to understand football data more deeply. In this section, we will examine the variables in conducting football analysis.
Let’s get started, if you want to see the full code of this article, please visit my github. In this article, we will use the Jupyter Notebook as code editor. If you are not familiar with the Jupyter Notebook, you can see the following article. You are free to use another code editor, do not have to follow what I use.
jupyter-notebook-beginner-guide.readthedocs.io
First of all, we must download the required data. In this article, We will use the dataset used in the research of Pappalardo et al., (2019). You can download the dataset here.
If you have downloaded the required dataset, the next step is to import the package that will be used. Here we will use the collections and json packages which are built-in functions of python and matplotlib and numpy.
If you have not installed one or both of the packages that I mentioned last, please install them first using the commands below.
$ pip install numpy$ pip install matplotlib
After the package is installed, enter the package that was mentioned earlier.
%matplotlib inlinefrom collections import defaultdictimport matplotlib.pyplot as pltimport numpy as npimport json
We can load the dataset by writing the code below. Data sets are imported from the figshare repository and stored in JSON format. In this session, we import matches, players, competitions, and team data. You can also see this Github for reference and as an inspiration for me to make this article.
def load_dataset(tournament):    matches, events = {}, {}    matches = json.load(open('./data/matches/matches_{}.json'.format(tournament)))    events = json.load(open('./data/events/events_{}.json'.format(tournament)))    players = json.load(open('./data/players.json'))    competitions = json.load(open('./data/competitions.json'))    teams = json.load(open('./data/teams.json'))    return matches, events, players, competitions, teams
After that, we define functions to get every data that we need. Start from matches and events.
def get_match(matches, events):    match_id2events = defaultdict(list)    match_id2match = defaultdict(dict)    for event in events:        match_id = event['matchId']        match_id2events[match_id].append(event)                                             for match in matches:        match_id = match['wyId']        match_id2match[match_id] = match
Player.
def get_player(players):    player_id2player = defaultdict(dict)    for player in players:        player_id = player['wyId']        player_id2player[player_id] = player    return player_id2player
Competitions.
def get_competitions(competitions):    competition_id2competition = defaultdict(dict)    for competition in competitions:        competition_id = competition['wyId']        competition_id2competition[competition_id] = competition    return competition_id2competition
Teams.
def get_teams(teams):    team_id2team = defaultdict(dict)    for team in teams:        team_id = team['wyId']        team_id2team[team_id] = team    return team_id2team
Now, everything is set. Let’s load the data. Here we will only use 2018 World Cup data, so we will enter the World_Cup parameter.
matches, events, players, competitions, teams = load_dataset('World_Cup')
After that, we change the data into the dictionary form.
match_id2events, match_id2match = get_match(matches, events)player_id2player = get_player(players)competition_id2competition = get_competitions(competitions)team_id2team = get_teams(teams)
Here we will try to see the structure of each data. For example, we can display player-related information with index 32777 (player_id2player[32777]) in our dictionary. As you can see, we might have a lot of information about the player, from the passport area information to the player’s identifier in the Wyscout system.
{'passportArea': {'name': 'Turkey',  'id': '792',  'alpha3code': 'TUR',  'alpha2code': 'TR'}, 'weight': 78, 'firstName': 'Harun', 'middleName': '', 'lastName': 'Tekin', 'currentTeamId': 4502, 'birthDate': '1989-06-17', 'height': 187, 'role': {'code2': 'GK', 'code3': 'GKP', 'name': 'Goalkeeper'}, 'birthArea': {'name': 'Turkey',  'id': '792',  'alpha3code': 'TUR',  'alpha2code': 'TR'}, 'wyId': 32777, 'foot': 'right', 'shortName': 'H. Tekin', 'currentNationalTeamId': 4687}
We can also explore the structure of other data using the code below.
match_id2events[2058017]
And this is the result.
{'status': 'Played', 'roundId': 4165368, 'gameweek': 0, 'teamsData': {'9598': {'scoreET': 0,   'coachId': 122788,   'side': 'away',   'teamId': 9598,   'score': 2,   'scoreP': 0,   'hasFormation': 1,   'formation': {'bench': [{'playerId': 69964,      'assists': '0',      'goals': 'null',      'ownGoals': '0',      'redCards': '0',      'yellowCards': '0'},... 'seasonId': 10078, 'dateutc': '2018-07-15 15:00:00', 'winner': 4418, 'venue': 'Olimpiyskiy stadion Luzhniki', 'wyId': 2058017, 'label': 'France - Croatia, 4 - 2', 'date': 'July 15, 2018 at 5:00:00 PM GMT+2', 'groupName': '', 'referees': [{'refereeId': 378051, 'role': 'referee'},  {'refereeId': 378038, 'role': 'firstAssistant'},  {'refereeId': 378060, 'role': 'secondAssistant'},  {'refereeId': 377215, 'role': 'fourthOfficial'}], 'duration': 'Regular', 'competitionId': 28}
You can try yourself for other data. If you need an example, I have provided it on Github.
We can make interesting data analysis by plotting data to a more interactive view. For example, we can take all the heights of players and plot them into a histogram.
heights = [player['height'] for player in player_id2player.values()   if player['height'] > 0]fig,ax = plt.subplots(1,1)ax.hist(heights)ax.set_title('histogram of height')ax.set_xlabel('height [cm]')ax.set_ylabel('frequency')plt.show()
This is the result.
Or you can make this.
Besides analyzing player statistics, we can also make other interesting analyzes by finding events in match data. Write the code below to create a function to identify the data structure of the match event.
event_key = []sub_event_key = []for events in match_id2events.values():    for event in events:        event_key.append(event['eventName'])        sub_event_key.append(event['subEventName'])        event_key = list(set(event_key))sub_event_key = list(set(sub_event_key))event_stats = []sub_event_stats = []for events in match_id2events.values():    event_stat = {key: 0 for key in event_key}    sub_event_stat = {key: 0 for key in sub_event_key}        for event in events:        event_stat[event['eventName']] += 1        sub_event_stat[event['subEventName']] += 1    event_stats.append(event_stat)    sub_event_stats.append(sub_event_stat)
This is the result of the event_key variable.
{'Duel', 'Foul', 'Free Kick', 'Goalkeeper leaving line', 'Offside', 'Others on the ball', 'Pass', 'Save attempt', 'Shot'}
And this is the result of the sub_event_key variable.
{'', 'Acceleration', 'Air duel', 'Clearance', 'Corner', 'Cross', 'Foul', 'Free Kick', 'Free kick cross', 'Free kick shot', 'Goal kick', 'Goalkeeper leaving line', 'Ground attacking duel', 'Ground defending duel', 'Ground loose ball duel', 'Hand foul', 'Hand pass', 'Head pass', 'High pass', 'Late card foul', 'Launch', 'Out of game foul', 'Penalty', 'Protest', 'Reflexes', 'Save attempt', 'Shot', 'Simple pass', 'Simulation', 'Smart pass', 'Throw in', 'Time lost foul', 'Touch', 'Violent Foul'}
We can see that the match event has some interesting variables. We can analyze each variable to get a more significant match analysis. For example, we can print some event statistics in the first game of our dictionary (event_stats[0]).
{'Goalkeeper leaving line': 0, 'Duel': 468, 'Free Kick': 105, 'Offside': 4, 'Others on the ball': 128, 'Foul': 32, 'Shot': 18, 'Pass': 827, 'Save attempt': 7}
Just like before, we can also create a histogram of each variable in a match event to simplify the analysis process. Let’s make a histogram.
pass_stat = [event['Pass'] for event in event_stats if 'Pass' in event]fig,ax = plt.subplots(1,1)ax.hist(pass_stat)ax.set_title("histogram of passes")ax.set_xlabel('passes')ax.set_yticks([0, 2, 4, 6, 8, 10, 12, 14, 16, 18, 20])ax.set_ylabel('frequency')plt.show()
And this is the result.
You can also make this.
And the last, you also can create goal statistic.
scores = []for match in match_id2match.values():    score = 0    for team in match['teamsData'].values():        score += team['score']    scores.append(score)
Then use this code to see the statistics.
scores = np.array(scores)fig,ax = plt.subplots(1,1)ax.hist(scores, bins = [0, 1, 2, 3, 4, 5, 6, 7])ax.set_title("histogram of goals")ax.set_xticks([0, 1, 2, 3, 4, 5, 6, 7])ax.set_xlabel('goals')ax.set_yticks([0, 2, 4, 6, 8, 10, 12, 14, 16, 18, 20])ax.set_ylabel('frequency')plt.show()
And this is the result.
The tutorial above is an introduction to how football analysis works. In the next article, we will explore more complex data.
Pappalardo, Luca; Massucco, Emanuele (2019): Soccer match event dataset. figshare. Collection. https://doi.org/10.6084/m9.figshare.c.4415000.v5
Pappalardo et al., (2019) A public data set of spatio-temporal match events in soccer competitions, Nature Scientific Data 6:236, https://www.nature.com/articles/s41597-019-0247-7
Pappalardo et al. (2019) PlayeRank: Data-driven Performance Evaluation and Player Ranking in Soccer via a Machine Learning Approach. ACM Transactions on Intellingent Systems and Technologies (TIST) 10, 5, Article 59 (September 2019), 27 pages. DOI: https://doi.org/10.1145/3343172