Exploratory Data Analysis of International Football History
By: Dejoli Tientcheu Touko Landry
Tags: Exploratory Data Analysis, Machine Learning, Python, Data Visualization

What do you think about the picture?

Exploratory Data Analysis of International Football History

Before diving into Football(soccer), what is EDA(Exploratory Data Analysis abbreviation)? What is the purpose of doing EDA?

“Seek First to Understand, Then to be Understood”

Steven Covey — 7 Habits of Highly Effective People (Habit 5)

I was taking my online data science course as usual when the Teacher during the Data visualization lesson said our next project will be doing an Exploratory Data Analysis.

Honestly, that was the first time I heard about EDA. My first reaction was yeah! let’s try to come out with some graphs learned from the lesson then a few minutes later I was like but wait why doing it? what are we going to learn from it? where do I find the dataset? etc …

Then I went back to watch the lesson recording again, and what came to my mind first was, maybe EDA is helpful to describe the Data ? using graphs for better visualization of Information getting from the Data , make a fast and efficient Analysis of the Data ? a bunch of things was coming to my head.

After listening to the entire video session record I also try to ask my best friend when I'm stuck google.com , I end up with :

EDA was originally developed by American mathematician John Tukey in the 1970s. It’s used by data scientists to analyze and investigate data sets and summarize their main characteristics, often employing data visualization methods. It helps determine how best to manipulate data sources to get the answers you need, making it easier for data scientists to discover patterns, spot anomalies, test a hypothesis, or check assumptions.

It seems that I was not that far away from the right path, but why do it?

The main purpose of EDA is to help look at the data before making any assumptions. It can help identify obvious errors, as well as better understand patterns within the data, detect outliers or anomalous events, find interesting relations among the variables. When it’s done and insights are drawn, its features can then be used for sophisticated data analysis or modeling, including machine learning. (source)

Ok after all we finally have an idea about EDA, now try to make one. Which dataset to use? headache. How to find an interesting dataset that I will enjoy working on it and where to find it?

First where to find the dataset, Kaggle was always coming during the course as one of the best platforms to search the corresponding dataset through filters. To find an interesting dataset for my EDA project the essential thing that came to my mind was to find something I show more interest in and love which is Football.

Football just like Basketball is a worldwide well-known sports activity that could be a profession for others, a hobby, or a passion for some others. For me, it’s more than a passion it's a part of my life. Watching and playing football use to take me more time in my daily life that sometimes I do have some remorse like I could have spent that time on learning how to code, it’s true but difficult to stay far away from it.

This Exploratory Data Analysis is a simple EDA project about exploring FIFA statistics on International Man Football team(countries) information from 1872 still now.

WHY DOING THIS EDA?

Find helpful information that could give a superficial idea and statistics concerning International Football history, and could help to answer questions concerning man international Football matches.

HOW TO DO IT? WHERE TO START?

First, let’s talk about the dataset; It was downloaded from Kaggle which is one of the world’s largest data science communities with powerful tools and resources to help you achieve your data science goals.

What I do realize is the main dataset was not enough to answer commonly asked questions, which is why for this work 4 datasets containing information about FIFA registered international Football teams were used. The datasets used are:

This Analysis, through visualizing different trends, will give us an idea about international football history around the world and information about countries that participate in FIFA major tournaments. international-football-results-from-1872-to-2017 is a dataset made of more than 42363 rows and 9 columns. fifaworldranking have 63054 rows, 9 columns where football team ranks are listed since 1992. Country-code folder content 2 datasets countrycode.xls for FIFA registered international team countries code and continent.xls where countries are group according to continent and regions.

fifa-worl-cup-data had fifa-host.xls dataset with 18 rows 3 columns and fifa-team-performance.xls dataset having 78 rows 7 columns.

The EDA will be done using Python3, libraries such as Jovian to save, make public and share the notebook, opendatasets to download the dataset files from Kaggle, pandas which is a software library written for the Python programming language for data manipulation and analysis, it offers data structures and operations for manipulating numerical tables and time series. libraries such as matplotlib, plotly and seaborn, will be used for data visualization. GeoPandas, folium will be helpful to watch a geographical representation. Finally the ineluctable NumPy, it’s suitable when working with an array.

OUTLINE:

Download and Import libraries

Read the dataset

Data Preparation, Cleaning, and Adding

Exploratory Analysis and visualization

Asking and Answering Questions

Summary and Conclusion

Future Works

References

Install and import libraries we will use for the Project

Install libraries

!pip install jovian geopandas opendatasets numpy folium dash colorama --upgrade --quiet

!pip install plotly==4.7.1 --quiet

!pip install matplotlib==3.0.2 --quiet

Import libraries

import opendatasets as od

import pandas as pd

import matplotlib.pyplot as plt

import matplotlib

import seaborn as sns

import folium

import geopandas as gpd

import branca.colormap as cm

from google.colab import files

import io

import numpy as np

import plotly.express as px

from wordcloud import WordCloud

import plotly.graph_objects as go

import re

# import dash

# import dash_core_components as dcc

# import dash_html_components as html

# from dash.dependencies import Input, Output

import jovian

from colorama import Back, Fore, Style

Download the datasets using opendatasets

Get the dataset URL

# dataset url

download_url_man = "https://www.kaggle.com/martj42/international-football-results-from-1872-to-2017"

od.download(download_url_man)

download_url_rank = "https://www.kaggle.com/cashncarry/fifaworldranking"

od.download(download_url_rank)

download_country_data = "https://www.kaggle.com/dejolilandry/country-code"

od.download(download_country_data)

download_worldcup_data = "https://www.kaggle.com/dejolilandry/fifa-world-cup-data"

od.download(download_worldcup_data)

Get the dataset path

# dataset path in colab

dataset_man = "./international-football-results-from-1872-to-2017/results.csv"

perfromance = "./fifa-world-cup-data/fifa-team-performance.xls"

dataset_man_rank = "./fifaworldranking/fifa_ranking-2021-05-27.csv"

dataset_countrycode = "./country-code/country-code-latitude-longitude.xls"

dataset_continent = "./country-code/continent.xls"

fifa_host = "./fifa-world-cup-data/fifa-host.xls"

countries_geojson = 'https://raw.githubusercontent.com/johan/world.geo.json/master/countries.geo.json'

Data Loading, Preparation, and Cleaning

Load the CSV or excel file using the pandas' library

Look at some information about the dataset and paid attention to helpful pieces of information.

Add needed columns

What I learned is always try to look at few rows of the dataset information when working with a big dataset, which can be done just by using head(), tail() or selecting some sample using sample() .

We only have few nan values due to a recent matches information update.

As a football match can only have 3 possible results (win, draw, loss), we add two columns reflecting each country's match. We added home_final_result and away_final_result columns using the home_score and away_score columns respectively to set the corresponding result between win, draw, and loss.

What we can observe from the football_df dataset description is:

maximum home_score are 31 goals whereas maximum away_score is 21 goals, which is too much for a normal football match.

are 31 goals whereas is 21 goals, which is too much for a normal football match. The mean and standard deviation for home_score are a bit higher than away score .

and for are a bit higher than . minimum home and away scores are 0 which is normal as a team can end a match without scoring a goal.

home and away scores are 0 which is normal as a team can end a match without scoring a goal. mean values are higher than median which is the 50% percentile in columns.

values are higher than which is the percentile in columns. 50% and 75% percentiles are equal for home and away scores.

and percentiles are equal for home and away scores. There is a large difference between 75% percentile and the maximum

The ranges of values in the numerical columns seem reasonable (no negative score), which means less data cleaning or correction.

Let’s have a look at Few key insights by looking at categorical columns as follows:

The match result statistic is reasonable too as the total number of matches win should be equal to the total number of matches loss .

Let’s read other complementary datasets.

Combine continent_df and countrycode_df to avoid less confusion in the coming step. continent_df dataset countries names are not exactly written in the same way as it's in the main dataset football_df . ex: United state and United state of America

Let’s modify names that use to identify the same team but a bit different in writing style. we do manually replace the corresponding name, it's time-consuming but at this moment that is the faster and effective way.

Define the list of FIFA major tournaments

FIFA major tournaments include : FIFA World Cup , UEFA Euro , Copa América , African Cup of Nations , CONCACAF Gold Cup , CONCACAF Championship , AFC Asian Cup , Oceania Nations Cup . The Gold Cup ( also called CONCACAF Gold Cup ) is the new version of "CONCACAF Championship" since 1991 . It's North America , Central America and the Caribbean tournament.

In This work we use Country Name , Team , country , international Team and so on to express a Team .

Exploratory Analysis and Visualization

Let’s settle the fontsize and style we wish our figure to look like.

— Relation between columns of the main Dataset.

Have a look at numeric columns and see their distributions.

Home_team: what we can observe in the first graph(Home matches final result and score relation distribution) is when no goal is score the probability of losing the match is about twice the probability to draw. When scoring only one goal, the probability to get one of the 3 results is almost the same. Scoring more than 2 goals at home gives a higher probability to win the match.

away_team: we can observe that in the second graph(Away matches final result and score relation distribution)when no goal has been scored the probability of losing the game is 3 times higher than draw. When scoring one goal the probability to win is only about 22% and 47% to loss. When scoring 2 or more goal, the probability to win the game become higher(more than 50%).

In most the situations, winning a match away is not a piece of cake, especially when the home team does play at Home in front of their fans, in their stadium, with an environment they are used to(weather condition). Playing at home gives a significant advantage to the home team when both are from the same level.

Let’s observe the distribution of playing ground whether most matches happened locally or abroad. Neutral columns here is a categorical column.

This graph shows us the total matches home_team played in their respective country ( true ) or happened in a foreign location( false ). We observe that only a few matches are played localy this is normal as FIFA major tournaments are played only in host countries, in most situations friendly matches are also played in a foreign location.

— Top 20 teams that have the highest number of international matches.

To visualize the top 20 teams with the highest number of international matches played we use barplot to plot it. Here we use the football_df DataFrame which is the main DataFrame, then need to get total home_team played matches plus total away_team played matches then sum it. Let’s created a function name total_games() to extract the total matches played, then choose the top 20.

Observation: What we can observe is that most of the team are from Europe, then comes the South Americans team. In the top 20 we only have one Asian team South Korea and no team from Africa . Sweden and England had played more than 1000 matches since the creation of their team.

Compared to other Continent such as Europe and South America , African countries national team and national football federation were built a bit late than others even though people were already practicing football, they are also used to play fewer exhibition matches during FIFA break times or international friendly matches FIFA period.

— Number of FIFA major tournament matches played by each country's national team.

Let’s make a better representation according to Geomap using folium ; For that, we need to obtain each country Geometry localization, we are going to use load countries geo_jon. We created a function name total_fifa_games() to extract the total number of FIFA major tournament matches played by each team.

Observation: From the above geo_graph, we can observe that South Americans teams have the highest number of appearances in FIFA major tournaments and this is due to fact that they have a good and constant team and also to the fact that they do have started playing FIFA major tournaments long ago(first Copa America Happened in the 1916 , AFC Asian Cup 1956 , African Cup of Nations the first edition was 1957 , Euro Cup 1958 , Ocenia Gold cup 1973 , FIFA World Cup 1930 , CONCACAF Cup 1963 ).

— The evolution of the number of teams participating in each FIFA major tournament.

This part is just to show the history of how Fifa and other continent football federations increased the number of teams gradually. Many reasons have been given, like to give more chances to several countries to participate and make the competition more challenging and attractive.

Observation: From this graph showing the total number of teams during each cup edition, we can easily see how Fifa major tournaments participating teams increase with time.

Fifa world cup increased first from 16 teams in 1954 then 24 teams from 1994 to 32 teams in 1998. (inlight Fifa World Cup to have a better view)

increased first from 16 teams in 1954 then 24 teams from 1994 to 32 teams in 1998. (inlight Fifa World Cup to have a better view) Copa America had once an edition with 16 teams in 2016.

had once an edition with 16 teams in 2016. AFC Asian cup and Africans Cup of Nations switch from 16 to 24 teams from 2019

Just as mentioned in preview analysis, we can observe that South American do organize their competition long before others continent. CONCACAF Gold Cup (or Gold Cup ) is the new version of CONCACAF Championship from 1991 edition.

— Regroup Team registered in FIFA or continental federation, according to region and continent.

Here we arrange team accordingly to their geographical location from continent to regions they belong to we use the football_df DataFrame and country_continent_df .

Observation: Here we can observe that Europe cotinent has the highest number of matches with more than 23000 games, specially Nothern Europe with more than 8000 matches. my point of view is Europe has a higher number of matches played because, as many games are friendly games, European players can easily be regrouped, as most of their players play in Europe top leagues where Africans and some other continent countries need more time, that is the reason why most of the time Africans and south Americans play their friendly game in Europe, as most of the players are in Europe. Asian countries most of the time play with national players(players playing in national league) when playing friendly regional games.

— History Rank for each country in FIFA Ranking

Since 1992 after a certain period, Fifa publishes the world football team rank, this rank relies on team performances during a certain period including friendly matches. Here we use the team_rank_df_new DataFrame and continent_country_region_game_df .

Observation: The graph here display football team Fifa rank history what we can observe is that many european and South America countries are easy to be ranked in the top 10. Cameroon had the best rank for an African team in history with position number 10. Ocenia the best Rank have mostly being Autralia .The last rank here was in 2020-05-07 with 1.Belgium, 2.France, 3.Brazil, 4.England, 5.Portugal. 4/5 are European countries team that can also show us the evolution and development of talented football players in Europe compared to other continents such as Africa and Asian . Note that each circle represent a country or team and the color is according to cotinent , the size corresponds to the number of matches.

— Statistic of top 20 teams(Win, draw, loss)

Here we analyze the top 20 Teams' statistics by looking for their home played game and away played game result. We use the football_df DataFrame.

Observation: We can observe that most of the team with a higher number of win matches are form Europe about 14 teams , and 3 teams form South America where North America , Asia and Africa make each have 1 team . Let’s have a look on maximum , median and minimum of matches result like win , loss and draw statistic.

Observation: Here what we observe is that the maximum matches win for a Team is 637 where the median is only 56 (11.37 time less than maximum ), the minimum 0 reflecting team with no victory in their record, they are still fighting to get their first football match victory. The maximum draw 249 the median 43 (5.79 times less than maximum ) and minimum 0 which means some teams never had a draw as final match result. The maximum loss is 408 where the median is 100 (only 4.08 time less than maximum ) the minimum is 1 this means no team have 100% victory for matches played.

— Total number of international football matches played each year.

Here we need to show a graph that can give us an observation about how many matches have been played in a single year.

Observation: In this graph, we chose to use a plot instead of others such as a barplot to make it more readable and observe some variation. This graph shows that from 2000 we have nearly more than 800 international football matches in a year, but right before 2020 the number of international matches drastically decline this should be due to the covid19 as every activity stop for long during the pandemic lockdown, tournament such a euro Cup 2020 was postponed to 2021 .

Ask & Answer Questions

Q1. which cities(top10) hosted the most international football matches?

For this question, the needed information is in football_df DataFrame.

From this graph, we realize that the top3 cities where most international matches happened are from Asia ( Kuala Lumpur , Doha , Bangkok ). At the top10 We have cities from Europe ( London , Budapest , Vienna , Copenhagen , Glasgow )and only one from South America which is Montevideo . Honestly, it's really surprising, difficult to find the reason why many international matches are played in Asia might be Asia has a lot of minor tournaments between countries to challenge each other and develop national players or they have several Sponsors? Tourism?.

Q2. what is the ratio of scored goals per match in each different Edition Fifa major(Continental) tournament?

We can see That South America Copa America have recorded from 1916 , the smallest average goal was 2 in 1922 edition, where other continents have Edition with less than 2 goals average. This show how exciting Copa America is comparing to others where average scored goals is rarely more than 3. After 1980 beside Ocenia cup which has less challenging teams, others tournaments have an average goal less than 3.3, it could also be because nowadays many countries have done remarkable evolution in football development There is a real Challenge between teams now than before and matches are tougher too.

Q3. What is the probability of hosting and winning a FIFA World Cup?

To find out the probability of hosting and winning a FIFA World Cup we should first look for the total number of World cup editions that happened and the number of teams that won when they were the host.

Even though the probability is 28.571% they are more others features that could have an impact on winning the world cup like FiFA Rank position before the competition. Let's see what is the history Rank of the countries that won the world cup.

Through the graph, we can observe that except uruguay during a certain period, other teams are always among the top 20, most of the time in the top 10. In conclusion, having a good FIFA rank among the best 10 and been a host can be an advantage during the tournament.

Notice: FIFA rank depends on team performances during a certain period.

Here we took into consideration performances during World Cup years, ranking of the period(months of the year) before and after the competition.

Q4. What countries have already Won the FIFA World Cup in History? When and Which Edition?

Here we find teams that already won the FIFA World Cup, then find out the edition they won. Use host team code and host year as indicators to the corresponding edition.

This graph was inspired by Worldcupmatch's work. It’s a better representation of Team or countries that won the FIFA World Cup edition since 1930 still now with Brazil having the record with 5 times Champions . We can also see the host country code and realize that Brazil hosted 2 editions of FIFA World Cup 1950 and 2014 but never won at home like Uruguay , France , Argentina , italy , German and England did.

Summary and Conclusion

This project was very interesting to me as it's about my hobby football, but was tricky when it comes to extracting helpful information as the main dataset has very few numeric columns. There is a bunch of things to improve and make it more clear and professional. By looking at the graphs we can have a general idea concerning man National Football(soccer for Americans) team history. In summary, we can see how even though the first official international football match was played in 1872 in Glasgow between Scotland and England, South America Copa America was the first continental competition in FIFA major tournaments to happen in 1916. They have fewer countries but very good football teams, these teams can challenge european teams. Country like brazil , argentina , Uruguay in South America and Germany , Italy , France are dominating FIFA Competitions winner, this could be due to good youth football development strategy(politics) base on talent detection and training process.

We can’t finish without saying special thanks to Jovian Team, Slack mate as their works inspired me.

Hope readers will like it and give me more advice to improve the EDA project by asking questions.

Future work :

Add women's football information, analyze it and try to have an idea about the development of women's football respectively in different continents.

Find the probability to win the UEFA Euro Cup right after winning the previews FIFA World Cup .(example: France 1998-2000, spain 2010-2012)

right after winning the previews .(example: 1998-2000, 2010-2012) Find and add each country players performance achieved in their respective Football club mainly from the best League as players are called to the national team according to their performance in their club . Having many high-class players make the team stronger enough and increased the probability to win a match. Such data will add more meaningful features to the dataset.

. Having many high-class players make the team stronger enough and increased the probability to win a match. Such data will add more meaningful features to the dataset. Analyze the data features available, find out the correlation between them, and try to make the data useful to predict the next FIFA World Cup Semi-final teams. (Apply Machine Learning)

Reference

How to run the notebook step by step:

The notebook of this EDA project contains more visualization and can be found here. To be able to run the notebook efficiently, after loading and open the jupyter Notebook(or login your jovian account):

1. Do Select Kernel and Restart to clear output or Runtime>Restart runtime in Colab.

2. Download your Kaggle API key and upload into the files folder to avoid being asked to enter your Kaggle account username and key each when downloading dataset from Kaggle.

3. Start running step by step each level of the code(by pressing on the execution button) and see the output for better understanding.

Hope this EDA can help to have an idea about FIFA football tournaments' history. Please don't hesitate to let a comment, asking a question, it will help me improve the quality of the work done.