Catching that flight: Visualizing social network with Networkx and Basemap | by Tuan Nguyen Doan | Towards Data Science

“If you graph these numbers, patterns emerge. There are patterns everywhere in nature.”
Max Cohen (played by Sean Gullette, in Pi, a film by Darren Aronofsky)
Today, I will introduce very powerful tools to visualize network — Networkx and Basemap. Many interesting problems naturally arrive from or inspire some form of graph models — relationship between vertices (or nodes) and edges that connects these vertices. For example: the outlink and inlink structures of a website can be represented by a directed graph, in which the vertices represent web pages and directed edges represent links from one page to another. Another example is a friend circle, in which vertices represent different people and edges represent the type of relationship.
When it comes to complicated networks such as virus outbreaks, cash flows among countries, or seismic waves of the 2005 Earthquake, it remains a challenge to illustrate attributes of the network visually, descriptively and effectively. We want the audience to quickly grasp the network in its geographical context. Networkx and Basemap (a toolkit of the matplotlib package) provides a “end-to-end” solution. While the former creates network graphs and calculates various measures, the latter is capable of presenting neat visualizations that are both intuitive and informative.
In this example, we look at the flight route network between airports in the United States of America. The goal is to represent vertices (airports) and edges (flight route) and preserve the geographical relationships between different vertices (e.g: we want to look at the graph and tell that this vertex is JFK or Logan Airport or whatever)
At first we load the relevant packages:
import pandas as pdimport networkx as nximport matplotlib.pyplot as pltfrom mpl_toolkits.basemap import Basemap as Basemap
The matplotlib basemap toolkit is a library for plotting 2D data on maps in Python. Networkx is a comprehensive library to study network structure.
The first step is to acquire the data and process it. Here, I use the OpenFlight1 database to acquire information about airports and routes. They have very comprehensive data. Unfortunately, the route database is not very up-to-date. It currently contains 59,036 routes between 3,209 airports on 531 airlines spanning the globe. Until today, there are up to 17,678 commercial airports in the world. Nevertheless, the current datasets are good enough for our illustration purpose today.
There are two relevant datasets:
The airport.dat dataset contains geographic information of all the listed airport
The routes.dat dataset contains geographic information of all the listed airports
Networkx cannot read the data in its raw form, so our first job is to process the data to acquire a clean dataframe of routes that could be read by Networkx. Here, we use pandas to parse Excel files into dataframes ,extract and process the information. Notice how the two dataset are connected by the code of the airport (the three letter IATA code). You can find the full code to process the data in my source code here.
The aim of our data-processing step is to acquire the following two Panda dataframes:
A condensed routes_us data frame where each row represents a unique air route and the total number of airlines operate on that route (for example, there are 3 airlines that operate the flight from Lehigh Valley Airport (ABE) to Atlanta International Airport (ATL).
A condensed position dataframe where each row represent each USA airports with IATA code, and more importantly, longitudinal and latitudinal details
With the former dataframe, we are ready to draw our very first sketch of the flight networks.
At first, we translated our dataframe into a graph. Notice that our graph is a directed graph, that is, a graph with a set of vertices connected by edges having directions associated with them. This means that in our graph, the two routes JFK-ATL and ATL-JFK are distinct since even though they are connecting the same 2 nodes, the two routes have different (opposite) directions.
We use Networkx’s from_panda_dataframe() function to quickly import our graph. Here we create a graph from our dataframe routes_us, where the source is ‘Source Airport’ column, the target is ‘Dest Airport’ column using a Directed Graph model. edge_attr means that we can add information to the edges of the graph. I have added the number of airlines operated on a route as the edge attribute
graph = nx.from_pandas_dataframe(routes_us, source = 'Source Airport', target = 'Dest Airport', edge_attr = 'number of flights',create_using = nx.DiGraph())
Networkx does have a graphical tool that we can use to draw our network.
plt.figure(figsize = (10,9))nx.draw_networkx(graph)plt.savefig("./images/map_0.png", format = "png", dpi = 300)plt.show()
The problem with this rough network is that we really cannot tell which airport is which and how routes are related to one another. Maybe it is a better idea to plot the airport in the exact geographical position in a American map. How do we do that? Aha, Basemap !!!
Now, we need to help Basemap define the borderline of the USA. Let us define a relatively large map that includes Alaska and Puerto Rico. One thing to note is that there are many ways we can map a large area such as a continent onto a 2-D surface. For example, we can choose Equidistant Cylindrical projection, a simple projection which divides the globe into rectangulars of equal size. A more familiar method is the Mercator projection, which is a cylindrical, conformal projection with very large distortion at high latitudes. Yes, this is the wrongly misguided map in every classroom that has Alaska at the same size with the African continent
I also choose the familiar Mercator projection. This
plt.figure(figsize = (10,9))m = Basemap(projection='merc',llcrnrlon=-180,llcrnrlat=10,urcrnrlon=-50,urcrnrlat=70, lat_ts=0, resolution='l',suppress_ticks=True)
Now, we need to define the position of our airports on the Basemap. Until now, we only have their longitudinal and latitudinal information. We need to find their actual projection onto the Basemap surface. Notice how I call our position dataset, get the Long and Lat data, and project them onto the Basemap surface
mx, my = m(pos_data['Long'].values, pos_data['Lat'].values)pos = {}for count, elem in enumerate (pos_data['IATA']):     pos[elem] = (mx[count], my[count])
The next step is to ask Networkx to add the nodes, edges and their attributes to the Basemap. This can be done as follows:
nx.draw_networkx_nodes(G = graph, pos = pos, node_list = graph.nodes(),node_color = 'r', alpha = 0.8, node_size = 100)nx.draw_networkx_edges(G = graph, pos = pos, edge_color='g', alpha=0.2, arrows = False)
The last step is to draw the countries, coastlines, and statelines.
m.drawcountries(linewidth = 3)m.drawstates(linewidth = 0.2)m.drawcoastlines(linewidth=3)plt.tight_layout()plt.savefig("./images/map_1.png", format = "png", dpi = 300)plt.show()
Well, this plot is pretty anti-climatic. It looks fine, but not great. Besides the fact that the map looks pretty ugly, we cannot tell anything from the graph. For example, we want to see more information such as:
Which airports are busy?
Which air routes are more prominent?
To answer these questions, maybe it is a good idea to incorporate the total number of incoming and outgoing flights each airport has, and plot them as the size of the airport. For example, an airport with lots of incoming and outgoing flights will have a larger size and appear more visible on the map.
To do that, we repeat the same code, with a small tweak:
nx.draw_networkx_nodes(G = graph, pos = pos, node_list = graph.nodes(), node_color = 'r', alpha = 0.8, node_size = [counts['total_flight'][s]*3 for s in graph.nodes()])
This is a lot better, once you have been more familiar with using Networkx and Basemap, you can start using personalize the map according to your use cases. For example, here, I restrict my map to mainland airports and style the map a little differently
We can start to make all sorts of interesting observations: for example, a number of large airports are mostly located in the 2 coastal areas (and Vegas, Denver, Dallas/Fort Worth, Houston and Atlanta). We can start to see the domestically air routes are particularly more dense in the West Coast area, as compared to any other geographical places. Interestingly, airports such as DEN (Denver International Airport) appears to function as a hub, that is, it serve as transfer (or stop-over) points to get passengers to their final destination. In future posts, I will introduce Networkx tools to analyze the distribution of edges and characteristics of nodes in such a network using Networkx
If you enjoy this article, you may also enjoy my other article sport visualization:
Advanced sports visualization with Python, Matplotlib and Seaborn
For other deep dive analyses about interesting statistical facts and rules of thumbs:
A statistical rule to optimize your life: the Lindy’s Effect
Rules of Three: Calculating the probability of events that have not yet occurred
Making big bucks with a data-driven sport betting strategy
The entire Python script for this article can be found at my Github page.
Data Source: OpenFlight. Airport, airline and route data 2017 https://openflights.org/data.html