Graph Algorithms (Part 2). Main concepts, properties, and... | by Maël Fabien | Towards Data Science

Graphs are becoming central to machine learning these days, whether you’d like to understand the structure of a social network by predicting potential connections, detecting fraud, understand customer’s behavior of a car rental service or making real-time recommendations for example.
In this article, we’ll cover :
The main graph algorithms
Illustrations and use-cases
Examples in Python
This article was originally published on my personal blog: https://maelfabien.github.io/ml/#
I publish all my articles and the corresponding code on this repository :
github.com
If you haven't, make sure to check my first article of this series:
towardsdatascience.com
NEW: Part 3 is out!
towardsdatascience.com
For what comes next, open a Jupyter Notebook and import the following packages :
import numpy as npimport randomimport networkx as nxfrom IPython.display import Imageimport matplotlib.pyplot as plt
The following articles will be using the latest version 2.x ofnetworkx. NetworkX is a Python package for the creation, manipulation, and study of the structure, dynamics, and functions of complex networks.
I’ll try to keep a practical approach and illustrate each concept.
In the previous article, we covered the main kind of graphs, and the basic characteristics to describe a graph. We’ll now cover into more details graph analysis/algorithms and the different ways a graph can be analyzed.
In order to understand the context, here are some use cases for graph algorithms :
real-time fraud detection
real-time recommendations
streamline regulatory compliance
management and monitoring of complex networks
identity and access management
social applications/features
...
There are 3 main categories of graph algorithms that are currently supported in most frameworks (networkx in Python, or in Neo4J for example) :
Pathfinding: identify the optimal path depending on availability and quality for example. We’ll also include search algorithms in this category. This can be used to identify the quickest route or traffic routing for example.
Centrality: determine the importance of the nodes in the network. This can be used to identify influencers in social media for example or identify potential attack targets in a network.
Community detection: evaluate how a group is clustered. This can be used to segment customers and detect fraud for example.
We will also develop the basis of Machine Learning in Graphs and Graph Learning in a third article, coming out next week. All algorithms implemented in Networkx can be found here :
networkx.github.io
As you‘ll notice, we’ll only cover the basic and most common algorithms implemented in Networkx.
Pathfinding algorithms try to find the shortest path between two nodes by minimizing the number of hops.
Search Algorithms does not give the shortest path. Instead, they explore graphs considering neighbors or depths of a graph. This can be used for information retrieval.
There are two main graph search algorithms :
Breadth-First Search (BFS) which explores each node’s neighbor first, then neighbors of the neighbors...
Depth-First Search (DFS) which tries to go down a path as much as possible, and visit new neighbors if possible.
a. Shortest Path
Shortest Path calculates the shortest weighted (if the graph is weighted) path between a pair of nodes.
It is used to identify optimal driving directions or degree of separation between two people on a social network for example.
The are many ways to compute the shortest path in a graph, including the Dijkstra’s algorithm, the default algorithm in Networkx.
The pseudo-code for this algorithm according to Wikipedia is the following :
Mark all nodes of the graph as unvisited. Create a set of all the unvisited nodes called the unvisited set.Assign to every node a tentative distance value: set it to zero for our initial node and to infinity for all other nodes. Set the initial starting node as current.For the current node, consider all of its unvisited neighbors and calculate their tentative distances through the current node. Compare the newly calculated tentative distance to the current assigned value and assign the smaller one. For example, if the current node A is marked with a distance of 6, and the edge connecting it with a neighbor B has length 2, then the distance to B through A will be 6 + 2 = 8. If B was previously marked with a distance greater than 8 then change it to 8. Otherwise, keep the current value.When we are done considering all of the unvisited neighbors of the current node, mark the current node as visited and remove it from the unvisited set. A visited node will never be checked again.If the destination node has been marked visited (when planning a route between two specific nodes) or if the smallest tentative distance among the nodes in the unvisited set is infinity (when planning a complete traversal; occurs when there is no connection between the initial node and remaining unvisited nodes), then stop. The algorithm has finished.Otherwise, select the unvisited node that is marked with the smallest tentative distance, set it as the new “current node”, and go back to step 3.
Mark all nodes of the graph as unvisited. Create a set of all the unvisited nodes called the unvisited set.
Assign to every node a tentative distance value: set it to zero for our initial node and to infinity for all other nodes. Set the initial starting node as current.
For the current node, consider all of its unvisited neighbors and calculate their tentative distances through the current node. Compare the newly calculated tentative distance to the current assigned value and assign the smaller one. For example, if the current node A is marked with a distance of 6, and the edge connecting it with a neighbor B has length 2, then the distance to B through A will be 6 + 2 = 8. If B was previously marked with a distance greater than 8 then change it to 8. Otherwise, keep the current value.
When we are done considering all of the unvisited neighbors of the current node, mark the current node as visited and remove it from the unvisited set. A visited node will never be checked again.
If the destination node has been marked visited (when planning a route between two specific nodes) or if the smallest tentative distance among the nodes in the unvisited set is infinity (when planning a complete traversal; occurs when there is no connection between the initial node and remaining unvisited nodes), then stop. The algorithm has finished.
Otherwise, select the unvisited node that is marked with the smallest tentative distance, set it as the new “current node”, and go back to step 3.
If you’d like to read more on the shortest path problem, check this article: https://en.wikipedia.org/wiki/Shortest_path_problem
In Python, the implementation is straightforward :
# Returns shortest path between each nodenx.shortest_path(G_karate)
This returns a list of the minimal path between each node of the graph :
{0: {0: [0],    1: [0, 1],    2: [0, 2],    ...
b. Single Source Shortest Path
The Single Source Shortest Path (SSSP) finds the shortest path between a given node and all other nodes in the graph.
It is often used for routing protocol for IP networks for example.
c. All Pairs Shortest Path
The “All Pairs Shortest Path” (APSP) algorithm finds the shortest path between all pairs of nodes.
Although providing similar results, it is quicker than calling the Single Source Shortest Path for every pair of nodes. This algorithm can typically be used to determine traffic load expected on different segments of a transportation grid.
# Returns shortest path length between each nodelist(nx.all_pairs_shortest_path_length(G_karate))
Which returns :
[(0,    {0: 0,    1: 1,    2: 1,    3: 1,    4: 1,    ...
d. Minimum Weight Spanning Tree
A minimum spanning tree is a subgraph of the graph (a tree) with the minimum sum of edge weights that connects all nodes within a graph.
Minimum spanning tree should be applied to undirected graphs.
from networkx.algorithms import treemst = tree.minimum_spanning_edges(G_karate, algorithm='prim', data=False)edgelist = list(mst)sorted(edgelist)
Which returns :
[(0, 1),(0, 2),(0, 3),(0, 4),(0, 5),(0, 6),...
Community detection partitions the nodes into a several groups according to a given quality criterion.
It is typically used to identify social communities, customers behaviors or web pages topics.
A community is a set of connected nodes. There is however no universal definition that one can give to define communities, although the nodes within a community should be densely connected.
A common algorithm to find communities is the Girvan Newman algorithm. It identifies communities by progressively removing edges within the network. We’ll refer to betweenness as the “edge betweenness”. It is a score proportional to the number of shortest paths between pairs of nodes that go through this edge.
The steps of this algorithm are the following :
Compute the betweenness of all existing edges in the network.Remove the edge with the highest betweenness.Recompute the betweenness of all edges after the removal of this edge.Steps 2 and 3 are repeated until no edges remain.
Compute the betweenness of all existing edges in the network.
Remove the edge with the highest betweenness.
Recompute the betweenness of all edges after the removal of this edge.
Steps 2 and 3 are repeated until no edges remain.
To implement this in Python, you can use the following code :
from networkx.algorithms import communityk = 1comp = community.girvan_newman(G_karate)for communities in itertools.islice(comp, k):    print(tuple(sorted(c) for c in communities))
This heads a list of the nodes that belong to each community (k=1 means we expect 2 communities):
([0, 1, 3, 4, 5, 6, 7, 10, 11, 12, 13, 16, 17, 19, 21], [2, 8, 9, 14, 15, 18, 20, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33])
As stated above, this method does not really scale. For this reason, methods such as the Louvain method have been developed. Such methods are however really long to run on large graphs.
Before defining the Louvain method, it is important to introduce the notion of modularity. Modularity is a measure of how well groups have been partitioned into clusters :
The pseudo-code of the Louvain method is the following :
Assign a community to each node at first
Alternate the next 2 steps until convergence :
Create a new community with neighboring nodes to maximize modularity
Create a new weighted graph. Communities of the previous step become nodes of the graph
This might seem a bit confusing right now. In fact, the only thing we’re doing is to group the closest nodes so that we optimize the modularity criteria.
Note that there is no theoretical guarantee of the Louvain method, but it works well in practice. Louvain’s method is present as a sub-project of NetworkX, right here: https://python-louvain.readthedocs.io/en/latest/
First, install the package :
pip install python-louvain
Then, compute the best partition (based on the Louvain method):
import communitypartition = community.best_partition(G_karate)pos = nx.spring_layout(G_karate)plt.figure(figsize=(8, 8))plt.axis('off')nx.draw_networkx_nodes(G_karate, pos, node_size=600, cmap=plt.cm.RdYlBu, node_color=list(partition.values()))nx.draw_networkx_edges(G_karate, pos, alpha=0.3)plt.show(G_karate)
The Strongly Connected Components (SCC) algorithm finds groups of connected nodes in a directed graph. Note that each node must be reachable in both directions from any other node in the same group.
It is often used early in a graph analysis process to give us an idea of how our graph is structured, for example, to explore financial statements data when we look at who owns shared in what company (think about Panama papers for example).
The Weakly Connected Components, or Union Find algorithm finds sets of connected nodes in a directed graph where each node is reachable from any other node in the same set.
It only needs a path to exist between pairs of nodes in one direction, whereas SCC needs a path to exist in both directions. As with SCC, Union Find is often used early in analysis to understand a graph’s structure.
Union-Find is a pre-processing step that is essential before any kind of algorithm, to understand the graph’s structure.
We can test for connected directed graphs using :
nx.is_weakly_connected(G)nx.is_strongly_connected(G)
Or for undirected graphs using :
nx.is_connected(G_karate)
Which returns a boolean.
Make sure to check the Networkx documentation on the Connectivity for implementations.
In hierarchical clustering, we build a hierarchy of clusters. We represent the clusters under a form a dendrogram.
The idea is to analyze community structures at different scales. We usually build the dendrogram bottom-up. We start with a cluster at each node and merge the two “closest” nodes.
But how do we measure if clusters are close? We use similarity distances. Let d(i,j) be the length of the shortest path between i and j.
For the maximum linkage, at each step, the two clusters separated by the shortest distance are combined. The similarity distances can be illustrated as follows :
Back to our Karate example. Before applying hierarchical clustering, we need to define the matrix of distances between each node.
pcc_longueurs=list(nx.all_pairs_shortest_path_length(G_karate))distances=np.zeros((n,n))# distances[i, j] is the length of the shortest path between i and jfor i in range(n):    for j in range(n):        distances[i, j] = pcc_longueurs[i][1][j]
Now, we’ll use the AgglomerativeClustering function of sklearnto identify hierarchical clustering.
from sklearn.cluster import AgglomerativeClusteringclustering = AgglomerativeClustering(n_clusters=2,linkage='average',affinity='precomputed').fit_predict(distances)
And finally, draw the resulting graph with colors depending on the cluster :
nx.draw(G_karate,  node_color = clustering)
The clustering coefficient measures how well two nodes tend to cluster together.
The local clustering coefficient is a ratio of the number of triangles centered at node i over the number of triples centered at node i. In some sense, it measures how close a node i and its neighbors are to being a complete graph.
I have tried to illustrate this computation of clustering coefficients for the following graph :
A global coefficient measures the density of triangles (local clusters) in the graph :
In the graph shown above, the clustering coefficient is equal to :
For Erdos-Rényi random graphs, E[Clustering Coefficient]=E[Ci]=p where p the probability defined in the previous article.
For Baràbasi-Albert random graphs, the global clustering coefficient follows a power law depending on the number of nodes. The average clustering coefficient of nodes with degree k is proportional to the inverse of k:
Nodes with a low degree are connected to other nodes in their community. Nodes with high degrees are linked to nodes in different communities.
For a given graph, in networkx, the clustering coefficient can be easily computed. First, let’s begin with the local clustering coefficients :
# List of local clustering coefficientslist(nx.clustering(G_barabasi).values())
This should return something quite similar to :
0.13636363636363635,0.2,0.07602339181286549,0.04843304843304843,0.09,0.055384615384615386,0.07017543859649122,...
And average the results to find the global clustering coefficient of the graph :
# Global clustering coefficientnp.mean(list(nx.clustering(G_barabasi).values()))
Which heads :
0.0965577637155059
Centrality measures how important a node is. This is not a clear definition, but it’s useful when we want to identify important web pages, bottlenecks in transportation networks...
A walk is a path which can go through the same node several times. Centrality measures vary with the type of walk considered and the way of counting them.
PageRank estimates a current node’s importance from its linked neighbors and then again from their respective neighbors.
Although popularized by Google, it’s a way of detecting influential nodes in any network. It is for example used to suggest connections on social networks.
PageRank is computed by either iteratively distributing one node’s rank (originally based on the degree) over its neighbors or by randomly traversing the graph and counting the frequency of hitting each node during these walks.
PageRank is usually computed on directed graphs. However, it will also execute on undirected graphs by converting each edge in the directed graph to two edges.
For example, the PageRank of the Karate graph can be accessed by :
nx.pagerank(G_karate, alpha=0.9)
Where alpha is the damping parameter (by default 0.85). It should give you a list of rankings in return :
{0: 0.09923208031303203, 1: 0.0543403155825792, 2: 0.05919704684187155, 3: 0.036612460562853694,...
Degree Centrality counts the number of walks of length 1 ending at node i.
It measures incoming and outgoing relationship. It is given by C(Xi)=di. Degree Centrality is used to identify the most influential persons on a social network for example.
c_degree = nx.degree_centrality(G_karate)c_degree = list(c_degree.values())
Eigenvector Centrality is the number of walks of infinite length ending at node i.
This gives more importance to nodes with well-connected neighbors.
c_eigenvector = nx.eigenvector_centrality(G_karate)c_eigenvector = list(c_eigenvector.values())
Closeness Centrality detects nodes that are can spread information efficiently through a graph.
It can be used to identify fake news accounts or in terrorist cells to isolate the individuals that can spread information to the rest of the graph.
Closeness Centrality is inversely proportional to the sum of lengths of the shortest paths to other nodes.
c_closeness = nx.closeness_centrality(G_karate)c_closeness = list(c_closeness.values())
Betweenness Centrality detects the amount of influence a node has over the flow of information in a graph.
It is often used to find nodes that serve as a bridge from one part of a graph to another, for example in the package delivery processor in a telecommunication network, or in the propagation of fake news.
Where :
σjk the number of shortest paths between j and k
σjk(i) the number of shortest paths between j and k going through i
The betweenness centrality measures the number of times a node acts as a bridge between two nodes. For example :
c_betweenness = nx.betweenness_centrality(G_karate)c_betweenness = list(c_betweenness.values())
In Python, the implementation relies on the built-in functions of networkx :
# Plot the centrality of the nodesplt.figure(figsize=(18, 12))# Degree Centralityf, axarr = plt.subplots(2, 2, num=1)plt.sca(axarr[0,0])nx.draw(G_karate, cmap = plt.get_cmap('inferno'), node_color = c_degree, node_size=300, pos=pos, with_labels=True)axarr[0,0].set_title('Degree Centrality', size=16)# Eigenvalue Centralityplt.sca(axarr[0,1])nx.draw(G_karate, cmap = plt.get_cmap('inferno'), node_color = c_eigenvector, node_size=300, pos=pos, with_labels=True)axarr[0,1].set_title('Eigenvalue Centrality', size=16)# Proximity Centralityplt.sca(axarr[1,0])nx.draw(G_karate, cmap = plt.get_cmap('inferno'), node_color = c_closeness, node_size=300, pos=pos, with_labels=True)axarr[1,0].set_title('Proximity Centrality', size=16)# Betweenness Centralityplt.sca(axarr[1,1])nx.draw(G_karate, cmap = plt.get_cmap('inferno'), node_color = c_betweenness, node_size=300, pos=pos, with_labels=True)axarr[1,1].set_title('Betweenness Centrality', size=16)
We observe that the different nodes highlighted by the centrality measures are quite distinct. Betweenness centrality, for example, produces results far from the other methods, since they don’t measure the same thing.
We have now covered the introduction to graphs, the main types of graphs, the different graph algorithms and their implementation in Python with Networkx.
In the next article, we’ll cover graph learning which provides ways to predict nodes and edges in a graph to handle missing values or predict new relations.
Feel free to comment if you have any question or remark. Stay tuned, the last article of this series is coming out next week :)
A Comprehensive Guide to Graph Algorithms in Neo4j, Mark Needham & Amy E. Hodler
Networkx documentation, https://networkx.github.io/documentation/stable/
If you’d like to read more from me, my previous articles can be found here: