Graph Machine Learning with Python Part 1: Basics, Metrics, and Algorithms | by Ani Madurkar | Towards Data Science

Graph-based methods are some of the most fascinating and powerful techniques in the Data Science world today. Even so, I believe we’re in the early stages of widespread adoption of these methods. In this series, I’ll provide an extensive walkthrough of Graph Machine Learning starting with an overview of metrics and algorithms. I’ll also provide implementation code via Python to keep things as applied as possible. Before we get started, let’s discuss the value of graph-based methods.
Why Graphs?Network BasicsNetwork ConnectivityNetwork DistanceNetwork ClusteringNetwork Degree DistributionsNetwork CentralitySummary
Why Graphs?
Network Basics
Network Connectivity
Network Distance
Network Clustering
Network Degree Distributions
Network Centrality
Summary
Graphs are a general language for describing and analyzing entities with relations/interactions
Graphs are prevalent all around us from computer networks to social networks to disease pathways. Networks are often referred to as graphs that occur naturally, but the line is quite blurred and they do get used interchangeably.
Dr. Jure Leskovec, in his Machine Learning for Graphs course, outlines a few examples such as:
Graphs (as a representation):
Information/knowledge are organized and linked
Software can be represented as a graph
Similarity networks: Connect similar data points
Relational structures: Molecules, Scene graphs, 3D shapes, Particle-based physics simulations
Networks (also known as Natural Graphs):
Social networks: Society is a collection of 7+ billion individuals
Communication and transactions: Electronic devices, phone calls, financial transactions
Biomedicine: Interactions between genes/proteins regulate life
Brain connections: Our thoughts are hidden in the connections between billions of neurons
Representing data as a graph allows us to embed complex structural information as features. This yields higher performance in some domains as relational structure can provide a plethora of valuable information. In addition to a stronger feature representation, graph-based methods (specifically for Deep Learning) leverages representation learning to automatically learn features and represent them as an embedding. Due to this, a large amount of high dimensional information can be encoded in a sparse space without sacrificing speed/performance significantly. Another noteworthy benefit of leveraging graphs is the variety of tasks one can use them for. Dr. Leskovec provides insight into classic applications:
Node classification: Predict a property of a node. Example: Categorize online users/items
Link prediction: Predict whether there are missing links between two nodes. Example: Knowledge graph completion, recommender systems
Graph classification: Categorize different graphs. Example: Molecule property prediction
Clustering: Detect if nodes form a community. Example: Social circle detection
Graph generation: Drug discovery
Graph evolution: Physical simulation
I kept it brief here, but I highly recommend reviewing the slides from Dr. Leskovec’s first lecture if you’d like a deeper review of applications of Graph Machine Learning.
Although we are able to embed high-dimensional data to achieve higher performance models for a variety of tasks, networks can be incredibly complex. This is due to:
Arbitrary size and complex topological structureNo fixed node ordering or reference pointOften dynamic and have multimodal features
Arbitrary size and complex topological structure
No fixed node ordering or reference point
Often dynamic and have multimodal features
We often need statistical analysis, models, and algorithms to assist our ability to understand and reason from networks. Network models assist to simulate dispersion and cascade of information through a network due to its inherent relational structure. This yields tremendous insight of how knowledge, information, etc. propagates instead of just what propagates.
A network (or graph) is a representation of connections among a set of items. This representation is often written as G=(V,E) , where V={V1,...,Vn} is a set of nodes (also called vertices) and E={{Vk,Vw},..,{Vi,Vj}} is a set of two-sets (set of two elements) of edges (also called links), representing the connection between two nodes belonging to V. In a network visualization, distance and location carries no meaning.
Networks can also take a series of different structures and attributes.
Undirected: Have no direction, useful for cases where relationships are symmetric
Directed: Have direction, useful for asymmetrical relationships
Cyclic: Paths start and end at the same node, endless cycles may prevent termination without caution
Acyclic: Paths start and end at different nodes, basis of many algorithms
Weighted: Not all relationships are equal, some carry more weight
Unweighted: All relationships are equal (according to network) shown by equal/non-existent weighting
Sparse: Every node in the subset may not have a path to every other node
Dense: Every node in the subset has a path to every other node
Monopartite, bipartite, and k-partite: Whether nodes connect to only one other node type (e.g., users like movies) or many other node types (e.g., users like users who like movies)
Neo4J provides a great summary visualization for each:
Networks also have some basic properties that advanced methods and techniques build upon.
The order of a graph is the number of its vertices |V|. The size of a graph is the number of its edges |E|.
The degree of a vertex is the number of edges that are adjacent to it. The neighbors of a vertex v in a graph G is a subset of vertex Vi induced by all vertices adjacent to v.
The neighborhood graph (also known as an ego graph) of a vertex v in a graph G is a subgraph of G, composed of the vertices adjacent to v and all edges connecting vertices adjacent to v.
There are numerous datasets with a preloaded network structure available to do work on. This is due to the graciousness of the research and applied community sharing their work and datasets. To get started with our own network, we can load in one of these NetworkX’s datasets and as a sports fan I’ll choose the Football dataset (M. Girvan and M. E. J. Newman, Community structure in social and biological networks, Proc. Natl. Acad. Sci. USA 99, 7821–7826 (2002)). This dataset is open licensed by Girvan and Newman and shown on NetworkX Datasets.
Graph Summary:Number of nodes : 115Number of edges : 613Maximum degree : 12Minimum degree : 7Average degree : 10.660869565217391Median degree : 11.0...
A connected graph is a graph where every pair of nodes has a path between them. In a graph, there can be multiple connected components; these are subsets of nodes such that: 1. every node in the subset has a path to every other node, 2. no other node has a path to any node in the subset.
Strongly connected components are considered subsets of nodes that: 1. every node in the subset has a path to every other node, 2. no other node has a path to and from every node in the subset.
Weakly connected components are subsets of nodes such that replacing all of its directed edges with undirected edges produces a connected (undirected) graph, or all the components are connected by some path, ignoring direction.
Graph Summary:Number of nodes : 115Number of edges : 613Maximum degree : 12Minimum degree : 7Average degree : 10.660869565217391Median degree : 11.0Graph ConnectivityConnected Components : 1...
Distance between two nodes is the length of the shortest path between them. Path length is identified by the number of steps it contains from beginning to end to reach node y from x.
Finding this distance, especially with large scale graphs, can be really computationally expensive. There are two algorithms that are at the core of graph theory here:
Breadth-First Search (BFS): “discovers” nodes in layers based on connectivity. It starts at the root node and finds all nodes in the most immediate layer of connectivity before traversing the graph further.
Depth-First Search (DFS): “visits” nodes by traversing the graph from the root node all the way to its first leaf node before going down a different route in the graph.
When we want to aggregate this up to a graph level, there are two common ways to do so:
Average Distance: average distance between every pair of nodes. This usually is restricted to largest component when network is unconnected.Diameter: max distance between any pair of nodes.
Average Distance: average distance between every pair of nodes. This usually is restricted to largest component when network is unconnected.
Diameter: max distance between any pair of nodes.
They each should be used in pair with domain knowledge of the data you’re modeling as a graph.
Graph Summary:Number of nodes : 115Number of edges : 613Maximum degree : 12Minimum degree : 7Average degree : 10.660869565217391Median degree : 11.0Graph ConnectivityConnected Components : 1Graph DistanceAverage Distance : 2.5081617086193746Diameter : 4...
Clustering is an important assessment of networks to start decomposing and understanding their complexity. Triadic closure in a graph is the tendency for nodes who share edges to become connected. This is commonly done in social network graphs when person A is friends with person B and person B is friends with person C, so a recommendation to person A may be to befriend person C. This is founded is evidence that has shown in most real-world networks, mainly social networks, nodes tend to create tightly knit groups represented by a relatively high density of ties.
A way to measure the tendency of clustering in a graph is the clustering coefficient. There are two common ways to measure the clustering coefficient: local and global.
Local Clustering Coefficient: fraction of pairs of the node’s friends that are friends with each other.
Formula: # of pairs of A’s friends who are friends / # of pairs of A’s friends
Global Clustering Coefficient has two approaches:
Average local clustering coefficient over all nodes in the graph.Transitivity: percentage of “open triads” that are triangles in a network. This weights nodes with large degree higher.
Average local clustering coefficient over all nodes in the graph.
Transitivity: percentage of “open triads” that are triangles in a network. This weights nodes with large degree higher.
Graph Summary:Number of nodes : 115Number of edges : 613Maximum degree : 12Minimum degree : 7Average degree : 10.660869565217391Median degree : 11.0Graph ConnectivityConnected Components : 1Graph DistanceAverage Distance : 2.5081617086193746Diameter : 4Graph ClusteringTransitivity : 0.4072398190045249Average Clustering Coefficient : 0.40321601104209814
The degree of a node in an undirected graph is the number of neighbors it has. Degree distributions of a graph is the probability distribution of the degrees over the entire network.
In Undirected graphs, it’s simply referred to as degree but for Directed graphs we get in-degree and out-degree distributions. In-Degree distributions represent the distribution of in-links each node in the graph has. It has been argued, in real-world networks (notably social networks), when we plot the degree/in-degree distributions on a log-log scale it represents a power-law distribution. It has been debated that these scale-free networks are actually quite rare when using statistically rigorous techniques, which others have argued are overly restrictive to measure against. In any case, assessing the degree distribution is important to understand your network but it does not give insight into how the network may evolve over time.
Although we’re dealing with a very small and understandable network, these can easily scale up to uninterpretable complexity. When networks get that large it’s imperative to use centrality measures to guide us in understanding the data.
Centrality is a way to think about importance of nodes/edges in a graph. Depending on your domain/data, you should use different assumptions and this will naturally lead you to assess different centrality measures. Some of the ways you can quantify “importance” in a network: amount of degree of connectivity, average proximity to other nodes, fraction of shortest paths that pass through node, etc.
Some applications that centrality measures can be used for:
finding influential nodes in a social network
identifying nodes that disseminate information to many nodes or prevent epidemics
hubs in a transportation network
important pages on the web
nodes that prevent the network from breaking up
and many more!
There are a ton of centrality you can use; I’ll cover a handful key ones here, but I highly recommend reading NetworkX documentation of Graph literature to find key metrics that fit your domain.
For network visualizations, I’ll use nx-altair because it offers easy functionality for interaction and editing. Interaction can’t be seen in the images below, but if you run this code in your notebook you can add filters and hover pretty easily.
Assumption: important nodes have many connections.
This is the most basic measure of centrality: number of neighbors. It uses degree for Undirected networks and in-degree or out-degree for Directed networks. The degree centrality values are commonly normalized by dividing by the maximum possible degree in a simple graph n-1 where n is the number of nodes in G.
Assumption: important nodes are close to other nodes.
This is the reciprocal of the average shortest path distance to a node over all n-1 reachable nodes. If nodes are disconnected then you can either consider its closeness centrality based on only nodes that can reach it or you can consider only nodes that can reach it and normalize that value by the fraction of nodes it can reach.
Assumption: important nodes are close to other nodes.
Betweenness centrality of a node v is the sum of the fraction of all-pairs shortest paths that pass through v. The formula essentially looks at the number of shortest paths between nodes s and t that pass through node v and divides it by all number of shortest paths between s and t (and sums over all paths that don’t start or end with v). Betweenness centrality values will be larger in graphs with many nodes. To control for this, we divide centrality values by the number of pairs of nodes in the graph (excluding v). Complication for this metric arises when there’s multiple shortest paths in the network.
Since computation of this can be very expensive, it can be common to calculate this metric for a sample of node pairs. This metric can also be used to find important edges as well.
Assumption: important nodes are connected to central nodes.
Eigenvector centrality computes the centrality for a node based on the centrality of its neighbors. It measures the influence of a node in a network. Relative scores are assigned to all nodes in the network based on the concept that connections to high-scoring nodes contribute more to the score of the node in question than equal connections to low-scoring nodes. A high eigenvector score means that a node is connected to many nodes who themselves have high scores.
Assumption: important nodes are those with many in-links from other important nodes.
Algorithm:
Start on a random node.Assign all nodes a PageRank of 1/n.Choose an outgoing edge at random and follow it to the next node.Perform the Basic PageRank Update Rule: each node gives an equal share of its current PageRank to all the nodes it links to.The new PageRank of each node is the sum of all the PageRank it received from other nodes.Repeat k times.
Start on a random node.
Assign all nodes a PageRank of 1/n.
Choose an outgoing edge at random and follow it to the next node.
Perform the Basic PageRank Update Rule: each node gives an equal share of its current PageRank to all the nodes it links to.
The new PageRank of each node is the sum of all the PageRank it received from other nodes.
Repeat k times.
This is the infamous Google’s PageRank algorithm. It measures the importance of webpages from the hyperlink network structure. It assigns a score of importance to each node depending on how many links it has coming in from other nodes. In large networks, scaled PageRank is preferred as it comes with a “dampening parameter” alpha. This is a probability that an outgoing edge will be chosen at random to follow to another node in the algorithm which is especially beneficial when there’s a closed loop of outgoing nodes in a network.
Assumption: important nodes that have incoming edges from good hubs are good authorities, and nodes that have outgoing edges to good authorities are good hubs.
Algorithm:
Assign each node an authority and hub score of 1.Apply the Authority Update Rule: each node’s authority score is the sum of hub scores of each node that points to it.Apply the Hub Update Rule: each node’s hub score is the sum of the authority scores of each node that it points to.Normalize Authority and Hub scores of each node by the total score of each.Repeat k times.
Assign each node an authority and hub score of 1.
Apply the Authority Update Rule: each node’s authority score is the sum of hub scores of each node that points to it.
Apply the Hub Update Rule: each node’s hub score is the sum of the authority scores of each node that it points to.
Normalize Authority and Hub scores of each node by the total score of each.
Repeat k times.
The HITS algorithm starts with a root, or a set of highly relevant nodes (potential authorities). Then all nodes that link to a node in the root are potential hubs. The base is defined as root nodes and any node that links to a node in the root. All edges connecting nodes in the base set are considered, and this focuses on a specific subset of the network that is relevant to a particularly query.
The fundamentals of graph machine learning are connections between entities. As graphs get immensely large, it’s imperative to use metrics and algorithms to understand and get graph features. Depending on your context as well, different metrics and algorithms will prove useful and, more importantly, meaningful to your use case.
Furthermore, we can use these metrics as features in a supervised or unsupervised learning task but we have to be careful which we use because they can add as much noise as signal.
Other than many more metrics and algorithms, the depths of Graph ML covers a wide array of supervised and unsupervised learning tasks. From naïve to advanced techniques, we can use graph structure and inference to go beyond structural data. In future sections I’ll cover these machine learning tasks (node, edge, and graph level) on real data.
References
[1] Football dataset (M. Girvan and M. E. J. Newman, Community structure in social and biological networks, Proc. Natl. Acad. Sci. USA 99, 7821–7826 (2002))
[2] Claudio Stamile, Aldo Marzullo, Enrico Deusebio, Graph Machine Learning
[3] Mark Needham, Amy E. Hodler, Graph Algorithms
[4] Estelle Scifo, Hands-On Graph Analytics with Neo4j