Learning in Graphs with Python (Part 3) | by Maël Fabien | Towards Data Science

Graphs are becoming central to machine learning these days, whether you’d like to understand the structure of a social network by predicting potential connections, detecting fraud, understand customer’s behavior of a car rental service or making real-time recommendations for example.
In this article, we’ll cover :
The main graph learning algorithms
Use cases and implementations in Python
I publish all my articles and the corresponding code on this repository :
github.com
This is the final article on my Graph series. If you haven’t, make sure to check the first articles of this series:
towardsdatascience.com
towardsdatascience.com
For what comes next, open a Jupyter Notebook and import the following packages :
import numpy as npimport randomimport networkx as nxfrom IPython.display import Imageimport matplotlib.pyplot as plt
The following articles will be using the latest version 2.x ofnetworkx. NetworkX is a Python package for the creation, manipulation, and study of the structure, dynamics, and functions of complex networks.
I’ll try to keep a practical approach and illustrate most concepts.
There are three main tasks in graph learning that we will cover in this article:
Link prediction
Node label prediction
Graph embedding
Let’s start with Link prediction!
In Link Prediction, given a graph G, we aim to predict new edges. Predictions are useful to predict future relations or missing edges when the graph is not fully observed for example, or when new customers join a platform (e.g. a new LinkedIn user).
Link prediction for a new LinkedIn user would simply be a suggestion of people he might know.
In link prediction, we simply try to build a similarity measure between pairs of nodes and link the most similar nodes. The question is now to identify and compute the right similarity scores!
To illustrate the different similarity scores, let’s consider the following graph :
Let N(i) be a set of neighbors of node i. On the graph above, the neighbors of both nodes i and j can be represented as :
And the neighbors of i :
We can build several similarity scores for these two nodes based on their neighborhoods.
Common Neighbors: S(i,j)=∣N(i)∩N(j)∣, i.e the number of common neighbors. In this example, the score would be simply 12, since they share only 2 common neighbors.
Jaccard Coefficient: A normalized common neighbors version.
The intersection is the Common Neighbors, and the union is :
Therefore, the Jaccard Coefficient is given by the ratio of the pink over the yellow:
And the value is 2/7.
Adamic-Adar index: For each common neighbor of nodes i and j, we add 1 divided by the total number of neighbors of that node. The concept is that common elements with very large neighborhoods are less significant when predicting a connection between two nodes compared to elements shared between a small number of nodes.
Preferential attachment : S(i,j)=∣N(i,j)∣∗∣N(j)∣
We can also use community information when it is available.
How do we perform the evaluation of the link prediction? We must hide a subset of node pairs, and predict their links based on the rules defined above. This is the equivalent of the train-test-split in supervised learning.
We then evaluate the proportion of correct predictions for dense graphs, or use Area under the Curve criteria for Sparse graphs.
Let’s implement this in Python on the Karate graph we used in the 2 first articles! First of all, print the information about the graph :
n = G_karate.number_of_nodes()m = G_karate.number_of_edges()print("Number of nodes :", str(n))print("Number of edges :", str(m))print("Number of connected components :" str(nx.number_connected_components(G_karate)))
Then, plot the graph itself :
plt.figure(figsize=(12,8))nx.draw(G_karate)
Now, let’s remove some connections, e.g 25% of the nodes:
# Take a random sample of edgesedge_subset = random.sample(G_karate.edges(), int(0.25 * G_karate.number_of_edges()))# Remove some edgesG_karate_train = G_karate.copy()G_karate_train.remove_edges_from(edge_subset)
And plot the partially observed graph :
plt.figure(figsize=(12,8))nx.draw(G_karate_train)
You can print the number of edges we deleted and the number of edges remaining :
edge_subset_size = len(list(edge_subset))print("Deleted : ", str(edge_subset_size))print("Remaining : ", str((m - edge_subset_size)))
This heads :
Deleted : 15Remaining : 63
Jaccard Coefficient
We can first make a prediction using the Jaccard Coefficient :
prediction_jaccard = list(nx.jaccard_coefficient(G_karate_train))score, label = zip(*[(s, (u,v) in edge_subset) for (u,v,s) in prediction_jaccard])
The prediction looks like this, a first node, a second node, and a Jaccard score (or a label directly):
[(0, 32, 0.15),(0, 33, 0.125),(0, 3, 0.21428571428571427),(0, 9, 0.0),(0, 14, 0.0),(0, 15, 0.0),...
We can use the ROC-AUC criteria to compare the performance of the different models since we have both a label and probabilities.
Adamic-Adar
We can now repeat this for the Adamic-Adar Index :
prediction_adamic = list(nx.adamic_adar_index(G_karate_train))score, label = zip(*[(s, (u,v) in edge_subset) for (u,v,s) in prediction_adamic])
Preferential Attachment
And for the preferential attachment score :
prediction_pref = list(nx.preferential_attachment(G_karate_train))score, label = zip(*[(s, (u,v) in edge_subset) for (u,v,s) in prediction_pref])
We then select the model that maximizes the area under the curve for example, or any other criteria that you chose. So far, we covered the most common similarity scores for link prediction.
We’ll now cover into more details the node labeling algorithms.
Given a graph where some nodes are not labeled, we want to predict their labels. This is in some sense a semi-supervised learning problem.
One common way to deal with such problems is to make the assumption that there is a certain smoothness on the graph. The Smoothness assumption states that points connected via a path through high-density regions on the data are likely to have similar labels. This is the main hypothesis behind the Label Propagation Algorithm.
The Label Propagation Algorithm (LPA) is a fast algorithm for finding communities in a graph using network structure alone as its guide, without any predefined objective function or prior information about the communities.
A single label can quickly become dominant in a densely connected group of nodes, but it will have trouble crossing a sparsely connected region.
How does the semi-supervised label propagation work?
First, we have some data : x1, ..., xl, xl+1, ..., xn ∈ Rp, and labels for the first l points: y1,...,yl ∈ 1...C.
We define the initial label matrix Y ∈ R(n×C) such that Yij=1 if xi has label yi=j and 0 otherwise.
The algorithm will generate a prediction matrix F∈R(n×C) which we’ll detail below. Then, we predict the label of a node by finding the most likely label :
What is the prediction matrix F ?
The prediction matrix is a matrix F* that minimizes both smoothness and accuracy criteria. Therefore, there is a tradeoff to make between the smoothness and the accuracy of our result.
The problem expression is quite complex, so I won’t go into details. However, the solution is given by :
If you’d like to go further on this topic, check the notions of smoothness of a graph function and manifold regularization. I won’t cover the implementation of this part, but if you are interested, Stanford has a great set of labeled graphs that you can download: https://snap.stanford.edu/data/, and Networkx has a direct implementation of label propagation: https://networkx.github.io/documentation/latest/reference/algorithms/generated/networkx.algorithms.community.label_propagation.label_propagation_communities.html
When dealing with NLP or computer vision problems, we are used to working with embeddings of images or texts in a Deep Neural Net. One of the limitations of graphs with what we have seen so far is the absence of vector features. However, we can learn an embedding of the graph! There are several levels of embedding in a graph :
Embedding graph components (nodes, edges, features...) (Node2Vec)
Embedding sub-parts of a graph or a whole graph (Graph2Vec)
We will first focus on the embedding of graph components. Several approaches are possible to embed a node or an edge. For example, DeepWalk uses short random walks to learn representations for edges in graphs. We will talk about Node2Vec , a paper that was published by Aditya Grover and Jure Leskovec from Stanford University in 2016.
According to the authors: “node2vec is an algorithmic framework for representational learning on graphs. Given any graph, it can learn continuous feature representations for the nodes, which can then be used for various downstream machine learning tasks.”
The model learns low-dimensional representations for nodes by optimizing a neighborhood preserving objective, using random walks.
The code of Node2Vec is available on GitHub: https://github.com/eliorc/node2vec
To install the package, simply run:
pip install node2vec
Then, in your notebook, we’ll embed the Karate graph :
from node2vec import Node2Vec
Then, pre-compute the probabilities and generate walks :
node2vec = Node2Vec(G_karate, dimensions=64, walk_length=30, num_walks=200, workers=4)
We can then embed the nodes :
model = node2vec.fit(window=10, min_count=1, batch_words=4)
To get the vector of a node, say node ‘2’, use get_vector :
model.wv.get_vector(‘2’)
The outcome has the following form :
array([-0.03066591,  0.52942747, -0.14170371,  0.5471569 ,  0.07588464, -0.5693364 , -0.3017375 ,  0.21902356,  0.05244258 ...
It has a length of 64 since we defined the dimension as 64 above. What can we do with this embedding? One of the first options is for example to identify the most similar node!
model.wv.most_similar(‘2’)
It returns a list of the most similar nodes and the corresponding probabilities :
[('3', 0.6494477391242981), ('13', 0.6262941360473633), ('7', 0.6137452721595764),...
If the nodes have labels, we can train an algorithm based on the embedding and attach a label (node labeling, most similar node...)
Edges can also be embedded, and the embedding can be further used for classification.
from node2vec.edges import HadamardEmbedderedges_embs = HadamardEmbedder(keyed_vectors=model.wv)
Then, retrieve the vectors by specifying the name of the 2 linked nodes :
edges_embs[(‘1’, ‘2’)]
Which heads :
array([-8.1781112e-03, -1.8037426e-01,  4.9451444e-02,  2.8731486e-01...
Again, we can retrieve the most similar edge, which can be used for missing edges prediction for example :
edges_kv = edges_embs.as_keyed_vectors()edges_kv.most_similar(str((‘1’, ‘2’)))
This heads :
[("('2', '21')", 0.8726599216461182), ("('2', '7')", 0.856759786605835), ("('2', '3')", 0.8566413521766663),...
There are also ways to embed a graph or a sub-graph directly. This approach has been developed in the Graph2Vec paper and is useful to represent graphs or sub-graphs as vectors, thus allowing graph classification or graph similarity measures for example. I won’t dive deeper into this technique, but feel free to check the Github of this project :
github.com
To run the embedding, it’s as easy as :
python src/graph2vec.py --input-path data_folder/ --output-path output.csv
We have now covered the introduction to graphs, the main types of graphs, the different graph algorithms, their implementation in Python with Networkx, and graph learning techniques for node labeling, link prediction, and graph embedding.
Needless to say, this is only the tip of the iceberg. Graph Theory is ever-expanding, and I thought that listing some resources to go further could be useful :
Graph Convolution Networks : https://towardsdatascience.com/how-to-do-deep-learning-on-graphs-with-graph-convolutional-networks-7d2250723780
Geometric Deep Learning on Graphs and Manifolds : http://www.geometricdeeplearning.com/
A MOOC that could help: https://www.edx.org/course/advanced-algorithmics-and-graph-theory-with-python
If you have any other resources in mind which you feel could be useful, don’t hesitate to leave a comment and I’ll add it to the list!
I hope this series of articles was interesting to follow! Feel free to comment if you have any question or remark.
NB: This article was originally published on my personal blog: https://maelfabien.github.io/
A Comprehensive Guide to Graph Algorithms in Neo4j, Mark Needham & Amy E. Hodler
Networkx documentation, https://networkx.github.io/documentation/stable/
Graph Theory courses at Telecom Paris