Understanding Graph Mining. Your first baby step to learn Deep... | by Vincent Tatan | Towards Data Science

Imagine Facebook: How do you get connected within layers of friends?
Imagine Recommendation System: How do you know a person’s preference is closely related to its clusters?
Welcome to Graph Mining
Graph classification generates graphs among a vast amount of connected data (e.g: Social, Biological, and Payment) and uses the graphs to identify labels (supervised) or clusters (unsupervised).
If this sounds tough to you, you can look no further than your brain because it is the master at inferencing connections among graphs quickly.
For example, your brain knows how to get from point A to point B without even thinking. Imagine your last grocery trip after being on shopping duty for a long time. You can skim through the catalogues and make multiple beelines of different categories of aisles to compare products while bringing your children to shop.
All these actions require consolidations to infer the best action based on series of sub actions to reach your end goal. All done in the most effortless methods as possible.
Graph mining uses features to see how a set of observations are related from a user facing similarity signal.
Graphs represent relationships (edges) between entities (nodes) which are formulated based on distance.
There are two characteristics commonly found:
Natural graphs which come from an external source. For example payment networks, social media, and roadways.
Similarity Graphs which comes from measures of similarity distance between nodes. For example a blob of metadata then shares the blob structure via graph representation.
Depending on the characteristics of the graphs, we can classify each graph as
Simple homogeneous with one type of node and edge
Complex heterogeneous with multiple types of nodes, multiple types of edges. These can be directed or undirected.
Graphs are multipurpose. You can build supervised models by classifying surrounding neighbors, semi-supervised models by propagating existing labels to missing labels in the neighborhood, and unsupervised models by training node-level embeddings to describe structural role of the data
There are a wide variety of tasks you can complete with graphs. For example, you can classify nodes, graphs, and links depending on your use case.
These include multiple breakthroughs such as: Google DeepMind’s Super Segments Graph to predict Time of Arrival (ETA) and PinSage from Pinterest.
Graphs abstract local information and extract useful global information from data. Graph structure/topology can tell us a lot about our data such as uncovering clusters of data points, or providing distance measures for otherwise intangible concepts.
Graphs work flexibly by embedding and combining multiple types of features such as: textual (e.g: Jaccard similarity ), visual (e.g: image similarities), and other semantic attributes. For example, when analysts work on campaign rules detections, they would use several features such as similarity among regex setups and deploy the regex to capture these sites.
Graph Sampling is used to seek subgraphs on receptive fields. These subgraphs can be formulated on top of async distributions of workers: dedicated servers to fetch model parameters.
This scalability also gives ways to use federated learning in graphs to preserve the privacy of users (such as Android based ML training)
In this case study, we are going to use Cora data, which provides a citation network benchmark with 2708 nodes, 5429 edges, 7 topic bags of words, 140 training, 500 validation, and 1000 test observations.
Please feel free to use the following colab for your tutorial.
colab.sandbox.google.com
Spektral is used as the open source Python library for graph deep learning, based on the Keras and Tensorflow2. The main goal of this project is to provide a simple but flexible framework for creating graph neural networks (GNNs).
Spektral has a convenience function that will allow us to quickly load and preprocess standard graph representation learnings. It also allows functions to access feature matrix, adjacency, matrix, and mask arrays to understand which node belongs to the train, validation, and test set.
TF GNN library is Agnostic to Model API and integrates with DeepMind graphNet, Google Neural Structured Learning, TF GNN API based on MPNN paper. The graph can also be passed around Keras layers with batching/serialization supported with TPUs.
To install and import these libraries in colab is very simple
!pip install numpy!pip install tensorflow!pip install spektral==0.6.1import numpy as npimport tensorflow as tfimport spektral
adj refers to adjacency matrix, features to feature_matrix, labels to topics of paper, and mask sets to indicate which node belongs to which data sets.
adj, features, labels, train_mask, val_mask, test_mask = spektral.datasets.citation.load_data(dataset_name=’cora’)
We then replace the dense features to convert features to adjacency matrices while setting them up in float32 as a standard machine learning output format.
features = features.todense()adj = adj.todense() + np.eye(adj.shape[0])features = features.astype(‘float32’)adj = adj.astype(‘float32’)
We follow the following steps to train our GNN on Cora datasets:
Define 2 GNN layers (Input — units, output — 7 topics, with sumpooling)Implement GNN with the defined layers with Relu activation function.Use tf.GradientTape() to record operations for auto differentiation. This helps us watch variables and gradients to catch back propagation errorsApply optimizer to gradients (Adam opt is the standard)Specify logits and accuracy calculations based on features and adjacency matrices
Define 2 GNN layers (Input — units, output — 7 topics, with sumpooling)
Implement GNN with the defined layers with Relu activation function.
Use tf.GradientTape() to record operations for auto differentiation. This helps us watch variables and gradients to catch back propagation errors
Apply optimizer to gradients (Adam opt is the standard)
Specify logits and accuracy calculations based on features and adjacency matrices
You can experiment with parameters (units, epochs and learning rate) to define the validation accuracy and training/val loss.
train_cora(features,adj,gnn,32,200,0.01)## Results ##Epoch 0 | training loss: 1.9685615301132202 | val_accuracy: 0.20399999618530273 | test_accuracy:0.20799998939037323Epoch 1 | training loss: 2.1238698959350586 | val_accuracy: 0.30399999022483826 | test_accuracy:0.3100000023841858Epoch 2 | training loss: 1.6506766080856323 | val_accuracy: 0.6060000061988831 | test_accuracy:0.6150000095367432Epoch 3 | training loss: 1.504560112953186 | val_accuracy: 0.628000020980835 | test_accuracy:0.6359999775886536...Epoch 129 | training loss: 0.004622798413038254 | val_accuracy: 0.7520000338554382 | test_accuracy:0.7450000047683716
As you can see, it seems the more epochs and training step runs, the more accuracy we can obtain from it.
Done! Congrats for your very first Graph Network!
Vincent fights internet abuse with ML @ Google. Vincent uses advanced data analytics, machine learning, and software engineering to protect Chrome and Gmail users.
Apart from his stint at Google, Vincent is also Georgia Tech CS MSc Alumni, a triathlete, and a featured writer for Towards Data Science Medium to guide aspiring ML and data practitioners with 1M+ viewers globally.
Lastly, please reach out to Vincent via LinkedIn, Medium or Youtube Channel
Soli deo gloria
Google Graph Mining and Learning
Graph Neural Networks in TF and Keras with Spektral