A Beginner’s Guide to Graph Neural Networks Using PyTorch Geometric — Part 2 | by Rohith Teja | Towards Data Science

In my previous post, we saw how PyTorch Geometric library was used to construct a GNN model and formulate a Node Classification task on Zachary’s Karate Club dataset.
A graph neural network model requires initial node representations in order to train and previously, I employed the node degrees as these representations. But there are several ways to do it and another interesting way is to use learning-based methods like node embeddings as the numerical representations.
This is a small recap of the dataset and its visualization showing the two factions with two different colours.
Dataset: Zachary’s Karate Club.
Here, the nodes represent 34 students who were involved in the club and the links represent 78 different interactions between pairs of members outside the club. There are two different types of labels i.e, the two factions.
Our idea is to capture the network information using an array of numbers which are called low-dimensional embeddings. There exist different algorithms specifically for the purpose of learning numerical representations for graph nodes.
DeepWalk is a node embedding technique that is based on the Random Walk concept which I will be using in this example. In order to implement it, I picked the Graph Embedding python library that provides 5 different types of algorithms to generate the embeddings.
Firstly, install the Graph Embedding library and run the setup:
!git clone https://github.com/shenweichen/GraphEmbedding.gitcd GraphEmbedding/!python setup.py install
We use the DeepWalk model to learn the embeddings for our graph nodes. The variable embeddings stores the embeddings in form of a dictionary where the keys are the nodes and values are the embeddings themselves.
As I mentioned before, embeddings are just low-dimensional numerical representations of the network, therefore we can make a visualization of these embeddings. Here, the size of the embeddings is 128, so we need to employ t-SNE which is a dimensionality reduction technique. Basically, t-SNE transforms the 128 dimension array into a 2-dimensional array so that we can visualize it in a 2D space.
Note: The embedding size is a hyperparameter.
The visualization made using the above code looks like this:
We can see that the embeddings generated for this graph are of good quality as there is a clear separation between the red and blue points. Now we can build a graph neural network model which trains on these embeddings and finally, we will have a good prediction model.
I will reuse the code from my previous post for building the graph neural network model for the node classification task. The procedure we follow from now is very similar to my previous post. We just change the node features from degree to DeepWalk embeddings.
Here, we are just preparing the data which will be used to create the custom dataset in the next step. Notice how I changed the embeddings variable which holds the node embedding values generated from the DeepWalk algorithm.
In order to compare the results with my previous post, I am using a similar data split and conditions as before.
The data object now contains the following variables:
Data(edge_index=[2, 156], num_classes=[1], test_mask=[34], train_mask=[34], x=[34, 128], y=[34])
We can notice the change in dimensions of the x variable from 1 to 128.
We use the same code for constructing the graph convolutional network.
Now it is time to train the model and predict on the test set.
Using the same hyperparameters as before, we obtain the results as:
Train Accuracy: 1.0 Test Accuracy: 0.90
This is a huge improvement from before!
As seen from the results, we actually have a good improvement in both train and test accuracies when the GNN model was trained under similar conditions of Part 1.
Note: We can surely improve the results by doing hyperparameter tuning.
This shows that Graph Neural Networks perform better when we use learning-based node embeddings as the input feature. Now the question arises, why is this happening?
Answering that question takes a bit of explanation. So I will write a new post just to explain this behaviour. Stay tuned!
PyTorch Geometric: https://github.com/rusty1s/pytorch_geometricDeepWalk: https://arxiv.org/abs/1403.6652Graph Embedding library: https://github.com/shenweichen/GraphEmbeddingGCN code example: https://github.com/rusty1s/pytorch_geometric/blob/master/examples/gcn.py
PyTorch Geometric: https://github.com/rusty1s/pytorch_geometric
DeepWalk: https://arxiv.org/abs/1403.6652
Graph Embedding library: https://github.com/shenweichen/GraphEmbedding
GCN code example: https://github.com/rusty1s/pytorch_geometric/blob/master/examples/gcn.py
5. Link to Part 1 of this series. I strongly recommend checking this out:
towardsdatascience.com
I hope you enjoyed reading the post and you can find me on LinkedIn, Twitter or GitHub. Feel free to say hi!