Graph Neural Network (GNN) Architectures for Recommendation Systems
By: Santhosh Rajamanickam
Tags: Graph Neural Networks, Recommender Systems, Gnn, Machine Learning, Recommendation System

GNNs for recommendation

Recommendation systems are used to generate a list of recommended items for a given user(s). Recommendations are drawn from the available set of items (e.g., movies, groceries, webpages, research papers, etc.,) and are tailored to individual users, based on:

user’s preferences (implicit or explicit),

item features,

and/or user<->item past interactions.

The quantity and the quality of the user and item data determine the quality of the recommendations. Most of the current state-of-the-art recommender systems use deep learning techniques. For a comprehensive overview of these techniques, please check out Zhang, Shuai, et al’s survey paper from 2019 [4].

From the above description of recommender systems, one can model the data as a graph: with users and items as the nodes and the edges representing the relation between the nodes. The relationships can be extracted/inferred from the input data of most recommender systems.

There are models available to tackle sequential recommendation [2], where the system tries to recommend the next item based on the “interaction order” between the user and items in the previous time steps. For simplicity, I have focused only on general recommendations (the order of recommendations is independent).

Depending on how we model the available data as a graph, there are several types of GNN based models available to solve the recommendation problem. These models can broadly be classified into 3 types:

GNNs using user-item interaction information GNNs enhanced with social network information GNNs enhanced with knowledge graphs

To illustrate the three types of GNN models for recommendation, consider the problem of content recommendations to experts, which will be used throughout this post. The goal is to recommend content similar to the publications previously written by the expert. Information such as title, introduction, keywords, etc., could potentially be used as the features of the content. Additionally, we could model the experts by leveraging their research interests, conferences attended, etc.

A closer look: GNNs using user-item interaction information

In this category of models, the data is modelled as a bipartite graph. In our example, the experts constitute one set of nodes, the content constitutes the other set of nodes, and edges exist between the nodes if the expert is one of the original authors of the content.

The above image shows an example of a user-item interaction graph, which is a bipartite graph consisting of expert nodes and content nodes, with edges representing whether the expert was one of the authors of the content. Image by author.

For starters, the following are popular models in the category of GNNs using user-item interaction, whose implementations are available, that are easy to experiment with:

GCN

Graph Convolutional Network (GCN) [3] is one of the earliest works in GNN. Neural Graph Collaborative Filtering (NGCF) [5] is a GCN variant that uses the user-item interactions to learn the collaborative signal, which reveals behavioral similarity between users, to improve recommendations. Rating predictions on Yelp2018 and Amazon-book datasets were used to measure the performance of the model.

Implementation: GCN — PyG, NGCF: Tensorflow

GraphSage

GraphSage [6] is a framework that proposes sampling fixed-sized neighborhoods instead of using all the neighbors of each node for aggregation. It also provides min, max, or sum pooling as options for aggregators and uses concatenation operation to update node/edge/graph representations. PinSage [7] is a variant of GraphSage proposed as a solution to handle web-scale graphs. PinSage, from Pinterest, introduces a sampling technique that samples fixed-size neighborhoods based on the highest visit counts. Alternatively, visit counts can be substituted with any feature that differentiates the node’s importance.

Implementation: GraphSage — PyG, PinSage — DGL

GAT

Graph Attention Networks [8] uses an attention mechanism to learn the influence of neighbors; this influence is used to determine the contribution of neighbors during the aggregation step. Multi-Component Graph Convolutional Collaborative Filtering (MCCF) [9] is one such approach that learns latent purchasing motivation using an attention mechanism and combines it with features from explicit user-item interactions for better recommendations. The authors benchmarked MCCF’s performance on MovieLens, Amazon Product Recommendation, and Yelp datasets.

Implementation: GAT — PyG, MCCF — Pytorch

A closer look: GNNs enhanced with social network information

In this category of models, the data is modelled either as:

two graphs: a bipartite user-item graph and a separate social network graph made up of users and edges representing some relation between them, or a heterogeneous graph consisting of users and items interlinked with user-user and user-item edges.

Think back to our example of matching content to experts. We could model a social network graph based on their interests, common groups they might belong to, or any other social or professional ties. Defining social network graphs in such a way, and using the learned features, has shown to be beneficial for many downstream tasks including recommendation [2].

The above image shows an example of a social network graph. Image by author.

Below are some of the interesting models in this category (GNNs with Social Network Information) whose source code is available:

DiffNet++

DiffNet++ [10] proposes a network that learns better user embeddings from two separate graphs, with latent user interest signals coming from the user-item graph and user influence signals coming from the user-user graph. It was benchmarked on Yelp, Epinions, and Dianping.

Implementation: DiffNet++ — TensorFlow

GraphRec

GraphRec [11] is another framework that takes advantage of the social network of the users to get better recommendations. In this approach, the model captures the user-item interactions and user opinions from the same user-item heterogenous (non-bipartite) graph. The framework performs:

user modelling, where user embeddings are enriched, leveraging the user-item aggregation and user-user embeddings (social ties), and item modelling, where item embeddings are enriched using user-item embeddings (i.e., what multiple users think about that item).

For the downstream task of rating prediction, concatenation of user and item embeddings are used as input. The framework benchmarks on the Epinions, and Ciao dataset.

Implementation: GraphRec — PyTorch

A closer look: GNNs enhanced with knowledge graphs

Models in this category focus on improving the item representation, which in turn leads to better item recommendations based on the user’s past interaction(s) with comparable items. Knowledge graphs are common in recommendation domains which link or group items based on their attributes. However, these graphs can be challenging to process due to the complex structure of multi-type relations between items.

Back to our example of matching content to experts, a knowledge graph could be formed using various attributes of the content such as common themes or subjects.

The above image illustrates an example of a knowledge graph, where the nodes are contents interlinked by different relations. Such relations can be exploited to enrich the representation of both the experts and the content nodes. Image by author.

One particularly interesting model in the category of GNNs enhanced with knowledge graphs is KGAT:

KGAT

Knowledge Graph Attention Network for recommendation (KGAT) [12] is based on GAT. It constructs a heterogenous graph that consists of users, items, and attributes as nodes. It further recursively propagates the embeddings from a node’s neighbors to aggregate and updates each node embedding. The neighbors can be any type of node (e.g., user, item, or attribute) and use an attention mechanism to learn the importance of the neighbors. The rating prediction task was used in performance analysis and was measured on Amazon-books, Yelp2018, and Last-FM datasets.

Implementation: KGAT — Tensorflow