## 2 Background and Related Work
### Graph Neural Networks for Link Prediction  
Graph Neural Networks (GNNs) [25; 26] are a popular machine learning paradigm that lets traditional feature-based neural network learning account for the relational information of data samples organized into graphs. This is achieved through message-passing protocols that gather and aggregate latent representations of graph neighbors, which are then transformed with neural network layers shared between all nodes before being passed on. Many industry-level applications focus exclusively on GNNs that employ the shift operation of graph signal processing as the aggregation operation, since the latter performs a (weighted) averaging of graph neighbor representations that can be efficiently implemented with sparse matrix multiplication within GPUs. All GNNs start from initial matrices of node representations \(H^{(0)}\), whose rows \(H^{(0)}[u]\) correspond to features of nodes \(u\). These could be unsupervised embeddings obtained by multilayer architectures and trained end-to-end [27] or other pre-processed machine learning features, such as weighted bag-of-word vectors. Then, given normalized adjacency matrices \(\hat{A}\), convolutional GNNs average graph neighbor representations where these are weighted by corresponding edge weights. This kind of smoothing is understood as a natural extension of graph signal processing to vector-valued graph signals can be expressed in matrix form with the operation \(\hat{A}H^{(\ell)}\), where \(H^{(\ell)}\) are matrices of (latent) node representations. Most GNNs add computational stability to the graph shift operation with a practice dubbed the renormalization trick. This adds self-loops to all nodes before computing the normalized adjacency matrix and will also be used throughout this work. Compared to the original matrix, the renormalization trick computes \(\hat{A}=(I+D)^{-1/2}(I+A)(I+D)^{-1/2}\), where \(I\) the unit matrix. Since matrix multiplication can be efficiently computed by modern GPUs, especially if graphs are not fully connected and sparse representations can be leveraged to make computation time scales with the number of edges, convolutional GNNs have become a widely popular variety for analyzing the graphs of many nodes and edges [25; 26]. Original GNN approaches (e.g., the architecture of Kipf and Welling [28] that helped popularize the domain) defined graph convolutional layers per:  
\[H^{(\ell)}=\sigma(\hat{A}H^{(\ell-1)}W^{(\ell)}) \tag{5}\]  
where \(\sigma(\cdot)\) are nonlinear activation functions applied on matrices element-by-element, such as rectified linear unit activations \(ReLU(x)=\max\{x,0\}\)[29] and \(W^{(\ell)}\) are learnable weights that help determine the output of GNN layers \(\ell=1,\dots,L\). The output of the final layer is used for predictions, which for node classification have dimensions equal to the number of classes and arise from a softmax activation on the top layer to obtain an estimation of a binary one-hot encoding of class labels. On the other hand, for link prediction tasks, any number of latent representation dimensions can be outputted and compared pairwise to select the most similar pairs of nodes to recommend links for, for example, through a sigmoid activation of their dot product [30]. To avoid the oversmoothing of representations along multiple graph convolutions, state-of-the-art GNNs often include recurrent terms in the predictions. This is achieved either by adding feedback loops that trade-off between layer outputs (before being passed through the activation function) and \(H^{(\ell)}\) with linear or feature-specific terms [31; 32], or by aggregating the outcomes of all convolutional layers [27]. Recursive loops effectively inject the graph signal transformations of trained features in all layers. One popular link prediction framework using GNNs is NGCF [27] which is also employed by the aforementioned GRec library recommendation system.