# Nearest Neighbours Graph Variational AutoEncoder
## 2 Materials and Methods
### Nearest Neighbour Graph VAE
#### 2.1.2 ReNN Graph VAE Architecture  
A Variational AutoEncoder is a generative model with an encoder-decoder structure. First the encoder learns to represent high dimensional input data as a distribution in a low dimensional space, called \"latent space\". The decoder is then trained to recover the original data, sampling a point from such distribution. We refer the reader to the Appendix B and the original paper [2] for a detailed description of the model.  
In our architecture, the encoding consists of three subsequent repetitions of two operations: a graph convolution and a ReNN-Pool operation. For the graph convolution, we chose to use the GraphConv layer [11]. With this convolution, the new features of nodes are just linear combinations between their old features and the mean of the features of their neighbourhood, followed by a ReLu activation:  
\[x_{i}^{\prime}=ReLu\left(W_{1}x_{i}+W_{2}\frac{1}{|N(i)|}\sum_{j\in N(i)}x_{j} \right). \tag{1}\]  
Figure 2: **Un-Pooling.** The Un-Pool operation consists in embedding the nodes of a pooled graph in their initial position in the bigger original graph structure. All other restored nodes are initialized to 0.  
Eventually, to increase the expressive power of the network, one can also include edge features \(e_{ij}\) in the computation, considering instead:  
\[x_{i}^{\prime}=ReLu\left(W_{1}x_{i}+W_{2}\frac{1}{|N(i)|}\sum_{j\in N(i)}e_{ij}x _{j}\right). \tag{2}\]  
In particular, we consider edge features \(e_{ij}\) to be learned parameters of the Neural Network.  
The number of output channels of the GraphConv in the three subsequent repetitions is set to 16, 32, and 64.  
After the three graph encoding steps, the node features are put together and flattened. Then, the dimensionality of data is further reduced through a linear layer.  
The encoded data is processed as in a standard VAE architecture with Gaussian prior to that.  
Data points are mapped to Gaussian distributions in the latent space and, using the reparameterisation trick, a variable \(Z\) is sampled from those distributions.  
The decoding uses the same graph representations employed for the encoding, but in reverse order. After an initial decoding linear layer, we repeat for three times the series of an un-pooling layer and a graph convolution. For the convolutions, we employ again the GraphConv layers with the output channels' number set to 32, 16 and 1. In this way, the original dimensionality of the data is recovered. The activation function of the last convolutional layer is a sigmoid. The model is trained minimizing the standard \(\beta\)-VAE loss function, defined in Appendix B, with binary cross entropy as the reconstruction term.  
The concatenation of a graph convolution and a pooling operation can be thought of as an \"encoding block\", while the union of an un-pooling operation and a convolution can be thought of as a \"decoding block\". Various architectures can be built using different numbers of blocks. In Figure 3, for example, our VAE architecture is illustrated, but with only two blocks in the encoder and decoder.