# Continuous Latent Spaces Sampling for Graph Autoencoder
## 4 Present Work: colaGAE
### Training and Evaluating Setups  
The task of node classification involves predicting labels for unknown nodes. We evaluated the performance of our proposed colaGAE on eight standard benchmarks, including Cora, Citeseer, PubMed, and Ogbn-arxiv. These benchmarks are citation networks in which nodes represent documents and edges represent citations. The overall training process is summarized in Figure 1.  
The framework consists of a series of encoders, denoted as \(f_{0}\) to \(f_{n}\), where \(f_{0}\) is a GCN encoder and \(f_{1}\) to \(f_{n}\) are MLP encoders. The first encoder, \(f_{0}\), has a significant impact on performance, and we refer to it as \"the first encoder\" in subsequent sections for convenience. These encoders continuously encode node representations from one latent space into another, which can be viewed as a continuous sampling from different semantic spaces. Overall, colaGAE is a simple and effective framework for self-supervised graph autoencoder learning. First, we feed the entire graph \(\mathcal{G}=(\mathbf{X},A)\) into the series-encoder to generate different representations. In order to evaluate the effectiveness of the node representations learned by our model in reconstructing the adjacency matrix \(A\), we utilize both the reconstruction loss and relative distance loss, as outlined in Algorithm 1. The hyperparameters \(\alpha\) and \(\beta\) are used to adjust the weighting of these criteria in the overall performance of the model. Figure 1: The overall framework of colaGAE for SSL on graphs. ```
# A: adjacency matrix
# alpha: coefficient of reconstruction loss
# beta: coefficient of relative distance loss
model: GCN + mlp layers  class encoder():  def_init_(=self, n_layers = 1):  super(encoder, self)_init_()  basic_block = {  Linear(),igmoid(), BatchNorm1d()  }  basic_block == n_layers  self_proj = m.Sequential(+basic_block)  def forward(self, x):  x = self.proj(x)  return x

def reconstruction_loss(x,A):  zx = zx_T  loss = F.range_loss(zz,A)  loss += F.binary_cross_entropy_with_logits(zz,A)  return loss def relative_distance_loss(z,A):  distance = zx_T  loss = (distance*A).sum() / (distance*(1-A)).sum()  return -torch.log(loss)
to deal with large graphs, we need to sample their subGraphs for subgraph in Graph: # transfer subGraphs into low-dimensional representation z0 = GCN(subgraph)  z1 = encoder(zd) ; z2 = encoder(z1)  z3 = encoder(z2)  z4 = encoder(z3)  z5 = encoder(z4) ; z6 = encoder(z5)  z7 = encoder(zd) ; z8 = encoder(z7)  loss = 0  for z in [z0,z1,z2,z3,z4]:  # A_sub is the adjacency graph of the subGraph  re_loss = reconstruction_loss(z,A_sub)  rd_loss = relative_distance_loss(z,A_sub)  # ad lounes  loss += alpha*re_loss + beta*rd_loss
# Adam optimizer  loss.backward()  update(model.params) with torch_no_grad():  evaluation(model)
```  
**Algorithm 1** Pseudocode of colaGAE in Pytorch-like style. Graph data are typically sparse, with a density defined as \(\frac{2E}{N(N-1)}\), where \(E\) is the number of edges and \(N(N-1)\) is the maximum number of edges in a graph with \(N\) nodes. In contrast to text data, which often contain dense information, the densities of the Cora, Citeseer, and Pubmed datasets (see Table 1) are only 0.288%, 0.167%, and 0.046%, respectively. Due to this sparsity, the concatenated node-level representations from the series-encoder as output do not provide satisfactory results. Errors accumulate from \(\mathbf{Z}0\) to \(\mathbf{Z}n\) during training, with \(Dec(\mathbf{Z}n)\sim A\Rightarrow Dec(\mathbf{Z}0)\sim A\). However, the reverse is not necessarily true, indicating that \(\mathbf{Z}0\) is more robust than \(\mathbf{Z}n\). Hence, we use only \(\mathbf{Z}_{0}\) for evaluations and downstream tasks, especially for the node classification task in this paper. After training, all encoders in the series-encoder except for the first can be discarded. This approach enables colaGAE to be used with any other graph SSL methods.