SlowGuess/NeurIPS_2023_OCR huggingface dataset:
K Numerical Experiments
Here we provide more details on the numerical experiments presented in Section 5. All models were implemented with PyTorch [27] and PyTorch Geometric [10].
- We used torch_geometric.datasets.planetoid provided in PyTorch Geometric for the three homophilic datasets: Cora, CiteSeer, and PubMed with their default training and test splits.
- We used torch/geometric.datasets. WebKB provided in PyTorch Geometric for the three heterophilic datasets: Cornell, Texas, and Wisconsin with their default training and test splits.
- Dataset summary statistics are presented in Table 1.
Model details
- For GAT, we consider the architecture proposed in Velicković et al. [38] with each attentional layer sharing the parameter  $a$  in LeakyReLU( $a^\top[W^\top X_i||W^\top X_j]$ ),  $a \in \mathbb{R}^{2d'}$  to compute the attention scores.
Dataset	Type	#Nodes	%Nodes in LCC
Cora	2,708	91.8 %
CiteSeer	homophilic	3,327	63.7 %
PubMed	19,717	100 %
Cornell	183	100 %
Texas	heterophilic	183	100 %
Wisconsin	251	100 %
Flickr	large-scale	89,250	100 %
Table 1: Dataset summary statistics. LCC: Largest connected component.
- For GCN, we consider the standard random walk graph convolution  $D_{\deg}^{-1}A$ . That is, the update rule of each graph convolutional layer can be written as
$$
X ^ {\prime} = D _ {\deg} ^ {- 1} A X W,
$$
where  $X$  and  $X'$  are the input and output node representations, respectively, and  $W$  is the shared learnable weight matrix in the layer.
Compute We trained all of our models on a Telsa V100 GPU.
Training details In all experiments, we used the Adam optimizer using a learning rate of 0.00001 and 0.0005 weight decay and trained for 1000 epoch
Results on large-scale dataset In addition to the numerical results presented in Section 5, we also conducted the same experiment on a large-scale dataset Flickr [46] using torch_physical.datasets.Flickr. Figure 2 visualizes the results.
Figure 2: Evolution of  $\mu(X^{(t)})$  (in log-log scale) on the largest connected component of the large-scale benchmark dataset: Flickr.