SlowGuess/NeurIPS_2023_OCR huggingface dataset:
However, any  $\mathbf{R}^2$ -TGNN based on updating rules in Equation (53) can't distinguish these two nodes, so  $\mathbf{R}^2$ -TGNN is strictly weaker than  $\mathbf{R}^2$ -GNN  $\circ H$ .
Based on Theorem 31, we can consider logical classifier  $\varphi_{3} := \exists^{\geq 2}y(p_{1}^{1}(x,y)\wedge p_{1}^{2}(x,y))$ . Note that this classifier is just renaming version of Figure 1. Therefore  $\varphi_{3}$  can't be captured by  $\mathbf{R}^2$ -GNN  $\circ H$  not to say weaker framework  $\mathbf{R}^2$ -GNN by Theorem 31.
K Experiment Supplementary
K.1 Synthetic dataset generation
For each synthetic datasets, we generate 7000 graphs as training set and 500 graphs as test set. Each graph has  $50 - 1000$  nodes. In graph generation, we fix the expected edge density  $\delta$ . In order to generate a graph with  $n$  nodes, we pick  $\delta n$  pairs of distinct nodes uniformly randomly. For each selected node pair  $a,b$ , each timestamp  $t$  and each binary relation type  $r$ , we add  $r^t(a,b)$  and  $r^t(a,b)$  into the graph with independent probability  $\frac{1}{2}$ .
K.2 Statistical Information for Datasets
We list the information for synthetic dataset in Table 4 and real-world dataset in Table 5. Note that synthetic datasets contains many graphs, but real-world datasets only contains a single graph. Therefore, for real-world dataset, we have two disjoint node set as train split and test split for training and testing respectively. In training, the model can see the subgraph induced by train split and unlabelled nodes, in testing, the model can see the whole graph but only evaluate the performance on test split.
FC2classifier	φ1	φ2	φ3	φ4
R-GAT oH	100	61.4	88.6	82.0
R²-GAT oH	100	93.5	95.0	82.2
R²-GAT oF oH	100	98.2	100	95.8
Table 7: Extra results on synthetic datasets
AIFB	MUTAG	DGS	AM
# of nodes	8285	23644	333845	1666764
# of edges	29043	74227	916199	5988321
R-GCN	95.8	73.2	83.1	89.3
R-GAT	96.9	74.4	86.9	90.0
R-GNN	91.7	76.5	81.2	89.5
R²-GNN	91.7	85.3	85.5	89.9
R²-GNN oF	97.2	88.2	88.0	91.4
Table 8: Extra results for static real-world datasets.
Models	GRU-GCN·FT	TGN ·FT	R-TGNN	R-TGNN ·FT	R2-TGNN	R2-TGNN·FT
Brain-10	95.0	94.2	85.0	90.9	94.8	94.0
Table 9: Extra results for temporal real-world dataset Brain-10.
K.3 Hyper-parameters
For all experiments, we did grid search according to Table 6.
K.4 More Results
Apart from those presented in main part, we have some extra experimental results here:
1. Extra results on synthetic datasets but using different base model architecture, where R-GAT refers to Busbridge et al. [2019] and  $\mathbf{R}^2$ -GAT refers to its extension with global readout. Please Refer to Table 7. These results show the generality of our results on different base models within the framework.
2. Extra results for static real-world datasets. Add a base model R-GATBusbridge et al. [2019] and two larger real-world datasets DGS and AM from Schlichtkrull et al. [2018]. Please refer to Table 8. From the results for two bigger datasets DGM and AM, we can see our framework outperforms the other baselines, which confirms the scalability of our method and theoretical results. These results show our method is effective both on small and large graphs.
3. Extra results for temporal real-world dataset Brain-10. Please refer to Table 9. These results imply that our method is effective on different base models in temporal settings. Moreover, we can see separate improvements from global readout and graph transformation respectively. As we said in the main part, the drop in the last column may be due to the intrinsic drawbacks of current real-world datasets. Many real-world datasets can not be perfectly modeled as first-order-logic classifier. This non-logical property may lead to less convincing experimental results. As Barceló et al. [2020] commented, these commonly used benchmarks are inadequate for testing advanced GNN variants.