SlowGuess/NeurIPS_2023_OCR huggingface dataset:
is connected. This can boost the ability of geometric structure learning (as explained in Section 6.1).
$$
k - \mathrm {E} - \operatorname {D i s G N N}: h _ {\boldsymbol {v}} ^ {t + 1} = f _ {\mathrm {M P}} ^ {\mathrm {E}, t} \left(h _ {\boldsymbol {v}} ^ {t}, \left(\{\{(h _ {\boldsymbol {w}} ^ {t}, e _ {\boldsymbol {v} \backslash \boldsymbol {w}, \boldsymbol {w} \backslash \boldsymbol {v}} ^ {t}) \mid \boldsymbol {w} \in N _ {j} (\boldsymbol {v}) \} \} \mid j \in | k |\right)\right). \tag {10}
$$
Output Block. The output function  $t = f_{\mathrm{out}}\big(\{\{h_{\pmb{v}}^{T} \mid \pmb{v} \in V^{k}\}\} \big)$ , where  $T$  is the final iteration, injectively pools all the tuple representations, and generates the E(3) and permutation invariant geometric target  $t \in \mathbb{R}$ .
Like the conclusion about  $k$ -(F)WL for unweighted graphs, the expressiveness (in terms of approximating functions) of DisGNNs does not decrease as  $k$  increases. This is simply because all  $k$ -tuples are contained within some  $(k + 1)$ -tuples, and by designing message passing that ignores the last index, we can implement  $k$ -DisGNNs with  $(k + 1)$ -DisGNNs.
6 Rich Geometric Information Learned by  $k$ -DisGNNs
In this section, we aim to delve deeper into the geometry learning capability of  $k$ -DisGNNs from various perspectives. In Subsection 6.1, we will examine the high-order geometric features that the models can extract from geometric graphs. This analysis is more intuitive and aids in comprehending the high geometric expressiveness of  $k$ -DisGNNs. In Subsection 6.2, we will show that DimeNet and GemNet, two classical and widely-used GNNs for GDL employing invariant geometric representations, are special cases of  $k$ -DisGNNs, highlighting the generality of  $k$ -DisGNNs. In the final subsection, we will show the completeness (distinguishing geometric graphs) and universality (approximating functions over geometric graphs) of  $k$ -DisGNNs, thus answering the question posed in the title: distance matrix is enough for GDL.
6.1 Ability of Learning High-Order Geometry
We first give the concept of high-order geometric information for better understanding.
Definition 6.1.  $k$ -order geometric information is the  $E(3)$ -invariant features calculable from  $k$  nodes' 3D coordinates.
For example, in a 4-tuple, one can find various high-order geometric information, including distance (2-order), angles (3-order) and dihedral angles (4-order), as shown in Figure 2(A1). Note that high-order information is not limited to the common features listed above: we show some of other possible 3-order geometric features in Figure 2(A2).
We first note that  $k$ -DisGNNs can learn and process at least  $k$ -order geometric information. Theorem 3.1 states that we can reconstruct the whole geometry of the  $k$ -tuple from the embedding of its distance matrix. In other words, we can extract all the desired  $k$ -order geometric information solely from the embedding of the  $k$ -tuple's distance matrix, which is calculated at the initialization step of  $k$ -DisGNNs, with a learnable function.
Furthermore, both  $k$ -E-DisGNN and  $k$ -F-DisGNN can actually learn  $(k + 1)$ -order geometric information in their message passing layers. For  $k$ -E-DisGNN, information about  $(h_{\mathbf{v}}, h_{\mathbf{w}}, e_{\mathbf{v} \setminus \mathbf{w}, \mathbf{w} \setminus \mathbf{v}})$ , where  $\mathbf{w}$  is some  $j$ -neighbor of  $\mathbf{v}$ , is included in the input of its update function. Since the distance matrices of tuples  $\mathbf{v}$  and  $\mathbf{w}$  can be reconstructed from  $h_{\mathbf{v}}$  and  $h_{\mathbf{w}}$ , the all-pair distances of  $(v_1, v_2, \dots, v_k, w \setminus v)$  can be reconstructed from  $(h_{\mathbf{v}}, h_{\mathbf{w}}, e_{\mathbf{v} \setminus \mathbf{w}, \mathbf{w} \setminus \mathbf{v}})$ , as shown in Figure 2(B1). Similarly, the distance matrix of  $(v_1, v_2, \dots, v_k, j)$  can also be reconstructed from  $(h_{\mathbf{v}}, (h_{\mathbf{w}}^t | \mathbf{w} \in N_j^F(\mathbf{v}))$  in update function of  $k$ -F-DisGNN as shown in Figure 2(B2). These enable  $k$ -E-DisGNN and  $k$ -F-DisGNN to reconstruct all  $(k + 1)$ -tuples' distance matrices during message passing and thus learn all the  $(k + 1)$ -order geometric information contained in the graph.
With the ability to learn high-order geometric information,  $k$ -DisGNNs can learn geometric structures that cannot be captured by Vanilla DisGNNs. For example, consider the counterexample shown in Figure 1. The two geometric graphs have a different number of small equilateral triangles, which cannot be distinguished by Vanilla DisGNNs even with infinite message passing layers. However, 3-DisGNN and 2-E/F-DisGNN can easily distinguish the graphs by counting the number of small equilateral triangles, which is actually a kind of 3-order geometric information. This example also illustrates that high-order DisGNNs are strictly more powerful than Vanilla DisGNNs.
6.2 Unifying Existing Geometric Models with DisGNNs
There have been many attempts to improve GDL models by manually designing and incorporating high-order geometric features such as angles (3-order) and dihedral angles (4-order). These features are all invariant geometric features that can be learned by some  $k$ -DisGNNs. It is therefore natural to ask whether these models can be implemented by  $k$ -DisGNNs. In this subsection, we show that two classical models, DimeNet [Klicpera et al., 2020a] and GemNet [Gasteiger et al., 2021], are just special cases of  $k$ -DisGNNs, thus unifying existing methods based on hand-crafted features with a learning paradigm that learns arbitrary high-order features from distance matrix.
DimeNet embeds atom  $i$  with a set of incoming messages  $m_{ji}$ , i.e.,  $h_i = \sum_{j \in N_i} m_{ji}$ , and updates the message  $m_{ji}$  by
$$
m _ {j i} ^ {t + 1} = f _ {\mathrm {M P}} ^ {\mathrm {D}} \left(m _ {j i} ^ {t}, \sum_ {k} f _ {\text {i n t}} ^ {\mathrm {D}} \left(m _ {k j} ^ {t}, d _ {j i}, \phi_ {k j i}\right)\right), \tag {11}
$$
where  $d_{ji}$  is the distance between node  $j$  and node  $i$ , and  $\phi_{kji}$  is the angle  $kji$ . We simplify the subscript of  $\sum$ , same for that in Equation (12). The detailed ones are referred to Appendix B.2, B.1.
DimeNet is one early approach that uses geometric features to improve the geometry learning ability of GNNs, especially useful for learning the angle-relevant energy or other chemical properties. Note that the angle information that DimeNet explicitly incorporates into its message passing function is actually a kind of 3-order geometric information, which can be exploited from distance matrices by our 2-E/F-DisGNN with a learning paradigm. This gives the key insight for the following proposition.
Proposition 6.2. 2-E-DisGNN and 2-F-DisGNN can implement DimeNet.
GemNet is also a graph neural network designed to process graphs embedded in Euclidean space. Unlike DimeNet, GemNet incorporates both angle and dihedral angle information into the message passing functions, with a 2-step message passing scheme. The core procedure is as follows:
$$
m _ {c a} ^ {t + 1} = f _ {\mathrm {M P}} ^ {\mathrm {G}} \left(m _ {c a} ^ {t}, \sum_ {b, d} f _ {\text {i n t}} ^ {\mathrm {G}} \left(m _ {d b} ^ {t}, d _ {d b}, \phi_ {c a b}, \phi_ {a b d}, \theta_ {c a b d}\right)\right), \tag {12}
$$
where  $\theta_{cabd}$  is the dihedral angle of planes cab and abd. The use of dihedral angle information allows GemNet to learn at least 4-order geometric information, making it much more powerful than models that only consider angle information. We now prove that GemNet is just a special case of 3-E/F-DisGNN, which can also learn 4-order geometric information during its message passing stage.
Proposition 6.3. 3-E-DisGNN and 3-F-DisGNN can implement GemNet.
It is worth noting that while both  $k$ -DisGNNs and existing models can learn some  $k$ -order geometric information,  $k$ -DisGNNs have the advantage of learning arbitrary  $k$ -order geometric information. This means that we can learn different high-order geometric features according to specific tasks in a data-driven manner, including but not limited to angles and dihedral angles.
6.3 Completeness and Universality of  $k$ -DisGNNs
We have provided an illustrative example in Section 6.1 that demonstrates the ability of  $k$ -DisGNNs to distinguish certain geometric graphs, which is not achievable by Vanilla DisGNNs. Actually, all counterexamples presented in Appendix A can be distinguished by 3-DisGNNs or 2-E/F-DisGNNs. This leads to a natural question: Can  $k$ -DisGNNs distinguish all geometric graphs with a finite and small value of  $k$ ? Furthermore, can they approximate arbitrary functions over geometric graphs?
We now present our main findings that elaborate on the completeness and universality of  $k$ -DisGNNs with finite and small  $k$ , which essentially highlight the high theoretical expressiveness of  $k$ -DisGNNs.
Theorem 6.4. (informal) Let  $\mathcal{M}(\theta) \in \mathrm{MinMod}_{\mathrm{s}} = \{1\text{-round 4 - DisGNN}, 2\text{-round 3 - E / F - DisGNN}\}$  with parameters  $\theta$ . Denote  $h_m = f_{\mathrm{node}}\left(\left\{\left|h_{\mathbf{v}}^T \mid \mathbf{v}_0 = m\right\}\right)\in \mathbb{R}^{K'}$  as node  $m$ 's representation produced by  $\mathcal{M}$  where  $f_{\mathrm{node}}$  is an injective multiset function, and  $\mathbf{x}_m^c$  as node  $m$ 's coordinates w.r.t. the center. Denote  $\mathrm{MLPs}: \mathbb{R}^{K'} \to \mathbb{R}$  as a multi-layer perceptron. Then we have:
1. (Completeness) There exists  $\theta_0$  such that  $\mathcal{M}(\theta_0)$  can distinguish all pairs of non-congruent geometric graphs.
2. (Universality for scalars)  $\mathcal{M}(\theta)$  is a universal function approximator for continuous,  $E(3)$  and permutation invariant functions  $f:\mathbb{R}^{3\times n}\to \mathbb{R}$  over geometric graphs.
3. (Universality for vectors)  $f_{\mathrm{out}}^{\mathrm{equiv}} = \sum_{m=1}^{|V|} \mathrm{MLPs}\big(h_m\big)\mathbf{x}_m^c$  is a universal function approximator for continuous,  $O(3)$  equivariant and translation and permutation invariant functions  $f: \mathbb{R}^{3 \times n} \to \mathbb{R}^3$  over geometric graphs.
In fact, completeness directly implies universality for scalars [Hordan et al., 2023]. Since we can actually recover the whole geometry from arbitrary node representation  $h_m$ , universality for vectors can be proved with conclusions from Villar et al. [2021]. Also note that as order  $k$  and round number go higher, the completeness and universality still hold. Formal theorems and detailed proof are provided in Appendix B.3.
There is a concurrent work by Delle Rose et al. [2023], which also investigates the completeness of  $k$ -WL on distance graphs. In contrast to our approach of using only one tuple's final representation to reconstruct the geometry, they leverage all tuples' final representations. They prove that with the distance matrix, 3-round geometric 2-FWL and 1-round geometric 3-FWL are complete for geometry. This conclusion can also be extended to  $k$ -DisGNNs, leading to the following theorem:
Theorem 6.5. The completeness and universality for scalars stated in Theorem 6.4 hold for  $\mathcal{M}(\theta) \in \mathrm{MinMod}_{\mathrm{ext}} = \{3\text{-round } 2-E/F-DisGNN, 1\text{-round } 3-E/F-DisGNN\}$  with parameters  $\theta$ .
This essentially lowers the required order  $k$  for achieving universality on scalars to 2. However, due to the lower order and fewer rounds, the expressiveness of a single node's representation may be limited, thus the universality for vectors stated in Theorems 6.4 may not be guaranteed.
In this section, we evaluate the experimental performance of  $k$ -DisGNNs. Our main objectives are to answer the following questions:
Q1 Does 2/3-E/F-DisGNN outperform their counterparts in experiments (corresponding to Section 6.2)?
Q2 As universal models for scalars (when  $k \geq 2$ ), do  $k$ -DisGNNs also have good experimental performance (corresponding to Section 6.3)?
Q3 Does incorporating well-designed edge representations in the  $k$ -E-DisGNN result in improved performance?
The best and the second best results are shown in **bold** and **underline** respectively in tables. Detailed experiment configuration and supplementary experiment information can be found in Appendix D. Our code is available at https://github.com/GraphPKU/DisGNN.
MD17. MD17 [Chmiela et al., 2017] is a dataset commonly used to evaluate the performance of machine learning models in the field of molecular dynamics. It contains a collection of molecular
Table 1: MAE loss on MD17. Energy (E) in kcal/mol, force (F) in kcal/mol/Å. We color the cell if DisGNNs outperforms its counterpart. The improvement ratio is calculated relative to DimeNet.  $k$ -DisGNNs rank top 2 on force prediction (which determines the accuracy of molecular dynamics [Gasteiger et al., 2021]) on average.
Target	FCHL	PaiNN	NequIP	TorchMD	GNN-LF	DimeNet	2F-Dis.	GemNet	3E-Dis.
aspirin	E	0.182	0.167	-	0.124	0.1342	0.204	0.1305	-	0.1466
F	0.478	0.338	0.348	0.255	0.2018	0.499	0.1393	0.2168	0.2060
benzene	E	-	-	-	0.056	0.0686	0.078	0.0683	-	0.0795
F	-	-	0.187	0.201	0.1506	0.187	0.1474	0.1453	0.1471
ethanol	E	0.054	0.064	-	0.054	0.0520	0.064	0.0502	-	0.0541
F	0.136	0.224	0.208	0.116	0.0814	0.230	0.0478	0.0853	0.0617
malonal.	E	0.081	0.091	-	0.079	0.0764	0.104	0.0730	-	0.0739
F	0.245	0.319	0.337	0.176	0.1259	0.383	0.0786	0.1545	0.0974
naphthal.	E	0.117	0.166	-	0.085	0.1136	0.122	0.1146	-	0.1135
F	0.151	0.077	0.097	0.06	0.0550	0.215	0.0518	0.0553	0.0478
salicyl.	E	0.114	0.166	-	0.094	0.1081	0.134	0.1071	-	0.1084
F	0.221	0.195	0.238	0.135	0.1005	0.374	0.0862	0.1048	0.1186
toluene	E	0.098	0.095	-	0.074	0.0930	0.102	0.0922	-	0.1004
F	0.203	0.094	0.101	0.066	0.0543	0.216	0.0395	0.0600	0.0455
uracil	E	0.104	0.106	-	0.096	0.1037	0.115	0.1036	-	0.1037
F	0.105	0.139	0.173	0.094	0.0751	0.301	0.0876	0.0969	0.0921
Avg improv. Rank	E	11.58%	-2.09%	-	26.40%	17.05%	0.00%	18.18%	-	13.51%
5	7	-	1	3	6	2	-	4
Avg improv. Rank	F	31.85%	39.13%	29.86%	52.40%	63.53%	0.00%	69.68%	60.96%	65.27%
7	6	8	5	3	9	1	4	2
dynamics simulations of small organic molecules such as aspirin and ethanol. Given the atomic numbers and coordinates, the task is to predict the energy of the molecule and the atomic forces. We mainly focus on the comparison between 2-F-DisGNN/3-E-DisGNN and DimeNet/GemNet. At the same time, we also compare  $k$ -DisGNNs with the state-of-the-art models: FCHL [Christensen et al., 2020], PaiNN [Schütt et al., 2021], NequIP [Batzner et al., 2022], TorchMD [Thölke and De Fabritiis, 2021], GNN-LF [Wang and Zhang, 2022]. The results are shown in Table 1. 2-F-DisGNN and 3-E-DisGNN outperform their counterparts on 16/16 and 6/8 targets, respectively, with an average improvement of  $43.9\%$  and  $12.23\%$ , suggesting that data-driven models can subsume carefully designed manual features given high expressiveness. In addition,  $k$ -DisGNNs also achieve the best performance on 8/16 targets, and achieve the second-best performance on all the other targets, outperforming the best baselines GNN-LF and TorchMD by  $10.19\%$  and  $12.32\%$ , respectively. Note that GNN-LF and TorchMD are all complex equivariant models. Beyond these, we perform experiments on revised MD17, which has better data quality, and  $k$ -DisGNNs outperforms the SOTA models [Batatia et al., 2022, Musaelian et al., 2023] on a wide range of targets. The results are referred to Appendix D.2. The results firmly answer Q1 and Q2 posed at the beginning of this section and demonstrate the potential of pure distance-based methods for graph deep learning.
QM9. QM9 [Ramakrishnan et al., 2014, Wu et al., 2018] consists of 134k stable small organic molecules with 19 regression targets. The task is like that in MD17, but this time we want to predict the molecule's properties such as the dipole moment. We mainly compare 2-F-DisGNN with DimeNet on this dataset and the results are shown in Table 2. 2-F-DisGNN outperforms DimeNet on 11/12 targets, by  $14.27\%$  on average, especially on targets  $\mu$ $(65.0\%)$  and  $\langle R^2\rangle$ $(90.4\%)$ , which also answers Q1 well. A full comparison to other state-of-the-art models is included in Appendix D.2.
Effectiveness of edge representations. To answer Q3, we split the edge representation  $e_{ij}^{t}$  (Equation (9)) into two parts, namely the pure distance (the first element) and the tuple representations (the second element), and explore whether incorporating the two elements is beneficial. We conduct experiments on MD17 with three versions of 2-DisGNNs, including 2-DisGNN (no edge representation), 2-e-DisGNN (add only edge weight) and 2-E-DisGNN (add full edge representation). The results are shown in Table 3. Both 2-e-DisGNN and 2-E-DisGNN exhibit significant performance improvements over 2-DisGNN, highlighting the significance of edge representation. Furthermore, 2-E-DisGNN outperforms 2-DisGNN and 2-e-DisGNN on 16/16 and 12/16 targets, respectively, with average improvements of  $39.8\%$  and  $3.8\%$ . This verifies our theory that capturing the full edge representation connecting two tuples can boost the model's representation power.
Table 2: Comparison of 2-F-DisGNN and DimeNet on QM9.
Target	Unit	DimeNet	2F-Dis.
μ	D	0.0286	0.0100
α	a03	0.0469	0.0431
εHOMO	meV	27.8	21.81
εLUMO	meV	19.7	21.22
Δε	meV	34.8	31.3
〈R2〉	a02	0.331	0.0299
ZPVE	meV	1.29	1.26
U0	meV	8.02	7.33
U	meV	7.89	7.37
H	meV	8.11	7.36
G	meV	8.98	8.56
cv	cal/mol/K	0.0249	0.0233
Table 3: Effectiveness of edge representations on MD17. Energy (E) in kcal/mol, force (F) in kcal/mol/Å.
Target	2-Dis.	2e-Dis.	2E-Dis.
aspirin	E	0.2120	0.1362	0.1280
F	0.4483	0.1749	0.1279
benzene	E	0.1533	0.0723	0.0700
F	0.2049	0.1475	0.1529
ethanol	E	0.0529	0.0506	0.0502
F	0.0850	0.0533	0.0438
malonal.	E	0.0868	0.0736	0.0732
F	0.2124	0.0981	0.0861
naphthal.	E	0.1163	0.1133	0.1149
F	0.1318	0.0407	0.0531
salicyl.	E	0.1209	0.1089	0.1074
F	0.2723	0.0960	0.0806
toluene	E	0.1437	0.0924	0.0920
F	0.1229	0.0528	0.0431
uracil	E	0.1152	0.1041	0.1037
F	0.2631	0.0874	0.0933
Conclusions. In this work we have thoroughly studied the ability of GNNs to learn the geometry of a graph solely from its distance matrix. We expand on the families of counterexamples that Vanilla DisGNNs are unable to distinguish from their distance matrices by constructing families of symmetric and diverse geometric graphs, revealing the inherent limitation of Vanilla DisGNNs in capturing symmetric configurations. To better leverage the geometric structure information contained in distance graphs, we proposed  $k$ -DisGNNs, geometric models with high generality (ability to unify two classical and widely-used models, DimeNet and GemNet), provable completeness (ability to distinguish all pairs of non-congruent geometric graphs) and universality (ability to universally approximate scalar functions when  $k \geq 2$  and vector functions when  $k \geq 3$ ). In experiments,  $k$ -DisGNNs outperformed previous state-of-the-art models on a wide range of targets of the MD17 dataset. Our work reveals the potential of using expressive GNN models, which were originally designed for traditional GRL, for the GDL tasks, and opens up new opportunities for this domain.
Limitations. Although DisGNNs can achieve universality (for scalars) with a small order of 2, the results are still generated under the assumption that the distance graph is complete. In practice, for large geometric graphs such as proteins, this assumption may lead to intractable computation due to the  $O(n^{3})$  time complexity and  $O(n^{2})$  space complexity. However, we can still balance theoretical universality and experimental tractability by using either an appropriate cutoff, which can furthest preserve the completeness of the distance matrix in local clusters while cutting long dependencies to improve efficiency (and generalization), or using sparse but expressive GNNs such as subgraph GNNs [Zhang and Li, 2021]. We leave it for future work.
Muhan Zhang is partially supported by the National Natural Science Foundation of China (62276003) and Alibaba Innovative Research Program.
Gunnar Schmitz, Ian Heide Godtliebsen, and Ove Christiansen. Machine learning for potential energy surfaces: An extensive database and assessment of methods. The Journal of chemical physics, 150 (24):244113, 2019.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018.
Ilya O Tolstikhin, Neil Houlsby, Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Thomas Unterthiner, Jessica Yung, Andreas Steiner, Daniel Keysers, Jakob Uszkoreit, et al. Mlp-mixer: An all-mlp architecture for vision. Advances in neural information processing systems, 34:24261-24272, 2021.
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision, pages 10012-10022, 2021.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020.
Jun Xia, Chengshuai Zhao, Bozhen Hu, Zhangyang Gao, Cheng Tan, Yue Liu, Siyuan Li, and Stan Z Li. Mole-bert: Rethinking pre-training graph neural networks for molecules. 2023.
Shuqi Lu, Zhifeng Gao, Di He, Linfeng Zhang, and Guolin Ke. Highly accurate quantum chemical property prediction with uni-mol+. arXiv preprint arXiv:2303.16982, 2023.
Christopher Morris, Gaurav Rattan, Sandra Kiefer, and Siamak Ravanbakhsh. Speqnets: Sparsity-aware permutation-equivariant graph networks. arXiv preprint arXiv:2203.13913, 2022.
Albert P Bartók, Mike C Payne, Risi Kondor, and Gábor Csányi. Gaussian approximation potentials: The accuracy of quantum mechanics, without the electrons. Physical review letters, 104(13): 136403, 2010.
Albert P Bartók, Risi Kondor, and Gábor Csányi. On representing chemical environments. Physical Review B, 87(18):184115, 2013.
Alexander V Shapeev. Moment tensor potentials: A class of systematically improvable interatomic potentials. Multiscale Modeling & Simulation, 14(3):1153-1173, 2016.
Jörg Behler and Michele Parrinello. Generalized neural-network representation of high-dimensional potential-energy surfaces. Physical review letters, 98(14):146401, 2007.
Ralf Drautz. Atomic cluster expansion for accurate and transferable interatomic potentials. Physical Review B, 99(1):014104, 2019.
Chaitanya K Joshi, Cristian Bodnar, Simon V Mathis, Taco Cohen, and Pietro Lio. On the expressive power of geometric graph neural networks. arXiv preprint arXiv:2301.09308, 2023.
A Counterexamples for Vanilla DisGNNs
In this section, we will give some counterexamples, i.e., pairs of geometric graphs that are not congruent but cannot be distinguished by Vanilla DisGNNs. We will organize the section as follows: First, we will give some isolated counterexamples that can be directly verified; Then, we will give some counterexamples that can hold the property (i.e., to be counterexamples) after some simple continuous transformation; Finally, we will give a family of counterexamples that can be obtained by the combination of some basic units. Since the latter two cases cannot be directly verified, we will also give the detailed proof.
We say there are  $M$  kinds of nodes if there are  $M$  different labels after infinite iterations of Vanilla DisGNNs. Note that in this section, all the grey nodes and the "edges" are just for visualization purpose. Only colored nodes belong to the geometric graph.
Note that we provide verification programs in our code for all the counterexamples. The first type of counterexamples (Appendix A.1) is limited in quantity and can be directly verified. The remaining two types (Appendix A.2 and Appendix A.3) are families of counterexamples and have an infinite number of cases. Our code can verify them to some extent by randomly selecting some parameters. Theoretical proofs for these two types are also provided in Appendix A.2 and Appendix A.3, respectively.