SlowGuess/NeurIPS_2023_OCR huggingface dataset:
A.1 Counterexamples That Can Be Directly Verified
Since the distance graph, which DisGNNs take as input, is a kind of complete graph and contains lots of geometric constraints, if we want to get a pair of non-congruent geometric graphs that cannot be distinguished by DisGNNs, they must exhibit great symmetry and have just minor difference. Inspired by this, we can try to sample nodes from regular polyhedrons, which themselves exhibit high symmetry. The first pair of geometric graphs is sampled from regular icosahedrons, which is referred to the main body of our paper, see Figure 1. Since all the nodes from both graphs have the same unordered distance list, there is only 1 kind of nodes for both graphs. We note that it is the only counterexample we can sample from just regular icosahedron.
The following pairs of geometric graphs are all sampled from regular dodecahedrons. We will distinguish them by the graph size.
6-size and 14-size counterexamples.
A pair of 6-size geometric graphs (red nodes) and a pair of 14-size geometric graphs (blue nodes) that are counterexamples are shown in the figure above. They are actually complementary in terms of forming all vertices of a regular dodecahedron.
For the 6-size counterexample, the label distribution after infinite rounds of Vanilla DisGNN is shown in the picture above, and the nodes are colored differently to represent different labels. It can be directly verified that there are 2 kinds of nodes. Note that the two geometric graphs are non-congruent: The isosceles triangle of the two geometric graphs formed by one green point and two pink nodes have the same leg length but different base lengths.
For the 14-size counterexample, we give the conclusion that there are 4 kinds of nodes and the number of nodes of each kind is 2, 4, 4, 4 respectively. Since the counterexamples here and in the following are complex and not so intuitionistic, we recommend readers to directly verify them with our programs.
8-size and 12-size counterexamples.
A pair of 8-size geometric graphs (red nodes) and a pair of 12-size geometric graphs (blue nodes) that are counterexamples are shown in the figure above.
For the 8-size counterexample, note that it is actually obtained by carefully inserting two nodes into the 6-size counterexample mentioned just before. We note that there are 2 kinds of nodes in this counterexample, and the numbers of nodes of each kind are 4, 4 respectively.
For the 12-size counterexample, it is actually obtained by carefully deleting two nodes from the 14-size counterexample (corresponds to the way to obtain the 8-size counterexample) mentioned just before. There are 4 kinds of nodes in this counterexample, and the numbers of nodes of each kind are 4, 4, 2, 2 respectively.
Two pairs of 10-size counterexamples.
The above figure shows the first pair of 10-size counterexamples. There are 3 kinds of nodes and the numbers of nodes of each kind are 4, 4, 2 respectively.
The above figure shows the second pair of 10-size counterexamples. There is actually only 1 kind of nodes, all the nodes of both graphs have the same unordered distance list. It is interesting because
there are two regular pentagons in the left graph and a ring in the right graph, which means that the two graphs are non-congruent, and it is quite similar to the case shown in Figure 1, where there are two equilateral triangles in the left graph and also a ring in the right graph.
In the end, we note that the above counterexamples are probably all the counterexamples one can sample from a single regular polyhedron.
A.2 Counterexamples That Hold After Some Continuous Transformation
Inspired by the intuition mentioned in Appendix A.1, we can also sample nodes from the combination of regular polyhedrons, which also have great symmetry but with more nodes.
Cube + regular octahedron.
The first two counterexamples are sampled from the combination of a cube and a regular octahedron. We combine them in the following way: Bring the center of both polyhedrons together, and align the vertex of the regular octahedron, the center of the cube's face, and the center of the polyhedrons in a straight line. We do not limit the relative size of the two polyhedrons: one can scale the cube to any size as long as the constraints are not broken. In this way, a small family of counterexamples is given (we say small here because the dimension of the transformation space is small).
Proof of the red case. Let the side length of the cube be  $2a$ , the diagonal of the regular octahedron be  $2b$ . Initially, all nodes are labeled  $l_0$ . We now simulate Vanilla DisGNNs on both graphs.
At the first iteration, there are actually two kinds of unordered distance lists for both of the graphs: For the vertexes on the regular octahedron, the list is  $\left\{\left((2b,l_0),1\right),\left((\sqrt{3a^2 + b^2 + 2ab},l_0),2\right),\left((\sqrt{3a^2 + b^2 - 2ab},l_0),2\right)\right\}$ . For the vertexes on the cube, the list is  $\left\{\left((2a,l_0),1\right),\left((2\sqrt{2}a,l_0),1\right),\left((2\sqrt{3}a,l_0),1\right),\left((\sqrt{3a^2 + b^2 + 2ab},l_0),1\right),\left((\sqrt{3a^2 + b^2 - 2ab},l_0),1\right)\right\}$ . Thus they will be labeled differently at this step, let the labels be  $l_{1}$  and  $l_{2}$  respectively.
At the second iteration, the lists of all the vertexes on the regular octahedron from both graphs are still the same, i.e.  $\left\{\left((2b,l_1),1\right),\left((\sqrt{3a^2 + b^2 + 2ab},l_2),2\right),\left((\sqrt{3a^2 + b^2 - 2ab},l_2),2\right)\right\}$ , as well as the lists of all the vertexes on the cube from both graphs, i.e.  $\left\{\left((2a,l_2),1\right),\left((2\sqrt{2} a,l_2),1\right),\left((2\sqrt{3} a,l_2),1\right),\left((\sqrt{3a^2 + b^2 + 2ab},l_1),1\right),\left((\sqrt{3a^2 + b^2 - 2ab},l_1),1\right)\right\}$ .
Since Vanilla DisGNNs cannot subdivide the vertexes at the second step, they can still not at the following steps. So it cannot distinguish the two geometric graphs. But they are non-congruent: on the left graph, the nodes on the regular octahedron can only form isosceles triangles with the nodes on the face diagonal of the cube. On the right graph, they can only form isosceles triangles with nodes on the cubes that are on the same side. We note that the counterexample in Figure 1 is not a special case of this, because on the left graph of this case all the nodes are on the same surface, but none of graphs in Figure 1 have all nodes on the same surface.
Proof of the blue case. The proof of the blue case is quite similar to that of the red case; thus, we omit the proof here. Note that the labels will stabilize after the first iteration. At the end of Vanilla DisGNN, there are 2 kinds of nodes, each kind has 4 nodes. And the two geometric graphs are
non-congruent because the plane formed by four nodes on the regular octahedron is perpendicular to the plane formed by four nodes on the cube in the left graph, and is not in the right graph.
2 cubes.
The second case is the combination of 2 cubes. We combine them in the following way: Let the center of the two cubes coincide as well as the axis formed by the centers of two opposite faces. Also, we do not limit the relative size of the two cubes: one can scale the cube to any size as long as the constraints are not broken. The proof of this case is quite similar to that in Cube + regular octahedron; thus, we omit the proof here. Note that the labels will stabilize after the first iteration. If the sizes of the two cubes are the same, then at the end of Vanilla DisGNN, there is only 1 kind of node. If not, there are 2 kinds of nodes, each kind has 4 nodes.
A.3 Families of Counterexamples
In this section, we will prove that any one of the counterexamples given in Appendix A.1 can be augmented to a family. To facilitate the discussion, we will introduce some notations.
Notation. Consider an arbitrary counterexample  $(G_{ori}^{L}, G_{ori}^{R})$  given in Appendix A.1. It is worth noting that for each graph, the nodes are sampled from a regular polyhedron and are located on the same spherical surface. We can denote the geometric graph on this sphere as  $G_{ori,r}$ , where  $r$  represents the radius of the sphere. Since the nodes are sampled from some regular polyhedron (denoted by  $G_{all,r}$ ), there must be nodes that aren't sampled, which we call as complementary nodes. We denote the corresponding geometric graph as  $G_{com,r}$ . Given two graphs  $G_{Q_1,r_1}, G_{Q_2,r_2}$  where  $Q_i \in \{ori, com, all\}$  for  $i \in [2]$ , we say that we align them if we make sure that the spherical centers of the two geometric graphs coincide and one can get  $G_{Q_2,r_2}$  by just scaling all the nodes of  $G_{Q_2,r_1}$  along the spherical center. We use  $\mathcal{L} = \{(v, l_v) \mid v \in V\}$  to represent a label state for some graph  $G$ , and call it a stable state if Vanilla DisGNN cannot further refine the labels of the graph  $G$  with initial label  $\mathcal{L}$ . We say that  $\mathcal{L}$  is the final state of  $G$  if it is the label state obtained by performing Vanilla DisGNN on  $G$  with no initial labels for infinite rounds. Note that final state is a special stable state. We denote the final state of  $G_{ori}$  and  $G_{com}$  as  $\mathcal{L}_{ori}$  and  $\mathcal{L}_{com}$  respectively.
Now, let us introduce an augmentation method for arbitrary counterexample  $(G_{ori}^{L}, G_{ori}^{R})$  shown in Appendix A.1. For a given set  $\mathcal{Q}\mathcal{R}_k = \{(Q_1, r_1), (Q_2, r_2), \ldots, (Q_k, r_k)\}$  where  $r_i > 0$ ,  $Q_i \in \{ori, com, all\}$  and  $r_i \neq r_j$  for  $i \neq j$ , we get the pair of augmented geometric graphs  $(G^L, G^R) = \mathrm{AUG}_{\mathcal{Q}\mathcal{R}_k}(G_{ori}^L, G_{ori}^R)$  by the following steps (take  $G^L$  for example):
1. Generate  $k$  geometric graphs  $G_{Q_1,r_1}^L$ ,  $i \in [k]$  according to  $\mathcal{QR}_k$ .
2. Align the  $k$  geometric graphs.
3. Combine the  $k$  geometric graphs  $G_{Q_i,r_i}^L$ ,  $i \in [k]$  to obtain  $G^L$ .
We give an example of the augmentation in Figure 3. Now we give our theorem.
Theorem A.1. Consider an arbitrary counterexample  $(G_{ori}^{L}, G_{ori}^{R})$  given in Appendix A.1. Let  $\mathcal{QR}_k = \{(Q_1, r_1), (Q_2, r_2), \ldots, (Q_k, r_k)\}$  be an arbitrary set, where  $Q_i \in \{\text{ori}, \text{com}, \text{all}\}$  and
Figure 3: An example of the augmentation. In this example,  $(G_{ori}^{L}, G_{ori}^{R})$  are from Figure 1, and  $\mathcal{QR}_3 = \{(ori, r_r), (all, r_p), (com, r_g)\}$  (with  $r, p, g$  representing red, purple and green respectively). Nodes with different colors indicate that they are derived from different  $G_{Q,r}$  generations. The augmented graphs  $(G^L, G^R) = \mathrm{AUG}_{\mathcal{QR3}}(G_{ori}^L, G_{ori}^R)$  consist of all the colored nodes.
$r_i \neq r_j$  for  $i \neq j$ . If  $\exists i \in [k]$  such that  $Q_i \neq all$ , then the augmented geometric graphs  $(G^L, G^R) = \mathrm{AUG}_{\mathcal{QR}_k}(G_{ori}^L, G_{ori}^R)$  form a counterexample.
Before presenting the proof of the theorem, we will introduce several lemmas that will be used in the proof.
Lemma A.2. If Vanilla DisGNN cannot distinguish  $(G^L, G^R)$  with some initial label  $(\mathcal{L}^L, \mathcal{L}^R)$ , then Vanilla DisGNN cannot distinguish  $(G^L, G^R)$  without initial label (with identical labels).
Proof of Lemma A.2. We use  $l_u^t$  to represent the label of  $u$  after  $t$  iterations of Vanilla DisGNN without initial labels, and use  $l_u^{\mathcal{L},t}$  to represent the label of  $u$  after  $t$  iterations of Vanilla DisGNN with initial label  $\mathcal{L}$ . We use  $l_u^\infty$  and  $l_u^{\mathcal{L},\infty}$  to represent the corresponding final label. In order to prove this lemma, we first prove a conclusion: for arbitrary graphs  $G = (V,E)$  and arbitrary initial label  $\mathcal{L}$ , let  $u,v$  be two arbitrary nodes in  $V$ , then we have  $\forall t\in \mathbb{Z}^{+}$ ,  $l_u^t\neq l_v^t\to l_u^{\mathcal{L},t}\neq l_v^{\mathcal{L},t}$ . We'll prove the conclusion by induction.
-  $t = 1$  holds.
If  $\left(l_u^0,\{\{(d_{us},l_s^0)\mid s\in N(u)\}\right)\neq \left(l_v^0,\{\{(d_{vs},l_s^0)\mid s\in N(v)\}\right)$ , then we have  $\{(d_{us}\mid s\in$ $N(u))\} \neq \{(d_{vs}\mid s\in N(v))\}$  since  $l_{s}^{0}$  are all identical for  $s\in V$ . Then it is obvious that  $(l_u^{\mathcal{L},0},\{\{(d_{us},l_s^{\mathcal{L},0})\mid s\in N(u)\})\neq \left(l_v^{\mathcal{L},0},\{\{(d_{vs},l_s^{\mathcal{L},0})\mid s\in N(v)\}\right)$ .
- For  $\forall k \in \mathbb{Z}^{+}$ ,  $t = k$  holds  $\Rightarrow t = k + 1$  holds.
Assume  $t = k + 1$  does not hold, i.e.,  $\exists u, v \in V$ ,  $l_u^{(k+1)} \neq l_v^{(k+1)}$  but  $l_u^{\mathcal{L}, (k+1)} = l_v^{\mathcal{L}, (k+1)}$ . We can derive from  $l_u^{\mathcal{L}, (k+1)} = l_v^{\mathcal{L}, (k+1)}$  that
$$
\begin{array}{l} l _ {u} ^ {\mathcal {L}, (k + 1)} = l _ {v} ^ {\mathcal {L}, (k + 1)} \\ \Rightarrow \left(l _ {u} ^ {\mathcal {L}, k}, \{\{(d _ {u s}, l _ {s} ^ {\mathcal {L}, k}) \mid s \in N (u) \} \}\right) = \left(l _ {v} ^ {\mathcal {L}, k}, \{\{(d _ {v s}, l _ {s} ^ {\mathcal {L}, k}) \mid s \in N (v) \} \}\right) \\ \Rightarrow l _ {u} ^ {\mathcal {L}, k} = l _ {v} ^ {\mathcal {L}, k} \text {a n d} \left\{\left(d _ {u s}, l _ {s} ^ {\mathcal {L}, k}\right) \mid s \in N (u) \right\} = \left\{\left(d _ {v s}, l _ {s} ^ {\mathcal {L}, k}\right) \mid s \in N (v) \right\}. \tag {13} \\ \end{array}
$$
Since  $t = k$  holds, for  $\forall s_1, s_2$ , we have
$$
\begin{array}{l} l _ {s _ {1}} ^ {\mathcal {L}, k} = l _ {s _ {2}} ^ {\mathcal {L}, k} \\ \Rightarrow l _ {s _ {1}} ^ {k} = l _ {s _ {2}} ^ {k}. \\ \end{array}
$$
Together with Equation (13), we have  $l_{u}^{k} = l_{v}^{k}$  and  $\{(d_{us},l_s^k)\mid s\in N(u)\} = \{(d_{vs},l_s^k)\mid s\in N(v)\}$ . This means  $l_{u}^{(k + 1)} = l_{v}^{(k + 1)}$ , which is contradictory to assumptions.
This means that the conclusion holds for  $\forall t\in \mathbb{Z}^{+}$ , as well as  $t = \infty$ . In other words, specifying an initial label  $\mathcal{L}$  will only result in the subdivision of the final labels.
Back to Lemma A.2, if Vanilla DisGNN cannot distinguish  $(G^L,G^R)$  even when the final labels are furthermore subdivided, it can still not when the final labels are not subdivided, i.e., cannot distinguish  $(G^{L},G^{R})$  without initial labels.
Lemma A.3. Consider an arbitrary counterexample  $(G_{ori}^{L}, G_{ori}^{R})$  given in Appendix A.1. We have that  $(\mathcal{L}_{ori}^{L} \cup \mathcal{L}_{com}^{L}, \mathcal{L}_{ori}^{R} \cup \mathcal{L}_{com}^{R})$  is a pair of stable states of  $(G_{all}^{L}, G_{all}^{R})$ . Denote the labels as  $\mathcal{L}_{all}^{L}$  and  $\mathcal{L}_{all}^{R}$  respectively. Moreover, we have that Vanilla DisGNN cannot distinguish  $(G_{all}^{L}, G_{all}^{R})$  with initial labels  $\mathcal{L}_{all}^{L}$  and  $\mathcal{L}_{all}^{R}$ .
Proof of Lemma A.3. Since the number of situations is finite, this lemma can be directly verified using a computer program. It is worth noting that we have also included the verification program in our code.
This lemma allows us to draw several useful prior conclusions. In the graph  $G_{all}$ , the node set  $V_{all}$  can be split into two subsets by definition, namely  $V_{ori}$  and  $V_{com}$ . Now for arbitrary node  $u$  from  $G_{all}^{L}$  with initial label  $\mathcal{L}_{all}^{L}$  and  $v$  from  $G_{all}^{R}$  with initial label  $\mathcal{L}_{all}^{R}$ , if the initial labels of node  $u$  and node  $v$  are the same, they must be in the same subset, i.e., respectively in  $V_{ori}^{L}$  and  $V_{ori}^{R}$  or  $V_{com}^{L}$  and  $V_{com}^{R}$ . Without loss of generality, let us assume that both  $u$  and  $v$  are in  $V_{ori}$ . Since  $(\mathcal{L}_{all}^{L},\mathcal{L}_{all}^{R})$  is a stable state of  $(G_{all}^{L},G_{all}^{R})$ , we can obtain the following equation:
$$
\left(l _ {u}, \{\{d _ {u s}, l _ {s} \} \mid s \in V _ {a l l} ^ {L} \}\right) = \left(l _ {v}, \{\{(d _ {v s}, l _ {s}) \mid s \in V _ {a l l} ^ {R} \}\right). \tag {14}
$$
Since  $(\mathcal{L}_{ori}^{L},\mathcal{L}_{ori}^{R})$  is a stable state of the induced subgraphs  $(G_{ori}^{L},G_{ori}^{R})$  where the node sets are  $(V_{ori}^{L},V_{ori}^{R})$ , we can obtain the following equation:
$$
\left(l _ {u}, \{\{(d _ {u s}, l _ {s}) \mid s \in V _ {o r i} ^ {L} \}\}\right) = \left(l _ {v}, \{\{(d _ {v s}, l _ {s}) \mid s \in V _ {o r i} ^ {R} \}\}\right). \tag {15}
$$
Notice that in Equation (14, 15), we can obtain a new equation by replacing  $V_{ori}^{L}$  or  $V_{all}^{L}$  with  $(V_{ori}^{L} - u)$  or  $(V_{all}^{L} - u)$  and doing the same for the  $v$  side, while keeping the equation valid, since  $l_{u} = l_{v}$ . By subtracting Equation (15) from Equation (14), we obtain the following equation:
$$
\left(l _ {u}, \{\{(d _ {u s}, l _ {s}) \mid s \in V _ {c o m} ^ {L} \}\right) = \left(l _ {v}, \{\{(d _ {v s}, l _ {s}) \mid s \in V _ {c o m} ^ {R} \}\right). \tag {16}
$$
It is important to note that the same conclusion holds if both  $u$  and  $v$  are from type com. These equations provide valuable prior knowledge that we can use to prove Theorem A.1.
Now, let us prove Theorem A.1. Our proof is divided into four steps:
1. Construct an initial state  $(\mathcal{L}^L,\mathcal{L}^R)$  for the augmented geometric graphs  $(G^{L},G^{R})$
2. Prove that  $(\mathcal{L}^L,\mathcal{L}^R)$  is a pair of stable states for  $(G^{L},G^{R})$
3. Explain that Vanilla DisGNN cannot distinguish  $(G^L, G^R)$  with initial labels  $(\mathcal{L}^L, \mathcal{L}^R)$ .
4. Explain that  $(G^L,G^R)$  are non-congruent.
By applying Lemma A.2, we can get the conclusion from step 2 and 3 that Vanilla DisGNN cannot distinguish  $(G^{L}, G^{R})$  without initial labels. Since  $(G^{L}, G^{R})$  are non-congruent (as established in step 4), it follows that  $(G^{L}, G^{R})$  is a counterexample.
Proof of Theorem A.1.
Step 1. We first construct an initial state for the augmented graphs  $(G^L, G^R)$ .
For simplicity, we omit the superscripts, as the rules are the same for both graphs. For each layer (i.e., the sphere of some radius) of the graph  $G$ , we label the nodes with  $\mathcal{L}_{tp}$  if the layer is of type  $tp$ , for  $tp \in \{ori, com, all\}$ . Note that we also ensure that the labels in different layers are distinct.
Step 2. Now we prove that  $(\mathcal{L}^L,\mathcal{L}^R)$  is a stable state for  $(G^{L},G^{R})$ . This is equivalent to proving that for arbitrary node  $u$  in  $(G^{L},\mathcal{L}^{L})$  and node  $v$  in  $(G^{R},\mathcal{L}^{R})$ , if  $l_{u}^{0} = l_{v}^{0}$ , then  $l_{u}^{1} = l_{v}^{1}$ .
Since  $G^{L}$  is considered as a complete distance graph by DisGNNs, the neighbors of node  $u$  are all the nodes in  $G^{L}$  except itself. We denote the neighbor nodes from the layer with radius  $r_i$  by  $N_{r_i}(u)$ .
This is similar for node  $v$ . Now we need to prove that  $\left(l_u^0, \{(d_{us}, l_s^0) \mid s \in \bigcup_{i \in [k]} N_{r_i}(u)\}\right) = \left(l_v^0, \{(d_{vs}, l_s^0) \mid s \in \bigcup_{i \in [k]} N_{r_i}(v)\}\right)$ . Since  $l_u^0 = l_v^0$ , we only need to prove that  $\{(d_{us}, l_s^0) \mid s \in \bigcup_{i \in [k]} N_{r_i}(u)\} = \{(d_{vs}, l_s^0) \mid s \in \bigcup_{i \in [k]} N_{r_i}(v)\}$ .
We split the multiset  $\{(d_{us},l_s^0)\mid s\in \bigcup_{i\in [k]}N_{r_i}(u)\}$  into  $k$  multisets, namely  $\{(d_{us},l_s^0)\mid s\in$ $N_{r_i}(u)\} ,i\in [k]$  , and do the same for the multiset of node  $v$  . Our goal is to prove that  $\{(d_{us},l_s^0)\mid$ $s\in N_{r_i}(u)\} = \{(d_{vs},l_s^0)\mid s\in N_{r_i}(v)\}$  for each  $i\in [k]$
Let the coordinates of node  $s$  in a spherical coordinate system be  $(r_s, \theta_s, \phi_s)$ . Since nodes  $u$  and  $v$  have the same initial label, they must be from the same layer, meaning that  $r_u = r_v$ . Additionally, we always realign the coordinate systems of  $G^L$  and  $G^R$  to ensure that the direction of the polyhedra is the same. The distance between node  $u$  and node  $s$  in a spherical coordinate system is given by  $\sqrt{r_u^2 + r_s^2 - 2r_ur_s\left(\sin\theta_u\sin\theta_s\cos(\phi_u - \phi_s) + \cos\theta_u\cos\theta_s\right)}$ . We denote the angle term  $\left(\sin\theta_u\sin\theta_s\cos(\phi_u - \phi_s) + \cos\theta_u\cos\theta_s\right)$  by  $\Theta_{us}$  for simplicity.
For each value of  $r_i$ , it can be one of three types: ori, com, or all. Similarly, nodes  $u$  and  $v$  can belong to one of three categories: ori, com, all. Regardless of the combination, these situations can always be found in the Equations (14, 15, 16) derived from Lemma A.3 and can produce the following conclusion:
$$
\left(l _ {u} ^ {0}, \{\left(\Theta_ {u s}, l _ {s} ^ {0}\right) \mid s \in N _ {r _ {i}} (u) \} \}\right) = \left(l _ {v} ^ {0}, \{\left(\Theta_ {v s}, l _ {s} ^ {0}\right) \mid s \in N _ {r _ {i}} (v) \} \}\right).
$$
$$
\begin{array}{l} \left(l _ {u} ^ {0}, \{\{\left(\sqrt {r _ {u} ^ {2} + r _ {i} ^ {2} - 2 r _ {u} r _ {i} \Theta_ {u s}} , l _ {s} ^ {0}\right) \mid s \in N _ {r _ {i}} (u) \} \}\right) \\ = \left(l _ {v} ^ {0}, \{\{\left(\sqrt {r _ {v} ^ {2} + r _ {i} ^ {2} - 2 r _ {v} r _ {i} \Theta_ {v s}}, l _ {s} ^ {0}\right) \mid s \in N _ {r _ {i}} (v) \}\}\right) \\ \end{array}
$$
since  $r_u = r_i$ . This means that for all values of  $r_i$ ,  $\{(d_{us}, l_s^0) \mid s \in N_{r_i}(u)\} = \{(d_{vs}, l_s^0) \mid s \in N_{r_i}(v)\}$ . By merging all the multisets of radius  $r_i$ , we can get  $\{(d_{us}, l_s^0) \mid s \in \bigcup_{i \in [k]} N_{r_i}(u)\} = \{(d_{vs}, l_s^0) \mid s \in \bigcup_{i \in [k]} N_{r_i}(v)\}$ , which concludes the proof.
Step 3. Since the stable states  $(\mathcal{L}^L,\mathcal{L}^R)$  are obtained by assigning the stable states of each layer, i.e.  $\mathcal{L}_{com},\mathcal{L}_{ori}$  or  $\mathcal{L}_{all}$ , that cannot be distinguished by Vanilla DisGNN, the histogram of both graphs are exactly the same, which means that Vanilla DisGNN cannot distinguish the two graphs.
Step 4. Notice that the construction of an isomorphic mapping requires that the radius of the layer of node  $u$  in  $G^L$  and node  $v$  in  $G^R$  must be the same. If there exists a pair of layers in the two geometric graphs that are non-congruent, then the two geometric graphs are non-congruent. According to the definition of  $(G^L, G^R)$ , such layers always exist, therefore the two geometric graphs are non-congruent.
B.1 Proof of Proposition 6.3
The main proof of our proposition is to construct the key procedures in GemNet [Gasteiger et al., 2021] using basic message passing layers of 3-E/F-DisGNN, and the whole model can be constructed by stacking these message passing layers up. Since the  $k$ -E-DisGNN and  $k$ -F-DisGNN share the same initialization and output blocks and have similar update procedure (they can both learn 4-order geometric information), we mainly focus on the proof of E version, and one can check easily that the derivation for F version also holds.