SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Algorithm 1: Reconstruction algorithm
1  $S\coloneqq \emptyset$    
2 for  $\{z\} \in L_u\cup L_v$  do // Restoring points from the lines  $(0,u)$  and  $(0,v)$    
3 Put  $z$  into  $S$  .   
4 Remove  $\{z\}$  from  $L_{u}\cup L_{v}$  .   
5 end for   
6  $F\coloneqq \mathrm{Cone}(u,v)$    
7 while  $F\neq \mathbb{R}^2$  do   
8 for  $\{z_1,z_2\} \in L_u\cup L_v$  do // Restoring points that have one candidate location in the forbidden area if  $z_{1}\in F$  or  $z_{2}\in F$  then   
9 1 Set  $y = z_{1}$  if  $z_{2}\in F$  and  $y = z_{2}$  if  $z_{1}\in F$  .   
11 Put  $y$  into  $S$  .   
12 Remove  $\{y,\mathrm{Refl}_uy\}$  from  $L_{u}$  and  $\{y,\mathrm{Refl}_vy\}$  from  $L_{v}$  .   
13 end for // Updating forbidden area   
14  $F = F\cup \mathrm{Refl}_uF\cup \mathrm{Refl}_vF;$    
15 end while   
16 Output S;
After  $k$  such "updates",  $F$  will consist of  $2k + 1$  adjacent angles, each of size  $\alpha_{uv}$ , with  $\mathrm{Cone}(u,v)$  being in the center. In each update, we replace  $F$  with  $F \cup \mathrm{Refl}_u F \cup \mathrm{Refl}_v F$ . This results in adding two angles of size  $\alpha_{uv}$  to both sides of  $F$ . Indeed, if we look at the ray  $(0,u)$ , it splits our current  $F$  into two angles, one of size  $(k + 1)\alpha_{uv}$  and the other of size  $k\alpha_{uv}$ . Under  $\mathrm{Refl}_u$ , the part whose size
(c)
Figure 1: Growth of the "forbidden" area.
is  $(k + 1)\alpha_{uv}$  adds an angle of size  $\alpha_{uv}$  to the other part. Analogously,  $\mathrm{Refl}_v$  adds an angle of size  $\alpha_{uv}$  from the opposite side of  $F$ . See Figure 1 for the illustration of this process.
Within at most  $1 + \frac{\pi}{\alpha_{uv}}$  such steps,  $F$  covers all angular directions, thus completing the reconstruction of  $S$ .
Initialization step. We explain how to obtain the Initialization Data about  $S$  from  $\mathcal{M}_1^{(3)}(S)$ .
We start by observing that from the first iteration of 1-WL, we can compute  $\| x\|$  for all  $x\in S$ . As the following lemma shows, this holds in any dimension, with the same proof. We temporarily omit the current hypothesis  $b = 0$ , in order to use the lemma later without this hypothesis.
Lemma 3.1 (The Barycenter Lemma). Take any  $n$ -point cloud  $S \subseteq \mathbb{R}^d$  and let
$$
D _ {x} = \{\{d (x, y) \mid y \in S \} \}.
$$
Then for every  $x \in S$ , knowing  $D_x$  and the multiset  $\{[D_y \mid y \in S]\}$ , one can determine the distance from  $x$  to the barycenter of  $S$ .
Proof. Consider the function  $f\colon \mathbb{R}^d\to [0, + \infty)$  defined as  $f(x) = \sum_{y\in S}\| x - y\| ^2$ , namely  $f(x)$  is the sum of the squares of all elements of  $D_{x}$  (with multiplicities). It follows that  $\sum_{y\in S}f(y)$  is determined by  $\{(D_y\mid y\in S)\}$ . The lemma is thus proved if we prove the following equality
$$
\left\| x - b \right\| ^ {2} = \frac {1}{n} \left(f (x) - \frac {1}{2 n} \sum_ {y \in S} f (y)\right). \tag {2}
$$
To prove the above, we first write:
$$
\begin{array}{l} f (x) = \sum_ {y \in S} \| x - y \| ^ {2} = \sum_ {y \in S} \| (x - b) + (b - y) \| ^ {2} \\ = \sum_ {y \in S} \left(\| x - b \| ^ {2} + 2 \langle x - b, b - y \rangle + \| b - y \| ^ {2}\right) \\ = n \cdot \| x - b \| ^ {2} + 2 \langle x - b, n \cdot b - \sum_ {y \in S} y \rangle + \sum_ {y \in S} \| b - y \| ^ {2} \\ = n \cdot \| x - b \| ^ {2} + \sum_ {y \in S} \| b - y \| ^ {2} \quad (\text {b y}) \\ \end{array}
$$
Denote  $\Gamma = \sum_{y\in S}\| b - y\|^2$ . Substituting the expression for  $f(x)$  and  $f(y)$  from above into the right-hand side of (2), we get:
$$
\begin{array}{l} \frac {1}{n} \left(f (x) - \frac {1}{2 n} \sum_ {y \in S} f (y)\right) = \frac {1}{n} \left(n \cdot \| x - b \| ^ {2} + \Gamma - \frac {1}{2 n} \sum_ {y \in S} \left(n \cdot \| y - b \| ^ {2} + \Gamma\right)\right) \\ = \frac {1}{n} \left(n \cdot \| x - b \| ^ {2} + \Gamma - \frac {1}{2 n} (n \Gamma + n \Gamma)\right) = \| x - b \| ^ {2}, \\ \end{array}
$$
By definition,  $\chi_{1,S}^{(1)}(x)$  determines the multiset  $D_x = \{\{d(x,y) \mid y \in S\}\}$  of distances from  $x$  to points of  $S$ . Since we are given the multiset  $\mathcal{M}_1^{(3)}(S)$ , we also know the multset  $\mathcal{M}_1^{(1)}(S) = \{\{x \mid x \in S\}\}$  (labels after the third iterations determine labels from previous iterations). In particular, this gives us the multiset  $\{\{D_x \mid x \in S\}\}$ . Overall, due to the Barycenter lemma, we conclude that  $\chi_{1,S}^{(1)}(x)$  can be converted into  $\|x\|$ .
Now, remember that
$$
\chi_ {1, S} ^ {(2)} (x) = \left(\chi_ {1, S} ^ {(1)} (x), \{\{(d (x, y), \chi_ {1, S} ^ {(1)} (y)) \mid y \in S \}\}\right).
$$
By converting  $\chi_{1,S}^{(1)}(y)$  into  $\| y\|$  for all  $y\in S$  here, one can convert  $\chi_{1,S}^{(2)}(x)$  into the multiset
$$
M _ {x} = \left\{ \right.\left\{ \right.\left( \right.d (x, y), \| y \|) \mid y \in S \left. \right\}\left. \right\}.
$$
We need one more iteration to find  $d(u,v), M_u, M_v$  for some  $u, v \in S$  satisfying the cone condition. In fact, we only need
$$
\chi_ {1, S} ^ {(3)} (u) = \left(\chi_ {1, S} ^ {(2)} (u), \{\{d (u, y), \chi_ {1, S} ^ {(2)} (y)) \colon y \in S \} \}\right)
$$
for arbitrary  $u \in S$  with  $u \neq 0$ . Since  $\chi_{1,S}^{(3)}(u)$  determines  $\| u \|$ , such  $\chi_{1,S}^{(3)}(u)$  can indeed be selected from  $\mathcal{M}_1^{(3)}(S)$  (and since we assume that  $S$  has more than one point, we know that there are points in  $S$  that are different from 0).
Due to the fact that  $\chi_{1,S}^{(2)}(y)$  can be converted to  $M_y$ , we can in turn convert  $\chi_{1,S}^{(3)}(u)$  into the multiset  $\mathcal{A} = \{\{(d(u,y),M_y):y\in S\}\}$ . In particular, since  $y = u$  is the only point for which  $d(u,y) = 0$ , we can compute  $M_u$  from  $\mathcal{A}$ . Once we have  $M_u$ , the rest of the initialization goes as follows. First note that for a given element  $(d(u,y),M_y)$  in  $\mathcal{A}$  with  $d(u,y) > 0$  (so that  $y\neq u$ ), we can look in  $M_y$  for the only element with 0 as the first entry, whose second entry is then  $\| y\|$ . So we can obtain  $(d(u,y),\| y\|)$ . As in the Reconstruction Algorithm, we then have only two possibilities for the location of  $y$  relative to  $u$ , say  $y_1$  and  $y_2 = \mathrm{Refl}_u(y_1)$ . It follows that the absolute value of the angle  $\alpha_{uy}$  between  $u$  and  $y$  is uniquely determined (if  $\| y\| = 0$ , we set  $\alpha_{uy} = 0$ ), and we can compute it from  $\mathcal{A}$ . In order to select  $v$ , we go through  $\mathcal{A}$  and look for  $v$  such that  $\alpha_{uv}$  is the smallest angle among  $\{\alpha_{uy}\mid y\in S,0 < \alpha_{uy} < \pi \}$ . If such a  $v$  does not exist, all points of  $S$  must lie on the line connecting 0 and  $u$ . In this case, the cone condition is satisfied, for example, for  $u$  and  $v = u$ . Thus, we can initialize with  $d_0 = 0$ ,  $M = M^{\prime} = M_u$ . If  $v$  as above exists, there can be no point in the interior of  $\mathrm{Cone}(u,v)$ , since otherwise there would be  $y$  with  $0 < \alpha_{uy} < \alpha_{uv} < \pi$ , contradicting the minimality of  $\alpha_{uv}$ . Thus, the cone condition is satisfied for  $u,v$ . We can then set  $d_0 = d(u,v)$ ,  $M = M_u$  and  $M^{\prime} = M_v$ .
4 Proof of Main Theorem for  $d > 2$
We now present the proof for the case  $d > 2$ . The strategy of the proof has the same structure as for  $d = 2$ . Since the objects involved now are more general, it will be convenient to introduce some terminology. Let  $\mathbf{x} = (x_{1},\ldots ,x_{k})\in (\mathbb{R}^{d})^{k}$  be a  $k$ -tuple of points in  $\mathbb{R}^d$ . The distance matrix of  $\mathbf{x}$  is the  $k\times k$  matrix  $A$  given by  $A_{ij} = d(x_i,x_j),i,j = 1,\dots ,k$ .
Now, let  $S \subseteq \mathbb{R}^d$  be a finite set. Then the distance profile of  $\mathbf{x}$  w.r.t.  $S$  is the multiset
$$
D _ {\mathbf {x}} = \{\{d (x _ {1}, y), \dots , d (x _ {k}, y) \} \mid y \in S \}.
$$
As before, we let  $b = \frac{1}{|S|}\sum_{y\in S}y$  denote the barycenter of  $S$ . For a finite set  $G\subset \mathbb{R}^d$ , we denote by LinearSpan(  $G$ ) the linear space spanned by  $G$ , and by AffineSpan(  $G$  ) the corresponding affine one. Their respective dimensions will be denoted by LinearDim(  $G$  ) and AffineDim(  $G$  ).
As for the case  $d = 2$ , we start by distilling the Initialization Data required for the reconstruction, which is described relative to the barycenter  $b$  of  $S$ . For convenience, we have decided not to assume at this stage that  $S$  has been translated first to put  $b$  at the origin, as we did for the sake of the exposition in the case  $d = 2$ . This is now the task of the isometry we apply to  $S$  when choosing locations for its points, which we now completely relegate to the reconstruction phase.
Definition 1. Let  $S \in \mathbb{R}^d$  be a finite set and let  $b$  be its barycenter. A  $d$ -tuple  $\mathbf{x} = (x_1, \ldots, x_d) \in S^d$  satisfies the cone condition if
- AffineDim  $(b, x_1, \ldots, x_d) = \text{AffineDim}(S)$ ;
- if AffineDim  $(S) = d$ , then there is no  $x \in S$  such that  $x - b$  belongs to the interior of  $\operatorname{Cone}(x_1 - b, \ldots, x_d - b)$ .
Definition 2. For a tuple  $\mathbf{x} = (x_{1},\dots ,x_{d})\in S^{d}$ , we define its enhanced profile as
$$
E P (x _ {1}, \dots , x _ {d}) = (A, M _ {1}, \dots , M _ {d}),
$$
where  $A$  is the distance matrix of the tuple  $(b, x_1, \ldots, x_d)$  and  $M_i = D_{\mathbf{x}[b / i]}$  is the distance profile of the tuple  $(x_1, \ldots, x_{i-1}, b, x_{i+1}, \ldots, x_d)$  with respect to  $S$ .
Definition 3. Let  $S \in \mathbb{R}^d$  be a finite set and let  $b$  be its barycenter. An initialization data for  $S$  is a tuple  $(A, M_1, \ldots, M_d)$  such that  $(A, M_1, \ldots, M_d) = EP(x_1, \ldots, x_d)$  for some  $d$ -tuple  $\mathbf{x} = (x_1, \ldots, x_d) \in S^d$  satisfying the cone condition.
Remark 4.1. The interested reader can verify that the Initialization Data condition extends the definition for  $d = 2$ . Note that the first bullet of the Cone Condition is automatically verified for  $d = 2$ , but is nontrivial for  $d > 2$ .
The fact that an initialization data  $(A, M_1, \ldots, M_d)$  can be recovered from  $\mathcal{M}_{d-1}(S)$  is ensured by the following proposition.
Proposition 4.2 (Initialization Lemma). Take any  $S \subseteq \mathbb{R}^d$ . Then, knowing the multiset  $\{\chi_{d-1,S}^{(3)}(\mathbf{s}) \mid \mathbf{s} \in S^{d-1}\}$ , one can determine an initialization data for  $S$ .
We now proceed with the reconstruction phase.
4.1 Reconstruction Algorithm
Assume an Initialization Data  $(A,M_1,\ldots ,M_d)$  for a finite  $S\subset \mathbb{R}^d$  is given. Our first task is to choose, up to isometry, positions for the points in the  $(d + 1)$ -tuple  $(b,x_{1},\dots ,x_{d})$  corresponding to the matrix  $A$ . We use the following classical lemma, whose proof is given e.g. in [4, Sec. 2.2.1].
Lemma 4.3 (Anchor Lemma). If  $(u_{1},\ldots ,u_{k})\in \mathbb{R}^{d}$  and  $(v_{1},\dots,v_{k})\in \mathbb{R}^{d}$  have the same distance matrix, then there exists an isometry  $f\colon \mathbb{R}^d\to \mathbb{R}^d$  such that  $f(u_{i}) = v_{i}$  for all  $i = 1,\ldots ,k$
As for  $d = 2$ , we put the barycenter of the cloud at the origin. Then, we simply apply the Anchor Lemma to any collection of points  $z_{1},\ldots ,z_{d}\in \mathbb{R}^{d}$  such that our given  $A$  is also the distance matrix of the tuple  $(0,z_1,\dots ,z_d)$ . The Lemma then gives us an isometry  $f\colon \mathbb{R}^d\to \mathbb{R}^d$  such that  $f(b) = 0,f(x_{1}) = z_{1},\ldots ,f(x_{d}) = z_{d}$ . As distance profiles are invariant under isometries, our given  $M_{i}$  is also the distance profile of the tuple  $(z_{1},\ldots ,z_{i - 1},0,z_{i + 1},\ldots ,z_{d})$  w.r.t.  $f(S)$ . Our task now is, from  $M_1,\ldots ,M_d$ , to uniquely determine the locations of all points in  $f(S)$ . This would give us  $S$  up to an isometry. Since now we have locations for  $(z_{1},\ldots ,z_{d})$ , we can in fact compute:
$$
\operatorname {A f f i n e D i m} (S) = \operatorname {A f f i n e D i m} (b, x _ {1}, \dots , x _ {d}) = \operatorname {A f f i n e D i m} (0, z _ {1}, \dots , z _ {d}) = \operatorname {L i n e a r D i m} (z _ {1}, \dots , z _ {d}),
$$
where the first equality is guaranteed by the cone condition. The reconstruction algorithm depends on whether AffineDim  $(S) = d$  or not.
Consider first the case when  $\operatorname{AffineDim}(S) = \operatorname{LinearDim}(z_1, \ldots, z_d) < d$ . Then there exists  $i \in \{1, \ldots, d\}$  such that  $\operatorname{LinearDim}(z_1, \ldots, z_d) = \operatorname{LinearDim}(z_1, \ldots, z_{i-1}, z_{i+1}, \ldots, z_d)$ . This means that  $f(S) \subset \operatorname{AffineSpan}(z_1, \ldots, z_{i-1}, 0, z_{i+1}, \ldots, z_d)$ , since otherwise  $f(S)$  would have larger affine dimension. We claim that, in this case, from  $M_i$  we can restore the location of all points in  $f(S)$ . Indeed, from  $M_i$  we know, for each  $z \in f(S)$ , a tuple with the distances from  $z$  to  $z_1, \ldots, z_{i-1}, 0, z_{i+1}, \ldots, z_d$ . As the next lemma shows, this information is enough to uniquely determine the location of  $z$ .
Lemma 4.4. Take any  $x_1, \ldots, x_m \in \mathbb{R}^d$ . Assume that  $a, b \in \operatorname{AffineSpan}(x_1, \ldots, x_m)$  are such that  $d(a, x_i) = d(b, x_i)$  for all  $i = 1, \ldots, m$ . Then  $a = b$ .
It remains to reconstruct  $f(S)$  when AffineDim  $(S) = d$ , in which case our pivot points  $z_{1},\ldots ,z_{d}$  are linearly independent. Recall that  $M_{i}$  is the distance profile of the tuple  $(z_{1},\dots ,z_{i - 1},0,z_{i + 1},\dots ,z_{d})$  w.r.t.  $f(S)$ . Moreover, since no  $x$  in  $S$  is such that  $x - b$  lies in the interior of  $\mathrm{Cone}(x_1 - b,\dots ,x_d - b)$ , we know that  $f(S)$  must be disjoint from the interior of  $\mathrm{Cone}(z_1,\dots ,z_d)$ . As the next proposition shows, this information is enough to reconstruct  $f(S)$  in this case as well, which finishes the proof of Theorem 2.1 for  $d > 2$ .
Proposition 4.5 (The Reconstruction Lemma). Assume that  $z_{1},\ldots ,z_{d}\in \mathbb{R}^{d}$  are linearly independent. Let  $T\subseteq \mathbb{R}^d$  be finite and disjoint from the interior of  $\operatorname {Cone}(z_1,\dots ,z_d)$ . If, for every  $i = 1,\dots ,d$ , we are given  $z_{i}$  and also the distance profile of the tuple  $(z_{1},\ldots ,z_{i - 1},0,z_{i + 1},\ldots ,z_{d})$  w.r.t.  $T$ , then we can uniquely determine  $T$ .
5 On the distinguishing power of one iteration of  $d$ -WL
In this section, we discuss a somewhat different strategy to reconstruct  $S$ . It is clear that if for a point  $z \in \mathbb{R}^d$  we are given the distances from it to  $d + 1$  points in general position with known coordinates, then the position of  $z$  is uniquely determined (see e.g. Lemma 4.4). Since  $d$ -WL colors  $d$ -tuples of points in  $S$ , a natural strategy to recover  $S$  is to use the barycenter as an additional point. By Lemma 3.1, we know that distances to the barycenter from points of  $S$  can be obtained after one iteration of  $d$ -WL. However, the information that allows us to match  $d(z, b)$  to the distances from this  $z$  to a  $d$ -tuple, is readily available only after two iterations of  $d$ -WL. It follows that this simple strategy can be used to directly reconstruct  $S$  from the second iteration of  $d$ -WL. We remark that this strategy is similar to the one used in [9] to uniquely determine  $S$  up to isometries when the coloring we are initially given corresponds to certain Gram Matrices for  $d$ -tuples of points. Essentially, after one interaction of  $d$ -WL over this initial data, we obtain enough information to directly determine the location of each  $z$  relative to a collection of  $d + 1$  points. In fact, it is not hard to show that from the first iteration of  $d$ -WL, applied to the distance graph of  $S$ , one can compute these Gram Matrices, thus providing an alternative proof that two iterations suffice for distinguishing geometric graphs.
We show instead that only one iteration suffices. Our approach differs and depends on certain geometric principles that allow us to simplify the problem by conducting an exhaustive search across an exponentially large range of possibilities.
Theorem 5.1. For any  $d \geq 1$  and for any finite set  $S \subseteq \mathbb{R}^d$ , knowing the multiset  $\{\{\chi_{d,S}^{(1)}(\mathbf{s})| \mathbf{s} \in S^d\}\}$ , one can determine  $S$  up to an isometry.
6 Final remarks
Open problems. An interesting open question about our work is what is the minimum number of rounds needed for the  $(d - 1)$ -WL test to be complete with respect to point clouds in  $\mathbb{R}^d$ . Our result shows that three rounds suffice, but we do not know the completeness status of the test when only one or two rounds are allowed. Another open problem is the completeness status of the  $(d - 2)$ -WL test for  $d$ -dimensional point clouds when  $d > 3$ .
Limitations. A consequence of our main result and is that distance-based  $d$ -MPGNNs possess sufficient expressive capability to learn  $d$ -dimensional point clouds up to isometry. However, the computational complexity of implementing  $d$ -MPGNNs is a major concern, as it involves  $O(n^{d})$  operations per iteration, where  $n$  denotes the number of nodes in the graph. This computational cost can quickly become unmanageable, even for relatively small values of  $d$ , such as  $d = 3$ . It remains to be studied which kind of optimizations on higher-order GNNs can be implemented for improved performance without much sacrifice on their expressive power.
Barceló and Kozachinskiy are funded by ANID-Millennium Science Initiative Program - ICN17002. All authors are funded by the National Center for Artificial Intelligence CENIA FB210017, Basal ANID. Delle Rose is funded by ANID Fondecyt Postdoctoral 3230263. Petrache is funded by ANID Fondecyt Regular 1210462.
[1] Pablo Barceló, Mikhail Galkin, Christopher Morris, and Miguel A. Romero Orth. Weisfeiler and leman go relational. In  $LOG$ , volume 198, page 46. PMLR, 2022.
[2] Ilyes Batatia, David Péter Kovács, Gregor N. C. Simm, Christoph Ortner, and Gábor Csányi. Mace: Higher order equivariant message passing neural networks for fast and accurate force fields, 2022.
[3] Jin-yi Cai, Martin Fürer, and Neil Immerman. An optimal lower bound on the number of variables for graph identification. Comb., 12(4):389-410, 1992.
[4] Trevor F. Cox and Michael A. A. Cox. Multidimensional Scaling (2nd edition). Chapman and Hall / CRC, 2001.
[5] Martin Fürer. Weisfeiler-lehman refinement requires at least a linear number of iterations. In Automata, Languages and Programming: 28th International Colloquium, ICALP 2001 Crete, Greece, July 8–12, 2001 Proceedings 28, pages 322–333. Springer, 2001.
[6] Justin Gilmer, Samuel S. Schoenholz, Patrick F. Riley, Oriol Vinyals, and George E. Dahl. Neural message passing for quantum chemistry. In ICML, 2017.
[7] Martin Grohe. The logic of graph neural networks. In LICS, pages 1-17. IEEE, 2021.
[8] William L. Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. In NeurIPS, pages 1024-1034, 2017.
[9] Snir Hordan, Tal Amir, Steven J. Gortler, and Nadav Dym. Complete neural networks for euclidean graphs, 2023.
[10] Chaitanya K Joshi, Cristian Bodnar, Simon V Mathis, Taco Cohen, and Pietro Lio. On the expressive power of geometric graph neural networks. arXiv preprint arXiv:2301.09308, 2023.
[11] Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In ICLR, 2017.
[12] Zian Li, Xiyuan Wang, Yinan Huang, and Muhan Zhang. Is distance matrix enough for geometric deep learning?, 2023.
[13] Christopher Morris, Yaron Lipman, Haggai Maron, Bastian Rieck, Nils M. Kriegle, Martin Grohe, Matthias Fey, and Karsten M. Borgwardt. Weisfeiler and leman go machine learning: The story so far. CoRR, abs/2112.09992, 2021.
[14] Christopher Morris, Martin Ritzert, Matthias Fey, William L. Hamilton, Jan Eric Lenssen, Gaurav Rattan, and Martin Grohe. Weisfeiler and leman go neural: Higher-order graph neural networks. In AAAI, pages 4602-4609, 2019.
[15] Sergey N. Pozdnyakov and Michele Ceriotti. Incompleteness of graph convolutional neural networks for points clouds in three dimensions. CoRR, abs/2201.07136, 2022.
[16] Sergey N. Pozdnyakov, Michael J. Willatt, Albert P. Bartok, Christoph Ortner, Gábor Csányi, and Michele Ceriotti. Incompleteness of atomic structure representations. Phys. Rev. Lett., 125:166001, Oct 2020.
[17] Boris Weisfeiler and A.A. Leman. The reduction of a graph to canonical form and the algebra which appears therein, 1968.
[18] Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? In ICLR, 2019.
[19] Bohang Zhang, Shengjie Luo, Liwei Wang, and Di He. Rethinking the expressive power of gnns via graph biconnectivity. arXiv preprint arXiv:2301.09505, 2023.
A.1 Proofs for Initialization
Here we provide the proof of Proposition 4.2. We will need the following extension of the Barycenter Lemma.
Lemma A.1. Let  $b$  be the barycenter of  $S \subset \mathbb{R}^d$ ,  $d > 2$ . For any  $\mathbf{x} = (x_1, \ldots, x_{d-1}) \in S^{d-1}$ , knowing  $\chi_{d-1,S}^{(1)}(\mathbf{x})$  and the multiset  $\{\{\chi_{d-1,S}^{(1)}(\mathbf{y}) \mid \mathbf{y} \in S^{d-1}\} \}$ , one can determine the tuple of distances  $(d(x_1, b), \ldots, d(x_{d-1}, b))$ .
Proof. Let  $D_x$  be as in Lemma 3.1. We claim that we can determine  $(D_{x_1},\ldots ,D_{x_{d - 1}})$  and  $\{ \{D_y|y\in S\} \}$  from the information given in the lemma statement. By Lemma 3.1, this allows to determine  $(d(x_{1},b),\dots ,d(x_{d - 1},b))$ .
$$
\chi_ {d - 1, S} ^ {(1)} (\mathbf {x}) = \left(\chi_ {d - 1, S} ^ {(0)} (\mathbf {x}), \{\{\left. \chi_ {d - 1, S} ^ {(0)} (\mathbf {x} [ y / 1 ]), \dots , \chi_ {d - 1, S} ^ {(0)} (\mathbf {x} [ y / (d - 1) ])\right) \mid y \in S \}\}\right).
$$
By definition, from  $\chi_{d-1,S}^{(0)}(\mathbf{x}[y/1])$  one can determine the tuple of distances  $(d(x_2,y),\ldots,d(x_{d-1},y))$ . Hence, from the multiset  $\{\{\chi_{d-1,S}^{(0)}(\mathbf{x}[y/1]) \mid y \in S\}\}$  one can determine  $(D_{x_2},\ldots,D_{x_{d-1}})$ . In turn,  $D_{x_1}$  can be determined from, say,  $\{\{\chi_{d-1,S}^{(0)}(\mathbf{x}[y/2]) \mid y \in S\}\}$ . We have just shown that  $\chi_{d-1,S}^{(1)}(\mathbf{x})$  uniquely determines  $D_{x_1}$ . Hence, from the multiset  $\{\{\chi_{d-1,S}^{(1)}(\mathbf{y}) \mid \mathbf{y} \in S^{d-1}\}$ , one can compute the multiset  $\{\{D_{y_1} \mid \mathbf{y} = (y_1,\ldots,y_{d-1}) \in S^{d-1}\}$ . But this multiset coincides with the multiset  $\{\{D_y \mid y \in S\}\}$  except that all multiplicities are  $|S|^{d-2}$  times larger.
Now, we show that after the second iteration, we can restore the distance profile of  $(b,x_{1},\ldots ,x_{d - 1})$  for all  $(x_{1},\ldots ,x_{d - 1})\in S^{d - 1}$
Lemma A.2. For any  $\mathbf{x} = (x_{1},\ldots ,x_{d - 1})\in S^{d - 1}$ , knowing  $\chi_{d - 1,S}^{(2)}(\mathbf{x})$  and the multiset  $\{\{\chi_{d - 1,S}^{(1)}(\mathbf{y})\mid \mathbf{y}\in S^{d - 1}\}\}$ , one can determine the distance profile of  $(b,x_1,\dots ,x_{d - 1})$  w.r.t.  $S$ .
Proof. Since  $d > 2$ , we have
$$
\chi_ {d - 1, S} ^ {(2)} (\mathbf {x}) = \left(\chi_ {k, S} ^ {(1)} (\mathbf {x}), \{\{\left(\chi_ {d - 1, S} ^ {(1)} (\mathbf {x} [ y / 1 ]), \dots , \chi_ {d - 1, S} ^ {(1)} (\mathbf {x} [ y / k ])\right) \mid y \in S \}\}\right).
$$
From the tuple  $\left(\chi_{d - 1,S}^{(0)}(\mathbf{x}[y / 1]),\ldots ,\chi_{d - 1,S}^{(0)}(\mathbf{x}[y / k])\right)$  we can restore the tuple of distances
$$
(d (y, x _ {1}), \dots , d (y, x _ {d - 1})).
$$
In turn, by Lemma A.1, from  $\chi_{d - 1,S}^{(1)}(\mathbf{x}[y / 1])$  and  $\{\{\chi_{d - 1,S}^{(1)}(\mathbf{y})\mid \mathbf{y}\in S^{d - 1}\}\}$ , we can restore  $d(y,b)$ . Hence, we can restore the whole distance profile of  $(b,x_1,\ldots ,x_{d - 1})$ .
Finally, we show that knowing  $\chi_{d - 1,S}^{(3)}(\mathbf{x})$  for  $\mathbf{x} = (x_{1},\ldots ,x_{d - 1})$  and the multiset  $\{\{\chi_{d - 1,S}^{(1)}(\mathbf{y})\mid \mathbf{y}\in S^{d - 1}\} \}$ , we can compute