SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Three Iterations of  $(d - 1)$ -WL Test Distinguish Non Isometric Clouds of  $d$ -dimensional Points
Valentino Delle Rose $^{1,2}$ , Alexander Kozachinskiy $^{1,3}$ , Cristóbal Rojas $^{1,2}$
Mircea Petrache $^{1,2,4}$ , Pablo Barceló $^{1,2,3}$
$^{1}$  Centro Nacional de Inteligencia Artificial, Chile  
 $^{2}$  Instituto de Ingeniería Matemática y Computacional, Universidad Católica de Chile  
 $^{3}$  Instituto Milenio Fundamentos de los Datos, Chile  
 $^{4}$  Departamento de Matemática, Universidad Católica de Chile  
valentino.dellerose@cenia.cl, alexander.kozachinskyi@cenia.cl  
cristobal.rojas@mat.uc.cl, mpetrache@mat.uc.cl, pbarcelo@uc.cl
The Weisfeiler-Lehman (WL) test is a fundamental iterative algorithm for checking the isomorphism of graphs. It has also been observed that it underlies the design of several graph neural network architectures, whose capabilities and performance can be understood in terms of the expressive power of this test. Motivated by recent developments in machine learning applications to datasets involving three-dimensional objects, we study when the WL test is complete for clouds of Euclidean points represented by complete distance graphs, i.e., when it can distinguish, up to isometry, any arbitrary such cloud. Our main result states that the  $(d - 1)$ -dimensional WL test is complete for point clouds in  $d$ -dimensional Euclidean space, for any  $d \geq 2$ , and only three iterations of the test suffice. Our result is tight for  $d = 2, 3$ . We also observe that the  $d$ -dimensional WL test only requires one iteration to achieve completeness.
Context. Recent work in machine learning has raised the need to develop effective and efficient tests for checking if two three-dimensional point clouds, i.e., finite sets of points in  $\mathbb{R}^3$ , are isometric [15, 9, 2, 12]. Recall that, given two such point clouds  $P$  and  $Q$ , an isometry is a distance-preserving bijection between the points in  $P$  and  $Q$ . The importance of these tests is that they provide the foundations for designing neural network architectures on point clouds that are capable of fully exploiting the structure of the data [18, 14]. It has been observed that the incompleteness of any such an architecture, i.e., the inability to recognize a point cloud up to isometry, can affect its learning performance [16]. Understanding which is the simplest test that allows detecting isometries in this scenario is thus essential not only for developing "complete" architectures but also to make them as efficient as possible in terms of the computational resources they need to use.
Point clouds can be represented as complete graphs in which each edge is labeled with the distance between the corresponding points. Under this representation, detecting the isometry of two point clouds is reduced to detecting an isomorphism between their graph representation. Not surprisingly, then, much of the work on developing so-called geometric tests for detecting isometries over point clouds is inspired by the literature on isomorphism tests from graph theory. Of particular importance
in this context has been the use of geometric versions of the well-known Weisfeiler-Lehman test (WL test) for graph isomorphism [17].
Intuitively, the  $\ell$ -dimensional geometric WL test ( $\ell$ -WL test), for  $\ell \geq 1$ , iteratively colors each tuple  $\bar{v}$  of  $\ell$  points in a point cloud. The color of  $\bar{v}$  in round 0 is a complete description of the mutual distances between the points that belong to the tuple. In round  $t + 1$ , for  $t \geq 0$ , the color of  $\bar{v}$  is updated by combining in a suitable manner its color in iteration  $t$  with the color of each one of its neighbors, i.e., the tuples  $\bar{v}'$  that are obtained from  $\bar{v}$  by exchanging exactly one component of  $\bar{v}$  with another point in the cloud. The dimensionality of the WL test is therefore a measure of its computational cost: the higher the  $\ell$ , the more costly it is to implement the  $\ell$ -dimensional WL test.
For checking if two point clouds are isometric, the geometric WL test compares the resulting color patterns. If they differ, then we can be sure the point clouds are not isometric (that is, the test is sound). An important question, therefore, is what is the minimal  $\ell \geq 1$  for which the geometric  $\ell$ -WL test is complete, i.e., the fact that the color patterns obtained in two point clouds are the same implies that they are isometric.
There has been important progress on this problem recently: (a) Pozdnyakov and Ceriotti have shown that the geometric 1-WL test is incomplete for point clouds in 3D; that is, there exist isometric point clouds in three dimensions that cannot be distinguished by the geometric 1-WL test [15]; (b) Hordan et al. have proved that 3-WL test is complete in 3D after 1 iteration when initialized with Gram matrices of the triples of points instead of the mutual distances in these triples. A similar result has recently been obtained in [12]. Hordan et al. also gave a complete "2-WL-like" test, but this test explicitly uses coordinates of the points.
As further related results, in [10], geometric WL tests have been compared to the expressivity of invariant and equivariant graph neural networks. Non-geometric related results include e.g. [5], where for explicit graphs on  $n$  vertices it is shown that  $\ell$ -WL requires  $O(n)$  iterations for distinguishing them, whereas  $2\ell$ -WL requires  $O(\sqrt{n})$  and  $(3\ell - 1)$ -WL requires  $O(\log n)$  iterations, and [19], which has a promising proposal for generalized distances on non-geometric graphs, based on biconnectivity.
Main theoretical results. Our previous observation shows an evident gap in our understanding of the problem: What is the minimum  $\ell$ , where  $\ell = 2, 3$ , for which the geometric  $\ell$ -dimensional WL test is complete over three-dimensional point clouds? Our main contributions are the following:
- We show that for any  $d > 1$  the geometric  $(d - 1)$ -WL test is complete for detecting isometries over point clouds in  $\mathbb{R}^d$ . This is the positive counterpart of the result in [14] (namely, that geometric 1-WL test is incomplete in dimension  $d = 3$ ), by showing that geometric 1-WL is complete for  $d = 2$  and that geometric 2-WL test is complete for  $d = 3$ . Further, only three rounds of the geometric  $(d - 1)$ -WL test suffice to obtain this completeness result.
- We provide a simple proof that a single round of the geometric  $d$ -WL test is sufficient for identifying point clouds in  $\mathbb{R}^d$  up to isometry, for each  $d \geq 1$ . This can be seen as a refinement of the result in [9], with the difference that our test is initialized with the mutual distances inside  $d$ -tuples of points (as in the classical setting) while theirs is initialized with Gram matrices of  $d$ -tuples of points. In other words, the initial coloring of [9], for each  $d$ -tuple of points, in addition to their pairwise distances, includes their distances to the origin, while in our result we do not require this additional information.
These results, as well as previously mentioned results obtained in the literature, are all based on the standard folklore version of the  $\ell$ -WL test (as defined, e.g., in [3]). This is important because another version of the test, known as the oblivious  $\ell$ -WL test, has also been studied in the machine learning literature [14, 13]. It is known that, for each  $\ell \geq 1$ , the folklore  $\ell$ -WL test has the same discriminating power as the oblivious  $(\ell + 1)$ -WL test [7].
Table 1 summarizes the state of the art concerning the distinguishing power of the geometric  $\ell$ -WL test; note that it is not known whether the  $(d - 2)$ -WL test is incomplete for  $d \geq 4$ .
Relationship with graph neural networks. Graph Neural Networks (GNNs) are specialized neural networks designed to process data structured as graphs [8, 11]. Among GNNs, Message Passing GNNs (MPGNNs) use a message-passing algorithm to disseminate information between nodes in a graph [6]. The relationship between the 1-WL test and MPGNNs is now a fundamental subject in
Is  $\ell$ -WL complete for  $\mathbb{R}^d$ ?
1-WL	2-WL	3-WL
R2	Complete in 3 iterations Theorem 2.1	Complete in 1 iteration Theorem 5.1
R3	Incomplete [14]	Complete in 3 iterations Theorem 2.1	Complete in 1 iteration Theorem 5.1
R4	Open	Complete in 3 iterations Theorem 2.1
(d-2)-WL	(d-1)-WL	d-WL
Rd	Open	Complete in 3 iterations Theorem 2.1	Complete in 1 iteration Theorem 5.1
Table 1: What is known about the distinguish power of geometric  $\ell$  -WL.
this field. Seminal research has shown that these two approaches are essentially equivalent in their ability to distinguish non-isomorphic graph pairs [14, 18]. Additionally, Morris et al. [14] proposed higher-dimensional  $\ell$ -MPGNNs that have the same discriminating power as the oblivious  $\ell$ -WL test, and hence as the folklore  $(\ell - 1)$ -WL test, for  $\ell > 1$ .
The geometric WL test studied here corresponds to a particular case of the relational WL test, i.e., a suitable version of the WL test that is tailored for edge-labeled graphs. Connections between the relational WL test and so-called relational MPGNNs have recently been established by [1]. In particular, relational  $\ell$ -MPGNNs have the same discriminating power as the (folklore) geometric  $(\ell - 1)$ -WL test, for  $\ell > 1$ . Our main result implies then that there is no need to create specialized GNN architectures for distinguishing non-isometric point clouds in  $\mathbb{R}^d$ . Instead, relational  $d$ -MPGNNs are sufficient for this task.
Experimental evaluation of MPGNNs, based on the folklore 2-WL test, on data sets from molecular physics, and their comparison with state-of-art models, was recently performed in [12].
2 Formal Statement of the Main Result
Consider a cloud of  $n$  points  $S = \{p_1, \ldots, p_n\}$  in  $\mathbb{R}^d$ . We are interested in the problem of finding representations of such clouds that completely characterize them up to isometries, while at the same time being efficient from an algorithmic point of view. Our main motivation is to understand the expressiveness of the WL algorithm when applied to point clouds in euclidean space seen as complete distance graphs. Let us briefly recall how this algorithm works.
A function whose domain is  $S^{\ell}$  will be called an  $\ell$ -coloring of  $S$ . The  $\ell$ -WL algorithm is an iterative procedure which acts on  $S$  by assigning, at iteration  $i$ , an  $\ell$ -coloring  $\chi_{\ell,S}^{(i)}$  of  $S$ .
Initial coloring. The initial coloring,  $\chi_{\ell ,S}^{(0)}$  assigns to each  $\ell$  -tuple  $\mathbf{x} = (x_{1},\dots ,x_{\ell})\in S^{\ell}$  the color  $\chi_{\ell ,S}^{(0)}(\mathbf{x})$  given by the  $\ell \times \ell$  matrix
$$
\chi_ {\ell , S} ^ {(0)} (\mathbf {x}) _ {i j} = d (x _ {i}, x _ {j}) \quad i, j = 1, \dots , \ell
$$
of the relative distances inside the  $\ell$ -tuple (for  $\ell = 1$  we have a trivial coloring by the 0 matrix).
Iterative coloring. At each iteration, the  $\ell$ -WL algorithm updates the current coloring  $\chi_{\ell,S}^{(i)}$  to obtain a refined coloring  $\chi_{\ell,S}^{(i+1)}$ . The update operation is defined slightly differently depending on whether  $\ell = 1$  or  $\ell \geq 2$ .
- For  $\ell = 1$ , we have:
$$
\chi_ {1, S} ^ {(i + 1)} (x) = \left(\chi_ {1, S} ^ {(i)} (x), \{\{(d (x, y), \chi_ {1, S} ^ {(i)} (y)) \mid y \in S \}\}\right).
$$
That is, first,  $\chi_{i+1}(x)$  remembers the coloring of  $x$  from the previous step. Then it goes through all points  $y \in S$ . For each  $y$ , it stores the distance from  $x$  to  $y$  and also the coloring
of  $y$  from the previous step, and it remembers the multiset of these pairs. Note that one can determine which of these pairs comes from  $y = x$  itself since this is the only point with  $d(x,y) = 0$ . We also note that  $\chi_{1,S}^{(1)}(x)$  corresponds to the multiset of distances from  $x$  to the points of  $S$ .
- To define the update operation for  $\ell \geq 2$ , we first introduce additional notation. Let  $\mathbf{x} = (x_{1},\ldots ,x_{\ell})\in S^{\ell}$  and  $y\in S$ . By  $\mathbf{x}[y / i]$  we mean the tuple obtained from  $\mathbf{x}$  by replacing its  $i$ th coordinate by  $y$ . Then the update operation can be defined as follows:
$$
\chi_ {\ell , S} ^ {(i + 1)} (\mathbf {x}) = \left(\chi_ {\ell , S} ^ {(i)} (\mathbf {x}), \{\{\left(\chi_ {\ell , S} ^ {(i)} (\mathbf {x} [ y / 1 ]), \dots , \chi_ {\ell , S} ^ {(i)} (\mathbf {x} [ y / \ell ])\right) \mid y \in S \} \}\right). \tag {1}
$$
In other words, first,  $\chi_{\ell,S}^{(i+1)}(\mathbf{x})$  remembers the coloring of  $\mathbf{x}$  from the previous step, as before. Then, it goes through all  $y \in S$  and considers the  $\ell$  tuples  $\mathbf{x}[y/1], \ldots, \mathbf{x}[y/\ell]$ . It then takes the colorings of these tuples from the previous step and puts them into a tuple. The new coloring now remembers the multiset of these tuples.
In this paper we show that the coloring obtained after 3 iterations of  $(d - 1)$ -WL is a complete isometry invariant for point clouds in  $\mathbb{R}^d$ . More precisely, we show:
Theorem 2.1 (Main Theorem). For any  $d \geq 2$  and for any finite set  $S \subseteq \mathbb{R}^d$ , the following holds. Let  $\chi_{d-1,S}^{(3)}$  be the coloring of  $S^{d-1}$  obtained after 3 iterations of the  $(d-1)$ -WL algorithm on the distance graph of  $S$ . Then, knowing the multiset
$$
\mathcal {M} _ {d - 1} ^ {(3)} (S) = \{\{\chi_ {d - 1, S} ^ {(3)} (\mathbf {s}) \mid \mathbf {s} \in S ^ {d - 1} \} \},
$$
one can determine  $S$  up to an isometry.
Our proof is constructive in the sense that we exhibit an algorithm which, upon input  $\mathcal{M}_{d-1}^{(3)}(S)$ , computes the coordinates of a point cloud  $S'$  which is isometric to  $S$ . In particular, if  $\widetilde{S} \subset \mathbb{R}^d$  is another point cloud such that  $\mathcal{M}_{d-1}^{(3)}(\widetilde{S}) = \mathcal{M}_{d-1}^{(3)}(S)$ , then  $S$  and  $\widetilde{S}$  are isometric.
Our result is also true for point clouds that are multisets  $S \subseteq \mathbb{R}^d$  (for which distance graphs can have edges, labeled by 0, connecting two nodes, representing the same point in space), but for simplicity, we now present the argument for sets (although no significant changes are needed).
Organization of the paper. We start in Section 3 with the proof of the two-dimensional case, which while somewhat simpler, will allow us to introduce the general strategy. In Section 4 we explain how to implement this strategy in general for  $d > 2$ . Then, in Section 5, we discuss the completeness of one round of  $d$ -WL. Finally, in Section 6 we present open problems and limitations. Due to space constraints, some proofs have been relegated to the appendix.
3 Three iterations of 1-WL distinguish clouds in the plane
Let  $S \subseteq \mathbb{R}^2$  be a cloud of  $n$  points in the plane. Our task is to reconstruct  $S$  up to an isometry, using as input the information contained in  $\chi_{1,S}^{(3)}$ . This means to find a point cloud  $S'$  in the plane which is an image of  $S$  under some isometry. Our proof has two main steps: Initialization and Reconstruction. In the Initialization step we show how to extract from  $\chi_{1,S}^{(3)}$  the relevant information we need, which we call initialization data. In the Reconstruction step, we describe an algorithm that, given some initialization data, computes the coordinates of the desired isometric cloud.
In our reconstruction algorithm, we employ the notion of the barycenter of a point cloud (also known as the center of mass), which we denote by  $b$ , and is defined by:
$$
b = \frac {1}{n} \sum_ {w \in S} w.
$$
For simplicity, we translate  $S$  by  $-b$  so that the new barycenter sits at  $b = 0$ . Notice that since our reconstruction is up to isometries, this assumption does not affect the generality of our result. For each  $w \in S$ , let  $\| w \|$  denote its norm (its distance to  $b = 0$ ).
We say that two points  $u, v \in S$  satisfy the cone condition if  $u \neq 0$ ,  $v \neq 0$ , and, moreover,
- if  $0, u, v$  lie on the same line, then all points of  $S$  lie on this line;
- if  $0, u, v$  do not lie on the same line, then the interior of
$\operatorname{Cone}(u, v) = \{\alpha u + \beta v : \alpha, \beta \in [0, +\infty)\}$  does not contain points from  $S$  (see the first picture on Figure 1, the red area between  $(0, u)$  and  $(0, v)$  is disjoint from  $S$ ).
In order to initialize our reconstruction algorithm, we need the following information about  $S$ . We assume that  $S$  has more than 1 point (otherwise there is nothing to do).
Initialization Data: the initialization data consists of a real number  $d_0 \geq 0$  and two multisets  $M, M'$  such that for some  $u, v \in S$ , satisfying the cone condition, it holds that  $d_0 = d(u, v)$  and
$$
M = M _ {u} = \left\{\left(d (u, y), \| y \|): y \in S \right\} \right\}; \quad M ^ {\prime} = M _ {v} = \left\{\left(d (v, y), \| y \|): y \in S \right\} \right\}.
$$
We will start by describing the Reconstruction Algorithm, assuming that the initialization data is given. We will then show how to extract this data from  $\chi_{1,S}^{(3)}$  in the Initialization Step bellow.
Reconstruction algorithm. Assume that initialization data  $(d_0, M, M')$  is given. Our task is to determine  $S$  up to isometry. Note that from  $M$  we can determine  $\|u\|$ . Indeed, in  $M$  there exists exactly one element whose first coordinate is 0, and this element is  $(0, \|u\|)$ . Likewise, from  $M'$  we can determine  $\|v\|$ . We are also given  $d_0 = d(u, v)$ . Overall, we have all the distances between 0,  $u$ , and  $v$ . Up to a rotation of  $S$ , there is only one way to put  $u$ , it has to be somewhere on the circle of radius  $\|u\|$ , centered at the origin. We fix any point of this circle as  $u$ . After that, there are at most two points where we can have  $v$ . More specifically,  $v$  belongs to the intersection of two circles: one of radius  $\|v\|$  centered at the origin, and the other of radius  $d(u, v)$  centered at  $u$ . These two circles are different (remember that the cone condition includes a requirement that  $u \neq 0$ ). Hence, they intersect by at most two points. These points are symmetric w.r.t. the line that connects the centers of the two circles, i.e., 0 and  $u$ . Thus, up to a reflection through this line (which preserves the origin and  $u$ ), we know where to put  $v$ .
Henceforth, we can assume the coordinates of  $u$  and  $v$  are known to us. Note that so far, we have applied to  $S$  a translation (to put the barycenter at the origin), a rotation (to fix  $u$ ), and a reflection (to fix  $v$ ). We claim that, up to this isometry,  $S$  can be determined uniquely.
Let  $\mathrm{Refl}_u$  and  $\mathrm{Refl}_v$  denote the reflections through the lines  $(0, u)$  and  $(0, v)$ , respectively. We first observe that from  $M$  we can restore each point of  $S$  up to a reflection through the line  $(0, u)$ . Likewise, from  $M'$  we can do the same with respect to the line  $(0, v)$ . More precisely, we can compute the following multisets:
$$
L _ {u} = \left\{\left\{\left\{y, \operatorname {R e f l} _ {u} y \right\} \mid y \in S \right\} \right\}, \quad L _ {v} = \left\{\left\{\left\{y, \operatorname {R e f l} _ {v} y \right\} \mid y \in S \right\} \right\}.
$$
Indeed, consider any  $(d(u,y),\| y\|)\in M$ . What can we learn about  $y\in S$  from this pair of numbers? These numbers are distances from  $y$  to  $u$  and to 0. Thus,  $y$  must belong to the intersection of two circles: one with the center at  $u$  and radius  $d(u,y)$  and the other with the center at 0 and radius  $\| y\|$ . Again, since  $u\neq 0$ , these two circles are different. Thus, we obtain at most two points  $z_{1},z_{2}$  where one can put  $y$ . We will refer to these points as candidate locations for  $y$  w.r.t.  $u$ . They can be obtained one from the other by the reflection  $\mathrm{Refl}_u$  through the line connecting 0 and  $u$ . Hence,  $\{z_1,z_2\} = \{y,\mathrm{Refl}_uy\}$ . To compute  $L_{u}$ , we go through  $(d(u,y),\| y\|)\in M$ , compute candidate locations  $z_{1},z_{2}$  for  $y$ , and put  $\{z_1,z_2\}$  into  $L_{u}$ . In a similar fashion, one can compute  $L_{v}$  from  $M^{\prime}$ .
Let us remark that elements of  $L_{u}$  and  $L_{v}$  are sets of size 2 or 1. A set of size 2 appears as an element of  $L_{u}$  when some  $y$  has two distinct candidate locations w.r.t.  $u$ , that is when  $y$  does not lie on the line  $(0, u)$ . In turn, when  $y$  does lie on this line, we have  $z_{1} = z_{2} = y$  for two of its candidate locations, giving us an element  $\{y\} \in L_{u}$ , determining  $y$  uniquely. The same thing happens with respect to  $L_{v}$  for points that lie on the line  $(0, v)$ ,
The idea of our reconstruction algorithm is to gradually exclude some candidate locations so that more and more points get a unique possible location. What allows us to start is that  $u$  and  $v$  satisfy the cone condition; this condition gives us some area that is free of points from  $S$  (thus, one can exclude candidates belonging to this area).
The easy case is when  $0, u$ , and  $v$  belong to the same line. Then, by the cone condition, all points of  $S$  belong to this line. In this case, every point of  $S$  has just one candidate location. Hence, both multisets  $\bar{L}_u$  and  $L_v$  uniquely determine  $S$ .
Assume now that  $0, u$ , and  $v$  do not belong to the same line. As in the previous case, we can uniquely restore all points that belong to the line connecting 0 and  $u$ , or to the line connecting 0 and  $v$  (although now these are two different lines). Indeed, these are points that have exactly one candidate location w.r.t.  $u$  or w.r.t.  $v$ . They can be identified by going through  $L_{u}$  and  $L_{v}$  (we are interested in points  $z$  with  $\{z\} \in L_u \cup L_v$ ).
The pseudocode for our reconstruction algorithm is given in Algorithm 1. We now give its verbal description. Let us make a general remark about our algorithm. Once we find a unique location for some  $y \in S$ , we remove it from our set in order to reduce everything to the smaller set  $S \setminus \{y\}$ . This is implemented by updating the multisets  $L_{u}$  and  $L_{v}$  so that  $y$  is not taken into account in them. For that, we just remove  $\{y, \operatorname{Refl}_{u} y\}$  from  $L_{u}$  and  $\{y, \operatorname{Refl}_{v} y\}$  from  $L_{v}$  (more precisely, decrease their multiplicities by 1).
From now on, we assume that these two lines (connecting 0 and  $u$ , and 0 and  $v$ , respectively) are free of the points of  $S$ . These lines contain the border of the cone  $\operatorname{Cone}(u, v)$ . At the same time, the interior of this cone is disjoint from  $S$  due to the cone condition. Thus, in fact, the whole  $\operatorname{Cone}(u, v)$  is disjoint from  $S$ .
We now go through  $L_{u}$  and  $L_{v}$  in search of points for which one of the candidate locations (either w.r.t.  $u$  or w.r.t.  $v$ ) falls into the "forbidden area", that is, into  $\mathrm{Cone}(u,v)$ . After restoring these points and deleting them, we notice that the "forbidden area" becomes larger. Indeed, now in  $S$  there are no points that fall into  $\mathrm{Cone}(u,v)$  under one of the reflections  $\mathrm{Refl}_u$  or  $\mathrm{Refl}_v$ . In other words, the updated "forbidden area" is  $F = \mathrm{Cone}(u,v) \cup \mathrm{Refl}_u \mathrm{Cone}(u,v) \cup \mathrm{Refl}_v \mathrm{Cone}(u,v)$ . If the absolute angle between  $u$  and  $v$  is  $\alpha_{uv}$ , then,  $F$  has total amplitude  $3\alpha_{uv}$ . We now iterate this process, updating  $F$  successively. At each step, we know that after all removals made so far,  $S$  does not have points in  $F$ . Thus, points of  $S$  that fall into  $F$  under  $\mathrm{Refl}_u$  or under  $\mathrm{Refl}_v$  can be restored uniquely. After deleting them, we repeat the same operation with  $F \cup \mathrm{Refl}_u F \cup \mathrm{Refl}_v F$  in place of  $F$ .