Title: Three Iterations of (d − 1)-WL Test Distinguish Non Isometric Clouds of d-dimensional Points
Venue: NeurIPS 2023
Accepted: yes
Keywords: euclidean graphs

point clouds

WL test

graph neural networks
Abstract: The Weisfeiler-Lehman (WL) test is a fundamental iterative algorithm for checking the isomorphism of graphs. It has also been observed that it underlies the design of several graph neural network architectures, whose capabilities and performance can be understood in terms of the expressive power of this test. Motivated by recent developments in machine learning applications to datasets involving three-dimensional objects, we study when the WL test is {\em complete} for clouds of Euclidean points represented by complete distance graphs, i.e., when it can distinguish, up to isometry, any arbitrary such cloud. Our main result states that the $(d-1)$-dimensional WL test is complete for point clouds in $d$-dimensional Euclidean space, for any $d\ge 2$, and only three iterations of the test suffice. Our result is tight for $d = 2, 3$. We also observe that the $d$-dimensional WL test only requires one iteration to achieve completeness.

Three Iterations of ( d -1) -WL Test Distinguish Non Isometric Clouds of d -dimensional Points

Abstract

The Weisfeiler-Lehman (WL) test is a fundamental iterative algorithm for checking the isomorphism of graphs. It has also been observed that it underlies the design of several graph neural network architectures, whose capabilities and performance can be understood in terms of the expressive power of this test. Motivated by recent developments in machine learning applications to datasets involving threedimensional objects, we study when the WL test is complete for clouds of Euclidean points represented by complete distance graphs, i.e., when it can distinguish, up to isometry, any arbitrary such cloud. Our main result states that the ( d -1) -dimensional WL test is complete for point clouds in d -dimensional Euclidean space, for any d ≥ 2 , and only three iterations of the test suffice. Our result is tight for d = 2 , 3 . We also observe that the d -dimensional WL test only requires one iteration to achieve completeness.

1 Introduction

Context. Recent work in machine learning has raised the need to develop effective and efficient tests for checking if two three-dimensional point clouds, i.e., finite sets of points in R 3 , are isometric [15, 9, 2, 12]. Recall that, given two such point clouds P and Q , an isometry is a distance-preserving bijection between the points in P and Q . The importance of these tests is that they provide the foundations for designing neural network architectures on point clouds that are capable of fully exploiting the structure of the data [18, 14]. It has been observed that the incompleteness of any such an architecture, i.e., the inability to recognize a point cloud up to isometry, can affect its learning performance [16]. Understanding which is the simplest test that allows detecting isometries in this scenario is thus essential not only for developing 'complete" architectures but also to make them as efficient as possible in terms of the computational resources they need to use.

Point clouds can be represented as complete graphs in which each edge is labeled with the distance between the corresponding points. Under this representation, detecting the isometry of two point clouds is reduced to detecting an isomorphism between their graph representation. Not surprisingly, then, much of the work on developing so-called geometric tests for detecting isometries over point clouds is inspired by the literature on isomorphism tests from graph theory. Of particular importance

2

in this context has been the use of geometric versions of the well-known Weisfeiler-Lehman test (WL test) for graph isomorphism [17].

Intuitively, the ℓ -dimensional geometric WL test ( ℓ -WL test), for ℓ ≥ 1 , iteratively colors each tuple ¯ v of ℓ points in a point cloud. The color of ¯ v in round 0 is a complete description of the mutual distances between the points that belong to the tuple. In round t +1 , for t ≥ 0 , the color of ¯ v is updated by combining in a suitable manner its color in iteration t with the color of each one of its neighbors , i.e., the tuples ¯ v ′ that are obtained from ¯ v by exchanging exactly one component of ¯ v with another point in the cloud. The dimensionality of the WL test is therefore a measure of its computational cost: the higher the ℓ , the more costly it is to implement the ℓ -dimensional WL test.

For checking if two point clouds are isometric, the geometric WL test compares the resulting color patterns. If they differ, then we can be sure the point clouds are not isometric (that is, the test is sound ). An important question, therefore, is what is the minimal ℓ ≥ 1 for which the geometric ℓ -WL test is complete , i.e., the fact that the color patterns obtained in two point clouds are the same implies that they are isometric.

There has been important progress on this problem recently: (a) Pozdnyakov and Ceriotti have shown that the geometric 1-WL test is incomplete for point clouds in 3D; that is, there exist isometric point clouds in three dimensions that cannot be distinguished by the geometric 1-WL test [15]; (b) Hordan et al. have proved that 3-WL test is complete in 3D after 1 iteration when initialized with Gram matrices of the triples of points instead of the mutual distances in these triples. A similar result has recently been obtained in [12]. Hordan et al. also gave a complete '2-WL-like' test, but this test explicitly uses coordinates of the points.

As further related results, in [10], geometric WL tests have been compared to the expressivity of invariant and equivariant graph neural networks. Non-geometric related results include e.g. [5], where for explicit graphs on n vertices it is shown that ℓ -WL requires O ( n ) iterations for distinguishing them, whereas 2 ℓ -WL requires O ( √ n ) and (3 ℓ -1) -WL requires O (log n ) iterations, and [19], which has a promising proposal for generalized distances on non-geometric graphs, based on biconnectivity.

Main theoretical results. Our previous observation show an evident gap in our understanding of the problem: What is the minimum ℓ , where ℓ = 2 , 3 , for which the geometric ℓ -dimensional WL test is complete over three-dimensional point clouds? Our main contributions are the following:

We show that for any d > 1 the geometric ( d -1) -WL test is complete for detecting isometries over point clouds in R d . This is the positive counterpart of the result in [14] (namely, that geometric 1-WL test is incomplete in dimension d = 3 ), by showing that geometric 1-WL is complete for d = 2 and that geometric 2-WL test is complete for d = 3 . Further, only three rounds of the geometric ( d -1) -WL test suffice to obtain this completeness result.

We provide a simple proof that a single round of the geometric d -WL test is sufficient for identifying point clouds in R d up to isometry, for each d ≥ 1 . This can be seen as a refinement of the result in [9], with the difference that our test is initialized with the mutual distances inside d -tuples of points (as in the classical setting) while theirs is initialized with Gram matrices of d -tuples of points. In other words, the initial coloring of [9], for each d -tuple of points, in addition to their pairwise distances, includes their distances to the origin , while in our result we do not require this additional information.

These results, as well as previously mentioned results obtained in the literature, are all based on the standard folklore version of the ℓ -WL test (as defined, e.g., in [3]). This is important because another version of the test, known as the oblivious ℓ -WL test, has also been studied in the machine learning literature [14, 13]. It is known that, for each ℓ ≥ 1 , the folklore ℓ -WL test has the same discriminating power as the oblivious ( ℓ +1) -WL test [7].

Table 1 summarizes the state of the art concerning the distinguishing power of the geometric ℓ -WL test; note that it is not known whether the ( d -2) -WL test is incomplete for d ≥ 4 .

Relationship with graph neural networks. Graph Neural Networks (GNNs) are specialized neural networks designed to process data structured as graphs [8, 11]. Among GNNs, Message Passing GNNs (MPGNNs) use a message-passing algorithm to disseminate information between nodes in a graph [6]. The relationship between the 1-WL test and MPGNNs is now a fundamental subject in

Is

ℓ

-WL complete for

R

d

?

1-WL

2-WL

3-WL

Complete

Complete

R

2

in 3 iterations

in 1 iteration

Theorem 2.1

Theorem 5.1

Incomplete

Complete

Complete

R

3

in 3 iterations

in 1 iteration

[14]

Theorem 2.1

Theorem 5.1

Complete

R

4

Open

in 3 iterations

Theorem 2.1

(

d

-

2)

-WL

(

d

-

1)

-WL

d

-WL

Complete

Complete

R

d

Open

in 3 iterations

in 1 iteration

Theorem 2.1

Theorem 5.1

this field. Seminal research has shown that these two approaches are essentially equivalent in their ability to distinguish non-isomorphic graph pairs [14, 18]. Additionally, Morris et al. [14] proposed higher-dimensional ℓ -MPGNNs that have the same discriminating power as the oblivious ℓ -WL test, and hence as the folklore ( ℓ -1) -WL test, for ℓ > 1 .

The geometric WL test studied here corresponds to a particular case of the relational WLtest, i.e., a suitable version of the WL test that is tailored for edge-labeled graphs. Connections between the relational WL test and so-called relational MPGNNs have recently been established by [1]. In particular, relational ℓ -MPGNNs have the same discriminating power as the (folklore) geometric ( ℓ -1) -WLtest, for ℓ > 1 . Our main result implies then that there is no need to create specialized GNN architectures for distinguishing non-isometric point clouds in R d . Instead, relational d -MPGNNs are sufficient for this task.

Experimental evaluation of MPGNNs, based on the folklore 2-WL test, on data sets from molecular physics, and their comparison with state-of-art models, was recently performed in [12].

2 Formal Statement of the Main Result

Consider a cloud of n points S = { p 1 , . . . , p n } in R d . We are interested in the problem of finding representations of such clouds that completely characterize them up to isometries, while at the same time being efficient from an algorithmic point of view. Our main motivation is to understand the expressiveness of the WL algorithm when applied to point clouds in euclidean space seen as complete distance graphs. Let us briefly recall how this algorithm works.

A function whose domain is S ℓ will be called an ℓ -coloring of S . The ℓ -WL algorithm is an iterative procedure which acts on S by assigning, at iteration i , an ℓ -coloring χ ( i ) ℓ,S of S .

Initial coloring. The initial coloring, χ (0) ℓ,S , assigns to each ℓ -tuple x = ( x 1 , . . . , x ℓ ) ∈ S ℓ the color χ (0) ℓ,S ( x ) given by the ℓ × ℓ matrix

of the relative distances inside the ℓ -tuple (for ℓ = 1 we have a trivial coloring by the 0 matrix).

Iterative coloring. At each iteration, the ℓ -WL algorithm updates the current coloring χ ( i ) ℓ,S to obtain a refined coloring χ ( i +1) ℓ,S . The update operation is defined slightly differently depending on whether ℓ = 1 or ℓ ≥ 2 .

For ℓ = 1 , we have:

That is, first, χ i +1 ( x ) remembers the coloring of x from the previous step. Then it goes through all points y ∈ S . For each y , it stores the distance from x to y and also the coloring

of y from the previous step, and it remembers the multiset of these pairs. Note that one can determine which of these pairs comes from y = x itself since this is the only point with d ( x, y ) = 0 . We also note that χ (1) 1 ,S ( x ) corresponds to the multiset of distances from x to the points of S .

To define the update operation for ℓ ≥ 2 , we first introduce additional notation. Let x = ( x 1 , . . . , x ℓ ) ∈ S ℓ and y ∈ S . By x [ y/i ] we mean the tuple obtained from x by replacing its i th coordinate by y . Then the update operation can be defined as follows:

In other words, first, χ ( i +1) ℓ,S ( x ) remembers the coloring of x from the previous step, as before. Then, it goes through all y ∈ S and considers the ℓ tuples x [ y/ 1] , . . . , x [ y/ℓ ] . It then takes the colorings of these tuples from the previous step and puts them into a tuple. The new coloring now remembers the multiset of these tuples.

In this paper we show that the coloring obtained after 3 iterations of ( d -1) -WL is a complete isometry invariant for point clouds in R d . More precisely, we show:

Theorem 2.1 (Main Theorem) . For any d ≥ 2 and for any finite set S ⊆ R d , the following holds. Let χ (3) d -1 ,S be the coloring of S d -1 obtained after 3 iterations of the ( d -1) -WL algorithm on the distance graph of S . Then, knowing the multiset

one can determine S up to an isometry.

Our proof is constructive in the sense that we exhibit an algorithm which, upon input M (3) d -1 ( S ) , computes the coordinates of a point cloud S ′ which is isometric to S . In particular, if ˜ S ⊂ R d is another point cloud such that M (3) d -1 ( ˜ S ) = M (3) d -1 ( S ) , then S and ˜ S are isometric.

Our result is also true for point clouds that are multisets S ⊆ R d (for which distance graphs can have edges, labeled by 0 , connecting two nodes, representing the same point in space), but for simplicity, we now present the argument for sets (although no significant changes are needed).

Organization of the paper. We start in Section 3 with the proof of the two-dimensional case, which while somewhat simpler, will allow us to introduce the general strategy. In Section 4 we explain how to implement this strategy in general for d > 2 . Then, in Section 5, we discuss the completeness of one round of d -WL. Finally, in Section 6 we present open problems and limitations. Due to space constraints, some proofs have been relegated to the appendix.

3 Three iterations of 1-WL distinguish clouds in the plane

Let S ⊆ R 2 be a cloud of n points in the plane. Our task is to reconstruct S up to an isometry, using as input the information contained in χ (3) 1 ,S . This means to find a point cloud S ′ in the plane which is an image of S under some isometry. Our proof has two main steps: Initialization and Reconstruction . In the Initialization Step we show how to extract from χ (3) 1 ,S the relevant information we need, which we call initialization data . In the Reconstruction step, we describe an algorithm that, given some initialization data, computes the coordinates of the desired isometric cloud.

In our reconstruction algorithm, we employ the notion of the barycenter of a point cloud (also known as the center of mass), which we denote by b , and is defined by:

For simplicity, we translate S by -b so that the new barycenter sits at b = 0 . Notice that since our reconstruction is up to isometries, this assumption does not affect the gerenality of our result. For each w ∈ S , let ∥ w ∥ denote its norm (its distance to b = 0 ).

̸

̸

We say that two points u, v ∈ S satisfy the cone condition if u = 0 , v = 0 , and, moreover,

if 0 , u, v lie on the same line, then all points of S lie on this line;

if 0 , u, v do not lie on the same line, then the interior of

Cone ( u, v ) = { αu + βv : α, β ∈ [0 , + ∞ ) } does not contain points from S (see the first picture on Figure 1, the red area between (0 , u ) and (0 , v ) is disjoint from S ).

In order to initialize our reconstruction algorithm, we need the following information about S . We assume that S has more than 1 point (otherwise there is nothing to do).

Initialization Data : the initialization data consists of a real number d 0 ≥ 0 and two multisets M,M ′ such that for some u, v ∈ S , satisfying the cone condition, it holds that d 0 = d ( u, v ) and

We will start by describing the Reconstruction Algorithm, assuming that the initialization data is given. We will then show how to extract this data from χ (3) 1 ,S in the Initialization Step bellow.

̸

Reconstruction algorithm. Assume that initialization data ( d 0 , M, M ′ ) is given. Our task is to determine S up to isometry. Note that from M we can determine ∥ u ∥ . Indeed, in M there exists exactly one element whose first coordinate is 0 , and this element is (0 , ∥ u ∥ ) . Likewise, from M ′ we can determine ∥ v ∥ . We are also given d 0 = d ( u, v ) . Overall, we have all the distances between 0 , u , and v . Up to a rotation of S , there is only one way to put u , it has to be somewhere on the circle of radius ∥ u ∥ , centered at the origin. We fix any point of this circle as u . After that, there are at most two points where we can have v . More specifically, v belongs to the intersection of two circles: one of radius ∥ v ∥ centered at the origin, and the other of radius d ( u, v ) centered at u . These two circles are different (remember that the cone condition includes a requirement that u = 0 ). Hence, they intersect by at most two points. These points are symmetric w.r.t. the line that connects the centers of the two circles, i.e., 0 and u . Thus, up to a reflection through this line (which preserves the origin and u ), we know where to put v .

Henceforth, we can assume the coordinates of u and v are known to us. Note that so far, we have applied to S a translation (to put the barycenter at the origin), a rotation (to fix u ), and a reflection (to fix v ). We claim that, up to this isometry, S can be determined uniquely.

Let Refl u and Refl v denote the reflections through the lines (0 , u ) and (0 , v ) , respectively. We first observe that from M we can restore each point of S up to a reflection through the line (0 , u ) . Likewise, from M ′ we can do the same with respect to the line (0 , v ) . More precisely, we can compute the following multisets:

̸

Indeed, consider any ( d ( u, y ) , ∥ y ∥ ) ∈ M . What can we learn about y ∈ S from this pair of numbers? These numbers are distances from y to u and to 0 . Thus, y must belong to the intersection of two circles: one with the center at u and radius d ( u, y ) and the other with the center at 0 and radius ∥ y ∥ . Again, since u = 0 , these two circles are different. Thus, we obtain at most two points z 1 , z 2 where one can put y . We will refer to these points as candidate locations for y w.r.t. u . They can be obtained one from the other by the reflection Refl u through the line connecting 0 and u . Hence, { z 1 , z 2 } = { y, Refl u y } . To compute L u , we go through ( d ( u, y ) , ∥ y ∥ ) ∈ M , compute candidate locations z 1 , z 2 for y , and put { z 1 , z 2 } into L u . In a similar fashion, one can compute L v from M ′ .

Let us remark that elements of L u and L v are sets of size 2 or 1. A set of size 2 appears as an element of L u when some y has two distinct candidate locations w.r.t. u , that is when y does not lie on the line (0 , u ) . In turn, when y does lie on this line, we have z 1 = z 2 = y for two of its candidate locations, giving us an element { y } ∈ L u , determining y uniquely. The same thing happens with respect to L v for points that lie on the line (0 , v ) ,

The idea of our reconstruction algorithm is to gradually exclude some candidate locations so that more and more points get a unique possible location. What allows us to start is that u and v satisfy the cone condition; this condition gives us some area that is free of points from S (thus, one can exclude candidates belonging to this area).

The easy case is when 0 , u , and v belong to the same line. Then, by the cone condition, all points of S belong to this line. In this case, every point of S has just one candidate location. Hence, both multisets L u and L v uniquely determine S .

Assume now that 0 , u , and v do not belong to the same line. As in the previous case, we can uniquely restore all points that belong to the line connecting 0 and u , or to the line connecting 0 and v (although now these are two different lines). Indeed, these are points that have exactly one candidate location w.r.t. u or w.r.t. v . They can be identified by going through L u and L v (we are interested in points z with { z } ∈ L u ∪ L v ).

The pseudocode for our reconstruction algorithm is given in Algorithm 1. We now give its verbal description. Let us make a general remark about our algorithm. Once we find a unique location for some y ∈ S , we remove it from our set in order to reduce everything to the smaller set S \ { y } . This is implemented by updating the multisets L u and L v so that y is not taken into account in them. For that, we just remove { y, Refl u y } from L u and { y, Refl v y } from L v (more precisely, decrease their multiplicities by 1).

From now on, we assume that these two lines (connecting 0 and u , and 0 and v , respectively) are free of the points of S . These lines contain the border of the cone Cone ( u, v ) . At the same time, the interior of this cone is disjoint from S due to the cone condition. Thus, in fact, the whole Cone ( u, v ) is disjoint from S .

We now go through L u and L v in search of points for which one of the candidate locations (either w.r.t. u or w.r.t. v ) falls into the 'forbidden area', that is, into Cone ( u, v ) . After restoring these points and deleting them, we notice that the 'forbidden area' becomes larger. Indeed, now in S there are no points that fall into Cone ( u, v ) under one of the reflections Refl u or Refl v . In other words, the updated 'forbidden area' is F = Cone ( u, v ) ∪ Refl u Cone ( u, v ) ∪ Refl v Cone ( u, v ) . If the absolute angle between u and v is α uv , then, F has total amplitude 3 α uv . We now iterate this process, updating F successively. At each step, we know that after all removals made so far, S does not have points in F . Thus, points of S that fall into F under Refl u or under Refl v can be restored uniquely. After deleting them, we repeat the same operation with F ∪ Refl u F ∪ Refl v F in place of F .

Algorithm 1: Reconstruction algorithm

̸

After k such 'updates', F will consist of 2 k +1 adjacent angles, each of size α uv , with Cone ( u, v ) being in the center. In each update, we replace F with F ∪ Refl u F ∪ Refl v F . This results in adding two angles of size α uv to both sides of F . Indeed, if we look at the ray (0 , u ) , it splits our current F into two angles, one of size ( k +1) α uv and the other of size kα uv . Under Refl u , the part whose size

0

u

v

(a)

0

u

v

(b)

0

u

v

(c)

is ( k +1) α uv adds an angle of size α uv to the other part. Analogously, Refl v adds an angle of size α uv from the opposite side of F . See Figure 1 for the illustration of this process.

Within at most 1+ π α uv such steps, F covers all angular directions, thus completing the reconstruction of S .

Initialization step. We explain how to obtain the Initialization Data about S from M (3) 1 ( S ) .

We start by observing that from the first iteration of 1-WL, we can compute ∥ x ∥ for all x ∈ S . As the following lemma shows, this holds in any dimension, with the same proof. We temporarily omit the current hypothesis b = 0 , in order to use the lemma later without this hypothesis.

Lemma 3.1 (The Barycenter Lemma) . Take any n -point cloud S ⊆ R d and let

Then for every x ∈ S , knowing D x and the multiset { { D y | y ∈ S } } , one can determine the distance from x to the barycenter of S .

Proof. Consider the function f : R d → [0 , + ∞ ) defined as f ( x ) = ∑ y ∈ S ∥ x -y ∥ 2 , namely f ( x ) is the sum of the squares of all elements of D x (with multiplicities). It follows that ∑ y ∈ S f ( y ) is determined by { { D y | y ∈ S } } . The lemma is thus proved if we prove the following equality

To prove the above, we first write:

Denote Γ = ∑ y ∈ S ∥ b -y ∥ 2 . Substituting the expression for f ( x ) and f ( y ) from above into the right-hand side of (2), we get:

as required.

By definition, χ (1) 1 ,S ( x ) determines the multiset D x = { { d ( x, y ) | y ∈ S } } of distances from x to points of S . Since we are given the multiset M (3) 1 ( S ) , we also know the multset M (1) 1 ( S ) = { { χ (1) 1 ,S ( x ) | x ∈ S } } (labels after the third iterations determine labels from previous iterations). In particular, this gives us the multiset { { D x | x ∈ S } } . Overall, due to the Barycenter lemma, we conclude that χ (1) 1 ,S ( x ) can be converted into ∥ x ∥ .

Now, remember that

By converting χ (1) 1 ,S ( y ) into ∥ y ∥ for all y ∈ S here, one can convert χ (2) 1 ,S ( x ) into the multiset

We need one more iteration to find d ( u, v ) , M u , M v for some u, v ∈ S satisfying the cone condition. In fact, we only need

̸

for arbitrary u ∈ S with u = 0 . Since χ (3) 1 ,S ( u ) determines ∥ u ∥ , such χ (3) 1 ,S ( u ) can indeed be selected from M (3) 1 ( S ) (and since we assume that S has more than one point, we know that there are points in S that are different from 0 ).

̸

Due to the fact that χ (2) 1 ,S ( y ) can be converted to M y , we can in turn convert χ (3) 1 ,S ( u ) into the multiset A = { {( d ( u, y ) , M y ) : y ∈ S } } . In particular, since y = u is the only point for which d ( u, y ) = 0 , we can compute M u from A . Once we have M u , the rest of the initialization goes as follows. First note that for a given element ( d ( u, y ) , M y ) in A with d ( u, y ) > 0 (so that y = u ), we can look in M y for the only element with 0 as the first entry, whose second entry is then ∥ y ∥ . So we can obtain ( d ( u, y ) , ∥ y ∥ ) . As in the Reconstruction Algorithm, we then have only two possibilities for the location of y relative to u , say y 1 and y 2 = Refl u ( y 1 ) . It follows that the absolute value of the angle α uy between u and y is uniquely determined (if ∥ y ∥ = 0 , we set α uy = 0 ), and we can compute it from A . In order to select v , we go though A and look for v such that α uv is the smallest angle among { α uy | y ∈ S, 0 < α uy < π } . If such a v does not exist, all points of S must lie on the line connecting 0 and u . In this case, the cone condition is satisfied, for example, for u and v = u . Thus, we can initialize with d 0 = 0 , M = M ′ = M u . If v as above exists, there can be no point in the interior of Cone ( u, v ) , since otherwise there would be y with 0 < α uy < α uv < π , contradicting the minimality of α uv . Thus, the cone condition is satisfied for u, v . We can then set d 0 = d ( u, v ) , M = M u and M ′ = M v .

4 Proof of Main Theorem for d > 2

We now present the proof for the case d > 2 . The strategy of the proof has the same structure as for d = 2 . Since the objects involved now are more general, it will be convenient to introduce some terminology. Let x = ( x 1 , . . . , x k ) ∈ ( R d ) k be a k -tuple of points in R d . The distance matrix of x is the k × k matrix A given by A ij = d ( x i , x j ) , i, j = 1 , . . . , k .

Now, let S ⊆ R d be a finite set. Then the distance profile of x w.r.t. S is the multiset

As before, we let b = 1 | S | ∑ y ∈ S y denote the barycenter of S . For a finite set G ⊂ R d , we denote by LinearSpan ( G ) the linear space spanned by G , and by AffineSpan ( G ) the corresponding affine one. Their respective dimensions will be denoted by LinearDim ( G ) and AffineDim ( G ) .

As for the case d = 2 , we start by distilling the Initialization Data required for the reconstruction, which is described relative to the barycenter b of S . For convenience, we have decided not to assume at this stage that S has been translated first to put b at the origin, as we did for the sake of the exposition in the case d = 2 . This is now the task of the isometry we apply to S when choosing locations for its points, which we now completely relegate to the reconstruction phase.

Definition 1. Let S ∈ R d be a finite set and let b be its barycenter. A d -tuple x = ( x 1 , . . . , x d ) ∈ S d satisfies the cone condition if

AffineDim ( b, x 1 , . . . , x d ) = AffineDim ( S ) ;

if AffineDim ( S ) = d , then there is no x ∈ S such that x -b belongs to the interior of Cone ( x 1 -b, . . . , x d -b ) .

Definition 2. For a tuple x = ( x 1 , . . . , x d ) ∈ S d , we define its enhanced profile as

where A is the distance matrix of the tuple ( b, x 1 , . . . , x d ) and M i = D x [ b/i ] is the distance profile of the tuple ( x 1 , . . . , x i -1 , b, x i +1 , . . . , x d ) with respect to S .

Definition 3. Let S ∈ R d be a finite set and let b be its barycenter. An initialization data for S is a tuple ( A,M 1 , . . . , M d ) such that ( A,M 1 , . . . , M d ) = EP ( x 1 , . . . , x d ) for some d -tuple x = ( x 1 , . . . , x d ) ∈ S d satisfying the cone condition.

Remark 4.1 . The interested reader can verify that the Initialization Data condition extends the definition for d = 2 . Note that the first bullet of the Cone Condition is automatically verified for d = 2 , but is nontrivial for d > 2 .

The fact that an initialization data ( A,M 1 , . . . , M d ) can be recovered from M d -1 ( S ) is ensured by the following proposition.

Proposition 4.2 (Initialization Lemma) . Take any S ⊆ R d . Then, knowing the multiset { { χ (3) d -1 ,S ( s ) | s ∈ S d -1 } } , one can determine an initialization data for S .

We now proceed with the reconstruction phase.

4.1 Reconstruction Algorithm

Assume an Initialization Data ( A,M 1 , . . . , M d ) for a finite S ⊂ R d is given. Our first task is to choose, up to isometry, positions for the points in the ( d +1) -tuple ( b, x 1 , . . . , x d ) corresponding to the matrix A . We use the following classical lemma, whose proof is given e.g. in [4, Sec. 2.2.1].

Lemma 4.3 (Anchor Lemma) . If ( u 1 , . . . , u k ) ∈ R d and ( v 1 , . . . , v k ) ∈ R d have the same distance matrix, then there exists an isometry f : R d → R d such that f ( u i ) = v i for all i = 1 , . . . , k .

As for d = 2 , we put the barycenter of the cloud at the origin. Then, we simply apply the Anchor Lemma to any collection of points z 1 , . . . , z d ∈ R d such that our given A is also the distance matrix of the tuple (0 , z 1 , . . . , z d ) . The Lemma then gives us an isometry f : R d → R d such that f ( b ) = 0 , f ( x 1 ) = z 1 , . . . , f ( x d ) = z d . As distance profiles are invariant under isometries, our given M i is also the distance profile of the tuple ( z 1 , . . . , z i -1 , 0 , z i +1 , . . . , z d ) w.r.t. f ( S ) . Our task now is, from M 1 , . . . , M d , to uniquely determine the locations of all points in f ( S ) . This would give us S up to an isometry. Since now we have locations for ( z 1 , . . . , z d ) , we can in fact compute:

AffineDim ( S ) = AffineDim ( b, x 1 , . . . , x d ) = AffineDim (0 , z 1 , . . . , z d ) = LinearDim ( z 1 , . . . , z d ) , where the first equality is guaranteed by the cone condition. The reconstruction algorithm depends on whether AffineDim ( S ) = d or not.

Consider first the case when AffineDim ( S ) = LinearDim ( z 1 , . . . , z d ) < d . Then there exists i ∈ { 1 , . . . , d } such that LinearDim ( z 1 , . . . , z d ) = LinearDim ( z 1 , . . . , z i -1 , z i +1 , . . . , z d ) . This means that f ( S ) ⊂ AffineSpan ( z 1 , . . . , z i -1 , 0 , z i +1 , . . . , z d ) , since otherwise f ( S ) would have larger affine dimension. We claim that, in this case, from M i we can restore the location of all points in f ( S ) . Indeed, from M i we know, for each z ∈ f ( S ) , a tuple with the distances from z to z 1 , . . . , z i -1 , 0 , z i +1 , . . . , z d . As the next lemma shows, this information is enough to uniquely determine the location of z .

Lemma 4.4. Take any x 1 , . . . , x m ∈ R d . Assume that a, b ∈ AffineSpan ( x 1 , . . . , x m ) are such that d ( a, x i ) = d ( b, x i ) for all i = 1 , . . . , m . Then a = b .

It remains to reconstruct f ( S ) when AffineDim ( S ) = d , in which case our pivot points z 1 , . . . , z d are linearly independent. Recall that M i is the distance profile of the tuple ( z 1 , . . . , z i -1 , 0 , z i +1 , . . . , z d ) w.r.t. f ( S ) . Moreover, since no x in S is such that x -b lies in the interior of Cone ( x 1 -b, . . . , x d -b ) , we know that f ( S ) must be disjoint from the interior of Cone ( z 1 , . . . , z d ) . As the next proposition shows, this information is enough to reconstruct f ( S ) in this case as well, which finishes the proof of Theorem 2.1 for d > 2 .

Proposition 4.5 (The Reconstruction Lemma) . Assume that z 1 , . . . , z d ∈ R d are linearly independent. Let T ⊆ R d be finite and disjoint from the interior of Cone ( z 1 , . . . , z d ) . If, for every i = 1 , . . . , d , we are given z i and also the distance profile of the tuple ( z 1 , . . . , z i -1 , 0 , z i +1 , . . . , z d ) w.r.t. T , then we can uniquely determine T .

5 On the distinguishing power of one iteration of d -WL

In this section, we discuss a somewhat different strategy to reconstruct S . It is clear that if for a point z ∈ R d we are given the distances from it to d +1 points in general position with known coordinates, then the position of z is uniquely determined (see e.g. Lemma 4.4). Since d -WL colors d -tuples of points in S , a natural strategy to recover S is to use the barycenter as an additional point. By Lemma 3.1, we know that distances to the barycenter from points of S can be obtained after one iteration of d -WL. However, the information that allows us to match d ( z, b ) to the distances from this z to a d -tuple, is readily available only after two iterations of d -WL. It follows that this simple strategy can be used to directly reconstruct S from the second iteration of d -WL. We remark that this strategy is similar to the one used in [9] to uniquely determine S up to isometries when the coloring we are initially given corresponds to certain Gram Matrices for d -tuples of points. Essentially, after one interaction of d -WL over this initial data, we obtain enough information to directly determine the location of each z relative to a collection of d +1 points. In fact, it is not hard to show that from the first iteration of d -WL, applied to the distance graph of S , one can compute these Gram Matrices, thus providing an alternative proof that two iterations suffice for distinguishing geometric graphs.

We show instead that only one iteration suffices. Our approach differs and depends on certain geometric principles that allow us to simplify the problem by conducting an exhaustive search across an exponentially large range of possibilities.

Theorem 5.1. For any d ≥ 1 and for any finite set S ⊆ R d , knowing the multiset { { χ (1) d,S ( s ) | s ∈ S d } } , one can determine S up to an isometry.

6 Final remarks

Open problems. An interesting open question about our work is what is the minimum number of rounds needed for the ( d -1) -WL test to be complete with respect to point clouds in R d . Our result shows that three rounds suffice, but we do not know the completeness status of the test when only one or two rounds are allowed. Another open problem is the completeness status of the ( d -2) -WL test for d -dimensional point clouds when d > 3 .

Limitations. A consequence of our main result and is that distance-based d -MPGNNs possess sufficient expressive capability to learn d -dimensional point clouds up to isometry. However, the computational complexity of implementing d -MPGNNs is a major concern, as it involves O ( n d ) operations per iteration, where n denotes the number of nodes in the graph. This computational cost can quickly become unmanageable, even for relatively small values of d , such as d = 3 . It remains to be studied which kind of optimizations on higher-order GNNs can be implemented for improved performance without much sacrifice on their expressive power.

References

Pablo Barceló, Mikhail Galkin, Christopher Morris, and Miguel A. Romero Orth. Weisfeiler and leman go relational. In LOG , volume 198, page 46. PMLR, 2022.

Ilyes Batatia, Dávid Péter Kovács, Gregor N. C. Simm, Christoph Ortner, and Gábor Csányi. Mace: Higher order equivariant message passing neural networks for fast and accurate force fields, 2022.

Jin-yi Cai, Martin Fürer, and Neil Immerman. An optimal lower bound on the number of variables for graph identification. Comb. , 12(4):389-410, 1992.

Trevor F. Cox and Michael A. A. Cox. Multidimensional Scaling (2nd edition) . Chapman and Hall / CRC, 2001.

Martin Fürer. Weisfeiler-lehman refinement requires at least a linear number of iterations. In Automata, Languages and Programming: 28th International Colloquium, ICALP 2001 Crete, Greece, July 8-12, 2001 Proceedings 28 , pages 322-333. Springer, 2001.

Justin Gilmer, Samuel S. Schoenholz, Patrick F. Riley, Oriol Vinyals, and George E. Dahl. Neural message passing for quantum chemistry. In ICML , 2017.

Martin Grohe. The logic of graph neural networks. In LICS , pages 1-17. IEEE, 2021.

William L. Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. In NeurIPS , pages 1024-1034, 2017.

Snir Hordan, Tal Amir, Steven J. Gortler, and Nadav Dym. Complete neural networks for euclidean graphs, 2023.

Chaitanya K Joshi, Cristian Bodnar, Simon V Mathis, Taco Cohen, and Pietro Lio. On the expressive power of geometric graph neural networks. arXiv preprint arXiv:2301.09308 , 2023.

Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In ICLR , 2017.

Zian Li, Xiyuan Wang, Yinan Huang, and Muhan Zhang. Is distance matrix enough for geometric deep learning?, 2023.

Christopher Morris, Yaron Lipman, Haggai Maron, Bastian Rieck, Nils M. Kriege, Martin Grohe, Matthias Fey, and Karsten M. Borgwardt. Weisfeiler and leman go machine learning: The story so far. CoRR , abs/2112.09992, 2021.

Christopher Morris, Martin Ritzert, Matthias Fey, William L. Hamilton, Jan Eric Lenssen, Gaurav Rattan, and Martin Grohe. Weisfeiler and leman go neural: Higher-order graph neural networks. In AAAI , pages 4602-4609, 2019.

Sergey N. Pozdnyakov and Michele Ceriotti. Incompleteness of graph convolutional neural networks for points clouds in three dimensions. CoRR , abs/2201.07136, 2022.

Sergey N. Pozdnyakov, Michael J. Willatt, Albert P. Bartók, Christoph Ortner, Gábor Csányi, and Michele Ceriotti. Incompleteness of atomic structure representations. Phys. Rev. Lett. , 125:166001, Oct 2020.

Boris Weisfeiler and A.A. Leman. The reduction of a graph to canonical form and the algebra which appears therein, 1968.

Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? In ICLR , 2019.

Bohang Zhang, Shengjie Luo, Liwei Wang, and Di He. Rethinking the expressive power of gnns via graph biconnectivity. arXiv preprint arXiv:2301.09505 , 2023.

A Appendix

A.1 Proofs for Initialization

Here we provide the proof of Proposition 4.2. We will need the following extension of the Barycenter Lemma.

Lemma A.1. Let b be the barycenter of S ⊂ R d , d > 2 . For any x = ( x 1 , . . . , x d -1 ) ∈ S d -1 , knowing χ (1) d -1 ,S ( x ) and the multiset { { χ (1) d -1 ,S ( y ) | y ∈ S d -1 } } , one can determine the tuple of distances ( d ( x 1 , b ) , . . . , d ( x d -1 , b )) .

Proof. Let D x be as in Lemma 3.1. We claim that we can determine ( D x 1 , . . . , D x d -1 ) and { { D y | y ∈ S } } from the information given in the lemma statement. By Lemma 3.1, this allows to determine ( d ( x 1 , b ) , . . . , d ( x d -1 , b )) .

We have

By definition, from χ (0) d -1 ,S ( x [ y/ 1]) one can determine the tuple of distances ( d ( x 2 , y ) , . . . , d ( x d -1 , y )) . Hence, from the multiset { { χ (0) d -1 ,S ( x [ y/ 1]) | y ∈ S } } one can determine ( D x 2 , . . . , D x d -1 ) . In turn, D x 1 can be determined from, say, { { χ (0) d -1 ,S ( x [ y/ 2]) | y ∈ S } } . We have just shown that χ (1) d -1 ,S ( x ) uniquely determines D x 1 . Hence, from the multiset { { χ (1) d -1 ,S ( y ) | y ∈ S d -1 } } , one can compute the multiset { { D y 1 | y = ( y 1 , . . . , y d -1 ) ∈ S d -1 } } . But this multiset coincides with the multiset { { D y | y ∈ S } } except that all multiplicities are | S | d -2 times larger.

Now, we show that after the second iteration, we can restore the distance profile of ( b, x 1 , . . . , x d -1 ) for all ( x 1 , . . . , x d -1 ) ∈ S d -1 .

Lemma A.2. For any x = ( x 1 , . . . , x d -1 ) ∈ S d -1 , knowing χ (2) d -1 ,S ( x ) and the multiset { { χ (1) d -1 ,S ( y ) | y ∈ S d -1 } } , one can determine the distance profile of ( b, x 1 , . . . , x d -1 ) w.r.t. S .

Proof. Since d > 2 , we have

From the tuple ( χ (0) d -1 ,S ( x [ y/ 1]) , . . . , χ (0) d -1 ,S ( x [ y/k ]) ) we can restore the tuple of distances

In turn, by Lemma A.1, from χ (1) d -1 ,S ( x [ y/ 1]) and { { χ (1) d -1 ,S ( y ) | y ∈ S d -1 } } , we can restore d ( y, b ) . Hence, we can restore the whole distance profile of ( b, x 1 , . . . , x d -1 ) .

Finally, we show that knowing χ (3) d -1 ,S ( x ) for x = ( x 1 , . . . , x d -1 ) and the multiset { { χ (1) d -1 ,S ( y ) | y ∈ S d -1 } } , we can compute

Since d > 2 , we have

For every y ∈ S , knowing χ (2) d -1 ,S ( x ) and ( χ (2) d -1 ,S ( x [ y/ 1]) , . . . , χ (2) d -1 ,S ( x [ y/k ]) ) , we have to compute EP ( x 1 , . . . , x d -1 , y ) , that is, the distance matrix of ( b, x 1 , . . . , x d -1 , y ) and the distance profiles of the tuples

Distance profiles can be computed by Lemma A.2. The distances amongst elements of { x 1 , . . . , x d -1 , y } can be computed by definition from χ (0) d -1 ,S ( x ) and ( χ (0) d -1 ,S ( x [ y/ 1]) , . . . , χ (0) d -1 ,S ( x [ y/k ]) ) . Distances to b from these points can be computed by Lemma A.1 from χ (1) d -1 ,S ( x ) and ( χ (1) d -1 ,S ( x [ y/ 1]) , . . . , χ (1) d -1 ,S ( x [ y/k ]) ) .

Now that we have the enhanced profiles, we have to select one for which ( x 1 , . . . , x d ) satisfies the cone condition. We first observe that, knowing EP ( x 1 , . . . , x d ) , we can reconstruct ( b, x 1 , . . . , x d ) up to an isometry by Lemma 4.3 (because inside EP ( x 1 , . . . , x d ) we are given the distance matrix of ( b, x 1 , . . . , x d ) ). This means that from EP ( x 1 , . . . , x d ) we can compute any function of b, x 1 , . . . , x d which is invariant under isometries. In particular, we can compute AffineDim ( b, x 1 , . . . , x d ) . We will refer to AffineDim ( b, x 1 , . . . , x d ) as the dimension of the corresponding enhanced profile.

We show that AffineDim ( S ) is equal to the maximal dimension of an enhanced profile. Indeed, first notice that AffineDim ( S ) = AffineDim ( { b } ∪ S ) because b is a convex combination of points of S . In turn, AffineDim ( { b } ∪ S ) is equal to the maximal k for which one can choose k points x 1 , . . . , x k ∈ S such that x 1 -b, . . . , x k -b are linearly independent. Obviously, k is bounded by the dimension of the space. Hence, there will be an enhanced profile with the same maximal dimension k .

If AffineDim ( S ) < d , any enhanced profile with maximal dimension satisfies the initialization requirement, and we are done. Assume therefore that AffineDim ( S ) = d . Our task is to output some EP ( x 1 , . . . , x d ) such that

AffineDim ( b, x 1 , . . . , x d ) = d , and

there is no x ∈ S such that x -b belongs to the interior of Cone ( x 1 -b, . . . , x d -b ) .

For that, among all d -dimensional enhanced profiles, we output one which minimizes the solid angle at the origin, defined as

(the solid angle is invariant under isometries, and hence can be computed from EP ( x 1 , . . . , x d ) ).

We have to show that for all x ∈ S we have that x -b lies outside the interior of Cone ( x 1 -b, . . . , x d -b ) . To prove this, we need an extra lemma. We will say that a cone C is simple if C = Cone ( u 1 , . . . , u d ) , for some linearly independent u 1 , . . . , u d . Observe that if C = Cone ( u 1 , . . . , u d ) is a simple cone, then the interior of C is the set

Note also that the boundary of C consists of d faces

Lemma A.3. Let C = Cone ( u ) ⊆ R d for u = ( u 1 , . . . , u d ) be a simple cone and let y belong to the interior of C . Then Cone ( u [ y/ 1]) is a simple cone and Angle ( u [ y/ 1]) < Angle ( u ) .

Proof. Since y belongs to the interior of Cone ( u ) , we have that

for some λ 1 > 0 , . . . , λ d > 0 . The fact that λ 1 > 0 implies that y, u 2 , . . . , u d are linearly independent, and hence Cone ( u [ y/ 1]) is a simple cone. Since y ∈ Cone ( u ) , we have that Cone ( u [ y/ 1]) ⊆ Cone ( u ) . Thus, to show that Angle ( u [ y/ 1]) < Angle ( u ) , it suffices to show that the volume of

is positive. We claim that for any point x = µ 1 u 1 + . . . + µ d u d ∈ Cone ( u [ y/ 1]) we have µ 1 > 0 = ⇒ µ 2 /µ 1 ≥ λ 2 /λ 1 . This is because x can be written as a non-negative linear combination of y, u 2 , . . . , u d . Since µ 1 > 0 , the coefficient in front of y in this linear combination must be positive. Now, if the coefficient in front of u 2 is 0 , then the ratio between µ 2 and µ 1 is exactly as the ratio between λ 2 and λ 1 , and if the coefficient before u 2 is positive, µ 2 can only increase.

This means that no point of the form

belongs to Cone ( u [ y/ 1]) . It remains to show that the set of points that satisfy (4) and lie in { x ∈ Cone ( u ) : ∥ x ∥ ≤ 1 } has positive volume.

Indeed, for any ε > 0 and δ > 0 , consider a d -dimensional parallelepiped:

Regardless of ε and δ , we have that P is a subset of Cone ( u ) and its volume is positive. For all sufficiently small ε, δ , we have that P is a subset of the unit ball { x ∈ R d | ∥ x ∥ ≤ 1 } . In turn, by choosing ε to be sufficiently big compared to δ , we ensure that all points of P satisfy (4).

Now we can finish the proof of Proposition 4.2. Let x ∈ S d minimize Angle ( x 1 -b, . . . , x d -b ) amongst x ∈ S d such that AffineDim ( b, x 1 , . . . , x d ) = d . Assume that x does not satisfy the cone condition, and let x ∈ S be such that x -b ∈ Int ( Cone ( x 1 -b, . . . , x d -b )) holds. Then, by Lemma A.3, Angle ( x -b, x 2 -b . . . , x d -b ) is strictly smaller than Angle ( x 1 -b, x 2 -b . . . , x d -b ) . As Cone ( x -b, x 2 -b . . . , x d -b ) is a simple cone, x -b, x 2 -b, . . . , x d -b are linearly independent, and thus AffineDim ( b, x, x 2 , . . . , x d ) = d , contradicting the minimality hypothesis on x , as desired.

A.2 Proofs for Reconstruction

Proof of Lemma 4.4. We claim that for every i = 2 , . . . , m we have ⟨ a -x 1 , x i -x 1 ⟩ = ⟨ b -x 1 , x i -x 1 ⟩ . This is because a -x 1 and b -x 1 have the same distance to x i -x 1 (which is equal to d ( a, x i ) = d ( b, x i ) ) and, moreover, the norms of a -x 1 and b -x 1 coincide (and are equal to d ( a, x 1 ) = d ( b, x 1 ) ). Hence, both a -x 1 and b -x 1 are solutions to the following linear system of equations:

This system has at most 1 solution over x ∈ LinearSpan ( x 2 -x 1 , . . . , x m -x 1 ) . Moreover, a -x 1 and b -x 1 are both from LinearSpan ( x 2 -x 1 , . . . , x m -x 1 ) because a, b ∈ AffineSpan ( x 1 , . . . , x m ) . Hence, a -x 1 = b -x 1 , and a = b .

Proof of Proposition 4.5. Let P i = AffineSpan ( z 1 , . . . , z i -1 , 0 , z i +1 , . . . , z d ) . As the following lemma shows, knowing the distances from s ∈ T to z 1 , . . . , z i -1 , 0 , z i +1 , z d , we can determine the position of s uniquely up to the reflection through P i .

Lemma A.4 (The Symmetric Lemma) . Let x 1 , . . . , x m ∈ R d be such that AffineSpan ( x 1 , . . . , x m ) has dimension d -1 . Assume that a, b ∈ R d are such that d ( a, x i ) = d ( b, x i ) for all i = 1 , . . . , m . Then either a = b or a and b are symmetric w.r.t. AffineSpan ( x 1 , . . . , x m ) .

Proof. As in the proof of Lemma 4.4, we have that ⟨ a -x 1 , x i -x 1 ⟩ = ⟨ b -x 1 , x i -x 1 ⟩ for every i = 2 , . . . , m . Consider orthogonal projections of a -x 1 and b -x 1 to LinearSpan ( x 2 -x 1 , . . . , x m -x 1 ) . Both these projections are solutions to the system

This system has at most one solution over x ∈ LinearSpan ( x 2 -x 1 , . . . , x m -x 1 ) . Hence, projections of a -x 1 and b -x 1 coincide. We also have that ∥ a -x 1 ∥ = ∥ b -x 1 ∥ , which implies that a -x 1 and b -x 1 have the same distance to LinearSpan ( x 2 -x 1 , . . . , x m -x 1 ) . Since the dimension of LinearSpan ( x 2 -x 1 , . . . , x m -x 1 ) is d -1 , we get that either a -x 1 = b -x 1 or they can be obtained from each other by the reflection through LinearSpan ( x 2 -x 1 , . . . , x m -x 1 ) . After translating everything by x 1 , we obtain the claim of the lemma.

In fact, if s belongs to P i , then there is just one possibility for s . Thus, we can restore all the points in T that belong to the union ⋃ d i =1 P i . Let us remove these points from T and update distance profiles by deleting the tuples of distances that correspond to the points that we have removed.

From now on, we may assume that T is disjoint from ⋃ d i =1 P i . Hence, T is also disjoint from the boundary of C = Cone ( x 1 , . . . , x d ) , not only from its interior (every face of this cone lies on P i for some i ).

For x ∈ R d , we define ρ ( x ) = min i =1 ,...,d dist ( x, P i ) . Since T is disjoint from ⋃ d i =1 P i , we have that ρ ( s ) > 0 for every s ∈ T . Moreover, from, say, the distance profile of (0 , z 2 , . . . , z d ) , we can compute some ε > 0 such that ρ ( s ) ≥ ε for all s ∈ T . Indeed, recall that from the distance profile of (0 , z 2 , . . . , z d ) , we get at most 2 potential positions for each point of T . This gives us a finite set T ′ (at most 2 times larger than T ) which is a superset of T . Moreover, as T is disjoint from ⋃ d i =1 P i , we have that T ′ \ ⋃ d i =1 P i ⊇ T Thus, we can define ε as the minimum of ρ ( x ) over T ′ \ ⋃ d i =1 P i ⊇ T .

We conclude that T is disjoint from

(moreover, the set A 0 is known to us).

Our reconstruction procedure starts as follows. We go through all distance profiles, and through all tuples of distances in them. Each tuple gives 2 candidates for a point in T (that can be obtained from each other by the reflection through P i ). If one of the candidates lies in A 0 , we know that we should take the other candidate. In this way, we may possibly uniquely determine some points in T . If so, we remove them from T and update our distance profiles.

Which points of T will be found in this way? Those that, for some i , fall into A 0 under the reflection through P i . Indeed, these are precisely the points that give 2 candidates (when we go through the i th distance profile) one of which is in A 0 . In other words, we will determine all the points that lie in ⋃ d i =1 Refl i ( A 0 ) , where Refl i denotes the reflection through P i . After we remove these points, we know that the remaining T is disjoint from A 1 = A 0 ∪ ⋃ d i =1 Refl i ( A 0 ) .

We then continue in exactly the same way, but with A 1 instead of A 0 , and then with A 2 = A 1 ∪ ⋃ d i =1 Refl i ( A 1 ) , and so on. It remains to show that all the points of T will be recovered in this way. In other words, we have to argue that each point of T belongs to some A i , where

We will show this not only for points in T but for all points in R d . Equivalently, we have to show that for every x ∈ R d there exists a finite sequence of reflections τ 1 , . . . , τ k ∈ { Refl 1 , . . . , Refl d } which brings x inside A 0 , that is, τ k ◦ . . . ◦ τ 1 ( x ) ∈ A 0 .

We construct this sequence of reflections as follows. Let x be outside A 0 . In particular, x is outside the cone C = Cone ( z 1 , . . . , z d ) . Then there exists a face of this cone such that C is from one side of this face and x is from the other side. Assume that this face belongs to the hyperplane P i . We then reflect x through P i , and repeat this operation until we get inside A 0 . We next show that the above process stops within a finite number of steps. For that, we introduce the quantity γ ( x ) = ⟨ x, z 1 ⟩ + . . . + ⟨ x, z d ⟩ . We claim that with each step, γ ( x ) increases by at least c · ε , where

Note that c > 0 because, for every i = 1 , . . . , d , we have that z i / ∈ P i , by the linear independency of z 1 , . . . , z d . Also observe that γ ( x ) ≤ | x | ∑ i | z i | and reflections across the subspaces P i do not change | x | . Hence, γ ( x ) cannot increase infinitely many times by some fixed positive amount.

It remains to show that γ ( x ) increases by at least c · ε at each step, as claimed. Note that reflection of x across some P i does not change the scalar product of x with those vectors among z 1 , . . . , z d that belong to P i . The only scalar product that changes is ⟨ x, z i ⟩ , and the only direction which contributes to the change is the one orthogonal to P i . Before the reflection, the contribution of this direction to the scalar product was -d ( x, P i ) · d ( z i , P i ) (remember that x and z i were from different sides of P i because z i ∈ C ). After the reflection, the contribution is the same, but with a positive sign. Thus, the scalar product increases by 2 d ( x, P i ) · d ( z i , P i ) . Now, we have d ( z i , P i ) ≥ c/ 2 by definition of c and d ( x, P i ) ≥ ε if x is not yet in A 0 .

B Proofs for Theorem 5.1

Proof of Theorem 5.1. From χ (1) d,S ( s ) , we can determine the tuple s = ( s 1 , . . . , s d ) up to an isometry, since χ (1) d,S ( s ) gives us χ (0) d,S ( s ) , which is the distance matrix of s . In order to determine S , we consider two cases:

Case 1 : For all s ∈ S d it holds AffineDim ( s ) < d -1 . Then take χ (1) d,S ( s ) with maximal AffineDim ( s ) , and fix locations of points from s compatible with the distance specification, according to Lemma 4.3. All points of S belong to AffineSpan ( s ) , otherwise we could increase AffineDim ( s ) . Indeed, since AffineDim ( s ) < d -1 , we could throw away one of the points from the tuple without decreasing the dimension and replace it with a point outside AffineSpan ( s ) . We now can reconstruct the rest of S uniquely up to an isometry. Indeed, in χ (1) d,S ( s ) we are given the multiset of d -tuples of distances to s = ( s 1 , . . . , s d ) from the points of S , and it remains to use Lemma 4.4.

Case 2 : There are tuples with AffineSpan ( s ) = d -1 . We first observe that from the multiset { { χ (0) d,S ( s ) | s ∈ S d } } we can compute the pairwise sum of distances between the points in S , i.e.,

Indeed, from χ (0) d,S (( s 1 , . . . , s d )) , we determine d ( s 1 , s 2 ) . Hence, we can compute the sum:

In our reconstruction of S , we go through all χ (1) d,S ( s ) with AffineDim ( s ) = d -1 . For each of them, we fix positions of the points of the tuple s in any way that agrees with the distance matrix of this tuple. As before, χ (1) d,S ( s ) gives us the multiset of d -tuples of distances to s from the points of S . We call 'candidates for S given s " the set of point clouds S ′ which have one point associated with each such d -tuple of distances, and realizing these distances to points in s . We aim to find s for which, exactly one of these candidates can be isometric to S . We start with the following lemma:

Lemma B.1. For any finite set S ⊆ R d with AffineDim ( S ) ≥ d -1 there exist x 1 , . . . , x d ∈ S with AffineDim ( x 1 , . . . , x d ) = d -1 such that all points of S belong to the same half-space with respect to the hyperplane AffineSpan ( x 1 , . . . , x d ) .

Proof. The general idea of the proof is the following. If AffineDim ( S ) = d -1 then the extreme points of the convex hull of S contain an affinely independent set of cardinality d , which then gives the desired s . The half-space condition in the lemma is automatically verified in this case. If AffineDim ( S ) = d then to find s we can proceed by moving a ( d -1) -plane from infinity towards S until it touches S in at least one point, then iteratively we rotate the plane around the subspace containing the already touched points of S , until a new point in S prohibits to continue that rotation. We stop within at most d iterations, when no further rotation is allowed, in which case the plane has an affinely independent subset in common with S .

Formally, we need to find a hyperplane H such that, first, all points of S belong to the same half-space w.r.t. H , and second, AffineDim ( H ∩ S ) = d -1 .

̸

To start, we need to find a hyperplane H such that, first, all points of S belong to the same halfspace w.r.t. H , and second, H ∩ S = ∅ . For instance, take any non-zero vector α ∈ R d , consider m = max x ∈ S ⟨ α, x ⟩ and define H by the equation ⟨ α, x ⟩ = m . Now, take any x 1 ∈ H ∩ S . After translating S by -x 1 , we may assume that x 1 = 0 .

Now, among all hyperplanes H that contain x 1 = 0 and satisfy the condition that all points of S lie in the same half-space w.r.t. H , we take one that contains most points of S . We claim that AffineDim ( H ∩ S ) = d -1 for this H . Assume for contradiction that AffineDim ( H ∩ S ) < d -1 . Define U = AffineSpan ( H ∩ S ) . Since H ∩ S contains x 1 = 0 , we have that U ⊆ H is a linear subspace, and its dimension is less than d -1 . Hence, since AffineDim ( S ) ≥ d -1 , there exists x 2 ∈ S \ U . Note that x 2 / ∈ H because otherwise x 2 belongs to H ∩ S ⊆ U .

Let α be the normal vector to H . Since all points of S lie in the same half-space w.r.t. H , w.l.o.g. we may assume that ⟨ α, s ⟩ ≥ 0 for all s ∈ S . In particular, ⟨ α, x 2 ⟩ > 0 because x 2 / ∈ H .

Let U ⊥ denote the orthogonal complement to U . Since α is the normal vector to H ⊇ U , we have that α ∈ U ⊥ . We need to find some β ∈ U ⊥ which is not a multiple of α but satisfies ⟨ β, x 2 ⟩ > 0 . Indeed, the dimension of U is at most d -2 , and hence the dimension of U ⊥ is at least 2. Now, since ⟨ α, x 2 ⟩ > 0 , we can take any β ∈ U ⊥ which is sufficiently close to α .

For any λ ≥ 0 , let H λ be the hyperplane, defined by ⟨ α -λβ, x ⟩ = 0 (this is a hyperplane and not the whole space because β is not a multiple of α ). We claim that for some λ > 0 , we have that H λ has more points of S than H while still all points of S lie in the same half-space w.r.t. H λ . This would be a contradiction.

Indeed, define S β = { s ∈ S | ⟨ s, β ⟩ > 0 } . Note that S β , by definition of β , contains x 2 and hence is non-empty. Moreover, S β is disjoint from H ∩ S . This is because H ∩ S ⊆ U and β ∈ U ⊥ .

Define

First, H λ ⊇ U ⊇ H ∩ S because α -λβ ∈ U ⊥ . Moreover, H λ contains at least one point of S which is not in H . Namely, it H λ contains any s ∈ S β , establishing the minimum in the definition of λ (and recall that S β is disjoint from H ∩ S ). Indeed, for this s we have λ = ⟨ α,s ⟩ ⟨ β,s ⟩ . Hence, ⟨ α, s ⟩ -λ ⟨ β, s ⟩ = 0 = ⟨ α -λb, s ⟩ = ⇒ s ∈ H λ .

It remains to show that all points of S lie in the same half-space w.r.t. H λ . More specifically, we will show that ⟨ α -λβ, s ⟩ ≥ 0 for all s ∈ S . First, assume that ⟨ s, β ⟩ = 0 . Then ⟨ α -λβ, s ⟩ = ⟨ α, s ⟩ ≥ 0 because all points of S lie in the 'non-negative' half-space w.r.t. α . Second, assume that ⟨ s, β ⟩ > 0 . Then s ∈ S β . Hence, by definition of λ , we have λ ≤ ⟨ α,s ⟩ ⟨ β,s ⟩ . This means that ⟨ α -λβ, s ⟩ = ⟨ α, s ⟩ -λ ⟨ β, s ⟩ ≥ 0 , as required.

Next, consider the following simple geometric observation:

Lemma B.2. Let P ∈ R d be a hyperplane and consider two points a, b ∈ R d \ P that lie in same half-space w.r.t. P . Let a ′ , b ′ be the reflections of a, b through P . Then d ( a ′ , b ′ ) = d ( a, b ) < d ( a, b ′ ) .

Proof. It suffices to restrict to the plane AffineSpan ( { a, b, a ′ } ) , and thus we take d = 2 and up to isometry we may fix P to be the x -axis, a = (0 , y ) , b = ( x, y ′ ) , b ′ = ( x, -y ′ ) , with y, y ′ > 0 . Then it follows that d ( a, b ) 2 = x 2 +( y -y ′ ) 2 < x 2 +( y + y ′ ) 2 = d ( a, b ′ ) 2 . As reflections are isometries, d ( a, b ) = d ( a ′ , b ′ ) .

Claim: If s is as in Lemma B.1, then we have the following:

Exactly one of the candidates for S given s , up to reflection across AffineSpan ( s ) , is completely contained in one of the half-spaces determined by AffineSpan ( s ) .

A candidate S ′ as in the previous point is the only one of the candidates for S given s , up to reflection across AffineSpan ( s ) , for which D S ′ = D S .

To prove the first item, we use only the property that AffineDim ( s ) = d -1 , with which by Lemma A.4, each point of a candidate for S given s , has either two possible locations (related by a reflection across AffineSpan ( s ) ) or a single possible location if it belongs to AffineSpan ( s ) . For the second item, let S ′ be as above and let S ′′ be a candidate for S given s , which is not completely contained in one of the halfspaces determined by AffineSpan ( s ) . We now consider each term d ( x ′ , y ′ ) in the sum defining D S ′ , comparing the corresponding term d ( x ′′ , y ′′ ) from D S ′′ , where x ′′ = x ′ or is a reflection across s of x ′ and similarly for y ′′ and y ′ . By Lemma B.2, either d ( x ′′ , y ′′ ) = d ( x ′ , y ′ ) in case x ′′ , y ′′ are in the same half-space determined by AffineSpan ( s ) , or d ( x ′′ , y ′′ ) > d ( x ′ , y ′ ) otherwise. Summing all terms, by the property of S ′ , S ′′ we find D S ′ < D S ′′ . By the same reasoning with S instead of S ′′ , since we are assuming that s satisfies Lemma B.2, we have D S = D S ′ , completing the proof of the second item and of the claim.

The reconstruction of S in Case 2, can therefore be done as follows: we run through all s such that AffineDim ( s ) = d -1 , and for each such s we construct all candidates ˜ S for S given s , and calculate D ˜ S for each of them. Lemma B.1 guarantees that we run into some s for which, up to isometry, only one such ˜ S realizes D ˜ S = D S . This is our unique reconstruction of S .

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies the completeness of $l$-WL test for Euclidean point sets. It shows an algorithm that certifies two Euclidean point sets of dimension $d$ are isometric using $(d-1)$-WL test, where only three iterations suffice. The results extend to $d$-WL test which only requires one iteration.\n\nThe punchline is to study \"how pair-wise distances determine the identity of all points\", which is well-developed for Euclidean scenarios. Taking the plane as an example, the proposed method shows that storing single-source all-pair distances of 2 points and the norm of the rest points suffice to uniquely reconstruct the whole point set. Such data can be obtained through several rounds of WL-test, which completes the claim that WL-test is able to distinguish the isometry of Euclidean point sets.\n\nSTRENGTHS:\n- The problem is well-motivated, most of the paper is written clearly.\n- The results characterize well on three parameters involved for isometry-test of Euclidean point sets: the data dimension $d$, the WL-test dimension $l$ and number of rounds needed ($r=3$ for $l=d-1$, $r=1$ for $l=d$). It should be an interesting result to be announced.\n\nWEAKNESSES:\nI think overall the paper is not hard to follow, but the presentation can be better\n- The contents on page 4 and 5 would be a lot easier to understand if there is a picture. \n- The algorithms deserve a box highlighting each step either in description or pseudocode, though I personally prefer just words. \n\nAnother issue is that it seems knowing the distance profiles is sufficient for the isometry test. Then why do we have to converge to WL test? The time and space consumption of WL-test is non-trivial. Since we have the two point sets at our hand, just consider a simple algorithm calculating the profile tuple $(A, M_1, M_2, ...)$ and use the proposed reconstruction algorithm, what is wrong with this? I believe the space might be the same, but the time would be much better.\n\nminor: line 268, $d$-tuple?.\n\nAnother suggestion on Table 1.  I think using $l>d$ WL test is no longer of interest, maybe you can replace the dots with a slash. You can consider different colors for previous work, this work, and open. Last, there could be general $d$.\n\nQUESTIONS:\nSee above.\n\nLIMITATIONS:\nI would say the result itself is interesting, only that as a theory paper the technical novelty is quite limited, especially if this can be done even without WL-test. The connection to GNN is in a phase of \"Good to know\" but does not seem to have application implications.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors rigorously prove that applying the d-1 WL to the distance matrices of point clouds in d dimensions is complete: that is, two point clouds are related by an isometry (and relabeling of the points) if and only if they are not separated by the d-1 WL test\n\nSTRENGTHS:\nThe questions the authors address is a fundamental theoretical question in the study of geometric machine learning. They give a rigorous and non-trivial proof that essentially solves the question. Very good work.\n\nWEAKNESSES:\nNo significant weaknesses.\n\nQUESTIONS:\nIn lines 43-49 in the discussion of Hordan et al.: The authors of that paper discuss not only a 3-WL algorithm but also what the call in the abstract `a 2-WL-like algorithm', when d=3. To avoid confusion it may be helpful  to address this claim in the discussion somehow. \n\nI had some issues with Section 6: \n(a) Firstly, defining MPGNNs for point clouds has been discussed in various ways in previous works: in  [9]-[10] cited in the paper and in \"On the expressive power of geometric graph neural networks\" by Joshi et al which should be cited as well. It would make sense to discuss the relationships to the definitions there or at least mention they exist\n(b) In the described MPGNN each x is given a `one hot encoding'. Since there are infinite possible x, how exactly is this accomplished?\n(c) I was confused at why Corollary 6.1 was stated for (d-1)-MPGNN instead of d-MPGNN. Later I saw that this is explained in lines 75-80 and is due to the differences between WL and Folklore-WL but I think it should be retierated in Corollary 6.1 \n(d) Generally, I feel like this paper does a great job re the WL tests themselves, but the discussion of MPNNs is somewhat short and non-convincing, and perhaps it would be better just to ref to the other papers mentioned above which discussed these issues in more depth (at the authors discretion, I support acceptance either way)\n\nI think the proof of Lemma 3.1 can be shortened and simplified. Once the equation in line 211 is established, you can immediately show that plugging it into the right hand side of (2), for both f(x) and f(y), gives the equality you want.\n\nWith the space the freed up, you could consider adding an illustration for the cone condition (lines 137-138)\n\nTiny comments:\nLine 129: `Algorithm' should not be capitalized\nLine 137: **line** on this line\nLine 142: Initialization Data: instead of `it consists..' I would prefer something like `the initialization data consists..'\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe expressive power of GNNs has long been a central topic in the GNN community. WL test serves as a fundamental algorithm in guiding designing expressive GNNs. For each k-WL, it has been shown that there always exist non-isomorphic graphs that cannot be distinguished by k-WL, and thus k-WL is incomplete for any k. However, things become different for geometric graphs, which are point clouds lying on a finite d-dimensional Euclidean space and the isomorphism is characterized by isometry. This paper makes a significant contribution by proving that k-WL is complete for distinguishing (k+1)-dimensional geometric graphs. Moreover, a constructive method shows that three-iteration suffices, which contrasts to a well-known result that there exist non-isomorphic graphs that cannot be distinguished by standard k-WL within o(n) iterations where n is the number of nodes. The authors further proved that k-WL can distinguish k-dimensional geometric graphs in only one iteration.\n\nSTRENGTHS:\n1. **Fundamental problem**. I believe characterizing the upper and lower bound of k such that k-WL is complete for distinguishing d-dimensional geometric graphs is a fundamental problem. This is due to a series of reasons.\n   - First, k-WL is a very elegant algorithm and applies to geometric graphs straightforwardly.\n   - Second, there have been debates whether equivariant architectures that use coordinate-wise information is necessary for learning geometric graphs, or purely distance information suffices. This paper answers this question timely and affirmatively.\nOverall, the problem formulation is very *clean* and are likely to have decent impact in the geometric deep learning community.\n\n2. **Strong theoretical result**. After going through the proof technique, I feel that the proof is non-trivial, despite presentation is very clear, well-organized, and rigorous. After checking several of the proofs, I am confident that the proofs are correct.\n   - Related to prior works: to my knowledge, the theoretical result seems to be new. While I mainly focus on standard GNN expressivity theory, I have read several works related to this paper, such as Pozdnyakov & Ceriotti et al., Hordan et al., and Zhang et al..\n   - Regarding completeness of theoretical result: this paper only gives upper bounds on the required k for distinguishing d dimensional geometric graphs. However, the bound is tight for 2 or 3 dimensional data. Other cases are highlighted in the limitation part and left as open problems. In this sense, the contribution seems to be sufficient for an acceptance. \n\n3. **Great presentation**. This paper is very well-written and easy to read. The organization is great, the proof sketch is carefully written, and the proof in the Appendix is also well-written (I couldn't even find a typo). I really enjoy reading this paper.\n\nWEAKNESSES:\n1. This is mainly a theoretical paper, without any experimental evaluation. However, I understand that showing experimental result is not very necessary given this paper focuses on a fundamental theoretical problem.\n2. Regarding theoretical results, I have several concerns and questions. While all of them are not major weaknesses, I would like the authors to answer these questions and revise the manuscript accordingly in the camera-ready version.\n   - The authors focus on the setting where all points are distinct, i.e., S is a set rather than multiset. The proof in Line 146-157 requires such a condition. But I think similar result should hold when S is a multiset. The authors may add a brief remark to illustrate this point.\n   - The authors wrote that Hordan et al. have proved that in the same setting, the geometric 3-WL test is in fact complete. However, I cannot find this result in their paper. Instead, they consider a different algorithm that requires the coordinates as input (although the output is invariant under isometric transformation). Moreover, they also considered higher dimensions. So could the authors give an explanation for the paragraph in Line 43-49 (if I miss something)?\n   \n3. The number of related work seems to be a bit insufficient. I can list several works which I think is relevant to this paper.\n   - Martin Furer. Weisfeiler-Lehman Refinement Requires at Least a Linear Number of Iterations.\n     \n     This paper proved that, for any k, there exist non-isomorphic graphs that cannot be distinguished by standard k-WL within o(n) iterations where n is the number of nodes and can be distinguished by standard k-WL within $\\Theta(n)$ iterations. Moreover, the same pair of graphs can be distinguished within much fewer iterations by increasing k. These results surprisingly parallels your results.\n   - Bohang Zhang, et al.. Rethinking the Expressive Power of GNNs via Graph Biconnectivity.\n\n     This paper proposed the expressive power of generalized distance WL test, which is basically the same as 1-WL in this paper while using different distance metrics.\n\n4. Minor issue: in Line 54, the citation [12] seems to be wrong. Do you mean [13]?\n\nQUESTIONS:\nSee the weakness part above.\n\nLIMITATIONS:\nThe limitation has been clearly stated in the paper.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-07"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper addresses the question of testing the existence of a one-to-one application between two point clouds such that the distances are preserved. This information can be used, for example, to build better structured graph neural network architectures to optimize their performance. The starting point is that the distances between elements of a point cloud can be used to label the edges of a graph connecting them. Hence, the problem boils down to the detection of isometry between two graphs, for which the Weisfeiler-Lehman test is the classic tool. However, the latter makes it possible to conclude if two clouds are not isometric, but not necessarily that they are, depending on the computation cost. This paper sheds light on the issue in terms of the size of the ambient space\n\nSTRENGTHS:\nThe document is fairly well written and the contributions are clearly stated. Since the results are theoretical in nature, they essentially consist of a succession of proofs. But they seem to be rigorously executed.\nNot being a specialist in the field, I can't really appreciate the impact of such a theoretical result. This brings me to my next comment on the weaknesses.\n\nWEAKNESSES:\nThe stated result, i.e. the ability to answer affirmatively if isometry between point clouds is proven from the WL test, is in itself very interesting. It squarely fits into \"understanding ML results\".\nHowever, the authors stressed the importance of their results in improving the design of neural net architecture. This is unfortunately not clarified and it should be nice if the authors could comment more on that.\n\nQUESTIONS:\nThe proofs are sometimes very verbose and would benefit from geometric illustration rather than prosaic text. The proof of lemma 3.1 (which seems connected to Konig-Huygens theorem) can easily go in the appendix to save space. As it is, I find it difficult to follow the entire logic of the proofs beyond a line-by-line follow-up.\n\nThe corollary 6.1 seems constructive. It woulds be helpful to describe the algorithm in a pseudo-code and maybe illustrate the proposed MPGNN. \n\nI believe this will make the paper more accessible to a wider audience. \nSince I don't have the necessary knowledge, I can hardly judge the impact and practical importance of such a result.\n\nLIMITATIONS:\nNA", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 1, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-07"}