On the Completeness of Invariant Geometric Deep Learning Models
Authors: Zian Li
Xiyuan Wang
Shijia Kang
Muhan Zhang
Venue: ICLR 2025 Poster (2025)
Keywords: geometric deep learning
invariant models
completeness
expressiveness
graph neural network
subgraph graph neural network
Primary area: learning on graphs and other geometries & topologies

Abstract: Invariant models, one important class of geometric deep learning models, are capable of generating meaningful geometric representations by leveraging informative geometric features in point clouds. These models are characterized by their simplicity, good experimental results and computational efficiency. However, their theoretical expressive power still remains unclear, restricting a deeper understanding of the potential of such models. In this work, we concentrate on characterizing the theoretical expressiveness of a wide range of invariant models under *fully-connected* conditions. We first rigorously characterize the expressiveness of the most classic invariant model, message-passing neural networks incorporating distance (DisGNN), restricting its unidentifiable cases to be only highly symmetric point clouds. We then prove that GeoNGNN, the geometric counterpart of one of the simplest subgraph graph neural networks, can effectively break these corner cases' symmetry and thus achieve E(3)-completeness. By leveraging GeoNGNN as a theoretical tool, we further prove that: 1) most subgraph GNNs developed in traditional graph learning can be seamlessly extended to geometric scenarios with E(3)-completeness; 2) DimeNet, GemNet and SphereNet, three well-established invariant models, are also all capable of achieving E(3)-completeness. Our theoretical results fill the gap in the expressive power of invariant models, contributing to a rigorous and comprehensive understanding of their capabilities.

Published as a conference paper at ICLR 2025
ON THE COMPLETENESS OF INVARIANT GEOMETRIC DEEP LEARNING MODELS
Zian Li! Xiyuan Wang S Kang', Muhan Zhang!* Institute for Artificial Intelligence, Peking University Schooo of Intelligence Science and Technology, Peking University
ABSTRACT
Invariant models, one important class of geometric deep learning models, are capable of generating meaningful geometric representations by leveraging infor- mative geometric features in point clouds. These models are characterized by their simplicity. good experimental results and computational efficiency. How- ever, their theoretical expressive power still remains unclear, restricting deeper understanding of the potential of such models. In this work, we concentrate on char- acterizing the theoretical expressiveness of a wide range of invariant models under fully-connected conditions. We first rigorously characterize the expressiveness of the most classic invariant model, message- pasaagggggaiingggggg neural networks incorporating distance (DisGNN), restricting its unidentifiable cases to be only highly symmetric point clouds. We then prove that GeoNGNN, the geometric counterpart of one of the simplest subgraph graph neural networks, can effectively break these corner cases' symmetry and thus achieve E(3) cc3plletpssssssss By leveraging GeoNGNN theoretical tool, we further prove that: 1) most subgraph GNNs developed in traditional graph learning can be seamlessly extended to geometric scenarios with compleeeees,, DimeNet, GemNet and SphereNet, three well-established invariant models, are also all capable of achieving compleettesss Our theoret- ical results fill the gap in the expressive power of invariant models, contributing to a rigorous and comprehensive understanding of their capabilities.
INTRODUCTION
Learning geometric structural information from 3D point clouds constitutes a fundamental require- ment for various real-world applications, including molecular property prediction, physical simulation. and point cloud Dassification/segmnnatiion (Schmitz et al. 2019: Sanchez-Gonzzlezz et al. 2020
Jumper et al. 2021 Guo et al.) 2020).
In the context of designing geometric models for point clouds, a pivotal consideration involves ensuring that the model respects both permutation symmetry and Euclidean symmetry (i.e., symmetry of SE(3) or E(3) group). This requirement necessitates the incorporation of appropriate inductive biases into the architecture, ensuring that the model's output remains invariant or equivariant to point reordering and Euclidean transformation.
However, these restrictions on symmetry can impose limitations on the ability of models to approxi- mate a broad range of functions. For example. Message Passing Neural Networks (MPNNs) (Gilmer) al., al.,[2011 2017). a representative class of Graph Neural Networks (GNNs), learn permutation symmetric functions over graphs. However, it has been shown that such functions are not universal (Xu et al. 2018a; Morris et al., 2019), leading to the development of more expressive GNN frameworks (Maron al.] 2018 Morris et al. 2019 Zhang al., 2023). Similar challenges arise in the geometric setting, where models must additionally respect the Euclidean symmetry, and thus achicving geometric universality (B(3)-completeness) becomes non-trivial task well.
In this work, we systematically investigate the expressiveness of a wide range of invariant models under fully-connected conditions, where interactions occur among all points. We begin by revisiting DisGNN (Li et al. 2024), the simplest invariant model augmenting MPNNs with Euclidean distances
"Correpooniigg author: Muhan Zhang (muhan @ku.ccd..c).

---

Published as a conference paper at ICLR 2025
between nodes additional edge features. We seek answer the question: how close is DisGNN to completeness? Through theoretical analysis, we show that DisGNN is unidentifiable cases can be restricted to O-measure subset containing well-defined sygmey--ymmmetric point clouds only. Our results extend the findings of previous works that solely concentrated on individual hand-crafted counterexamples illustrating the incompleteness of DisGNN (Li et al., 2024
whose
Pozdnyakov & Ceriotti 2022; Pozdnyakov et al. 2020), and a contemporary work that provided a coarser characterization of DisGNN's expressiveness (Hordan et al. 2024b).
We then show that GeoNGNN. the geometric counterpart of NGNN (Zhang & Li 2021) (one simple subgraph GNN), can effectively break all DisGNN's ' unidentifiable cases' symmetry through node marking, thereby achieving Eo3)-comlssseness We then leverage GeoNGNN as a theoretical tool, proceeding to establish the E(3)-completeness of a wide range of invariant models by showeasing their ability to implement GeoNGNN. Specifically, we systematically define the geometric counterparts of subgraph GNNs within Zhang et al. 2023)s y's framework by generalizing their local operations to incorporate geometric information, and then prove that all of these geometric subgraph GNNs are c(3)-comllete. Furthermore, we establish that three well-established invariant geometric models, DimeNet Gasttigerr et al. 2019). SphereNet (Liu et al. 2021) and GemNet (Gasteiger et al. 2021 are also all capable of achieving cooleemmpesssss
These established complet models, except for GemNet (Gasteiger et al. 2021 are either known to be strictly weaker than 2-FWI (Cai et al. 1992) in traditional graph learning setting, or utilize less informative aggregation schemes than 2-FWLe geometric models (Li et al. 2024 Delle Rose et al. [2033 Hordan et al., (024bba) a) (See Section for details). However, our investigation reveals that in the geometric setting, these models are equally E(3)-complete. These surprising findings can deepen our understanding of the power of invariant models, while also suggesting that the bottleneck of invariant models may lie in generalization instead of expressiveness. As side effect, by generalizing subgraph GNNs to the geometric setting, we greatly enlarge the design space of geometrie deep learning models, which we hope could inspire future, more powerful model designs.
2 RELATED WORKS
Designing complete invariant descriptors for point clouds plays crucial role for scientific prob- lems (Nigam et al. 2024), since complete invariants directly allow universal function approximations when coupled with MLPs (Hordan et al. 2024b Li et al. 2024). Previous works, such as Widdowson & Kurlin 2022 2023): Kurlin (2023), have proposed polynomial-time algorithms to compute and compare complete invariants that uniquely determine point clouds. However, these invariants are often highly structured, such as sets of matrices (Kurlin, 2023), and the absence of corresponding neural-network formulations limits their practical applicability.
In the neural-network context, most prior works propose powerful models but typically with "weaker completeness" aaateesss""Waaggggaa (Wang et al. 2022 Puny et al. 2021 Duval et al. [0233] Du al. 2024 Villar et al.) 2021). For example, Com Net (Wang el al. 2022) uses nearest neighbors as references for computing higher-order geometric information, which is infeasible for symmetric point clouds where nodes can have multiple nearest neighbors, thereby limiting its completeness over asymmetric subset of point Frame-based methods, such as FAENet (Duval et al. 2023), LEFTNet (Du et al. 2024), and others (Wang& & Zhang. 2022 Puny et al. 2021). project coordinates or vector features onto local or global equivariant frames and employ powerful structures like MLPs (Hornik et al. 1989) to ensure expressiveness. However, these frames
clouds[] [
degenerate in symmetric structures (c.g., structures with non-distinet eigenvectors (Puny et al. 2021 Duval et al. 2023)), limitation that these methods do not carefully address and theoretically characterize. Although symmetric point clouds often represent zero-measuee mee-ure cases (Hordan et al.) (024b), they can significantly impact downstream continuous
function learning, as demonstrated in (Pozdnyakov & Ceriotti 2022
Hordan et al. 2024b).
Several previous works (Dym & Maron 2020 Joshi et al. 2023
Hordan et al. 2024b Liet et al. 2024
Delle Rose et al.
2023 Hordan et al. 2024a) have characterized geometric models' expressiveness in
a more rigorous sense. The seminal work (Dym & Maron 2020) proved the universality of structures like TFN (Thomas et al. 2018), while such universality requires arbitrarily high-order tensors that are computationally expensive in practice. Another related work, GWL (Toshi et al. 2023), offers many
Arbitrarily selecting one when multiple nearest neighbors exist could break permutation invariance, fundamental principle ensuring stable outputs. We focus only on invariant models.
2

---

Published as a conference paper at ICLR 2025
insightful theoretical conclusions, including that GWL provides upper bounds the expressiveness of many equivariant models (Batatia et al. 2022; Satorras et al. 2021). However, GWL relies on injective multiset functions involving equivariant features, which do not exhibit practical polynomial- time neural forms like scalar multiset functions proposed in (Dym & Gortler 2024] Amir et al. 2024), thus serving more conceptual upper bounds. Poshi et al. (2023) also does not directly address completeness of models as we do. A recent work (Cen et al. 2024 characterizes the symmetry of
point clouds from point group perspective, and investigate the expressiveness through degeneration
of equivariant functions. Notably, recent works (Li et al. 2024; Delle Rose et al. 2023 Hordan et al. 2024b have proposed provably complete geometric models that can produce complete invariant features. Nevertheless, these methods are predominantly built upon the 2-FWL framework (Cai et al., 1992), and the completeness of a broader class of invariant models-many of which employ weaker aggregation schemes (Gasteiger et al.
remains an open question. In this work, we focus on significantly broader range of invariant methods (Schütt et al. 2018 Gasteiger et al. 2019 2021 Liu et al., [021]] Zhang & Li 202 Zhang et al. (2023), some of which newly proposed based on subgraph GNNs in traditional graph learning (Zhang & Li 2021 Zhang et al. 2023). with the aim to fully characterize their completeness rigorously.
PRELIMINARY
3.1 NOTATIONS AND DEFINITIONS
We investigate the expressiveness of invariant models for unlabeled point clouds consisting of node denoted by P E Raxs, The coordinates of the i-th node in P are represented by We use denote and { denote multisets, with the set 1,2,..., represented as [n.. The Euclidean distance between nodes and j is denoted as dzy.
Two point clouds P1 E R and P2 E R are isomorphic if n1 = n2 = n, and there exists a rotation matrix R O()), translation vector t E RR, and a permutation matrix P E Rx"x such that (PP)) + t = Ps. Invariant models considered in this study are all permutation- and [(3)-invariant, meaning they embed isomorphic point clouds into the same k-dimensional representation E BR.
Definition 3.1 (Distinguish, Identify, and E(3) B(3)-Completeness) Let f U g2x3 + E be permutation- and E(3)-invariant function that maps point clouds to a k-dimensional representation. We define the following concepts:
Distinguish: Given two non-isomorphic point clouds P1 and P2, if f(Pi) + f(22), we say f can distinguish P1 and P2-
Identify: For a point cloud Pt, for any non-isomorphic Pz. we always have f(Pi) + f(P2). we say f can identify Pt.
Comploempes:::ss:: If for any pair of non-isomorphic point clouds P1 and Pz, always have f(P:) # ((2)), we say f is E(3)-complete.
Notably, we introduce the novel concept of "Identify"; which allows for a fimer-grained analysis of model expressiveness than commonly adopted "Distinguish": By definition, each "identify" operation encompasses an infinite number of "distinguish" pairs. It is evident that identifying all point clouds implics E(3)-completeness. Furthermore, since distinguishing point clouds with different node counts is straightforward, we focus solely on the case where point clouds have the same finite size.
For a parametric model fe. investigate its maximal expressive form where all intermediate functions are injective, following typical ways (Delle Rose et al., 2023 Li et al., 2024).. Notably, recent work by Dym & Gortler (2024); Amir et al. (2024) proves the existence of neural function forms and parametrizations that ensure injectivity for multiset functions commonly used in geometric models (Gasteiger al. 2019 2021 Li al. 2024), allowing us to achieve such completeness in polynomial time and under (for Lebesgue) almost every parametrization.
Unlabeled point clouds represent one of the most complex and general cases in geometric expressive- ness (Kurlin 2023 Widdowson & Kurlin 2023), surpassing the complexity of point clouds with initial features (e.g., molccules). We consider cases where P contains more than one point, trivial cases arise otherwise.
3

---

Published as a conference paper at ICLR 2025
3.2 DISGNN
To leverage the rich geometric information present in point clouds, a straightforward approach is to model the point cloud distance graph, where edges are present between nodes if their Euclidean distance is below a certain cutoff Temoff- and edge weights are Euclidean distance dyz. Then, an MPNN is applied to it. We call models with such framework as DisGNN, and SchNet (Schütt et al., 2018) is representative of such works.
To be specific, DisGNN models first embed nodes based on their initial node features X E Ruxd (if there exists), such as atomic numbers, obtaining initial node embedding 14 for node i. They then perform message passing according the following equation iteratively:
(1+1) updatte
(NE (1) 1 (I(N)) d:y) (VOB))
h
where 1, represents the node embedding for node i layer (i) denotes the neighbor set of node i. The global graph embedding is then obtained by aggregating the representations of all nodes using invariant function fou ( i E [m]B), where L is the total number of layers and is the total number of nodes.
permutation intruatation-ivariaatt
In the theoretical analysis for DisGNN (Section
assume fully-connected distance graph modeling for point clouds, i.e., (i) is [n] by default, and sufficient iterations until convergence. This ensures maximal geometric ex presssiveeesss of DisGNN and aligns with existing works (Pozdnyakov
& Ceriotti 2022 Hordan et al. 2024b Li et al. 2024).
4 How POWERFUL IS DISGNN?
In PPoddnykkov & Ceriotti 2022); Li et al. (2024), researchers have carefully constructed pairs of symmetric point clouds that DisGNN cannot distinguish to illustrate its E(3)-incompleteness. However, rather than relying on hand-crafted finite pairs of counterexamples and intuitive illustration, more significant question arises: what properties do the point clouds that DisGNN cannor identify share in common? Only after theoretically characterizing these corner cases can we describe and bound the expressiveness of DisGNN in rigorous way, and then design more powerful geometric models accordingly by "solving" these corner cases.
Naively, to determine whether a point cloud P can be identified by DisGNN requires iterating through all other point clouds P" within the entire cloud set I according to its definition. Here, we propose a simple sufficient condition for a point cloud to be identifiable by DisGNN, which focuses solely on P. To achieve this, we first rigorously define the concept of A-symmetry.
Definition 4.1 (A-symmetry)). Given point cloud P E gax3 let A be permutation-cquuiaarinn and E(3)-invariant algorithm taking P as input and produces node-level features X'A A(P)) E Raxd (we use 2dd E Rdd to denote the i-th nodes' feature). We define center set of as A (P))
Exeln) m(r Jp: m Rd + IR, Exe(ni m(x7) + 0}. If [A)) = 1, then we say P is A- symmetric; otherwise, we say P is A-asymmetric.
Exe m(*7)
Intuitively, A assigns features to nodes of the point cloud based on their spatial properties, while m represents a "mass" function that maps these features to corresponding "masses" Thus, ( denotes the collection of "barycenters" associated with different "mass" assignments. For example, we can consider A as the distance encoding function, denoted as C, which computes the distance from each node the geometric center as the node feature. A C-asymmetric case shown in Figure aa).
Obviously, the cardinality of (Att(()) is solely dependent on the partition of node features A(P) and is independent of the specific feature values. We also propose more powerful algorithm, DisGNN encoding, denoted as D, which applies DisGNN to point clouds iteratively until the partition of node features stabilizes. Notably, earlier studies by Delle Rose et al. (2023); : Li al. (2024) have shown that D guarantees a node feature partition that is no coarser than that of C. Consequently, D-symmetry always implies C-symmetry, representing stricter form of symmetry.
Based on these definitions, we now present the primary theorem demonstrating that all such asym- metric point clouds can be identified by DisGNN.
Note that the geometric center of P always included in (P) by defining m as constant function.
4

---

Published as a conference paper at ICLR 2025
(b)
R 128
X
Ceee
C-ym
Paay
Unitentfable
Cases
Figure (a) C-asymmetric point cloud. The blue small node represents the geometric center, and each green node represents an element in (P). The large nodes constitute the point cloud, with different node colors denoting different node features. (b) Illustration of the relations among different subsets of the point cloud. A-sym parts encompass all A-symmetric graphs. Each subset has a measure of and strictly contains their sub-parts (proper subset relation).
Theorem 4.2 (Asymmetric point clouds are identifiable). Let denote the center distance encoding and D the DisGNN encoding. Then, given arbitrary point cloud P E TRRx33 P C-asymmetric + P is D-asymmetric = can be identified by DisGNN.
Key idea. Given point cloud P, when it has two global-level distinct anchors, such geometric center or other centers that can be determined in permutation-invant DisGNN has the potential learn triangular distance encoding for each node w.r.t. and these two global anchors, and produces complete representation for P by leveraging such encoding information (Lemma B.6). For C- or D-asymmetric point clouds, their center sets C (P) or D have more than element, within which any pair of distinct centers can be utilized by DisGNN for such encoding (Lemma B.4), thereby facilitating DisGNN to fully represent and identify P.
and E(3)-cquivariant way,
We provide rigorous proof in Appendix B.3 and encourage readers to refer to it, where we show the superior global geometric learning ability of DisGNN and can inspire the development of new complete model designs in the triangular distance encoding perspective, exemplified by GeoNGNN proposed in the following section.
Theorem shows that to test if point cloud is identifiable by DisGNN, one can first check its C-symmetry, which is quick and intuitive. If C-asymmetric, identifiable; otherwise, one can test for the more powerful but costlier D-symmetry, Notably, although the latter test still requires running DisGNN, it demonstrates that single execution of DisGNN on can provide an indication of identifiability, offering insights without necessitating exhaustive search over all P' R
Importantly, Theorem restrrics the unidentifiable set of DisGNN to a constrained, highly symmetric subset, illustrated in Figure [[). In Theorem 4 we provide rigorous analysis demonstrating that the unidentifiable set of DisGNN has zero measure. These results sufficiently demonstrate that, despite its extremely simple and naive design, DisGNN is nearly c30mpleeee
Theorem 4.3 (Unidentifiable set of DisGNN has measure zero). The Lebesgue measure on Trxx3 of the C-symmetric, D-symmetric, and unidentifiable point cloud sets is zero.
Finally, we note that a recent study (Hordan et al. 2024b) also indicates the near-completeness of DisGNN, by showing that can distinguish arbitrary pairs of point clouds from an asymmetric point cloud I Nevertheless, our findings represent significant advancement by introducing a strictly larger asymmetric subset and employing the concept of "identify" instead of "pair-wise wise distinguish". This provides a finer characterization of the near completeness of DisGNN. Please refer to Appendix C.2 for a detailed comparison.
ON THE COMPLETENESS OF INVARIANT GEOMETRIC MODELS
In this section, we move to demonstrate and prove the E(3) E(3)-completeness of broad range of invariant models that leverage more geometric features or more complicated aggregation schemes than DisGNN under fully-connected conditions. These models can effectively identify all those highly-symmetric point clouds that DisGNN can not. We first introduce a new simple invariant model design, which can precisely break such symmetry and achieve completeness. Consequently, show how this new design serves as a theoretical tool for proving o a wide range of invariant models' expressiveness. Proof for this section is provided in Appendix B
5

---

Published as a conference paper at ICLR 2025
5.1 GEONGNN: BREAKING SYMMETRY THROUGH NODE MARKING
According to Theorem 4.2 (along with the lemmas emphasized in its key idea), the essential factor in identifying& a point cloud by DisGNN is the existence of two distinct anchors. Although every point cloud inherently possesses one, i.c.. the geometric center, the other may not exist in some highly symmetric cases, such subsection, we show that GeoNGNN, the geometric counterpart of Nested GNN (NGNN) ZZangg Li (02T). can fill in this last piece of the puzzle by applying DisGNN on point clouds with an additionally marked node, which breaks the symmetry and exactly acts as the other anchor.
sphere. This hinders DisGNN from achieving completeness. In this
GeoNGNN employs two-level hierarchical framework by nesting DisGNN. Specifically, processes a given point cloud follows: 1) The first level, referred to as the inner GNN, operates independently on each point's 's Tsub-Sizs -ized sub- sub-point cloud and aggregates the local sub-point cloud information into an initial embedding for the corresponding point in the original point cloud. 2) The second level, referred to as the outer GNN, processes the original point cloud based on the embeddings generated by the inner GNN, thereby producing final point-level and cloud-level representations.
Formally, for the inner GNN, the representation of the node j in node i's sub-point cloud at the 1-th layer, denoted as hij is updated as:
(++1) = upddate, inmer (n Mij
(1]
aggr. inner (E day) K E(), du S (rau
for all byy satisfying dij Tuub and N()) represents all nodes it satisfying dej S reutoff- Here, Tsub and Teutoff are hyperparameters representing the subgraph size and interaction cutoff, respectively. (0) htg is initialized as:
(0) hag
iiit, inner (Tij. diy 1 liz));
where zj is the raw feature of node j. and di and i denote the position encoding and marking of the center point, respectively. After Nin iterations, the inner GNN summarizes the sub-point cloud as:
Sompput, inner (h
(Nu) 1 E [m]] dij S. Tsub B).
producing the initial point representations for the outer GNN. The outer GNN then updates these representations over Nour iterations following the framework in Equation (
By representing point clouds as distance graphs, GeoNGNN captures subgraph patterns instead of subtree patterns, which have been shown to be more expressive traditional graph learning (Zhang
& Li (021). We now show that GeoNGNN achieves geometric completeness under fully-connected conditions, thereby effectively addressing expressiveness limitations of DisGNN.
Theorem 5.1 Compleeemsss of GeoNGNN). When the following conditions are met, GeoNGNN is E(3)-compe:
Nm >= 5 and N >= O (where O indicates that the outer GNN only performs final pooling).
The distance graph is fully- colly-conneeted (Fcutoff -0))
All subgraphs are the original graph (Trub
+00).
Key idea. Given an arbitrary point cloud P. consider a specific node i within it that differs from the geometric center. As node is explicitly marked within its own subgraph (Equation 33, there are now two anchors in this subgraph, namely node and the geometric center, that facilitate triangular distance encoding by DisGNN. Consequently. DisGNN can fully represent and identify P through the representation of node i's subgraph, as underscored in the key idea of Theorem 4.2 Notice that in a point cloud with more than two nodes, node that is distinct from the geometric center always exists, and through the outer pooling, overall completeness and permutation invariance are guaranteed.
While Theorem 5.1 establishes that infinite subgraph radius and distance cutoff guarantee com- pleteness over all point clouds, in Appendix D.2 we show that finite values also lead to boosted expressiveness compared to DisGNN. And we notice that the conditions specified in Theorem 5.] ultimately result in polynomial- time complexity w.r.t. the size of the point cloud even when consider- ing the complexity involved in obtaining intermediate injective functions, which aligns with prior studies (Kurlin 2023: Widdowson & Kurlin 2023) [Hrddn et al., 2024b [ et al. 2024 Delle Rose
et al. 2023). The detailed complexity analysis is provided in Appendix D.I

---

Published as a conference paper at ICLR 2025
5.2 COMPLETENESS OF GEOMETRIC SUBGRAPH GNNS
GeoNGNN provides simple approach to extend traditional subgraph GNNs to geometric scenarios with remarkable geometric expressiveness. Indeed, the realm of traditional graph learning literature contains multitude of subgraph GNNs such as DSS-GNN (Bevilacqua et al. (021). GNN-AK (Zhao jet al. 202T), OSAN (Qian et al., 2022] and so on (You et 20.1 Frasca et al.) 2022). Extending these models to geometric settings can significantly enlarge the design space of geometric models, and potentially introduce valuable inductive biases. In this section, we take pioneering step to do this, subsequently establishing the geometric completeness of all these models.
We first define the general geometric subgraph GNN, a broad family of geometric models, by slightly adapting the unweighted graph models from Zhang al. (2023) to handle geometric scenarios. For simplicity, we provide an intuitive overview and outline the modifications here, while self-contined and formal definitions can be found in Appendix R
Definition 5.2 (General geometric subgraph GNN, informal). A general geometric subgraph GNN takes point clouds P E _=x3 (potentially with node features X E Raxd, such as atomic numbers) as input, treats as a distance graph with interaction cutoff Teutoff, and:
Utilizes node marking with T'ub "ize ego subgraph the subgraph generation policy.
Stacks multiple geometric subgraph GNN layers, each following general subgraph GNN layer defined in ZZbag et al. (2023). which could arbitrarily include single-point, global, and local operations for aggregating graph information different levels. The only modification is to the local operations: when updating hav (the representation of node e in u's subgraph), distance information is additionally integrated, thus the local operations aggregate information {f (Nuor dew)) w E
M(u)B B and if (hwe. duw) | w E (W) } respectively.
It adopts vertex- vertex-subgraph or subgraph-vertes pooling schemes (Zhang et al. 2023) to summarize node and subgraph features and produce the final graph representation.
noteworthy that GeoNGNN (without outer layers) represents particular instance of general geometric subgraph GNNs, focusing solely on intra-subgraph message propagation. Beyond this scope, general geometric subgraph GNNs have the capability to engage in inter-subgraph message passing in diverse manners and adopt different pooling schemes. Now, we show that the whole family, with at least one local aggregation, is complete under exactly the same conditions as GeoNGNN.
Theorem 5.3 (Completeness for general geometric subgraph GNNs). When the general geometric subgraph layer number is larger than some constant C (irrelevant to the node number), and the last two conditions specified in Theorem 3.1 are met, all general geometric subgraph GNNs in Definition 5.2 with least one local aggregation E(3)-complete.
Key idea. It can be shown that any type of general geometric subgraph GNNs, which could employ any combination of aggregation and pooling schemes, can implement (Frasca et al., 2022) GeoNGNN, which is established an c(3)-complete model in Theorem [.1] Consequently, any general geometric subgraph GNN can also achieve B(3)-completenes under the same conditions as GeoNGNN.
Concerning prior research traditional subgraph GNNs (Bevilacqua et al., 2021 Zhao al. 2021] Qian et al. 2022 You et al. 2021 Frasca et al., 2022), their geometric counterparts can be defined correspondingly. by substituting all local aggregation schemes with distance-aware counterparts. We denote their geometric counterpart by prefixing Geo to their original names, for instance, GeoSUN |rasca et al. (2022). Consequently, it can be established that some of them precisely fall into the general definition 5.2 such as GeoOSAN (Qian et al. 2022), and for the others such as GeoGNN-AK (Zhao et al. 2021 and GeoSUN Frasca al. (2022). ). while even though they do not exactly match the general definition, they can still implement GeoNGNN, thereby exhibiting F(3)-completeness (Theorem (..).. Please refer to Appendix F..2 for details about all of these.
We note that all subgraph GNNs in this section have been proven to be strictly weaker than 2- FWL (Cai et al., [992) in traditional graph learning context (Zhang et al. 2023). Specifically, there exist pairs of unweighted graphs that can be distinguished by 2-FWL but remain indistinguishable by these subgraph GNNs. Moreover, these subgraph GNNs themselves establish a strict expressiveness hierarchy (Zhang et al. 2023). Interestingly. our findings reveal that all of these discrepancies diminish when these models are extended to geometric scenarios with point clouds by leveraging distance graphs. This phenomenon could be attributed to the low-rank nature of distance graphs,
7

---

Published as a conference paper at ICLR 2025
whose intrinsic dimension is less than the point cloud space dimension 3n rather than equals their ambient dimension n²
5.3 COMPLETENESS OF WELL-ESTABLISHHR INVARIANT MODELS
Based on the completeness of GeoNGNN, we move on to establish the -o0plleelsss of several well-established invariant models, including DimeNet (Gasteiger et al. 2019), GemNet (Gasteiger et al., (021)) and SphereNet (Liu et al. 2021). These models do not exactly learn on subgraphs, however, can still be mathematically aligned with GeoNGNN, Formal descriptions of these models can be found in Appendix B.6
Theorem 5.4 (EC))-Comlesteness of DimeNet, SphereNet, GemNet). When the following conditions are met, DimeNet, SphereNe and GemNel are E(3 -complleee
The aggregation layer number is larger than some constant C' (irrelevant to the node number). They initialize and update all edge representations, i.e., Tembed +00.
They interact with all neighbors, i.e., Viu = +00
Key idea. The key insight underlying is that all these models track edge representations (hi for edge (i. which can be mathematically aligned with the node-subgraph representations tracked in GeoNGNN (hig, subg for node in subgraph i). Moreover, the additionally incorporated features, such as angles, can all be equivalently expressed by multiple distances. Consequently, these three models can all implement GeoNGNN, and thereby achieving completeness.
Note that DimeNet is relatively simpler invariant model that aggregates neighbor information in weaker manner compared to existing 2-FWL-like complete geometric models such as 2F-DisGNN (Li et al. 2024). Specifically, when updating hag. DimeNet can be equivalently considered aggregating (hes: daj) for a specific neighbor k; while 2F-DisGNN aggregates (ha,, hag). The latter incorporates more informative hidden representation hug- which is essential in the proof of 2F-DisGNN's completeness (Li et al. 2024 Delle Rose et al. 2023), instead of dæj in the former. Nevertheless, our findings first show that these models are equally co3)lceee under fully-connected conditions. Consequently, future developments like GemNet that integrate higher-order geometry could be unnecessary in terms of boosting theoretical expressiveness under such conditions.
5.4 SUMMARIZATION AND DISCUSSION
Fully-Connected Condition. Thus far, we have characterized a broad collection of H(3)-complete invariant models under similar conditions. Among these conditions, the condition of fully-comectedd geometric graphs (or equivalently, infinite cutoffs) is required. Notably, this condition consistent with all prior works that rigorously characterize invariant models/descriptors in the same sense, including (Kurlin 2023 Widdowson & Kurlin 2022 2023
Li et al., 2024 Delle Rose et al. 2023 Hordan al. 2024bbai ab. And due to significant local information loss during invariant message passing (Joshi et al. 2023 3 Du et al. (004)), even under this condition the characterization remains highly nontrivial. Indeed, removing the fully-connected condition would require conditions about specific forms of sparsity, which would be considerably more demanding than the overall connectivity
typically required by equivariant models (Joshi et al. 2023 Du et al. 2024 Wang et al. 2024] Sverdlov & Dym 2024) that can maintain local information through equivariant features. Such
characterization is left for future work.
Theoretical Characterization vs. Practical Use. In practical scenarios, efficiency and generalization are often prioritized, and local/sparse connectivity can typically offer empirical advantages these aspects (Musaelian et al. 2023). Thus, strictly adhering to the complete condition which requires full connectivity is unnecessary, and finding a balance crucial, as demonstrated in our additional experiments in Appendix [ (Figure(6). However, this does not diminish the importance of theoretical characterization, since theoretical completeness provides an upper bound for the parametric model's
'Here in SphereNet we do consider the relative azimuthal angle y. since SphereNet with o is not F(3)-invariant, while F(3))completeness defined on F(3)-invariant models. Note that the exclusion of şo only results in weaker expressiveness.
In DimeNet and GemNet, nodes i or j are excluded when aggregating neighbors for edge ij. The condition requires the inclusion of these end nodes.
8

---

Published as a conference paper at ICLR 2025
potential-wher when combined with MLPs (Hornik et al. 1989), complete models can achieve universal
approximation over continuous invariant functions (Hordan et al. 2024b; Li et al. 2024). This is analogous to the universal approximation property for MLPs (Hormik et al. 1989 and the Turing completeness for RNNs (Siegelmann & Sontag 1992), where practical implementations do not (and typically cannot) satisfy all theoretical conditions, yet these results indicate their superiority over weaker structures that cannot achieve certain approximations even with unlimited resources.
CEm3)-complete Counterpart, Finally, we present provable SE(3)-complete variant of GeoNGNN, capable of distinguishing chiral molecules, by making a minor modification to the distance features through the addition of orientation sign. This further detailed in Appendix H
EXPERIMENTS
In this section, we conduct additional assessments to validate our theoretical claims. The first experiment aims to verify the conclusion that DisGNN is nearly complete by assessing the proportion of its unidentifiable cases in real- world point clouds. The second experiment designed to evaluate whether the complete models consistently demonstrate separation power for challenging pairs of point clouds, where numerical precision may influence the outcomes. We provide more evaluations of GeoNGNN on practical molecular-relevan tasks in Appendix F) which could offer further insights.
6.1 ASSESSMENT OF UNIDENTIFIABLE CASES OF DISGNN
In Theorem 4.3 we have rigorously shown the rarity of symmetric point cloud sets, which are supersets of the unidentifiable set of DisGNN. However, since real-world point clouds typically subject to slight noise, requiring exact symmetry would be overly restrictive. Therefore, we address more challenging setting by explicitly accounting for noise and evaluating the rarity of relaxed symmetric point clouds in practical scenarios. To this end, we first define two noise tolerances that allow us to relax the exact C- and D-symmetry.
Noise Tolerances. 1) The rounding number for distance-related calculations. When applying algorithms C and D, we round the distance values to decimal places for robustness noise. 2) The deviation error In Definition 4.1 we define symmetry by the set of extended " "mass" centers. We provide an equivalent definition Proposition C.4, which shows that a point cloud is A-symmetrie and only if the geometric centers of all sub- lub-point clouds, partitioned by distinct node features of A, coincide. This alternative definition allows us to define another noise tolerance: we now consider a point cloud as symmetric if all sub-point clouds' centers described above lie within ball of radius c. Together, r and c define a relaxed symmetry that accounts for noise real-world point clouds. When r > +oc and
the relaxed symmetry becomes the exact theoretical symmetry.
We are now ready to evaluate the proportion of relaxed symmetric point clouds in real-world datasets. We select representative datasets, namely QM9 (Ramakrishnan et al., 2014: Wu et al.) 2018) and ModelNet40 (Wu et al. 2015), for this assessment. The QM9 dataset comprises approximately 130K molecules, while the ModelNet40 dataset consists of roughly 12K real-world point clouds categorized into 40 classes, including objects such as chairs. We first rescale all point clouds, and then fix the rounding number to small values and evaluate the proportion of symmetric point clouds with respect to different values of Please refer to Appendix C.3 for detailed settings.
Results are shown in Figure Here are several key observations: 1) For the QM9 dataset, even with the largest deviation error (10-1), the proportion of C- and D-symmetric point clouds is less than ~0. 15% of the entire dataset. Since C- and D-symmetric point cloud sets are supersets of the unidentifiable set of DisGNN, the unidentifiable proportion is therefore the strictest deviation error, only 0.0046% of the graphs (6 out of ~ 130K) exhibit D-symmetry. 2) For the Med4 dataset, when the deviation error is less than 10 only I point cloud exhibit C-symmetry. Moreover, no D-symmetric point clouds are found across all deviation errors.
more than ~0.15%. With
To summarize, the statistical results suggest that the occurrence of unidentifiable graphs in DisGNN is practically negligible in real-world scenarios, even when the criterions are significantly relaxed. This supports the conclusion that DisGNN is almost E(3)-complete.

---

Published as a conference paper at ICLR 2025
QM9
ModelNet-00
0:40
C-ty mometic
Coymuntric symmetric
6:35-
D symmettic
0.18
0.80
Proportion (%6
Propoction bin 196)
0.25
0.06-
6:20
0.04
0.15
0.10
0.05
0.00
1o.4
10-4 16.5
86-2
4n-2
10/2
16.1
Deviation
Eeroc e
Deviation Error
Figure 2: Assessment of symmetric point clouds in real-world datasets. (a) Proportion of symmetric point clouds in QM9 (r = 2). (b) Proportion of symmetric point clouds in ModelNet40. The proportion of DDsyymmerric point clouds in ModelNet40 is zero across all deviation errors (r 1).
6.2 SEPARATION POWER ON SYNTHETIC POINT CLOUD PAIRS
To construct hard-to-distinguish counterexamples, we develop geometric expressiveness dataset based on the counterexamples proposed by Li et al. (2024). The synthetic dataset consists of 10 isolated and combinatorial counterexamples. Each counterexample is composed of a pair of highly symmetric point clouds (all of which are D-symmetric, as described in Section which are non- isomorphic yet indistinguishable by DisGNN. We provide some examples Figure 7 and please refer to Appendix G for further settings.
Table 1: Separation results the constructed geometric expressiveness dataset. Models for which we have theoretically established completeness are highlighted in gray.
Invaraint
Equivaraint
SchNer
DisGNN
DimeNer
SphereNet
GemNer
GeoNGNN
PaiNN
MACE
Isolated (10 cases)
0%
0%
100%
100%
100%
100%
100%
100%
Combined (7 cases)
0%
0%
100%
100%
100%
100%
100%
100%
Results are presented in Table [ As shown, our established complete models can all distinguish these challenging pairs effectively, whereas DisGNN and SchNet cannot. This supports theory and indicates that numerical precision not impacting these complete models' separation ability here. Interestingly, two equivariant models, PaiNN (Schütt et al.) 2021) and MACE (Batatia al. 2022) that leverages vectors and high-order tensors respectively, also effectively distinguish all the pairs. This prompts further question of whether they also complete, especially under sparse connections or finite tensor orders, and requires further investigation.
7 CONCLUSION AND LIMITATION
Conclusion. In this study, we thoroughly analyze a wide range of invariant models' theoretical expressiveness under fully-connected condition. Specifically, we rigorously characterize the ex- pressiveness of DisGNN, showcasing that all its unidentifiable cases exhibit D-symmetry and have a measure of O. We then establish a large family of E(3)-complete models, which encompasses GeoNGNN, geometric subgraph GNNs, as well as three established models DimeNet, GemNet, and SphereNet. This contributes significantly to a comprehensive understanding of invariant geometric models. Moreover, the newly introduced geometric subgraph GNNs notably enlarge the design space of expressive geometric models. Experiments further validate our theoretical findings.
Limitation. The rigorous EC3)-completeness characterization for invariant models is under the conditions of fully connected graphs, which can be limited and have been throughout discussed in Section 5.4 The extent to which invariant models can exhibit high expressiveness on general sparse graphs remains an open question that needs further investigation. Additionally, future research needed to characterize the expressiveness of vector models that rely solely 1-order equivariant representations and adopt atom-level representations, such as PaiNN (Schütt et al. (021). which show promising experimental results but still lack theoretical expressiveness characterization.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENT
This work is supported by the National Natural Science Foundation of China (62276003).
REFERENCES
Keir Adams, Lagnajit Pattanaik, and Connor W Coley. Learning 3d representations of molecular chi- rality with invariance to bond rotations. In International Conference on Learning Representations, 2021.
Tal Amir, Steven Gortler, Ilai Avni, Ravina Ravina, and Nadav Dym. Neural injective functions for multisets, measures and graphs via a finite witness theorem. Advances in Neural Information Processing Systems, 36, 2024.
Ilyes Batatia, David P Kovacs, Gregor Simm, Christoph Ortner, and Gábor Csányi. Mace: Higher order equivariant message passing neural networks for fast and accurate force fields. Advances in Neural Information Processing Systems, 35: 15:11422-11436 2022.
Simon Batzner, Albert Musaelian, Lixin Sun, Mario Geiger, Jonathan P Mailoa, Mordechai Kornbluth, Nicola Molinari, Tess Smidt, and Boris Kozinsky. E (3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials. Nature communications, 13(1):1-11, 202,,
Beatrice Bevilacqua, Fabrizio Frasca, Derek Lim, Balasubramaniam Srinivasan, Chen Cai, Gopinath Balamurugan, Michael M Bronstein, and Haggai Maron. Equivariant subgraph aggregation networks. In International Conference on Learning Representations, 2021.
Jin-Yi Cai, Martin Fürer, and Neil Immerman. An optimal lower bound on the number of variables for graph identification. Combinatorica, 12(4):389-44 410, 1992.
Jiacheng Cen, Anyi Li, Ning Lin, Yuxiang Ren, Zihe Wang, and Wenbing Huang. Are high- degree representations really unnecessary in equivariant graph neural networks? arXiv preprint arXiv:2410.11443 2024.
Stefan Chmiela, Alexandre Tkatchenko, Huziel E Sauceda, Igor Poltavsky, Kristof T Schütt, and Klaus-Robert Müller. Machine learning of accurate energy-conserving molecular force fields. Science advances, 36))e1603015, 2017.
Stefan Chmiela, Valentin dassile--Galinde Oliver T Unke, Adil Kabylda, Huziel E. Sauceda, Alexan- dre Tkatchenko, and Klaus-Robert Müller. Accurate global machine learning force fields for molecules with hundreds of atoms. Science Advances, 9())eadf00873, 2023.
Valentino Delle Rose, Alexander Kozachinskiy, Cristóbal Rojas, Mircea Petrache, and Pablo Barceló. Three iterations of (d w) test distinguish non isometric clouds of d-dimensional points. arXiv e-prints, pp. arXiv-2303, 2023.
Yuanqi Du, Limei Wang, Dieqiao Feng. Guifeng Wang, Shuiwang Ji, Carla P Gomes, Zhi-Ming Ma. al. A new perspective on building efficient and expressive 3d equivariant graph neural networks. Advances in Neural Information Processing Systems, 36, 2024.
Alexandre Agm Duval, Victor Schmidt, Alex Srrmández-Garefa Santiago Miret, Fragkiskos D Malliaros, Yoshua Bengio, and David Rolnick. Faenet: Frame averaging equivariant gnn for materials modeling, In International Conference on Machine Learning, pp. 9013-9033. PMLR. 2023.
Nadav Dym and Steven J Gortler. Low-dimensional invariant embeddings for universal geometric learning. Foundations of Computational Mathematics, pp. 1-41, 2024.
Nadav Dym and Haggai Maron. On the universality of rotation equivariant point cloud networks, In International Conference on Learning Representations, 2020.
Matthias Fey and Jan Eric Lenssen. Fast graph representation learning with pytorch geometric. arXiv preprint arXiv: 1903. 024288. 2019.
11

---

Published as a conference paper at ICLR 2025
Fabrizio Frasca, Beatrice Bevilacqua, Michael Bronstein, and Haggai Maron. Understanding and ex- tending subgraph gnns by rethinking their symmetries. Advances in Neural Information Processing Systems, 35:31376-31390, 2022.
Piotr Gaiński, Michal Koziarski, Jacek Tabor, and Marek Smieja. Chienn: Embracing molecular chirality with graph neural networks. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 36-52. Springer, 2023.
Johannes Gasteiger, Janek GroB, and Stephan Günnemann. Directional message passing for molecular graphs. In International Conference Learning Representations, 2019.
Johannes Gasteiger, Shankari Giri, Johannes T Margraf, and Stephan Günnemann. Fast and uncertainty-awart directional message passing for non-equilibrium molecules. arXiv preprint 1rXi::2011.141155 2020.
Johannes Gasteiger, Florian Becker, and Stephan Günnemann. Gemnet: Universal directional graph neural networks for molecules. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan (eds.), Advances in Neural Information Processing Systems, volume 34, pp. 6790-6802. Curran Associates, Inc., 2021. URL httphhttts////oceee///eeeeoo neuri cc/ Paper.
Johannes Gasteiger, Muhammed Shuaibi, Anuroop Sriram, Stephan Günnemann, Zachary Ward Ulissi, C Lawrence Zitnick, and Abhishek Das. Gemnet-oc: Developing graph neural networks for large and diverse molecular simulation datasets. Transactions Machine Learning Research, 2022.
Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl. Neural message passing for quantum chemistry. In International conference on machine learning, pp. 1263-1272. PMLR, 2017.
Yulan Guo, Hanyun Wang, Qingyong Hu, Hao Liu, Li Liu, and Mohammed Bennamoun. Deep learning for 3d point clouds: A survey. IEEE transactions on pattern analysis and machine intelligence, (3(12):4333-4366 2020.
Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. Advances in neural information processing systems, 30, 2017.
Snir Hordan, Tal Amir, and Nadav Dym. Weisfeiler leman for euclidean equivariant machine learning. arXiv preprint arXiv:2402.024480 2024a.
Snir Hordan, Tal Amir, Steven J Gortler, and Nadav Dym. Complete neural networks for complete cuclidean graphs. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 12482-12490, 2024b.
Kurt Hornik, Maxwell Stinchcombe, and Halbert White. Multilayer feedforward networks universal approximators. Neural networks, 2(5): -36, 1989.
Chaitanya K Joshi, Cristian Bodnar, Simon V Mathis, Taco Cohen, and Pietro Lio. On the expressive power of geometric graph neural networks. In International Conference Machine Learning, pp. 15330-15355. PMLR, 2023.
John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger, Kathryn Tunyasuvunkkool Russ Bates, Augustin Židek, Anna Potapenko, et al. Highly accurate protein structure prediction with alphafold. Nature, 596(7873):583-5888 -88, 2021.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv: 1412.6694 2014.
Thomas Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. 2016.
arXiv preprint arXiv:
Dávid Péter Kováes, Cas van der Oord, Jiri Kucera, Alice EA Allen, Daniel Cole, Christoph Ortner, and Gábor Csányi. Linear atomic cluster expansion force fields for organic molecules: beyond rmse. Journal of chemical theory and computation, 17(12):666--711, 2021.
12

---

Published as a conference paper at ICLR 2025
David Peter Kovacs, Ilyes Batatia, Eszter Sara Arany, and Gabor Csanyi. Evaluation of the mace force field architecture: from medicinal chemistry to materials science. arXiv preprint 2Xiv:2335.112477. 2023.
Vitaliy Kurlin. Polynomial-time algorithms for continuous metrics on atomic clouds of unordered points. Match: Communications in Mathematical and in Computer Chemistry, 2023.
Yunyang Li, Yusong Wang, Lin Huang, Han Yang, Xinran Wei, Jia Zhang, Tong Wang, Zun Wang, Bin Shao, and Tie Liu. Long-short-range pessaag-passsing A physics-informed framework to capture non-local interaction for scalable molecular dynamics simulation. arXiv preprint arXiv:2304. 13542, 2023.
Zian Li, Xiyuan Wang, Yinan Huang, and Muhan Zhang. Is distance matrix enough for geometric deep learning? Advances in Neural Information Processing Systems, 36, 2024.
Yi-Lun Liao and Tess Smidt. Equiformer: Equivariant graph attention transformer for 3d atomistic graphs. In The Eleventh International Conference on Learning Representations, 2022.
Yi Liu, Limei Wang, Meng Liu, Yuchao Lin, Xuan Zhang, Bora Oztekin, and Shuiwang Ji. Spherical message passing for 3d molecular graphs. In International Conference on Learning Representations, 2021.
Haggai Maron, Heli Ben-Hamu, Nadav Shamir, and Yaron Lipman. Invariant and equivariant graph networks. In International Conference on Learning Representations, 2018.
Boris Mityagin. The zero set of a real analytic function. arXiv preprint arXiv:1512.07276. 2015.
Christopher Morris, Martin Ritzert, Matthias Fey, William L Hamilton, Jan Eric Lenssen, Gaurav Rattan, and Martin Grohe Weisfeiler and leman go neural: Higher-order graph neural networks. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pp. 4602 4609, 2019.
Albert Musaelian, Simon Batzner, Anders Johansson, Lixin Sun, Cameron Owen, Mordechai Kornbluth, and Boris Kozinsky. Learning local equivariant representations for large-scale atomistic dynamics. Nature Communications, 14(1):579, 2023.
Jigyasa Nigam, Sergey N Pozdnyakov, Kevin K pleteness of atomic structure representations. API. Machine Learning, 2(1), 2024.
Dumittan, and Michele Ceriotti. Com-
Lagnajit Pattanaik, Octavian -otavian-Eugen Ganea, Ian Coley, Klavs F. Jensen, William H. Green, and Connor W. Coley. Message passing networks for molecules with tetrahedral chirality, 2020.
Sergey N Pozdnyakov and Michele Ceriotti. Incompleteness of graph convolutional neural networks for points clouds in three dimensions. arXiv preprint arXi::2201.07136 2022.
Sergey N Pozdnyakov, Michael J Willatt, Albert P Bartók, Christoph Ortner, Gábor Csányi, and Michele Ceriotti. Incompleteness of atomic structure representations. Physical Review Letters, 125(16))166001,, 2020.
Omri Puny, Matan Atzmon, Edward Smith, Ishan Misra, Aditya Grover, Heli Ben- Ben-Hamu, and Yaron Lipman. Frame averaging for invariant and equivariant network design. In International Conference on Learning Representations, 2021.
Chendi Qian, Gaurav Rattan, Floris Geerts, Mathias Niepert, and Christopher Morris. Ordered subgraph aggregation networks. Advances in Neural Information Processing Systems, 35:21030- 21045, 2022.
Raghunathan Ramakrishnan, Pavlo O Dral, Matthias Rupp, and O Anatole Von Lilienfeld. Quantum chemistry structures and properties of 134 kilo molecules. Scientific data, 1(1):1-7, 2014.
Alvaro
Jonathan Godwin, Tobias Pfaff, Rex Ying, Jure Leskovec. and Peter Battaglia. Learning simulate complex physics with graph networks. In International Conference on Machine Learning, pp. 8459 8468. PMLR, 2020.
Victor Garcia Satorras, Emiel Hoogeboom, and Max Welling. E (n) equivariant graph neural networks. In International conference on machine learning, pp. 9323-9332, PMLR, 2021.
13

---

Published as a conference paper at ICLR 2025
Gunnar Schmitz, Ian Heide Godtliebsen, and Ove Christiansen. Machine learning for potential energy surfaces: An extensive database and assessment methods. The Journal of chemical physics, 150 (24):244113, 2019.
Kristof Schütt, Oliver Unke, and Michael Gastegger. Equivariant message passing for the prediction of tensorial properties and molecular spectra. International Conference on Machine Learning, pp. 9377-9388. PMLR, 2021.
Kristof T Schütt, Huziel E Sauceda, P-J Kindermans, Alexandre Tkatchenko, and K-R Müller. Schnet-a deep learning architecture for molecules and materials. The Journal of Chemical Physics, 148(24)2241722, 2018.
Hava T Siegelmann and Eduardo D Sontag. On the computational power of neural nets. In Proceedings of the fifth annual workshop on Computational learning theory, pp. 440-449, 1992.
Yonatan Sverdlov and Nadav Dym. On the expressive power of sparse geometric mpnns. arXiv preprint arXiv:2407.02025. 2024.
Philipp Thölke and Gianni De Fabritüis. Equivariant transformers for neural network based molecular potentials. International Conference on Learning Representations, 2021.
Nathaniel Thomas, Tess Smidt, Steven Kearnes, Lusann Yang. Li Li, Kai Kohlhoff, and Patrick Riley. Tensor field networks: Rotation-and translation-equivariinn neural networks for 3d point clouds. arXiv preprint arXiv: 089299 2018.
Soledad Villar, David W Hogg, Kate Storey-Fisher Weichi Yao, and Ben Blum-Smith. Scalars universal: Equivariant machine learning, structured like classical physics. Advances in Neural Information Processing Systemx, 34:28848-28863, 2021.
Limei Wang, Yi Liu, Yuchao Lin, Haoran Liu, and Shuiwang Ji. Comenet: Towards complete and efficient message passing for 3d molecular graphs. Advances in Neural Information Processing Systems, 35:650-664, 2022.
Wang. Yung- Shag- Hsu, Justin Baker, Andrea L Bertozzi, Jack Xin, and Bao Wang. Rethinking the benefits of steerable features in 3d equivariant graph neural networks. In The Twelfth International Conference Learning Representations, 2024.
Xiyuan Wang and Muhan Zhang. Graph neural network with local frame for molecular potential energy surface. In Learning on Graphs Conference, pp. 19-1. PMLR, 2022.
Daniel Widdowson and Vitaliy Kurlin, Resolving the data ambiguity for periodic crystals. Advances in Neural Information Processing Systems, 15:24625-246388 2022.
Daniel Widdowson and Vitaliy Kurlin. Recognizing rigid patterns of unlabeled point clouds by complete and continuous isometry invariants with no false negatives and no false positives. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1275-1284, 2023.
Zhenqin Wu, Bharath Ramsundar, Evan N Feinberg, Joseph Gomes, Caleb Geniesse, Aneesh Pappu, Karl Leswing, and Vijay Pande. Moleculenet: benchmark for molecular machine learning. Chemical science, 9()):513-530, 2018.
Zhirong Wu, Shuran Song, Aditya Khosla, Fisher Yu, Linguang Zhang, Xiaoou Tang, and Jianxiong Xiao. 3d shapenets: A deep representation for volumetric shapes. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1912-1920, 2015.
Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? In International Conference on Learning Representations, 2018a.
Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? arXiv preprint arXiv: 2018b
Jiaxuan You, Jonathan M Gomes-Selman Rex Ying, and Jure Leskovec. Identity-aware graph neural networks. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pp. 10737-10745, 2021.
14

---

Published as a conference paper at ICLR 2025
Bohang Zhang, Guhao Feng, Yiheng Du, Di He, and Liwei Wang. A complete expressiveness hierarchy for subgraph gnns via subgraph ceisfeiler-ehmmal tests. arXiv preprint arXiv:2302.07099, 2023.
Muhan Zhang and Pan Li. Nested graph neural networks. Advances in Neural Information Processing Systems, 34:15734-15747, 2021.
Lingxiao Zhao, Wei Jin, Leman Akoglu, and Neil Shah. From stars to subgraphs: Uplifting any gnn with local structure awareness. In International Conference on Learning Representations, 2021.
15

---

Published as a conference paper at ICLR 2025
A EXTENDED DISCUSSION OF RELATED WORK
Invaraint geometric models To leverage the rich 3D geometric information contained in point clouds in permutation- and B(3)-invariant manner, early work Schüit al. (2018) integrated 3D Eu- clidean distance into the MPNN framework (Gilmer al. 2017) and aggregate geometric information iteratively. Nevertheless, this model exhibits a restricted capacity to distinguish isomooopip point clouds fully-connected graphs substantiated by et al. (2024); Pooddnyaky & Ceriotti (2022) Pozdnyakov et al. [2020)] Hordan et al. (2024b). Subsequent invariant models (Gasteiger
al. 2019 | [021 [022 Liu al. 2021 Wang al. 2022 Li al. 2024) have endeavored to
enhance their geometric expressiveness through the incorporation of carefully designed high-order
geometric features (Gasteiger et al. 2019 2020 2021
2022), adopting local spherical representa-
tions (Wang et al. 2022) or nigher-order distance features (Li et al. 2024). These designs have greatly improved the models' performance on downstream tasks, however, most of them lack theoretical guarantees of geometric completeness over the whole point cloud spaces.
B PROOF OF MAIN CONCLUSIONS IN MAIN BODY
B.1 MEASURE OF SYMMETRIC AND UNIDENTIFIABLE POINT CLOUDS
In this subsection, we aim to prove that the Lebesgue measure on Igax3 of the C-symmetric point cloud set, the D-symmetric point cloud set, and the unidentifiable point cloud set are all zero.
Theorem 4.3 (Unidentifiable set of DisGNN has measure zero) The Lebesgue measure on Rex3 of the C- C-symmetric, D-symmetric, and unidentifiable point cloud sets is zero.
We first show that the C-symmetric point cloud set has measure zero. We denote the set containing all C-symmetric point clouds as EC Note that
pax3 C
n super e == (P E 1R
3i ji E [n]] such that condi,j holds),
where
cond.,j == llp mcan, (px) ll - mean
which is essentially a non-trivial polynomial equality constraint. Here, mean: calculates the geometric center of the point cloud, and ||p: mean; (px) is essentially the distance between and IR the geometric center. The set super e is superset of IR," x3 which can be validated as follows:
Assume some element P belongs Rax3 but not to RRx3 C- Then, we have
Vi サ E [n]. cond;.j for P does not hold.
This essentially means that all the nodes are embedded with different features after applying the algorithm. Then, is obvious that P C-asymmetric, which contradicts the assumption that P E RC Rnx3 Hence, we conclude that
R
C
[ C*
The set TR
C defines algebraic manifold with non-trivial polynomial equality constraint, thus having dimension most 3n Therefore, according to (Mityagin) 2015 Hordan et al. 2024b). the set nxx
super e has measure zero, which implies that Ry also has measure zero.
According to Proposition C.1 since RR is the largest set under consideration, it follows that both the D-symmetric point cloud set and the unidentifiable point cloud set also have measure zero.
B.2 PREPARATION RECONSTRUCTION PROOF
In the following proof, particularly focus models' ability to encode input representations without significant information loss. We provide formal definition of an important concept derive below:
16

---

Published as a conference paper at ICLR 2025
Definition B.1. (Derive) Given input representations P P. let f: O1 and g : - 02 be two property encoders defined on P. If there exists a function h: 01 - O2 such that for all P E P. h(f(P)) = (PP), we say that f can derive g. With slight abuse of notation, we say that ((P can derive g( P), denoted as f(P) + g(P)
For example, consider the space P R comprising all point clouds of size n. Let f denote an encoder that computes f(P) = 1-==1 i=1 Pt. n.). which represents the sum of the nodes' coordinates and the number of nodes, while g being an encoder calculating the geometric center of P, follows that f(P) + (P),, since we can calculate the geometric center from the sum of the nodes' coordinates and the number of nodes. In the following analysis, we consider P = _2xx by default. Notice that derivation exhibits transitivity: if fi + f2 and f2 f3. it follows that + f3.
Intuitively, "derive" is a concept describing the relation of two properties, i.e., whether property S(P) contains all the information needed to calculate g(P). Obviously, if an encoder f can embed all the information needed for reconstructing a given point cloud, then it is com)-comlet:::
Proposition B.2. Consider R being the space of all point clouds of size n. and f satisfying permutation- and E(3) iivaainvv,,, then: f(P) + P up to permutation and Euclidean isometry f is coomlllee
Proof. We first prove the reverse direction. Since f is complete, it will give distinct graph embeddingssfeatures s E Id for non-isomen isomoorpeiicc finite-size point clouds. We can thus let h : R + R3x3 be a mapping which exactly maps the graph embeddings the original point clouds up to permutation and Euclidean isometry. The existence of such h implies that f(P) + P up to permutation and Euclidean isometry.
For the forward direction, we prove by contradiction. Assume that f is not c(3)-complet,, i.e., there exists two non-isomorphic point clouds Pt, P2 such that f(Pi) = ((P).. Since S(P) - P up to per- mutation and Euclidean isometry, we have that there exists a function h that can reconstruct the point cloud from the embedding. However, since f(P)) = f(P2), their reconstruction h(f()))), (Fp))) will be the same to permutation and Euclidean isometry, which contradicts the assumption that P1 and P2 are non-isomorphic. Therefore, f E(3)-comelet..
B.3 PROOF OF THEOREM 4.2
As indicated in Figure b). all D-symmetric point clouds are also C-symmetric, which is direct conclusion from Delle Rose et al. (2023): Li et al. (2024). We formally prove here.
Proposition B.3. (D-symmelry implies C-symmetry) For any point cloud P E Rax3 if P is D-symmetric, then P is C-symmetric.
Proof. As indicated by the conclusions of Delle Rose et al. 2023): Li et al. (2024), with the "derive" notations introduced in the previous subsection, have that: 2; + I.j where P R This essentially implies that DisGNN's node-level output contains the information of the distance between nodes to the geometric center. As direct consequence, for any point cloud P. the node partition based on node features X will be no coarser than that based on C(P).
If P is D-symmetric, the center set (P(((P) will obtain only one element. By definition, the cardinality of the center sets only depends on the node features partitions, and no coarser partition will lead to no smaller center set. Therefore, (P)) will also contain only one element, and thus P is C-symmetric.
Based on Proposition B.3 to prove Theorem 4.2 it suffices to prove that point clouds that are not D-symmetric are identifiable. To prove this, we first show the ability of DisGNN to learn global geometric information:
Lemma B.4. (Locate A centers) Given a point cloud m with node features X calculated by algorithm A. Denote the D-center calculated by mass function m as e E Rnx3, With a bit notation abuse, we use diam to represent the distance between node and cm
We now run a DisGNN on (P, X.A) and denote the node i's representations at layer as h (we let (0) A. Then we have:
17

---

Published as a conference paper at ICLR 2025
(Node-center distance) Given an arbitrary mass function m, (2) can derive di,em.
(Center-center distance) Given two arbitrary mass functions my and m2- ( (2) | [m]] can derive demi,eh .c
(Counting centers) (h; (2) i [n]] can derive 1 [PP(())=
This proposition essentially extends the Barycenter Lemma in Delle Rose al. (202): Li al. (2024) to the general case of centers defined by 'D features. Such information can enable DisGNN obtain many global geometric features, facilitating it to distinguish non-isomorphic point clouds. Moreover, the ability to count centers is crucial for DisGNN to distinguish any point cloud that is not A-symmetric from A-symmetric one, thereby achieving "identifiability". Now we give the formal proof.
Proof.
Node-center distance We first denote the weighted distance profile of node as f(m, i). which is
defined as f(m, i) = =E Lje(n m(x))) )dy According to the aggregation scheme of DisGNN, f(m, i) can be derived from h (1) since h! (14
(0) 8 ((h)) ( d.y)) i E (m]B) [( d:y) | [+]
Eje(n m(a4))d; yfy when is given. All the - here hold because we assume that DisGNN uses injective function forms and parameterizations (thereby resulting in no information loss). and j E [n] since DisGNN treats point clouds fully connected distance graphs, mentioned at the beginning of the main body.
f i) can be further decomposed
f(m, i)
Σ m(a3)) dd
jE[n]
Σ m(r3))) ) 11511
jE(m]
Σ m(x7))) ) - cm + c -pil12
jE[m]
Σ m(x71) ((pp))))(ppp)))))))))) c=1 |/p, - -cm111-22 -c -c,,p;" - c(
jE(M]
M [p: cc 1
Σ m 1 1 2(p; cm Σ m(x7)(p3 cm
JEIN]
je(n]
where M -M= EjEm m(x))) and () denotes the inner product.
By definition cm EjEnn) m(x (p; - cc = 0. Therefore, we have:
f(m, M MI|p cm 112 Σ
mm= c 1
JE(m]
Therefore, the distance from to ct can be calculated as:
1
[(m,i)
1
Σ m(=f)s(m,())
=1lp: 11ne=1112 d..am
(5)
M
2M jE(m)
And since
1 + (n. ()) ((h))) dby) e [n|]]
+ (f(m f), ((m(=f). li [n]]).
(6)
finally have: ((2) di
18

---

Published as a conference paper at ICLR 2025
Center-center distance Based the conclusion of Node-center distance, at round 2, each node is aware of its distance to the two centers defined by my and m2:
(2) h
(di dm))
(7)
Therefore,
1122) > 1|pi - -c 1|2 lp: c. 112 (c c ,2p; m. m)
Therefore, the multiset all node representations round can derive the distance between the two centers:
(2)
iE
[n]]
+ ((h)) (2)
m1(21
[nt]
1 Σ mi (1 )( (-ccm + c 22; - c - m2)
M
iE/nl
m + m
2
Σ
mi (xz))p:) c. cc
M1
iE(n]
{-cmm + c 2c c )
(-c" + m2 cm cm
-11cmm c=aa
(8)
where M1 = -ie m1 (x
Counting centers In this case, we are not given any specific mass function m. It seems like we need to iterate all possible mass functions to determine the number of centers, which is infeasible due to the infinite number of mass functions. However, thanks to Proposition C.4 we can determine whether A 's cardinality is (i.e., whether is A-symmetric) by checking whether the geometric centers of all corresponding sub point clouds coincide with each other.
Now, assume that there are K types of node features in XA, and we use my to represent the mass function that maps all x; if xz is the k-tth type of features and 0 otherwise, and use e solely to represent the coordinate of geometric center (unweighted average coordinates of all points).
Based on the conclusion of center-center distance part, we know that hh
(2)
[m] can derive the E [K]]. We can easily
distance between to all cm & E [K], i.e., | (2) I [n]] + {de.c'x
check the cardinality of the set {de,em> k [K] to determine whether the geometric centers of all
these K sub point clouds coincide, and according to Proposition C.4 this is equivalent to determining
whether (Aa)(P) =
Therefore, (h?²) i [n]] + [|4=()))=1-
Notice that DisGNN can itself calculate C and D feature, therefore, it can simply take unlabeled point clouds as input, and eventually is able to learn distance information related to all centers in set Dea(P), This is concluded in the following corollary:
Corollary B.5. Given an unlabelled point cloud E R DisGNN can derive the "node-center", "center-center" and "center count" information defined in Lemma B.4 w.r. center sets A (P) after rounds, where A and k can be:
= = NULL (i.e., A assigns the same features to all nodes), k = 2. In such case, A contains only the geometric center of P.
= C, k
A = D. k's upper bound is (n + 2) (n means that we need to first ensure that DisGNN stabilizes).
19

---

Published as a conference paper at ICLR 2025
In the case of where A is NULL, the conclusion degenerates to the Barycenter Lemma in Delle Rose et al. (2023); Li et al. (2024).
We now give essential lemma. In previous work (Pozdnyakov & Ceriotti 2022 Li al. 2024). it has been shown that DisGNN is E(3) ((3)-incomplcte i.c., there exist pairs of non-iso iso-orphi point clouds that DisGNN cannot distinguish. This essentially means that cannot prove that DisGNN's output can derive the point cloud P up to permutation and Euclidean isometry. However, in the following lemma and corollary, we will show that, if we restrict the underlying space to be all point clouds that are not A-symmetric for arbitrary A algorithm, we can prove that DisGNN's output can derive the input point cloud up to permutation and Euclidean isometry. This implies that DisGNN is complete on such subsets.
Lemma B.6. Given a point cloud P E R
A with node features X calculated by algoritim A. where A A contains all A-asymmetric point clouds in g Then, 4-round DisGNN's 's output can derive P up to permutation and Euclidean isometry.
Proof. By definition of "derive", we describe the reconstruction function that takes DisGNNs' output as input and produces the point cloud P up to permutation and Euclidean isometry. We initially assume that the point cloud does not degenerate into 2D, which is trivial case and will be discussed at the end of the proof.
Since P A-asymmetric, there are at least two centers in (P). We now take two arbitrary such centers and denote them Ci and cz. According to Corollary B.5 2-round DisGNN's node- level features can derive the distance information related to these two centers, namely, k: + (di.e) dz dei,cg) In the following, we show how DisGNN can reconstruct the whole geometry based on triangular distance encoding.
By defnition of DisGNN and the injectivity assumption, its output can derive the multiset (1.0) i E [n]]. And since h24)) + h. + h (dr.es di,cr dey,es), we know that from h we can reconstruct the triangle formed by i, c1, Consequently, it is feasible to determine the following dihedral angle list from n,(0)
b/th) HASH(()) (3) ((h))) .41))dd))11 (m]
(( _nj d.j) jE E [n]]
[(h?") 11 ,dd) 1j E [n|]
(diy, dies di dj dee 1 [n]] (0i
E [n]].
Here, (2ic1c,,j110 represents the dihedral angle formed by plane i0i02 and plane jeiez. If or lies on the line 0i 02, or the angle 9 is O, we define/overwrite f as +oc.
Given this observation, we now conduct search across the multiset (n E [m]] to find the node x which can derive the smallest ((zc1e2,) yc102) relative to some other node y (In case of multiple minimal angles, arbitrary æ is chosen). Denote the minimal angle as and we record the corresponding 1) And note that since (xx1) C2, y0102) is derived from (h 13 day), the corresponding (11 , ,day) in the calculation of h can also be recorded. We now aim to prove that the entire geometry can be reconstructed using (4) and the corresponding (h. (3) ,dz)).
First of all, having found the node y in the multiset. we calculate the exact 3D coordi- nates of nodes I, Ci, 02 up to Euclidean isometry given (Nu by day). This because (3) dzy) + (day, dz.er c dy dy.e2 dz),e)). At this stage, the coordinates of 2 (h(4)) byy
nodes remain undetermined.
We proceed to traverse all nodes in the multiset { (h (1) daj) 1 E [n]] that can be derived from h
by definition. For each node j. with information (A (1) h (1) dzy), we can derive (djz; dj.cos dz.c2)- Consequently, we can determine the 3D coordinates of to at most two positions, which are symmetrically mirroring w.r.t. the plane 02, Notably, j's position is unique if and only if it lies on
20

---

Published as a conference paper at ICLR 2025
the plane 22102 (denoted as z-plane). We first identify all nodes the x--plan, since their calculated positions are unique.
Additionally, from ng we can also derive (d dj,er ds for each node j. allowing us to identify all nodes on the 3y-plane.
The coordinates of these nodes are added to the "known" set of nodes K, while the remaining nodes are labeled as "unknown."
It is established that there are no "unknown" nodes in the interior (as eeeres is minimal) and border (since we have identified these points) of the x-plane and y-plane. We denote the interior and border of the two planes as Po-
Now, we reflect the y-plane mirrorly w.r.t. the x-pllane which yields the p- p-r--lane. The interior and border formed by p-1-plane and x-plane are denoted as P-1. Since there no "unknown" nodes in Po. the remaining undetermined nodes in P-1 can be determined from ( as their another possible position calculated by 1/6') lie in the "dead area" Po. where it is ensured to be empty (no undetermined nodes left). Similarly, reflecting the z-plane w.r.t. the y y-plane produces the p1 -1-plane, and the corresponding area by pi-plane and y-plane is denoted Pt. Nodes in Pi are determined likewise from h.)) This process continues by reflecting the P1 Pi-plane w.r.t. the x-plane to obtain the p-2--lane, determining the nodes in P-2, and so forth, until all P areas collectively cover the entire 3D space. Consequently, all nodes can be determined, and the geometric structure can be reconstructed.
When the point cloud degenerates into 2D, we can initially derive a node i's representation h; from the output of DisGNN that does not lie on the line c2 By leveraging h! we extract the necessary distance information to compute the coordinates of i, c1, and 02 up to Euclidean isometry. Given that h {(h)) (8) dez) j E [n]].. we can derive (dij, djj,, djA,) from each (h d.y). Consequently, the coordinates of node can be uniquely determined (2D setting). And therefore, the complete geometry can be reconstructed up to Euclidean isometry.
As an insightful takeaway from Lemma B.6 we notice that the key in reconstruction is the initial triangular distance encoding that can be captured by DisGNN. The triangular distance encoding enhances each node i's feature with distance information (dic, dic,, derez), where e₁ and 02 are two global anchors. In Lemma [B.6] the two anchors from A center sets, and DisGNN
capture the corresponding distance encoding according to Lemma B.4 In Section 5.1] we show that we can mark node as an additional anchor, which can also be captured by DisGNN due to the distinct mark. Other designs may also be proposed by considering this insightful idea.
Similariy to Corollary B.5, DisGNN can take unlabelled point clouds as input and finally reconstruct them when the unlabelled point clouds are C-asymmetric or D-asymmetric.
Corollary B.7 Given a point cloud P E Rax3,, assume that P is A-asymmetric. Then, k-round DisGNN's 's output s can derive up to permutation and Euclidean isometry, where A and k can be:
C, ks
A = DD k's upper bound is (n + +).
Now, we are ready to prove Theorem 4.2 based on all the above propositions and theorems.
Theorem 4.2 (Asymmetric point clouds are identifiable) Let C denote the center distance encoding
and D the DisGNN encoding. Then, given an arbitrary point cloud P E xR P is C-asymmetric > P is D-asymmetric P can be identified by DisGNN.
Proof. The first + is proved in Proposition B.3
It suffices to show that P D-asymmetric + P can be identified by DisGNN.
Now, given another P' Rax3 there are two cases:
21

---

Published as a conference paper at ICLR 2025
P' is D- D-asymmetric. As direct consequence of Lemma B.6 Corollary tion B.2 DisGNN can distinguish P and P
B.7 and Proposi-
DisGNN's output for P' D-symmetric. According to Lemma B.4 and Corollary B.5
P and P' can derive [[D=(())]=1 and 1l (P))]= respectively. Since P is D-asymmetric while P' is, by definition, know that (P)] # [PP)(P)))=1 and thus DisGNN can distinguish P and Pl.
B.4 PROOF OF THEOREM 5.1
Theorem 5.1 Completemsss of GeoNGNN) When the following conditions met, GeoNGNN compleet:
Nm >= and Now >= (where indicates that the outer GNN only performs final pooling).
The distance graph is fully-connected (Feutoff = -00)
All subgraphs are the original graph (Tsub -00).
Proof. Based on the condition that r'ab oo, all subgraphs are exactly the original distance graph, with the exception that the central node (the subgraph around which is generated) is explicitly marked.
We consider the GeoNGNN with exactly inner layers and outer layers, and since we assume the injectiveness of intermediate functions, more layers will only lead to no-worse expressiveness.
Now consider node i's subgraph. We first assume that node distinct geometric center c. denote node j's representation in subgraph using hj instead of hij for brevity when the context is clear. We can now prove the following:
1. After I round of DisGNN, for all j e [m], h - di (since node i is explictly marked in its subgraph).
2. After rounds of DisGNN, according to Lemma B.4 for all j E [n], 1150) - djo.
3. After 3 rounds of DisGNN, for all E [n). ht h (2)
dic-
At this point, for all nodes j. h can derive dyis dic- and dic- thus completing the triangular distance encoding necessary for reconstruction in Lemma [.6] Therefore, similar to the proof of Lemma B.6 with another two rounds of DisGNN, the output can derive the point cloud up permutation and Euclidean isometry.
4. At round 5, the output of DisGNN the subgraph of node st Sompus (h [n] can derive the point cloud up to permutation and Euclidean isometry.
j
There is still potential problem with the above proof: node may coincide with the geometric center. In such case, we can not anymore obtain triangle distance information formed by each node j and c, which is essential in Lemma B.6 reconstruction. However, since P has more than 2 nodes, there is at least one node that satisfics the assumption.
Notice that GcoNGNN injectively pools all the subgraph representations in the outer GNN and obtains the final output s = foweer s E [ri]] H.. Each subgraph represnetation 8; can derive the distence between node i and the geometric center according to Lemma B.4 This can be leveraged as an indicator, telling us whether node coincides with the As consequence, all subgraph representations s; to find the one that does not coincide with the geometric center, then reconstruct the point cloud based on this subgraph representation according Lemma B.6 This finishes the proof.
following: s; - h - h,
h (2) dic-
derive the point cloud from follows: first, search across
22

---

Published as a conference paper at ICLR 2025
B.5 PROOF OF THEOREM 5.3
Theorem 5.3 (Completenes: for general geometric subgraph GNNs) Under the conditions specified
in Theorem 5.1 all general geomeiric subgraph GNNs Definition 5.2 with at least one local aggregation are E(3)-complete.
Proof. We establish by demonstrating that all general geometric subgraph GNNs GeoA(A, Pool, L, Tuub +o0, Teutoff +00) (Please Appendix E for formal defini- tions), abbreviated as GeoA hereafter, are capable of implementing (Frasca al. 2022) GeoNGNN (with Ninner inner layers, Tsub +oc, Teutoff too and without outer GNN) denoted complete GeoNGNN - with a fixed number of layers Since complete GeoNGNN E(3 ccmmplee when Ninner 2 5, GeoA can also achieve compleeeesssss when L is larger than some constant.
Two scenarios considered: when Geoaggu A and when Geoagg A.
In the scenario where Geoagg, A, single aggregation layer in GeoA implement complete that only maintains
GeoNGNN layer by learning an aggregation function (()) defined in Section E Geoaggu, P and Geoagg, while disregarding other aggregation operations.
Notably, complete GeoNGNN utilizes VS pooling, while Pool in GeoA encompasses the options of VS SV pooling. If Pool is VS, then consequently GeoA with L. Numer has the capability to implement complete GeoNGNN.
On the other hand, should Pool be SV, GeoA can deploy additional aggregation layer in conjunction with SV pooling to implement VS pooling. In the following, we mainly elaborate on this simulation process.
First, it is crucial to note that SV pooling learns the global representation hgg through the function pSV follows:
hgg = S ( (((LW))
VoB Val),
and VS pooling learns through yvS in the subsequent manner:
hag / ( (1 I Vo Vo
With Lsy Lvs 1, the last aggregation layer be utilized accumulate all subgraph information to the central node representation, i.e., & (Tssv) (u, u) = A ((nsss) u) + (1 E VG B.. given that Geoaggu AA. Notice that this is feasible since each subgraph is the original graph and is fully connected. Consequently:
$ ((((Ls))
Vo / Vc 1 Val
+ f (((tt)))
e Vol u V&
| (((es))
u Vcl.
indicating that fSV can acquire a function that initially converts if ( E Vo v Vo to I. Hhur v Vo u Vo and subsequently emulates fvs Thus, GeoA with L = Ninner + 1 has the capability to implement complete GeoNGNN.
Finally, consider the case where Geoagg!. & A. And since we assume the existence at least one local operation, follows that Geoagg, E A. By presenting proposition that essentially underscores symmetry, we can affirm that this case is equivalent to the first case, and therefore the conclusion still holds.
Proposition B.8. Let GeoA be any general geometric subgraph GNN defined in Section P Denote Aue as the aggregation scheme obtained from A by exchanging the element agg with agg exchanging Geoagg!. with Geoagg and exchanging agg with agg Then, GeoA(A, VS, Tub rawoff) and GeoA(Auuey ,S,, VV L, suby Teurof) implement each other.
Proof. Similar to the original proof of Proposition 4.5. in Zhang et al. (202)). the proof is almost trivial by symmetry: All functions within GeoA(A, VS) inherently lear ensure that (1)))
23

---

Published as a conference paper at ICLR 2025
in GeoA(A, VS) precisely corresponds to hew in GeoA(AutH, SV) for any arbitrary u, u,v,d, with the reverse equivalence also holding. Additionally, given the symmetry of the pooling method, consistency in the oulput can be guaranteed.
As direct consequence of Proposition B.8 all of the general subgraph GNN with Geoagg can implement another equivalent one with Geoaggu- and according to the previous proof, they are also E(3)-complete under the conditions specified in Theorem. This ends the proof.
B.6 PROOF OF THEOREM 5.4
Theorem 5.4 (E Completeness of DimeNet, SphereNet, GemNet) When the following conditions
met, DimeNet, SphereNet and GemNet are E(3)-complete.
The aggregation layer number is larger than some constant C' (irrelevant to the node number).
They initialize and update all edge representations, i.e., Tembed +00.
They interact with all neighbors, i.e., Tin +
The main idea to prove this theorem is to also show that DimeNet, SphereNet and GemNet can implement (Frasca et al. 2022) GeoNGNN, which is E(3)-complete, and thus they are E(3)-complete well. We elaborate these models in separate subsections.
B.6.1 E(3)-COMPLETENESS OF DIMENET
To begin, let us abstract the functions utilized in DimeNet. Fundamentally, DimeNet employs edge representations, iteratively updating these representations based on the neighbors of edges within a specified neighbor cutoff. During aggregation, DimeNet incorporates angle information formed by the center edge and its neighbor edges. Finally, all these edge representations are aggregated to produce graph-level output.
We formally state these procedures:
Initialization (DimeNet) In DimeNet, the initial representation his of the edge ij is initialized based on the tuple (i diz), where x; and æy are the features of nodes i and j, and di is the distance between these nodes:
hij
DimeNet (xi, T.j, dij)
init
Message Passing (DimeNet) The message passing in DimeNet updates the edge representation haj based on the features neighboring edges and their respective geometric information. It aggregates information from all neighbors & of node i:
hi
DDimeNet (hij, (Okis, hkg, dry) E (
uudatte
Note that hig is initialized by fusing d, and atomic information, therefore we have: hij - dij- And clearly dij, dia, dæj - Osij- Therefore, the above function is expressively equivalent to:
haj =
DimeNet (hiji Il (dkgg hys)) E ()))))
update
When the interaction cutoff of DimeNet is infinite, [n]. giving the final update function:
his =
DimeNet (hig, [l (hui, day) A [m]]]
Jupdate
(10)
Output Pooling (DimeNet) Finally, DimeNet pools over all node pairs to obtain the final represen- tation t:
oupput ( his
[m]} 1 (m())
(11)
GeoNGNN, subtype of subgraph GNN, initializes and updates the node j's represnetation in node i's subgraph based on its atomic number and its distance to node i. Representing the node j's representation in node i's subgraph as hij, then we can samely abstract its function forms.
24

---

Published as a conference paper at ICLR 2025
Initialization (GeoNGNN) GeoNGNN initializes buy based on node j's initial node feature and distance encoding w.r.t. t:
hag = GGooNGNN (Iz, dij)
init
(12)
Message Passing (GeoNGNN) GeoNGNN's inner GNN iteratively updates hig based on the following procedure:
hug
update
(hij) (hux, day) k E [m]]])
(13)
Note that k iterates all nodes, because according to Theorem 5.1 the complete version GeoNGNN has infinite subgraph size and message passing cutoff.
Output Pooling (GeoNGNN) In the absence of an outer GNN, GeoNGNN produces the scalar out- put by respectively aggregating all in-subgraph nodes' representations as the subgraph representation, and then aggregating all subgraphs' representations:
t
GGeNNGNN autput
( 1 E [n| i [m]
(14)
Implementing GeoNGNN with DimeNet Now we show how to use DimeNet to implement GeoNGNN. Since GeoNGNN is (33)ccomee,,, it can be observed that if DimeNet can successfully implement GeoNGNN, then DimeNet is also com)leemee
Let us start by examining the initialization step. Note that DDimeNet (Eq. 9 takes more information
Jinic
than init the term, DimeNet can achieve the same function form [mi
fGeuNGNN (Eq. 12 as input. Therefore, by learning a function inni
eDimeNet that simply disregards
[GeoNGNN
init
Next, we move on to the update (message passing) step. The only difference between Supaneeee (Eq. 10
Supalte and GooNNGNN (Eq. 13 lies in their input arrangements. Specifically, /DimeNet takes hgs, k [1]
as part of input, while A takes hak: & E [n] as the corresponding part of input. This index swap can be mathematically aligned: We can stack two DimeNet update layers to implement
one GeoNGNN update layer. The first DimeNet layer, starting from h calculates hij
((+5) to
store the information of h i.c., hij (1+)) + h This is feasible due to the property that in Eq 10 (dis: can be selected uniquely from the multiset by Supdate k J. Subsequently, the second DimeNet layer swaps the indices within the multiset, transforming
fDimeNot daj if and only if
(ny
$) k E [n]B] into (nl) [dde A la E [m]B). At this stage, the function (1+)) [ddg. ki
(1+
form aligns with that of &GeoNGNN (Eq 13
Supdate
Finally, we consider the output step. Similarly, the only distinction between DimeNet (Eq 11 and
GGeNNGNN (Eq 14h lies in the swapping of input indices. By stacking one DimeNet update layer and Jupdate one DimeNet output layer, we can similarly implement the GeoNGNN output layer, as discussed in the prior paragraph.
In conclusion, we have shown that DimeNet's 's layers can be utilized to implement GeoNGNN's layers. By doing so, we establish that DimeNet also B(3)-complete with a constant number of layers.
B.6.2 COM SPHERENET
We aim to demonstrate the compleettssss of SphereNet by illustrating that SphereNet can implement DimeNet, which has already been established as ((3)-ccm in the preceding subsection. We begin by abstracting the layers of SphereNet.
Initialization (SphereNet) Similar to DimeNet. SphereNet also initializes edge representations by integrating atom properties and distance information:
hij
SphereNet mit
(I,, (I; dzj)
(15)
Message Passing (SphereNet) During message passing, SphereNet aggregates neighbor edge information into the center edge, while additionally considering end nodes" embeddings and spherical
25

---

Published as a conference paper at ICLR 2025
coordinates. Since consider SphereNet", which drops the relative azimuthal angle P of end node k, the function form is highly similar to that of DimeNet:
his
I SphereNet (haj, U/; Uj &f (hya, dai, 0kiy) E [m]]}
update
Note that the original SphereNet can only be more powerful than SphereNet".
Since haj - dej. (diss dai, Ouij) dej. the above function is equivalently expressive
ESphereNet SSphereNet (huj. Ue, Uj. (haa, dkj) E [n]B)
hag
update
(16)
Output (SphereNet) The output block of SphereNet first aggregates edge representations into node representations as follows:
Dj
modee
SathereNat (1 hig | e [n]
Then, in order to calculate graph/global embedding, it further aggregates node representations as follows:
SphereNet
(u | [m]B)
graph
Therefore, the overall output function can be abstracted as:
f fuuput
SphereNet
(8 i [m]B j [n]})
(17)
Implementing DimeNet with SphereNet Now we show how to use SphereNet to implement DimeNet.
exhibit exactly the same function (Eq. 15 and DimeNet (Eq. 9 At the initialization step, Jini form.
Sphere
inni
At the update step, SppeereNet tions, than DiimeNot (Eq. 10 Therefore, by learning a function S that simply ignores node representations liy,
Supdate
(Eq. 16 takes strictly more information, i.e., the node representa-
SSphereNet
and fupineNee can share the same function form.
uupaate
.SphereNet (Eq, 17 and fomppa (Eq. share exactly the same function form. At the output step, Sourput
In conclusion, we have shown that SphereNet's layers can be utilized to implement DimeNet's 's layers.
By doing so, we establish that SphereNet is also E(3)-complete with a constant number of layers.
B.6.3 (OM)-COMPE GEMNET
GemNet(- Q) and DimeNet share the same initialization and output block. The difference between them lies in the update block. Specifically, GemNet adopts three-hop message passing to incorporate higher-order geometric information, dihedral angle. The update function can abstracted as:
h4g
Iupaaee (luj, I (hhas dej, daj, dab, Oighs Ojbar Oizba) b [n], a E [m(]]
Here, Oiyba that has subscripts represents the dihedral angle of plane ijb and jba. By learning function that simply selects all = from the multiset and ignores some terms (Note that this is feasible, since (hab, daj. Oiyb. Oiyba dej. Bybes dat) + (dey.0est. doj) - dois while only b = i results in du, 0), Jupdate can be simplified as follows:
hig =
eGem'eet (hejs & (fois, hai dy) Jupdate
[m]]]
(18)
Now, the simplified f has the same form as DiimeNet (Eq. 10 indicating that GemNet can implement DimeNet, and therefore also E(3)-complete.
C EXTENDED ANALYSIS OF DISGNN
C.I THE PROPER SUBSET RELATION IN FIGURE
Proposition C.I. (Proper Subset)
26

---

Published as a conference paper at ICLR 2025
D symmeerric point cloud set is proper subset C-symmetric point cloud set.
The unidentifiable point cloud set is proper subset of D-ssymmerrie point cloud set.
Proof. The first C relation is direct consequence of results in Delle Rose et al. (2023): Li et al. (2024), and reformulated in Proposition [B.3 The second S relation is a direct consequence of Theorem 4.2 Therefore, we only need to prove the relation for the two cases. To see this, construct a point cloud P for each case, showing that P is in the second set but not in the first set.
The first constructed P is shown in Figure
This point cloud P is C-symmetric but D-asymmetric.
(a)
(b)
(c)
Figure 3: A point cloud P that exhibits C-symmetry but not D-symmetry. (a) The point cloud P. which consists of an equilateral triangle and two additional nodes. (b) The labeled point cloud (P, (C) after calculating C features. Note that each node has the same distance to the geometric center, therefore the nodes still remain undivided. (c) The labeled point cloud (P. (P.VV XVD after calculating D features.
We describe the second
follows. Consider equilateral triangle P of arbitrary side length. For any other point cloud p' consisting of three points and non-isomorphic to P, there are only two possible scenarios: (1) P' consists of three nodes forming equilateral triangle with side length distinct from that of P; (2) P' consists
nodes arranged in a manner that does not form an equilateral triangle. Importantly, DisGNN possesses the capability to distinguish between P and P' for both cases, due to its ability to embed node numbers and all distance lengths. Consequently, DisGNN successfully identifies P (i.e., P is not in the unidentifiable set). However, it's easy to check that P is D-symmetric.
C.2 COMPARISON TO HORDAN ET AL. (2024B)
We first show that our established symmetric point cloud set, specifically the D-symmetric point cloud set, denoted as R is strictly smaller than the symmetric point cloud set Rax3 distinet
defined in Hordan et al. (2024b) (and restated in Definition C.3), as illustrated in Figure
R paxn
D-sym
Unidentifable
Cases
Figure 4: The relationship between our proposed symmetric point cloud sets and the symmetric set
R nnx3 distinct characterized in prior work (Hordan et al. 2024b). The D-symmetric point cloud set
a proper subset of pnx33 nx3 distinnt as stated in Proposition C.2
More precisely, if we denote the complement of the D-symmetric point cloud set as x the conclusion stated in Proposition C.2
not D. we arrive
Proposition C.2.
n distinet
27

---

Published as a conference paper at ICLR 2025
Proof. To establish this result, we first revisit the findings of Hordan et al. (2024b). which define an asymmetric point cloud subset Iaxa as follows:
Definition C.3 (Raxa distinet set (Hordan et al.
2024b ) We define Raxa
C R
Rax3 distinct
= {P Rax d(i, P) d(j. |n], (
where the geometric degree of defined as
d(i, P) {llp1 - pill. ..... Ilpr p:ll
is evident that Rax3 paxx
not D. since DisGNN embeds nodes based on their geometric degrees.
distinct
According to Definition
all nodes' geometric degrees are distinct, leading unique embedded
node features. Thus, these point clouds are D-asymmetric, meaning that distinct S P D
gnx3
x3
2x3 Furthermore, there exist many cases in nor that do not belong to distinet For example, see the case in Figure Other examples can be easily constructed, such as isosceles triangles where the leg length differs from the base length.
R
The key reason why Pux D contains significantly more point clouds than disttingt is that the latter imposes overly strict constraints on asymmetry, requiring all pairs of nodes to be distinct. In contrast, R D emphasizes the global asymmetry of the entire point cloud.
Eax3 3
In Theorem 2 of Hordan al, (2024b), the authors showed that given any two point clouds Pi.
Rix33 distinet: DisGNN can distinguish them. Our result is strictly stronger than that of Hordan et al. (2024b) the sense that our Theorem 4. identiies many more distinguishable pairs point clouds (by DisGNN) than their result, in the following sense:
Theorem 4.2 shows that arbitrary pairs of point clouds Pi, P2 from R D (which is strictly
larger than R pnx3
distinct as shown in Proposition C.2 can be distinguished by DisGNN.
Theorem 4.2 further establishes identifiability. meaning that for any P1 1 and any P/ from the entire point cloud set x2 DisGNN can still distinguish them.
C.3 ASSESSMENT OF UNIDENTIFIABLE CASES OF DISGNN
Here, we provide further details about the experiment in Section 6.1
first give a simple proposition, which can determine whether a given point cloud A-symmetric without the need to consider "mass" functions m. and subsequentially facilitate the noise tolerance setting.
Proposition C.4. (Equivalent definition for A-symmetry) Given arbitrary point cloud P E TR and a E(3) F(33)-ina and permutation-equivane node features in A(P), and consider K sub-point clouds each only contain nodes from P with the same node feature. Then we have: A-symmetric + all these sub-point clouds' geometric centers coincide.
algorithm A, let K denote the number of distinct
Proof. We first prove that P is A-symmetric = all these K sub-point clouds' geometric centers coincide. This is actually direct consequence of the original definition of A-symmetry: each geometric center of the K point clouds in (P). and is A-symmetric means that ( contains only one element, therefore all K sub-point clouds' geometric centers coincide.
We then prove the reverse direction. We denote Ig as the index set the nodes with the k-thh kind of node features in A(P). For arbitrary element A(P) (P)) from collections Ase(P)) calculated by
28

---

Published as a conference paper at ICLR 2025
Engrol m(a by definition, which can be decomposed as follows: 2igha)) m(x;7)
function m. have A(P)
A(P)
Eie(n m(x;))
ERE(K) Eier, m(a;));
ΣnE[K] Eier m(x+))
ΣKE(K] MaCK
(19)
ΣRE(K] Mr
where E RR denotes the geometric center of the k-th point cloud, M2 = Eier m(x;") denotes
the sum of "masses" associated with the k-th sub-point cloud. Since all c coincide, i.e., C1 = =
= CK. we have: A(P) et Ezeje M₁
ΣnE(Ky Mr = c1. Thus, all the possible elements from A (P)
coincide with e1. Obviously, since the geometric center c of the whole point cloud is also in A (P), C1 Therefore, P is A-symmetric.
Rescaling and criteria settings Since the two datasets under consideration exhibit different scales, the use of fixed tolerance emrors is inapplicable. To address this, preprocessing step is performed on both datasets by rescaling all point clouds to ensure that the distance between the geometric center and the farthest node is 1. For ModelNet40, we preprocess the data using farthest point sampling with 256 nodes. We then fix the rounding number to 2 for QM9 and for ModelNet40 (which is thus quite robust against noise in this scale), and conduct the assessment with the deviation error ranging from 1c-6 to le-1, 1, as shown in Figure Note that deviation error of le-1 is quite large for point cloud located within unit sphere. As a result, many asymmetric clouds may still be determined as symmetric under such criteria. We use this as rough upper bound only for reference.
D EXTENDED ANALYSIS OF GEONGNN
D.1 COMPLEXITY ANALYSIS
For a n-sized point cloud, without considering the complexity of achieving injective intermediate func- tions, GeoNGNN achieves theoretical completeness with an asymptotic time complexity of ( This complexity arises from the fact that there n subgraphs, each of which undergoes the complete- version DisGNN operation, resulting in an overall complexity of (nz²) per subgraph. Importantly, this time complexity is consistent with that of 2-F-DisGNN (Li et al., 2024), DimeNet (Gasteiger et al. 2019), and SphereNet (Liu et al. 202T). when their conditions for achieving E(3) (t3)lloomessssssss are met.
We leverage the findings introduced by Amir et al. (2024) to analyze the complexity involving the realization of injective neural functions Within each aggregation layer, GeoNGNN embeds the multiset {f (h d.y) j E [n]]] to update the representation of n As per the study conducted by Amir et al. 20044) an embedding dimension of O(k)) is sufficient for injectively embedding such a multiset, with & representing the embedding dimension of (h d.z). In the initial layer, (n.9) dey) possesses a constant dimension independent of n. Therefore, the sufficient embedding dimension for h0 is On)). Though it seems that the sufficient embedding dimension will grow exponentially with respect the layer number /, Hordan et al. (2024b) has demonstrated that the crucial dimension is the intrinsic dimension the multiset, which maintains (nn) throughout all layers, rather than kn, the ambient dimension. Consequently, the sufficient embedding dimension for any given layer is (()). By following the neural function form proposed by AAmir et al. (2024), we apply a shallow MLP individually to each element within the multiset and sum them up to obtain the multiset embedding.
This leads to a complexity of ((n²) ) x n O(n²), ), where ((n²) represents the complexity of forwardness of MLP whose input and output dimension are both (n)) and n represents the
'We note that for tractability, compleeompeet""sss" now should refer the model's ability distinguish hetween any pairs of non-isomorphic point clouds of size less than or equal to n.
29

---

Published as a conference paper at ICLR 2025
elements in the multiset. Considering the updating of all hi5 i,j [ni]. the complexity in each layer becomes Onn") X n² O(n²). During the final pooling stage, the nodes are initially pooled into subgraph representations, resulting in n x O(n²) x O(n*), where the first denotes subgraphs, and the last n denotes n nodes within each subgraph. Subsequently, all subgraph representations are further pooled into graph-level representation, culminating in complexity of ( where n represents n nodes. Consequently, the overall complexity amounts to O(n)).
O
D.2 THEORETICAL EXPRESSIVENESS WITH FINITE SUBGRAPII RADIUS AND DISTANCE CUTOFF
While Theorem 5.1 establishes that infinite subgraph radius and distance cutoff guarantees com- pleteness over all point clouds, this section explores how finite values can also enhance geometric expressiveness compared to DisGNN. We demonstrate this through example.
Take the left pair point clouds in Figure for example, and note that this pair of point clouds cannot be distinguished by DisGNN even when taking fully-connected point cloud input. Assuming that the subgraph radius and distance cutoff for each node are finite, only covering nodes' one-hop neighbors, illustrated in Figure 5 The representation of the green node produced by inner DisGNN will differ between the two point clouds, due to the presence of long-distance information on the left point cloud, whereas it is absent the right point cloud. Therefore, GeoNGNN with such small subgraph radius and distance cutoff can easily distinguish the two point clouds.
Figure 5: An example that illustrates the separation power of finite-uuggraph-size GeoNGNN. The green node represents the central node, while the green sphere depicts the subgraph environment surrounding the central node. The brown line signifies the distance information that will be aggregated during the message passing.
E GEOMETRIC SUBGRAPH GRAPH NEURAL NETWORKS
In this section, we provide self-contained definitions of general geometric subgraph GNNs as delineated in Definition (5.2n
5.22 as well as definitions of the geometric counterparts of well-established traditional subgraph GNNs, all of which are proven to be E(3)-complete under specific conditions.
E. BASIC DEFINITIONS
We first notice that the main definition of general geometric subgraph GNN is based on the general definitions of subgraph GNNs in traditional graph settings from Zhang al. (2023), where the input graphs are unweighted graphs. The difference is that the geometric subgraph GNN is applied to point clouds, treating it as distance graphs with distance cutoff cuutoff and applies geometric aggregations.
We separately introduce the main components in Definition 5.2 From now on, we follow the notation style of (Zhang et al. 2023), but for geometric settings.
We denote the original distance graph as = (Va, Ea), where Vo = [n] contains all nodes in the distance graph, and Eg {(u, v) weight (distance) less than Teutoft by definition of the distance graph. We denote the subgraph of node G (Van, E.-). We use No- (u) to denote the set of neighbors of in w's subgraph,
Ve. duv I Feusott) consists of all weighted edges with
30

---

Published as a conference paper at ICLR 2025
i.e., .G (u) (i i E Var, (i,u) E E We use buv denote the node embedding for node in node u's subgraph at the 1-th layer of the subgraph GNN.
Subgraph generation As defined in Definition 5.2 general geometric subgraph GNNs adopt node marking with SSub-Size ego subgraph the subgraph generation policy. This essentially means that node u's subgraph, G = (VG.. &g-) ), contains all the nodes and edges within Euclidean distance rsub: i.e., Vo- {i i Ve, diu Tuu Eg- = {(i,)) (1,5) E &g, i,j Vo-). J. And node in u's 's subgraph's initial representation is 1i = f1 Bus Fw)) where 1 uu= represents indicator function that equals when
and 0 otherwise, and Tuy By are the potential node features.
Geometric aggregation schemes As defined in Definition 5.2 geometric subgraph GNN adopts general geometric subgraph GNN layers as the basic aggregation layer, which is formally defined as follows:
Definition E.1. A general geometric subgraph GNN layer has the form
h ((+1) (((+1) op, (u, v, A ..,,..,, (u, v, (0) ,G),
where f((+1) is an arbitrary parameterized continuous function, and each atomic operation (u, v, h, G) can take any of the following expressions:
Single-point: hvar hwar or havi
Global: (how I w Vo or (hro w Vo
Local: {( dow) w E (a-(()) or If (Nwws duw) | w E Ne- (u)
Notice that h is always in the input to ensure that geometric subgraph GNN can always refine the node feature partition. agg aggw. agg aggu aggv, Geoagg., Geoagg!, respectively. operations with blue and name it with additional "Geo" prefix since all the aggregation schemes do not incorporate geometric information and are the same as those in Zhang et al. (2023). except for the local operations, which additionally incorporates distance information.
These operations are denoted as
Here, we color the local
Pooling layer After L geometric aggregation layers, geometric subgraph GNN outputs graph-level representation /(G) through two-level pooling of the collected features (1 : u, u E Vc the same as Zhang et al. (2023) do. Specifically, there are two approaches, named VS (vertex-subgraph pooling) and SV (subgraph-vertex pooling). The first approach first pools all node features in each subgraph GH to obtain the subgraph representation, i.c., h2 = ff (1 J E VaB). and then pools all subgraph representations obtain the final output hG = f (h,, | Voll). Here, fs and fo can be any parameterized function. The second approach first generates node representations
h2 = / (Krl uu | B) and then pools all these node representations to obtain the graph representation, i.e., ha = ( U Val).
We denote general geometric subgraph GNN with layers, each with aggregation scheme A BUage P and with parameters rsub; Teutoot as GeoA(A, Pool, L, Tsub, Teuoor), where Pool (VS,SV)
and
B c {agg agg agg agg agev Geoagg Geoagg!).
It is obvious that GeoNGNN without outer GNN, is specific kind of general subgraph GNN, which can be denoted as GeoA((ags Geoagg.)) VS, Nimer; Fsub, Teutott).
E.2 GEOMETRIC COUNTERPARTS OF WELL-KNOWN TRADITIONAL SUBGRAPH GNNs
In the main body, we have extended one of the simplest subgraph GNNs, NGNN Zhang & Li 2021), to geometric scenarios, and shown that GeoNGNN is already co3)llle Similar efforts can be made to other well known subgraph GNNs, including DS-GNN (Bevilacqua et al. 2021). DSS-GNN (Bevilacqua et al., 202T), GNN-AK (Zhao et al. (022I. OSAN (Qian et al. [022 and so We now give general definition of the geometric counterparts of these subgraph GNNs, and
31

---

Published as a conference paper at ICLR 2025
show that they, though does not exactly match the general forms in Definition 5.2 are also E(3)-comelete under exactly the same conditions as Theorem 5.3
Similar to Zhang et al. (2023), consider the following well-known subgraph GNNs: IDGNN Youu ctt al.
2021), DS-GNN [Bevilaqqua et al. 202T), OSAN (Qian et al, 2022), GNN-AK (Zhao et al.
2021 DSS-GNN (ESAN) (Bevilacqua al. 202Th, GNN-AK-ctx (Zhao et al. 2021). SUN (Frasca 2022). Generally, all these traditional subgraph GNNs involve local aggregations.
a manner akin to the modification detailed in Definition 5.2 to get the geometric counterpart of these models, we replace the node representations aggregated by local operations with node representations that integrate the distances between the aggregated node and the base node (i.e., the node performing the aggregation).
et al.
Notably, GNN-AK, GNN-AK-ctx, IDGNN, OSAN, DS-GNN, and DSS-GNN all employ base GNNs, which can operate on graphs, subgraphs, or combination of subgraphs (as in DSS-GNN). In these cases, the proposed adjustment simply involves replacing these base GNNs with DisGNN. In certain architectures like SUN, they may have more complicated operations, such as aggregating Ihone w' E (w),, w E V& to update law.. which involve combination of global (w) and local (w') aggregation. Consistent with the modification rule, we selectively incorporate distance information solely for these local operations. Therefore, the featured geometric counterpart aggregates H. (hwwae w', ) w/ E (u),,, E Ve B instead. One also incorporate distance information into those global operations. However, they are unnecessary in terms of boosting expressiveness, as current modifications
enough for them to achieve completeness, shown in the following. We name all the geometric counterparts with a Geo prefix, such as GeoSun.
Theorem E.2. (Completeness for geometric counterparts of well-known subgraph GNNs) Under the node marking with size ego subgraph policy, when the conditions described in Theorem 5.3 are met, GeoIDGNN, GeoDS-GNN, GeoOSAN, GeoGNN-AK, GeoDSS-GNN (ESAN), GeoGNN-AK-etx, GeoSUN are all Bo3)lleeee
Proof.
Among these models, GeolDGNN, GeoDS-GNN, GeoOSAN fall under the general definition of geometric subgraph GNNs (see Definition (52). and they all incorporate at least one local operation, therefore they are inherently complete given these conditions according to Theorem 5.3 (See proof in Appendix B.5).
Under node marking with original graph (r oo) policy, the other models can all be well abstracted and summarized, which have been done by Zhang et al. (202)). We list here for convenience (non-blue parts represent the geometric modification):
GeoGNN-AK (Zhao et al. 2021
f( (h(), h(9)
if
(1(()) wow drw) w (o)))))
((+1))
10 (1())
((h()) dow) : w w (o(t)), if==
( w E Vall
It adopts VS pooling. GeoGNN-AK-ctx (Zhao et al. 2021). The GNN aggregation scheme can be written
(()) (n()) an:
if + u,
(h uue dow) : w w No(v)))
1. (1+1)
f() (1())
H (h( dow) w (N()))) i
{ (h(() E Vol.
(h E Voll)
32

---

Published as a conference paper at ICLR 2025
It adopts VS pooling. GeoDSS-GNI Bevilacqua et al. 2021). written as
The aggregation scheme of DSS-GNN can be
N (1 HI (h drw) No(o)
(h w e Vc).
(( dwro) : w E Va, w' e (N())))
It adopts VS pooling.
GeoSUN (Frasca et al.
2022
The WL aggregation scheme can be written
hag
(1+1)
f0)
(h (1) No0 H (h dmw) (N)(
th we VcH.
( : :.:ee Vc).
(1 w,w dwr)) : w E Vo, w' (N())))
It can adopt VS or SV pooling.
Regarding these models, it observed that they all exhibit the local operation Geoagg thus enabling them implement the GeoNGNN inner layer with single aggregation layer, demonstrated in the proof of Theorem 5.3 Moreover, irrespective of whether they employ SV or VS pooling, they are all capable of implementing VS pooling, which is adopted by GeoNGNN, as evidenced in the proof of Theorem 5.3 Hence, under the given conditions, these models can also implement GeoNGNN under the given conditions and consequently achieve completeness, as GeoNGNN itself is complete.
F EXTENDED EVALUATION OF GEONGNN
In this section, further evaluate GeoNGNN on various datasets, including rMD17 (Chmicla ctt al., 2017). which assesses the model's 's ability to predict high-precision energy and forces based on molecular conformations; MD22 (Chmiela al. 2023), which requires models to efficiently handle large point clouds; 3BPA (Kovács et al.) 2021), which assesses the model's ability to generalize well on out-of-domain datasets; and QM9 (Ramakrishnan al. 2014 Wu al. 2018), which contains diverse properties for evaluating the models' universal learning capabilities. Please refer Appendix
G for further model architecture and experimental settings.
Since GeoNGNN is built upon DisGNN. direct comparison is made to DisGNN, which we implement as an enhanced SchNet (Schütt al., 2018) with additional residual layers, Additionally. we include two other complete models: DimeNet Gasttegger et al. 2019) and GemNet (Gasteiger et al. 202T), as well as the recent advanced invariant model 2-F-DisGNN N (Li et al. 2024). Furthermore, we incorporate equivariant models that leverage vectors or higher-order tensors, including PaiNN (Schütt
ct al. 2021), NequlP (Batzner et al. 2022). ) and MACE (Batatia al. 2022).
Overall, GeoNGNN demonstrates good experimental results, and in some cases, performs comparably to advanced models that incorporate equivariant high-order tensors, such as Allegro (Musaelian et al.) 2023). MACE [Batatia et al.) 2022), and PaiNN (Schütt al. 2021). specific tasks. This validates the effectiveness of capturing subgraph representations in molecular- provide even better inductive bias generalization ability in this scenario than the design like directional message passing in DimeNet (Gasteiger et al.) 2019). which is also complete our proof yet did not catch up with GeoNGNN's downstream performance.
tasks, which may
However, it does not consistently achieve the best performance, which may be attributed to its simple design, relying solely distance features and the weak base GNN it employs. Therefore, future
33

---

Published as a conference paper at ICLR 2025
Table 2: MAE loss on revised MD17. Energy (E) is in kcal/mol, and force (F) is in kcal/mol/A., The best and second-best results are shown in bold and underline. Results for GeoNGNN are highlighted in green if they outperform their base model, DisGNN, while models are grayed if they are our
characterized powerful models. The average rank is computed as the mean rank across all rows.
Equivariant models
Invariant models
Molecule
Target
PadNN
NequlP
MACE
DisGNN
2isDis.
DimeNet
GemNet
GeoNGNN
E
0.1591
0.0530
0.0507
0.1565
0.0465
0.1321
0,0502
Aspirin
F
0.3713
0.1891
0.1522
0.4855
0.1515
0.3549
0.2191
0.1720
E
0.0161
0.0277
0.2312
0.0315
0.1063
0.0315
Azobenzene
F
0.0669
0.0692
0,5050
0.1121
0.2174
0.1157
E
0.0009
0.0092
0.0308
0.0013
0.0061
0.0014
Benzene
0.0069
0.0069
0.2209
0.0085
0.0170
0.0115
0.0065
0.0623
0,0092
0,0092
0.0117
0.0065
0.0345
0.0074
Ethanol
0.2306
0,0646
0.0484
0.0774
0.0379
0.1859
0.0830
0.0482
0.0899
0.0184
0.0184
0.2814
0.0129
0.0507
0.0143
Malonaldehyde
0.3182
0.1176
0.0945
0.1661
0.0782
0.2743
0.1522
0.0875
E
1.76
0.0208
0.0115
0.1269
0.0103
0.0445
0.0069
Naphthalene
F
0.0830
0.0300
0.0369
0,4144
0.0478
0.15005
0.0438
0.0377
E
0.0323
0.0300
0.1534
0.0310
0.1176
0.0352
Paracetamol
F
0.1361
0.1107
0,4698
0.1178
0.3028
0.1385
E 0.1130
0.0161
0.0208
0.0791
0.0174
0.0590
0.0168
Salicylic acid
F 0.2099
0.0922
0.0715
0,3481
0.0860
0,2428
0.1222
0.0881
E
0.0969
0.0069
0.0115
0.0918
0.0051
0.0228
0.0069
Toluene
0.15
0.0369
0.0346
0.3070
0.0284
0. 0.1085
0.0507
0.0375
0,1038
0.0092
0,0115
0.0363
0.0139
0.0338
0.0096
Uracil
0.1407
0.0715
0.0484
0.1973
0.0828
0.1634
0.0876
0.0577
AVG RANK
6.71
2.80
2.50
6.55
2.25
5.70
5.00
2.55
improvements could focus on integrating vectors (Schütt et al. 2021 Thölke & De Fabritiis 2021)
higher-order tensors (Thomas et al. 2018) into GeoNGNN's ' base model, as these have been shown to enhance model generalization (Du et al. 2024 Musaelian et al. 2023).
Nevertheless, GeoNGNN shows promise, leaving open opportunities for future research by incorporat- ing more powerful base architectures within its nested framework to improve empirical performance, or developing theoretically more powerful models analogous to its underlying principles.
F.1 MOLECULE STRUCTURE LEARNING: REVISED MD17
We first evaluate on the rMD17 dataset (Chmiela et al. 2017), which poses a substantial challenge to models' geometric learning ability. This dataset encompasses trajectories from molecular dynamics simulations of several small molecules, and the objective is to predict the energy and atomic forces of a given molecule conformation, which contains all atoms' positions and atomic numbers.
The comprehensive results and comparison is shown in Table There are several important observa- tions: 1) GeoNGNN exhibits substantial improvements over DisGNN, demonstrating the effectiveness of higher expressiveness and the significance of capturing subgraph representations on practical tasks. 2) Though equally being E(3 E(3)-complete, with practical implements, DimeNet and GemNet show worse performance than GeoNGNN. This finding suggests that different inductive biases of different model designs can impact significantly on generalization ability, while learning more refined subgraph representations like GeoNGNN may be superior to directional message passing in certain molecule-relevant tasks. 3) GeoNGNN shows competitive performance in comparison to other well-designed advanced models. However, it does not consistently achieve the best performance.
F.2 SCALING TO LARGE GRAPHS: MD22
Geometric models face challenges in scaling to larger point clouds. Here, we evaluate GeoNGNN
MD22 (Chmiela et al., 2023). ) a dataset of molecules with up to 370 atoms. To enhance efficiency, we apply GeoNGNN with a finite subgraph cutoff rsub (5 Å) and message passing cutoff remoor (5 Å), and assess its practical performance.
In this table, we further include a recent SOTA model VisNet-LSRM (Li al. 2023 As shown in Table GeoNGNN consistently outperforms DisGNN across all targets. It also delivers competitive
34

---

Published as a conference paper at ICLR 2025
Table 3: MAE loss on MD22. Energy (E) in me force (F) in meV/Á. The best and the second best results are shown bold and underline. color the cell if GeoNGNN outperforms DisGNN. The average rank is the average of the rank of each row.
Equisarant models
Imvarant: modes
Mol
Targel
XGDML
PaiNN
ToN
Allegro
Equitormer
MACE
VisNer-LSR LSRM
DIONN
GeoNGNN
Decoschesacnnti
0.089
3.2
2.5
2.6
11.3
Stachyose
87
0,376
0.009
0.124
0.078
0.002
0.050
0U077
0455
10.1
8.3
4.2
3.0
3.8
1.3
4.1
AT-T
60
0.52
0.121
0.139
0,103
0.109
0,079
0,060
0.109
0.078
30
10.3
8.5
4,1
4.3
4.3
3,4
8.221
5.0
AT-AT-CG-CG
118
0.52
0.097
0,191
0,145
0.055
0,058
0,040
0.098
0.081
16.0
Backyfooll cancher
148
Double-waike
RANK
7.57
6.30
6.30
0.80
3.50
1.80
3.50
results compared other SOTA models, including MACE and VisNet-LSRM, ranking 2nd average. These results demonstrate the effectiveness of subgraph enhancement in practical GeoNGNN with finite configurations.
To explore how subgraph size affects experimental performance, we selected three molecules of different representative scales and observed the effects, as depicted Figure As anticipated. performance generally improves with increased subgraph size, roughly matching the improved theoretical expressiveness. However, the rule does not always hold: there are cases where increasing the subgraph size leads to degraded performance, which can be attributed to the potential noise and redundant information. It implies that practically, there could exist a conflict between theoretical expressiveness and experimental performance, and striking a balance between them is important.
0.25
9
Ac
AT
AT
Energy Error
0.20
Bu
Force Error
8
Bu
7
0.15
0.10
5
2.0
2.5
3.0
3.5
4.0
4.5
5.0
2.0
2.5
3.0
3.5
4.0
4.5
5.0
Subgraph Size
Subgraph Size
Figure 6: Effect of subgraph size for energy and force prediction Ac-Ala3 AT-AT and Buckyball catcher in MD22. Subgraph size in A, energy (E) in V/atom, force (F) in V/À.
We further evaluated the inference time and GPU memory consumption of GeoNGNN and several nigh-performance models. As shown in Table GeoNGNN is more than 2x faster than MACE and more than 8x faster than Equiformer, which both utilize high-order equivariant representations and perform tensor products. Additionally, GeoNGNN demonstrates efficient memory usage, particularly when compared Equiformer. This could be attributed the invariant representation and the simple architecture GeoNGNN adopts.
F.3 GENERALIZATION ON OUT-OF-DOMANE DATA: 3BPA
GeoNGNN has demonstrated remarkable expressiveness. However, higher expressiveness only implies better capability to fit data, and whether this leads to improved inductive bias and general- ization power remains uncertain. Therefore, we conducted further evaluations of GeoNGNN on the 3BPA dataset (Kovács al. 2021), which includes lots of out-of-domain data in its test set, providing
35

---

Published as a conference paper at ICLR 2025
Table 4: Efficiency Analysis of Models. The batch size for all molecules is set 16, except for Bu and Do where it is 4. Models marked with an asterisk (*) utilize a batch size of 8, except for Bu and Do, which use batch size of 2. The evaluation conducted on Nvidia A100.
Molecule
atoms
MACE
Equiformer*
GeoNGNN
0.166
0.305
0,070
Ac-Ala--HHMe
42
13400
38895
12997
0.209
0.397
0.088
Docosahexaenoic acid
17875
51565
16980
0.388
OOM
0.161
Stachyose
87
33146
OOM
32430
0.195
0.361
0.083
AT-AT
60
17310
50032
15864
0.437
OOM
0.175
AT-AT-CG-CG
118
38090
OOM
34951
0.183
0.074
Buckyball catcher
148
14466
47702
13891
0.545
OOM
0.180
Double-walles nanotube
370
44923
OOM
36849
good assessment of the generalization power of geometric models. The dataset comprises molecular dynamic simulations of the molecule (arpllxx))yrriii-2222 with the training set consisting of samples from the trajectory 300K, and the test sets containing samples from trajectories at 300K, 600K, and 1200K.
Table 5: Root-mean- Root-mean-square (RMSE) 3BPA. Energy (E) in meV, force (F) in m/V/Á.
Standard deviations are computed over three runs.
NequlP
Allegro
MACE
GeoNGNN
E
3.1=0.1
3.8420.08
320.2
3.76::0.29
300K
F
11.310.2
12.9810.77
8.810.3
11.7711.34
E
11.3-0.31
12.07+0.45
9.7+0.5
12.49+0.38
600K
F
27.3+0.3
29.1740.22
21.8+0.6
28.91+0.71
E
40.8+11.3
42.57+1.46
29.8+1100
44.82-10.8
1200K
86.411.5
82.9611.77
62.10.7
89.62.11.58
Consistent with prior studies, we present the Root Mean Square Error for energy and force pre- dictions, comparing GeoNGNN with MACE (Batatia et al. (022), NequlP (Batzner et al., 2022). and Allegro (Musaelian al. 2023). The results are detailed in Table GeoNGNN demonstrates competitiveness with Allegro, an equivariant model equipped with high-order tensors for learning local equivariant representations. However, when compared to the SOTA method MACE, GeoNGNN exhibits relatively worse performance, especially under higher temperatures, which signifies greater distribution shift between the testing and training sets. This underscores that despite GeoNGNN's strong theoretical expressiveness, its generalization power may lag behind that of those well-designed cquivariant models. As stated in the main text Section 6, this discrepancy could potentially be attributed the fact that GeoNGNN only adopts invariant representations, whereas equivariant representations could offer superior generalization abilities, particularly in sparse graph cases, as demonstrated in Du al. 2024).
F.4 HANDLING VARIOUS PROPERTY PREDICTIONS: QM9
Finally, we evaluate GeoNGNN on QM9 RRamakrishman et al. 2014: Wu et al. 2018), which contains
approximately 13k well-annotated small molecules, each paired with 12 properties predict. The results, shown in Table 6 indicate that GeoNGNN performs particularly well in energy-related predictions, such Ug and H.
36

---

Published as a conference paper at ICLR 2025
Table 6: MAE loss on QM9. The best and the second best results are shown bold and underline. We color the cell if GeoNGNN outperforms DisGNN. GeoNGNN demonstrates competitive performance
across all models and ranks first in terms of average ranking.
laget
Unit
SchNet
PhysNet
ComENet
SphereNet
2F-Dis.
FaiNN
Torchmd
DisGNN
GeeNGNN
0.033
0.053
0.025
0.025
0.010
0.012
0.002
0.015
0.012
0.235
0.062
0.045
0,045
0.043
0.045
0.010
0.057
0.044
CHOMO eporo A
m
41.0
32.9
22.3
21.8
21.2
34.0
23.8
meV
34.0
24.7
19.8
21.2
20.4
17.8
28.3
21.5
W
63.0
42.5
32.4
31.1
31.3
45.7
38.0
45.2
33.3
(R')
d6
0.073
0.765
0.259
0.268
0,030
0.066
0.015
0.160
0.046
ZPVE
meV
1.7
1.39
1.20
1.12
1.26
1.28
2.12
1.88
1.4
Up
WeV
14
8.15
6.59
6.26
7.33
5.85
6.24
9.60
5,29
8.34
7.37
6.30
9.88
5.35
H
8.42
6.86
6.33
7.36
5.98
6.48
9.82
5.35
meV
9,40
7.98
7.78
8.56
7.35
7.64
11.06
7.04
/al/mol/K
0,033
0.028
0,024
0.022
0.023
0.024
0.026
0.031
0.023
AVG RANK
8.42
7.25
4.58
3,42
3.67
3.92
3.17
7.42
2.92
G EXPERIMENT SETTINGS
G.1 DATASET SETTINGS
Synthetic dataset
We generate the synthetic dataset by implementing the counterexamples proposed
by Li et al. 2024)
The dataset includes 10 isolated cases and 7 combinatorial cases, where each case consists of a pair of symmetric point clouds that cannot be distinguished by DisGNN. Figure 7 shows two pairs of selected counterexamples, and more counterexamples are provided Figure
(a)
(b)
Figure 7: Selected counterexamples the synthetic dataset. Only colored nodes belong to the point clouds. The grey nodes and "edges" are for visualization purposes only. (a) One isolated case. (b) One combinatorial case.
The 10 isolated cases comprise the following: 6 pairs of point clouds sampled from regular dodecahe- drons, pair from icosahedrons, 2 pairs from combination of one cube and one regular octahedron, and pair from a combination of two cubes. In the combination cases, the relative size is 1/2.
The 7 combinatorial cases are obtained through the augmentation of the 6 pairs of point clouds sampled from regular dodecahedrons and pair from icosahedrons, using the method outlined in Theorem A. Li et al. (2024). When augmenting the base point cloud, we consider the original, complementary, and all types of the base graph (Li et al. 2024), respectively.
Given that the counterexamples are designed to test the maximal expressiveness of geometric models, we configure point clouds in the dataset be fully connected.
in rMD17 is 950/50/the rest, follow- ing related works such as Batatia et al. (2022) ; Gasteiger et al, (2021). For MD22, we adopt the data split specified in Chmiela et al. (2023). which is also consistent with the other works. We train the models using a batch size of 4.
rMD17 and MD22 The data split (trininn/validaationnessss
To optimize the parameters 0 of the model fo.
weighted loss function:
0fo(X,z)
((X,2) =(1 (11-))ff((x. 1(x,2) 2)1
P
(xx(x))))
m
37

---

Published as a conference paper at ICLR 2025
Figure 8: Pairs of point clouds that cannot be distinguished by DisGNN, taken from Li al. (2024). The nodes are arranged the surfaces of regular polyhedra, only red or blue nodes are part of point clouds; the grey nodes and the "edges" are included solely for vi- sualization purposes. For those subfigures with two node colors, nodes of the same color in the left and right point clouds represent one specific pair of indistinguishable point clouds by GeoNGNN. The label "Scale cube" signifies vary arbitrarily. Each subfigure, except the
that the relative sizes of the regular polyhedra bottom-rightmoss one, represents an isolated counterexample. The bottom--rghhmoss case illustrates a combinatorial counterexample, constructed by combining multiple instances of the top-left pair in an "orgginalllcoompleeennay pattern. Comprehensive explanations of these counterexamples are provided in Li Li al. (2024).
where E R and E IR represents the n atoms' coordinates and atomic number respectively. i and F represents the molecule's energy and forces acting on atoms respectively. The force ratio is chosen 0.99 0.999.
2022), GemNet (Gasteiger et al. 202T). ) are sourced from Musaelian et al. (2023). The result of Pai NN (Schütt al. 2021) sourced
In rMD17, the experimental results of NequlP
(Batzner et al.
from Batatia et al. (2022). The results of 2-F-DisGNN (Li et al.) 2024) and MACE (Batatia et al.
2022) are reported by their original papers. We trained and evaluated DimeNet (Gasteiger al.
2019) using its implementation in Pytorch Geometric (Fey & Lenssen 2019) since the original paper
did not report results on this dataset, and we use the default parameters (We find that the default parameters produce the best performance).
In MD22, the results of sGDML and MACE (Kovacs et al. 2023) are sourced from Kovacs et al. (2023). The results of PaiNN (Schütt et al.] (021). TorchMD-Net (Thölke & De Fabritis (021)). Allegro MMusaelian al. 2023). and Equiformer (Liao Smidt 2022) are obtained from the work
presented by Li al. (2033)
38

---

Published as a conference paper at ICLR 2025
3BPA Following Batatia et al. (2022). split the training set and validation set ratio of 450/50, utilizing pre-split test sets for evaluation. We use batch size of 4, am identical objective function to that in rMD17 and MD22, and a force ratio of 0.999 were employed. The results of all other models are sourced from Batatia et al. (2022).
QM9 Following Li et al. (2024), we split the training set and validation set at a ratio 110K/10K, utilizing pre-split test sets for evaluation. We use batch size of 32. See the following for details of model prper-parameters.
G.2
MODEL SETTINGS
RBF Following previous work, we use radial basis functions (RBF) febt R - RR to expand the euclidean distance between two nodes into vector, which is shown to be beneficial for inductive bias Gasteiger et al. (2019); Li et al. (2024). We use the same expnorm RBF function as Li ct al.
(2024), defined as
(eig)) = e - (pxp((<4))
(20)
where Bk are coefficients of the 1tth basis.
DisGNN In each stage of the DisGNN, we augment its expressiveness and experimental perfor- mance by stacking dense layers residual layers DisGNN's architecture is described in Figure
rbfy
d
Emegn)
Linear
Residual
Residual
Embedding
RBF
Residual
rbf
Residual
Conv
Residual
VD-Cond
mi
Output
VD Conv
hg
Dense
Output
Vanilla DisGNN
Residual
Residual
Dense*
Residual
VD-Conv
Residual
5
Figure 9: Architecture of DisGNN. 2, represent the atomic number, euclidean distance and the target output, respectively. Each Linear block represents affine transformation with learnable parameters W. b, while the Dense* block extends the Linear block by incorporating a non-linear activation function (pre-activation). C represents Hadamard product.
We set the number of VD-Conv layers to 7 for rMD17 and for MD22. The hidden dimension is set to 512 and the dimension of radial basis functions (RBF) is set 16. The message passing cutoff is set 13Å for rMD17 and 7Ä for MD22. We employ the polynomial envelope with p 6, as proposed in Gasteiger et al. (2019). along with the corresponding cutoff.
NGNN GeoNGNN based on the high-level framework of NGNN proposed in Zhang
2021b. NGNN learns topological graphs by nesting base GNN such GCN (Kipf & Welling 2016), GIN (Xu al. 2018b), or GraphSAGE (Hamilton et al. 2017), as illustrated in Figure I0 For completeness, include formal descriptions provided in (Zhagg & Li 2021).
Formally, given a topological graph G = E), where = {1. 2......... n)] represents the set of nodes and E C V V represents the set of edges, NGNN first defines k-hop ego subgraphs G for each node e E V. Each k-hop ego subgraph G is induced by the nodes within k-hops from (including itself) and the edges connecting these nodes.
39

---

Published as a conference paper at ICLR 2025
For each rooted subgraph G NGNN applies base GNN to perform -round of meesage passing. Let v any node appearing in G Denote the hidden state of at time t in subgraph G% as h ay The initial hidden state Ka,ay is typically set the node's raw features, such embeddings of atomic numbers. When explicit node marking is adopted, which is the default in GeoNGNN, K2,0; is further fused with a unique mark embedding to distinguish the root node within its subgraph.
After -oundss of message passing, subgraph pooling operation aggregates the node embeddings ! u E B into a single subgraph representation how:
h
Rsubgraph (1bT..r
| E GED)
where Rabgrrah is the subgraph pooling function that summarizes all node-level information within the subgraph. This subgraph representation hog serves as the final representation of the root node w in the original graph.
Subsequently, an outer GNN performs 7 rounds of message passing, obtaining the hidden repre- sentation ht, of node v at step where the initial representation h. = hag- Finally, the graph-level representation la is produced:
hg = Reraph (1h!""
U (UEV)):
where Rggaph is a global pooling function that aggregates node-level embeddings into a comprehensive graph-level representation. This process enables NGNN capture hierarchical representations of the graph, leveraging both local (subgraph-level) and global (graph-level) features.
Tnner (base) GNNs
Outer GNN
Graph
Perform passing within each subgraph
Graph
Apply (subgraph pooling
Use the outputs from inner GNNs final node represenaaiins
Extract a rooted subgruph uround each mode
Apply jrrah pooling to get a graph representation
Figure 10: Illustration NGNN Zhang & Li 2021), l. taken from (Zhang & Li 2021 NGNN operates by first extracting rooted subgraphs around each node in the original graph. A base GNN with subgraph pooling layer is then applied independently each rooted subgraph to compute its representation. The resulting subgraph representation serves as the original feature for the root node in the original graph. Subsequently, an outer GNN applied to the original graph, leveraging these node features and original graph structures to extract final graph-level representations.
GeoNGNN GeoNGNN serves the geometric counterpart to NGNN, operating on point clouds rather than topological graphs in Figure 10 The high-level architecture of GeoNGNN is described in the main body. The inner DisGNN, which runs the ego subgraph of each node, is similar the DisGNN block shown in Figure However, the inner DisGNN additionally performs node marking and distance encoding for each node node i's subgraph before passing their representations to VD-Conv layers:
hij < hy @ (LPs(R @ (L.Ps(Em((1,)))
(21)
where hij represents node j's representation in i's subgraph.
The outer DisGNN additionally fuses the subgraph representation i, produced by the inner DisGNN for node i with h; obtained from the embedding layer:
h MLPs((M,.. +.]))
(22)
The hidden dimension and message passing cutoff set to the same values as those in DisGNN.
The subgraph size is set to 13 Á for rMD17, 5 À for MD22, 6 Á for 3BPA, 13 Á for QM9 and the
40

---

Published as a conference paper at ICLR 2025
maximal subgraph size is set to 25. We set (Nin as (5, 2) for rMD17, (3.3) for MD22, (6. 2) for 3BPA and (4, 1) for QM9. It is worth mentioning that the total number convolutional layers is the same as that of DisGNN. This ensures that the receptive field of the two models is roughly the same, thereby guaranteeing a relatively fair comparison.
We optimize all models using the Adam optimizer (Kingma & Ba 2014), incorporating exponential decay and plateau decay learning rate schedulers, as well as a linear learning rate warm-up. To mitigate the risk of overfitting, we employ early stopping based on validation loss and apply exponential moving average (EMA) with a decay rate of 0.99 to the model parameters during the validation and testing phases. The models are trained on Nvidia RTX 4090 and Nvidia A 100 (80GB) GPUs. For rMD17, training hours for each molecule range from 20 to 70. For MD22, the training hours ranged from 35 to 150 on Nvidia 4090 for all molecules except Stachyose (due to longer convergence time) and Double-walled nanotube (the largest molecule necessitating an A 100), which required about 210 and 70 GPU hours on the Nvidia 4090 and 80GB A A100, respectively. For 3BPA, the training hours around 10 GPU hours on Nvidia 4090, For QM9, the training hour is around 36 to 100 on the Nvidia 4090.
H GEONGNN-C: SE(3)-COMPLETE VARIANT OF GEONGNN
H.1 DEFINITION AND THEORETICAL ANALYSIS OF GEONGNN-C
In certain chemical-related scenarios, it is important to guarantee that the outputs of models exhibit invariance under permutations, transformations, and rotations of point clouds, while exhibiting differentiation when the point cloud undergoes mere reflection. This distinction is crucial as pairs of enantiomers, may demonstrate vastly contrasting properties, such as binding affinity. In this section, propose the SE(3)-complete invariant of GcoNGNN, which is capable of embracing chirality and producing complete representations for downstream tasks.
Generally speaking, GeoNGNN-C is designed by simply replacing the undirected distance in GeoNGNN with the directed distance. The formal definition is:
Definition H.1. The fundamental architecture of GeoNGNN remains the same, with the exception that the inner GNN for node k's subgraph performs the subsequent message- message-passing operation:
h ki
(1+1) Supdate ( ki K (. duaas) i E (i)))).
where f represents node i's representation at layer / in node k's subgraph, and the directed distance, dy. is defined as:
dz.ij = faist (Fign(Fa: x Faj Fack,, diy),
where e cis the geometric center of the point cloud.
As can be easily validated, directed distance is invariant under SE(3) (e(3)frransooooaation while undergoing a sign reversal after reflecting the point cloud. Consequently, GeoNGNN-C retains - while possessing the potential to discriminate enantiomers. Significantly, we show that GeoNGNN-C is SE(3)-compltte, able to distinguish all pairs of enantiomers:
Theorem H.2. (SE(3)- Conepleeeesssseesss of GeoNGNN-C) When the conditions described in Theorem S.1 are met, GeoNGNN-C is SE(3)-complete.
Proof. We first note that the proof highly similar that in the proof of Theorem 5.1
Let us consider a subgraph containing a node i, where we assume that i is distinct from the geometric center c. By imitating the proof of Theorem 5.I we can ascertain the following:
After three rounds of the Chiral DisGNN (i.e., the DisGNN that incorporates the directed distance as geometric representation described in Definition H.1 node representations h,(3) for any node j can encode the directed distances dyt. djes and dic. This enables the completion of the triangular distance encoding necessary for reconstruction in Lemma B.6
Next, employ modified version of the proof from Lemma B.6 to demonstrate that with an additional two rounds of the Chiral DisGNN, we can derive the SE(3)- and permutation permutationninvariant identifier of the point cloud P.
41

---

Published as a conference paper at ICLR 2025
In the proof of Lemma B.6 first search across all the nodes to find two nodes and b that form the minimal dihedral angles 6 (aic, bic), and then identifies the coordinates of a, b, i, up to Euclidean isometry from the all- distances D {(m, n. dmm)) m, E {a, b, i, i, }. However, since Chiral DisGNN embeds the geometry using directed distance, we now have Dj = {(mm, n, de, de, m, n E {a, b, i, c)) instead. We are able to identify the coordinates of a, b, i, c,pup SE(33)-ransformaaton (i.e., Euclidean isometry without reflection) from D, in the following way:
1. Fix 2; = (0, 0, 0).
2. Fix Ze = (dic: 0,0)
3. Fix Ta in plane xOy+ and calculate its coordinates using dia dea.
4. Calculate the coordinates of a. using dib, deb, dab, and determine whether b is on 2+ side or z- side using the signal embedded in di.
Note that the orientation relevant to SE(3) transformation is already determined this stage. We proceed with the remaining steps of the proof outlined in Lemma B.6 and reconstruct the whole geometry in a deterministic way based on the known 4-tuple abic, and therefore reconstruct the whole geometry up to permutation and (E(3)trranfoormation.
Similarly, note that such subgraphs i, where i is distinct from the geometric center c, always exist as GeoNGNN-C traverses all node subgraphs. Hence, we have established the SE(3)-Completeness of GeoNGNN-C.
H.2 EXPERIMENTAL EVALUATION OF GEONGNN-C
Datasets and tasks We evaluate the ability of GeoNGNN-C to distinguish enantiomers and learn meaningful chirality-relevant representations through three tasks on two datasets (Adams et al.) 2021 Gaiński et al.) 2023): 1) Classification of tetrahedral chiral centers as R/S. R/S offers a fundamental measure of molecular chirality, and this classification task serves as an initial evaluation of the model's capacity to discriminate enantiomers. The underlying dataset contains a total of 466K conformers 78K enantiomers with single tetrahedral chiral center. 2) Enantiomer ranking task and binding affinity prediction task. The two tasks share the same underlying dataset, while the second task requires models to predict the binding affinity value for each enantiomer directly, and the first requires the model to predict which enantiomer between the corresponding pair exhibits higher such values. Chirality is essential here because enantiomers often exhibit distinct behaviors when docking in a chiral protein pocket, thus leading to differences in binding affinity. The underlying dataset contains 335K conformers from 69K enantiomers carefully selected and labeled by Adams et al. (2021).
Model architecture and training/evaluation details In the task R/S classification, GeoNGNN- C adopts three-layer outer GNN positioned before the inner GNNs. This configuration is designed to enhance the node features first, facilitating the subsequent extraction of chirality-related properties within the inner GNNs. For the other two tasks, GeoNGNN-C consists of five inner layers and one outer layer, with the outer GNN placed after the inner GNN. The model is trained with I.1 loss. Specifically, if predicted affinity difference between two enantiomers falls below the threshold of 0.001, it considered that the model lacks the capability to discriminate between the two enantiomers.
Note that all the evaluation criteria are consistent with previous works Adams et al. (021)): Gaiñski et al. (023) for fair comparisons.
Models to compare with We compare GeoNGNN-C against previous high-performance models, SphereNet (Liu et al. 2021), ChiRo (Adams et al.) 2021), ChiENN (Gaiñski et al. 2023), and other baseline models in (Pattanaik et al. 2020 Adams et al. 2021 Gainski et al. 2023). It is worth noting that, for the last two tasks, the docking scores are labeled at the stereoisomer level, meaning that all different conformers of a given enantiomer are assigned the same label, specifically the best score, by Adams et al. (2021). As result, 2D Graph Neural Networks (GNNs) such as DMPNN+tags and models that possess invariance to conformer-level transformations, such as bond rotations, including ChiRo (Adass et al. 202T), Tetra-DMPNN (Pattanaik et al. 2020 and
42

---

Published as a conference paper at ICLR 2025
Table 7: Results on chiral-sensitive tasks compared reference models. We bold the best result among models and underline the second best ones.
R/S
Enantiomer ranking
Binding affinity
Model
Accuracy
R. Accuracy
MAE
+MPNN+ags
0.701 000003
0.285:10.001
SphereNet
0.982.10.002
0.686110.00
Tetra-DMPNN
0.935+00.01
0.690=0.000
0.324++0.02
ChiRo
0.968+0.019
0.691-00.000
0.359++0.009
ChiENN
0.989=0.000
0.760=0.002
0.275=+0.003
GeoNGNN-C
0.980.10.002
0.701
0.254:1.0000
ChiENN (Ganski et al. 2023), inherently exhibit significantly better data efficiency and usually exhibit better performance. GeoNGNN-C and SphereNet do not exhibit such invariance, and trained on 5 conformers for each enantiomer as data augmentation.
The experimental results for the three tasks are presented in Table 7 be observed that GeoNGNN-C demonstrates a competitive performance compared to SOTA methods on the R/S classification task. Importantly, despite the simple and general model design, GeoNGNN-C outperforms models specifically tailored for the last two tasks, such as ChIRo and ChiENN, which incorporates invariance to bond rotations. successfully learns the approximate invariance from only 5 conformers per enantiomer and achieves the second-best result (very close to the best one) on the enantiomer ranking task and new SOTA results on the binding affinity prediction task. These results reveal the potential of GeoNGNN-C as simple but theoretically chirality- aware expressive geometric model.
Results
43


Reviews:
[{"review_id": "8pdrh5i12Z", "reviewer": "Reviewer_jQxs", "summary": "This paper explores the geometric completeness of a significant class of geometric deep learning models: invariant neural networks. These networks leverage invariant features to impose strong inductive biases on spatial information, yet their theoretical expressive power remains somewhat unclear. This study aims to bridge that gap, enhancing both our theoretical understanding and practical application of these models.\n\nThe authors first demonstrate that incorporating distance into message-passing neural networks (like DisGNN) allows for the identification of asymmetric point clouds but struggles with highly symmetric ones. They then investigate geometric extensions of subgraph-based GNNs and prove that these models, specifically GeoNGNN, can successfully distinguish symmetric point clouds, achieving E(3)-completeness.", "questions": "Please address the concerns I raised in the weaknesses section. Additionally, I recommend revising the introduction to better reflect the paper's contributions. In my view, the primary contribution is the proposal of the geometric counterpart of NGNN and the proof that this approach effectively resolves the limitation of DisGNN in identifying **symmetric point clouds when the graphs are even fully connected**. \nThe authors should also consider relevant experiments in this direction to emphasize the novelty and significance of this work.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 4, "strengths": "- The paper attempts to address a crucial problem that enhances our understanding of the potential of invariant neural networks and can guide future model design.\n- Investigating the geometric counterparts of subgraph GNNs is a novel contribution.\n- The results extend beyond specific cases, such as asymmetric point clouds, broadening our understanding of how these models perform on symmetric point clouds as well.", "weaknesses": "- The paper lacks clarity and structure in some areas. The detailed explanation of NGNN, which serves as the backbone of their main contribution, the GeoNGNN framework, is left in the appendix. I recommend the authors integrate key aspects of NGNN, such as its core equations or an architectural diagram, into the main text. Additionally, including a comparison with the original DisGNN would be helpful—highlighting the differences and explaining what enables NGNN (intuitively) to overcome the limitations of DisGNN. This would give readers a clearer understanding of how the proposed approach builds on previous work and addresses specific challenges without disrupting the flow. Instead, a brief overview of NGNN, along with key formulas and a comparison with GNN, could benefit the flow of the whole paper.\n\n- The results are primarily constrained to cases with global connectivity, which is often impractical in real-world applications due to the significant computational costs. Several studies [1], [2], [3], [4] have explored scenarios where the graph is not fully connected, underscoring the need to evaluate the performance of invariant neural networks in sparse graph settings. In practice, invariant neural networks tend to perform worse than equivariant ones in these cases. While the authors have left these cases in the future direction, it would greatly strengthen the paper if they could extend their analysis to sparse graphs or at least discuss how their completeness results may vary with different levels of graph sparsity. Providing theoretical bounds on performance degradation as connectivity decreases would also be valuable.\n\n- The experimental results, while showing some improvement, are relatively marginal, which limits the empirical impact of the work. I suspect this might be due to the sparsity of the graphs used in practical applications. The authors should aim to demonstrate the significance of their approach by clarifying in which specific cases their method outperforms existing methods. Providing examples or scenarios where GeoNGNN has a clear advantage would strengthen the empirical contributions.\n\n\n[1] Wang, L., Liu, Y., Lin, Y., Liu, H., & Ji, S. (2022). ComENet: Towards complete and efficient message passing for 3D molecular graphs. Advances in Neural Information Processing Systems, 35, 650-664.\n[2] Joshi, C. K., Bodnar, C., Mathis, S. V., Cohen, T., & Lio, P. (2023, July). On the expressive power of geometric graph neural networks. In International Conference on Machine Learning (pp. 15330-15355). PMLR.\n[3] Wang, S. H., Hsu, Y. C., Baker, J., Bertozzi, A. L., Xin, J., & Wang, B. (2024). Rethinking the benefits of steerable features in 3D equivariant graph neural networks. In The Twelfth International Conference on Learning Representations, ICLR\n[4] Sverdlov, Y., & Dym, N. (2024). On the Expressive Power of Sparse Geometric MPNNs.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "4gLpWNWAk8", "reviewer": "Reviewer_JK57", "summary": "The paper offers the following contributions:\n1) Introduces and defines the notion of \"Identify\" for invariant GNNs, positioned between distinguishability and completeness.\n2) Provides a characterization for the incompleteness of DisGNN\n3) Proposes GeoNGNN to ensure indentification of the cases where DisGNN is incomplete\n4) Demonstrates that several established invariant GNNs are capable of completeness", "questions": "1) The excessive use of bold text and the absence of a clear outline make the paper’s contributions difficult to discern. Could the authors consider restructuring the introduction for better clarity?\n\n2) I find that the statements in the section **Theoretical Characterization vs Practical Use** rely on the example C.2. How increasing the sparsity beyond this simple example is understudied despite the authors strong claims that relaxing the fully-connected condition leads to better expressiveness of GeoNGNN compared to DisGNN.\n\n3) There is no supporting evidence for GeoNGNN over existing architectures in the primary paper. Additionally, there is no comparative analysis involving node feature information generated by a complete invariant function. Could the authors address this gap?\n\n4) A significant portion of the QM9 dataset consists of non-symmetric structures. What are the proportions of indistinguishable data restricted to the subset of QM9 that includes only symmetric structures?\n\n5) In the QM9 noise study, the significant reduction in non-distinguishable point clouds occurs near what appears to be the level of reported error in the QM9 dataset. Given the reported error of 0.1Å, how is this error rescaled based on the applied scaling coefficient? \n\n6) Distinguishing structures on QM9, which lacks conformers, does not seem to be as important as datasets which contain conformers or very nearly isomorphic point clouds.The most compelling analysis appears to come from the study of MD17 but with mixed results. GeoNGNN appears to do particularly well on Benzene which is highly symmetric. How does Benzene behave under the noise tolerance study?\n\n7) Typically, ModelNet40 is sampled to avoid handling large point clouds. It is unclear from the text whether the entire mesh or a sampled version is used. If sampled uniformly, there is no guarantee that the symmetries are preserved. Could the authors clarify this in the text?\n\n8) The selection of ModelNet40 does not seem to rigorously test the theoretical claims of the paper, which focus on nearly isomorphic point clouds. Could the authors provide more rigorous testing on datasets that better align with their theoretical focus?\n\n9) It is unclear from the text and appendix what each structure in the synthetic dataset represents, how these structures were constructed, and why they are significant. Could the authors provide more detailed explanations on the construction and relevance of these synthetic structures?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 1, "contribution": 3, "strengths": "The paper introduces a novel conceptual framework for understanding the efficacy of certain invariant architectures. This is further supported through theoretical analysis and empirical studies. Additionally, it proposes a framework for the development of future architectures extending the impact and significance of the work.", "weaknesses": "The paper lacks sufficient empirical evidence to support its theoretical analysis, significantly reducing the overall significance and impact of the work. The selected real world experiments emphasize datasets which lack conformers or nearly isomorphic point clouds. Furthermore, the main text does not provide adequate evidence to demonstrate the advantages of GeoNGNN over the existing complete invariant architectures.\n\nAdditionally, the excessive use of bold text and the absence of a clear outline in the introduction make it challenging to follow and clearly understand the contributions of the paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "tivcui3DVB", "reviewer": "Reviewer_y3SR", "summary": "The authors of the paper prove that certain families of models are not only\ninvariant with respect to the Euclidian group and permutation group, but also\nthat classes of models distinguish the orbits of $\\mathbb{R}^3$ under the\naction of $E(3)$. An extended analysis of the expressivity of DisGNN is\nprovided and it is shown that this network architecture is nearly $E(3)$\ncomplete. As a last contribution an analysis is provided for various families\nof neural networks and conditions are provided under which they are\n$E(3)$-complete. The   theoretical   results   are   verified   by experiments\non  the QM9  dataset  and  a synthetic  dataset  with designed edge cases.", "questions": "- How does this method for expressivity generalize to different types of architectures? To the reviewer it seems that this method for showing is very specific and would be difficult to generalize to other types of equivariant architectures acting on point clouds.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The authors have provided both extensive proofs as well as extensive analysis\nto their claims. Overall the presentation and intend is clear and definitions\nare well-thought out and the authors provide a good heuristic insight with each\nintroduced theorem and definition which is nice. The extensive analysis of both\nDisGNN and GeoNGNN shows that the work is of good quality and looks to be of\ngood quality to the reviewer. All theorems come with extensive proofs and with\na intuition which is helpful for the non-mathmatical audience. The quality of\nthe content, such as originality and potential impact, is harder to asses since\nthe reviewer is not familiar expressivity research.", "weaknesses": "To the reader it seems that some of the definitions are somewhat convolved and\nsome simplification and clarity in the definitions might improve reading. Some\nof the definitions, while they might be customary in the machine learning\nliterature, are somewhat unfortunately choses from a mathematical perspective.\nCompleteness of a space in the mathematical sense implies that each Cauchy\nsequence has a limit within that space. A second example is the use of the term\nisomorphism. While not wrong, a better phrasing is to say that the two point\ncloud lie in the same orbit with respect to the action of the Euclidian group\nacting on the tensor product of copies of $\\mathbb{R}^3$. The current phrasing\nmight be better if is more in line with terminology used in machine learning.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "JT111Psumj", "reviewer": "Reviewer_PLLb", "summary": "This paper studies the expressiveness power of message-passing neural networks incorporating pairwise distance between graph nodes, showing the near E(3)-completeness. Furthermore, the authors study the subgraph graph neural networks, which can achieve E(3)-completeness. Therefore, it is possible to make DimeNet, GemNet, and SphereNet to achieve E(3)-completeness.", "questions": "I have two questions.\n\n1. Do you have any insight on achieving E(3)-completeness for frame-based approaches?\n\n2. Can you comment on achieving E(3)-completeness by using node features beyond pairwise distances, e.g., dihedral angles?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The paper symmetrically studies the problem of E(3)-completeness geometric graph neural networks.", "weaknesses": "The work is based on global connectivity assumption, and this assumption significantly limits this work. Also, the experimental results seem to be quite weak.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)