Title: Universally Invariant Learning in Equivariant GNNs
Venue: NeurIPS 2025
Accepted: yes
Keywords: Equivariant Graph Neural Networks

Geometric Isomorphism

Completeness
Abstract: Equivariant Graph Neural Networks (GNNs) have demonstrated significant success across various applications. To achieve completeness---that is, the universal approximation property over the space of equivariant functions---the network must effectively capture the intricate multi-body interactions among different nodes. Prior methods attain this via deeper architectures, augmented body orders, or increased degrees of steerable features, often at high computational cost and without polynomial-time solutions. In this work, we present a theoretically grounded framework for constructing complete equivariant GNNs that is both efficient and practical. We prove that a complete equivariant GNN can be achieved through two key components: 1) a complete scalar function, referred to as the canonical form of the geometric graph; and 2) a full-rank steerable basis set. Leveraging this finding, we propose an efficient algorithm for constructing complete equivariant GNNs based on two common models: EGNN and TFN. Empirical results demonstrate that our model demonstrates superior completeness and excellent performance with only a few layers, thereby significantly reducing computational overhead while maintaining strong practical efficacy.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper considers equivariant GNNs for geometric graphs (ones where the nodes / edges also have positional information). They extend existing work towards constructing complete GNNs (ones which can approximate any equivariant function) by reframing GNNs as sums over basis sets with weights that depend on the input graph. They provide two algorithms for constructing such GNNs, one of which is slower but works for all geometric graphs, the other of which requires that four non-coplanar reference nodes can be learned by an equivariant GNN. They evaluate their system against strong baselines across 7 datasets.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The paper builds well on existing work, and situates itself well within the literature. A great example of this is L249 \"FastEGNN [41], it is assumed that non-coplanar virtual nodes can always be found, but in the discussion later, we will see that this is not the case\"\n2. The paper provides a rigorous theoretical grounding and motivation.\n3. The theories are backed up by strong empirical performance over the baselines.\n4. The appendices are very thorough.\n\nWeaknesses:\n1. The theory in the paper was hard to follow as someone not familiar with the related work. It should ideally be more self-contained. For example, L128-L129 \"as lth-degree steerable features if they are transformable using D(l)\" -> This cannot be understood within the confines of the paper. What does \"transformable using D\" mean in this context, and what is the Wigner-D matrix? Also, L146-L147 \"the basis set that expands the function space of equivariant GNNs.\" -> it is also unclear what this means within the confines of the paper.\n2. In general, the figures are not well used and do not do much to help convey the ideas in the paper. E.g. the image in Figure 2 is not helpful, the math in Figure 1 is not understandable at this point of the paper, and Figure 3 is far too small to draw anything useful from.\n3. The datasets chosen for evaluation are quite limited: there is only one real-world dataset (MD17) and they are not tested on large graphs. The authors claim that the 100-body system dataset \"tests its scalability in large systems\"; however, it is not large in comparison to the other datasets used in \"Improving Equivariant Graph Neural Networks on Large Geometric Graphs via Virtual Nodes Learning\". It is just 100 nodes, compared to their datasets Proteins (800 nodes) and Water-3D (8000 nodes). Thus it is unclear from the experiments how the method scales to larger graphs and real-world scenarios.\n\nQUESTIONS:\nQuestions:\n1. L114 -> what is C_H? It has not been defined.\n2. L155 \"assume uniform node and edge features\" -> Is this a fair assumption? This will affect permutation invariance\n3. L295-L297 \"Strictly speaking, the virtual nodes forming the tetrahedron should also be included in Algo. 4, but we omit this term as it degrades performance in our experiments.\" -> Is the theory still valid without this? Do the authors have an intuition as to why this degraded performance?\n\nMinor comments:\n1. L211 \"geometrically isomorphic if\" -> \"geometrically isomorphic if they fulfil both of the following isomorphisms\"\n\nLIMITATIONS:\nLimitations are briefly discussed in Appendix C.\nI would suggest including them in the main section of the paper.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 2, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-06-28"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper describes an advanced, theoretical grounded method for obtaining equivariant predictions from point clouds. The derivation is such that the authors can guarantee any equivariant function can be parametrized through an canonicalization procedure (complete scalar function) and a corresponding complete steerable basis set. The paper supports such claims via 5 theorems and experiments in which geometric isomorphism tests, in which the model succeeds in all cases, where other models (not using the completeness procedure) fail.\n\nSTRENGTHS AND WEAKNESSES:\n# Strengths\n\n1.\tApart from some challenges to get into this rather technical paper (see comments), the exposition of the main method (the algorithm) and the derivation of a *complete* equivariant model is clear and the results of Theorem 3.2, 3.5-8 are an important contribution to the field of equivariant deep learning. The derivation seems sound, I only thoroughly went through the proof of 3.2, but seem intuitively correct and I trust the proofs given in the appendix based on my understanding of canonicalization methods and steerable/tensor field methods.\n2.\tThe method is thoroughly validated with controlled experiments that confirm the theory.\n3.\tThe code is provided as supplementary materials. It looks clean and the paper seems as such very reproducible (I didn’t not try running the code though).\n4. The paper is quite extensive in technical details, but could still benefit from some more intuitive clarifications\n\n# Weaknesses\n1.\tAlthough theoretically very strong, and with experiments supporting the theoretical claims, the practical impact is at this stage hard to assess as the tasks other than completeness-type tests still seem a bit artificial. The paper would be more impactful if demonstrates something on atomic point clouds, where equivariant models typically make most impact and where there is a strong need for efficient models.\n2.\tAlthough generally complete in discussion of literature on equivariance (and well written!), it misses in my opinion a clear connection between the scalarization methods and tensor-product based methods. They are unified through the group convolution viewpoint (cf [Vadgama et al 2024]) in which the scalar class can be converted into the other via point-wise SO(3)-Fourier transforms and working with CG products instead of otherwise constraind linear layers. As such, the distinction into two classes is a bit arbitrary and often comes down to an implementation preference. Related is the exposition of the method relative to canonicalization methods (here categorized as frame-based methods) can be made more explicit. It seems that the presented paper is a form of canonicalization which simultaneously expands the predictions in a particular basis. Could the authors clarify how the work should be considered relative to canonicalization methods (e.g. [31])?\n3.\tThe paper has many forward references to equations and algorithms which makes the reading sometimes a bit difficult. A striking example is “Actually, we have presented an algorithm …” which confuses me and I starting to doubt myself, have I missed something, or did the authors forget to put in a citation, or a reference to a section in the paper.\n\n# Minor comments\n1.\tIn 3.2 the authors talk about an “equivariant function”. I understand how an *operator* (or a map, or layer, or network) can be equivariant, namely via eq (1). But how is a function equivariant. E.g. the text states that spherical harmonics provide a compete basis for SO(3) equivariant functions, but I am not sure if I follow. The SHs provide a complete basis for functions over S^2, but not for SO(3), that would require all Wigner-D functions. Or are we talking about a basis for linear operators (like tensor products)? I don’t think so as the next line talks about “SE(3) equivariant functions over the sphere”. But isn’t any function over the sphere (regardless of basis) equiriant via the regular reprsentation phi(n) -> phi(R.n)? Or is this function to be considered vector valued, for example a convolution kernel? (cf Lang and Weiler, 2020 “A Wigner-Eckart Theorem for group Equivariant Convolution Kernels”, https://arxiv.org/abs/2010.10952 ). I am probably misinterpreting the statements here, but perhaps this can be clarified. It would help to make precise in what sense a spherical function is called equivariant. \n2.\tThe claim of “without loss of generality” on line 154 should be further justified, I would recommend doing this in an appendix, together with a response to the following comment:\n3.\tCould you please include a detailed derivation of Example 3.1? I’m worried that something is not correct, namely that TFN is of order \\nu=2 and EGNN of order \\nu=1. Aren’t both of order \\nu=2? TFN can be written in group convolution form, and only involves pairwise interactions (albeit with l>0). Same for Schnet (albeit with l=0). EGNN is not of convolution form due to the position update, but it’s still pairwise interactions. I don’t understand how TFN is different to EGN in this regard. The tensor products used only involve two quantities. Apart from this concern, it would be nice to see where the equation comes form because it seems quite fundamental to the rest of the exposition.\n4.\tIn definition 3.3 I don’t think doublestruck M is defined.\n5.\tWhen forward referencing to the algorithms, please mention where we can fined them (in the appendix)\n\n# References\n\nVadgama et al, 2024 “Probing Equivariance and Symmetry Breaking in Convolutional Networks” https://arxiv.org/abs/2501.01999\n\nQUESTIONS:\nSee numbered comments/questions above. Thank you.\n\nLIMITATIONS:\nAre adequately addressed. Checklist is included.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a novel dynamic method for constructing complete equivariant graph neural networks (GNNs) for geometric graphs. Unlike previous methods that use infinite basis to achieve completeness, the proposed method only needs a complete scalar function (of the geometric graph) and a full rank steerable basis, which are finite. This way, the architecture doesn't need to use higher-order representations or stacking multiple layers to achieve completeness. The authors introduce an efficient implementation of the method via geometric isomorphism for asymmetric graphs, and empirically demonstrate its expressiveness and performance through synthetic and real-world datasets.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**:\n1. Novelty and Significance. The author proposes a new framework for constructing complete equivariant maps for geometric graphs, which is fundamentally different from previous approaches and potentially expands the perspective on this important question. Plus, the proposed framework is clear, rigorous, and straightforward to implement.\n2. Experiment. The experiments and results are solid, covering different tasks, a strong set of baselines and insightful analysis. And the performance improvements brought by the newly proposed method are significant.\n\n**Weakness**:\n1. Clarity. The explanation of some concepts can be improved. For instance, in Thm 3.2, I'm not quite sure how does the $F^(l)(G)$ depends on G, I didn't see definition of it previously. Also, it's better to explain what a geometric graph and equivariance for geometric graph is in the introduction section. As I mainly work with topological graphs, I was initially confused by the description of completenss of equivariant GNNs. Another example is in line 145-146, I'm not quite sure what is \"steerable function\", is it just \"equivariant function\" w.r.t. O(3)?\n\nQUESTIONS:\n1. As mentioned in weakness, can you explain the definition of \"steerable function\" in line 146, and explain the Thm 3.2 in more detail, such as what is  $F^{(l)}(G)$ , how does it depends on G, how to construct it (and $V^{(l)}(G) )?\n\n2. For simpler dataset like tetrahedron center prediction, 5-body, 100-body system, the proposed dymaic method significantly improves over the baselines, but for MD17, the improvement becomes much less, with some even underperformed the baselines. Can you explain why?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a theoretically grounded and efficient framework for constructing complete equivariant graph neural networks (GNNs). While existing methods often rely on deep architectures, high-order body interactions, or high-dimensional steerable features to achieve universal approximation capability, they typically incur high computational costs. The authors show that completeness can be achieved through two key components: (1) a complete scalar function, referred to as the canonical form of the geometric graph, and (2) a full-rank steerable basis set. Based on this theoretical analysis, they propose an efficient algorithm for building complete equivariant GNNs using EGNN and TFN as base models. Experimental results demonstrate that the proposed approach achieves superior completeness and strong performance with only a few layers, significantly reducing computational overhead while maintaining practical effectiveness.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\n- 1. This paper systematically summarizes several tensor-product modules.\n\n- 2. The paper is clearly written and well-organized.\n\n- 3. The canonical form of scalar functions may help replace certain equivariant features, offering a new perspective on model design.\n\n**Weaknesses**\n\n- 1. The theorems in this paper mainly focus on proving completeness, but lack theoretical analysis regarding expressive power, similar to Theorem 2 in [1]. I would like to know whether this canonical form can only substitute for simple (low-order) equivariant functions in practical applications.\n\n- 2. Constructing GNNs using high-quality invariant features is also a common practice. In my opinion, the authors should compare their method with existing invariant representations, such as FPFH in point cloud processing or the embedding layer of GotenNet [2].\n\n- 3. The authors could try more complex real-world systems, such as QM9.\n\n- 4. Theorem 3.5 is more like a definition.\n\n[1] Nadav Dym and Haggai Maron. On the universality of rotation equivariant point cloud networks\n\n[2] Sarp Aykent and Tian Xia. GotenNet: Rethinking Efficient 3D Equivariant Graph Neural Networks\n\nQUESTIONS:\nSee Weaknesses.\n\nLIMITATIONS:\nYES", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}