Title: E2Former: An Efficient and Equivariant Transformer with Linear-Scaling Tensor Products
Venue: NeurIPS 2025
Accepted: yes
Keywords: Equivariant Neural network

quantum chemistry
Abstract: Equivariant Graph Neural Networks (EGNNs) have demonstrated significant success in modeling microscale systems, including those in chemistry, biology and materials science. However, EGNNs face substantial computational challenges due to the high cost of constructing edge features via spherical tensor products, making them almost impractical for large-scale systems. 
To address this limitation, we introduce E2Former, an equivariant and efficient transformer architecture that incorporates a Wigner $6j$ convolution (Wigner $6j$ Conv). By shifting the computational burden from edges to nodes, Wigner $6j$ Conv reduces the complexity from $O(| \mathcal{E}|)$ to  $O(| \mathcal{V}|)$ while preserving both the model's expressive power and rotational equivariance.
We show that this approach achieves a 7x–30x speedup compared to conventional $\mathrm{SO}(3)$ convolutions. Furthermore, our empirical results demonstrate that the derived E2Former mitigates the computational challenges of existing approaches without compromising the ability to capture detailed geometric information. This development could suggest a promising direction for scalable molecular modeling.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents the Wigner 6j Convolution (Wigner 6j Conv), a spherical equivariant neural network architecture that leverages Wigner 6-j symbols to shift the computation of tensor products from edges to nodes, enhancing the computational efficiency.\n\nThe authors’ response has fully addressed all of my concerns, resulting in a logical and coherent manuscript. Given the high efficiency and broad application prospects of the proposed method (Wigner6j-conv), I recommend a **Strong Accept**. However, I encourage the authors to carefully revise the manuscript to ensure that the narrative and exposition are fully commensurate with the quality of this excellent work.\n\nSTRENGTHS AND WEAKNESSES:\n### S. 1 Interesting Motivation\nLosslessly reducing the number of tensor product calculations is a valuable research discovery.\n\n### W.1 Ambiguous Presentation\nThe presentation and mathematical writing of this paper have a lot of room for improvement, which has forced me to spend a lot of effort trying to clarify it. Here is my understanding. Please clarify it:\n1. In Def. 2.1, why  normalization factor $k_m^{(l)}$ does not have subscript $m$ when $m\\leq 0$?\n2. In Def 2.2, why does the summarization traverse superscript $l_1, l_2$?\n3. In Line 122, should it be $Y^{(1)}(\\vec r_{ij})\\propto Y^{(1)}(\\vec r_{ic})-Y^{(1)}(\\vec r_{jc})$? There is a lack of decentralization and it is not strict equality.\n4. Fig. 1(a) in this paper is fancy and difficult to understand, and needs to be modified. The orange dashed line is difficult to distinguish, why not use different colors to distinguish. The background color of the formula is not distinguishable, the font is inconsistent, and text \"xx dependent\" is not conducive to understanding.\n\nThe above are some small details that greatly affect the reading experience. If I have misunderstood them and caused errors in the evaluation, please forgive me. Next, I need to point out that there are big problems with the overall presentation logic of this article.\n1. The core of this article is to reduce the number of tensor product calculations, which is somewhat different from reducing complexity. It is recommended to adjust the description.\n2. Simple mathematics takes up a lot of space, including most of Sec. 2, and the proofs of Thms. 3.2 and 3.3, which are trivial and can be placed directly in the appendix. In particular, the authors introduce spherical harmonics in trigonometric form, which is completely unnecessary - for the binomial expansion, spherical harmonics in the form of $x,y,z$ are more helpful.\n3. As an article whose title emphasizes the new model \"E2former\", but the main body of the article hardly discusses the model - the highlight module in Appendix H should be moved forward to the main text (exchanged with the mathematical proof). In addition, the introduction of the model in Appendix H lacks a clear mathematical form, which is inappropriate and should be clearly given.\n\nFrankly speaking, it is very abrupt to give the proof of Thms. 2.2-2.3 in the main text, and the completion of these two paragraphs is out of tune with the completion of other paragraphs. The most important model design part is given in the appendix, making the argument and the statement unable to support each other - it is like an article recycled many times and lost its original coherence and character.\n\n### W.2 Missing Relevant Reference\nI suggest moving \"Related Work\" after \"Introduction\". In addition, it does not reflect the relevance to this article. I think the following changes should be included.\n1. Combine all existing introductions to invariant and equivariant models. Focus on the pros and cons of tensor products, such as strong expressiveness but time-consuming. In addition, the latest spherical-scalarization models, such as SO3KRATES [NC'24], HEGNN[NeurIPS'24] and GotenNet[ICLR'25], should be discussed. They completely abandon tensor products, which is much more efficient than the one in this paper. What are the benefits of retaining tensor products in this paper?\n2. The design of equivariant transformers should be discussed separately, such as SE3-Transformer [NeurIPS'20], SO3KRATES [NeurIPS'22], MEAN [ICLR'23], EquiformerV2 [NeurIPS'23], Geoformer [NeurIPS'23], GotenNet [ICLR'25], DPA2 [NPJ'24], eSEN [arXiv:2502.12147].\n\n### W3. Experimental Design and Results\n1. In the results of OC-22 in the main text, E2former does not have an advantage, which cannot illustrate the potential of this method. In addition, without seeing the ablation experiment, it is difficult to determine which module is effective.\n2. The results in Tab. 5 are inferior to GotenNet, and lack comparisons with other indicators, which makes people worry about the effect of the model on small-scale systems.\n3. Although the authors say that \"our model is primarily designed for larger systems\", the three experiments in the main text are not large enough to be called large-scale. It is recommended to refer to VINE-GATr [ICLR'25 MLMP], Geometric Hyena [ICML'25] for additional experiments.\n\nQUESTIONS:\nSee weakness.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 5, "CLARITY": 2, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-06-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces E2Former, an SO(3)-equivariant transformer architecture designed for molecular modelling. The primary contribution is a new method which the authors call \"Wigner 6j convolution\", a reformulation of the computationally expensive tensor products that are central to equivariant GNNs based on SO(3) convolutions. By leveraging a binomial expansion of spherical harmonics and the Wigner 6j recoupling of angular momenta, the main computational burden is shifted from being dependent on the number of edges $\\\\lvert \\\\mathcal{E} \\\\rvert$ to being dependent on the number of nodes $\\\\lvert \\\\mathcal{V} \\\\rvert$. Reducing the complexity of the core tensor product operation from $\\\\mathcal{O}(\\\\lvert \\\\mathcal{E} \\\\rvert)$ to $\\\\mathcal{O}(\\\\lvert \\\\mathcal{V} \\\\rvert)$ results in a large computational efficiency gain, because $\\\\lvert \\\\mathcal{V} \\\\rvert$ is typically much smaller than $\\\\lvert \\\\mathcal{E} \\\\rvert$ (even in sparse graphs). The authors build the E2Former architecture around this efficient convolution and demonstrate its effectiveness through extensive experiments on various benchmarks, including OC20, OC22, and SPICE. The results show that E2Former achieves accuracy competitive with or superior to state-of-the-art models while being significantly faster in terms of both training and inference time. The practical utility of the model is further demonstrated through a stable, long-time molecular dynamics simulation that accurately reproduces DFT-level vibrational spectra.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n1. The paper tackles a widely recognised bottleneck in the field of equivariant models for molecular systems, namely the computational cost of SO(3) tensor products. By providing a principled method to achieve $\\\\mathcal{O}(\\\\lvert \\\\mathcal{V} \\\\rvert)$ scaling, this work makes the core operation of many existing (and future) architectures relying on SO(3) convolutions more efficient.\n2. The core technical contribution (Wigner 6j convolution) is highly original. While the mathematical tools used (Wigner 6j symbols and angular momentum theory) are well-established, their application to reformulate and accelerate equivariant convolutions in a GNN is novel and insightful. Leveraging a \"binomial local expansion\" combined with the recoupling scheme is a clever and powerful idea and an elegant mathematical reformulation.\n3. The paper is well-written and organised. Despite the mathematical density of the topic, the authors do an excellent job of building intuition. Figure 1 provides a clear visual overview of the core idea before diving into the formal mathematics. The main text provides proof sketches that convey the key steps, while the appendix contains the full details.\n\n**Weaknesses**\n1. I might be wrong about this (in which case I apologise!), but I believe there is a rather large mistake in the manuscript (which, however, should be relatively easy to fix). The definition of spherical harmonics $Y^{(\\\\ell)}$ (Definition 2.1) seems correct to me, but the construction used to obtain the Wigner 6j convolution relies on a clever trick which seems incompatible with spherical harmonics. Rather, it seems that solid harmonics $R^{(\\\\ell)} = \\\\lVert \\\\vec{r} \\\\rVert^{\\\\ell} Y^{(\\\\ell)}$ are required to make it work. Namely, the identity $Y^{(1)}(\\\\vec{r}_{ij})=Y^{(1)}(\\\\vec{r}_i)-Y^{(1)}(\\\\vec{r}_j)$ (line 122) is crucial for the derivation, but it does not hold for spherical harmonics (it holds only for solid harmonics). The spherical harmonics $Y^{(\\\\ell)}_m$ (using the notation from the paper) for degree 1 are given by (using Definition 2.1 with Racah normalisation): $Y^{(1)}_1 = \\\\frac{x}{r}$, $Y^{(1)}\\_{-1} = \\\\frac{y}{r}$, and $Y^{(1)}_0 = \\\\frac{z}{r}$, where $r=\\\\sqrt{x^2 + y^2 + z^2}$. It is easy to see that the required linearity condition does not hold for these. However, it *does* hold for the solid harmonics $R^{(1)}_1 = x$, $R^{(1)}\\_{-1} = y$, and $R^{(1)}_0 = z$. In fact, the polynomials given for $Y^{(2)}$ in the text (lines 70 and 71) are *actually* expressions for $R^{(2)}$, because they lack the normalisation by $r^2$. I therefore believe that the authors have (unknowingly?) used solid harmonics for their implementation already (otherwise it would likely not work). The empirical results are thus (very likely) unaffected, but the authors should correct the description of their method in the revised manuscript. This also has some practical consequences for architecture design, which the authors may want to discuss, so that readers that want to use Wigner 6j convolutions are aware of them: Typical SO(3) convolutions use \"filters\" of the general form $g\\_{\\\\ell}(r)Y^{(\\\\ell)}(\\\\vec{r})$, where $g\\_{\\\\ell}(r)$ are (learnable) radial functions for the different degrees $\\\\ell$. The fact that Wigner 6j convolutions must instead use filters of the form $g\\_{\\\\ell}(r)R^{(\\\\ell)}(\\\\vec{r})$  is important in so far that the \"weight\" of \"interactions\" between nodes implicitly increases by a factor $r^{\\\\ell}$, which is undesirable for typical applications (usually, the influence of distant nodes should *decrease*). Fortunately, this should not be an issue in practice, because it can be compensated for by scaling down the radial components of filters accordingly (i.e. $g\\_{\\\\ell}(r)Y^{(\\\\ell)}(\\\\vec{r})=r^{-\\\\ell}g\\_{\\\\ell}(r)R^{(\\\\ell)}(\\\\vec{r})$),  but readers should probably be made aware of this fact.\n2. Typically, SO(3) convolutions use filters of the form $g\\_{\\\\ell}(r)Y^{(\\\\ell)}(\\\\vec{r})=r^{-\\\\ell}g\\_{\\\\ell}(r)R^{(\\\\ell)}(\\\\vec{r})$ (see above), because it makes the operation much more expressive (the radial component allows to discriminate between nodes based on distance). When this construction is used, then the whole Wigner 6j convolution *technically* still scales $\\\\mathcal{O}(\\\\lvert \\\\mathcal{E} \\\\rvert)$, because in general, $g\\_{\\\\ell}(r)$ needs to be evaluated for all $r_{ij}$. Of course, the cost of the expensive tensor product is still greatly reduced (so a large speedup is to be expected anyway), but this detail should be discussed in the text. Perhaps it would be more precise to state that only the tensor product component of the convolution is reduced to $\\\\mathcal{O}(\\\\lvert \\\\mathcal{V} \\\\rvert)$.\n3. The methodology is heavily reliant on advanced concepts from representation theory and quantum angular momentum theory (irreps, Clebsch-Gordan coefficients, Wigner 3j/6j symbols). This high barrier to entry may limit the immediate understanding and adoption of the method by members of the broader NeurIPS community who are not specialised in this subfield. This is more of an inherent characteristic of the topic than a flaw in the paper's execution, but it should still be considered. While the authors provide code for their implementation, I imagine an additional standalone script that exemplifies the core idea on a simple \"toy task\" (perhaps showing numerically that Wigner 6j convolution and ordinary SO(3) convolution are equivalent for a small point cloud) would be helpful to many readers. This way, many advanced concepts could be \"demystified\", for example, solid harmonics are just very simple polynomials in Cartesian coordinates, Clebsch-Cordan coefficients are just a tensor of certain numerical constants etc.\n\nQUESTIONS:\n1. Can the authors please comment on the point outlined in Weakness 1? In particular, if I am mistaken, I'd appreciate a detailed explanation of what I am missing. If I am right about this, I hope the authors agree with me that this error should be fixed in the manuscript.\n\n2. The paper cites \"Euclidean Fast Attention\" (Frank et al., 2024), which also achieves linear scaling for equivariant attention. Could you elaborate on the conceptual differences between the Wigner 6j convolution and the approach in that work? My understanding is that your method provides a general factorization of the tensor product itself, which could be used in any context (e.g., in a MACE-like model without attention), whereas their method is specific to linearizing the dot-product attention mechanism. Is this interpretation correct? A brief comparison in the related work section could be beneficial.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-06-13"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces the E2Former, a new spherically equivariant architecture for molecular modeling. At its core the model uses the novel Wigner 6j convolution, which is mathematically equivalent to conventional SO(3) convolutions, but scales with the number of nodes instead of the number of edges, significantly reducing computational cost. E2Former shows competitive results on several benchmark datasets and is also demonstrated to perform well in molecular dynamics simulations.\n\nSTRENGTHS AND WEAKNESSES:\n- Strengths:\n    - The paper tackles a relevant problem, namely the acceleration of equivariant GNN architectures in order to scale them to increasingly large molecular systems. The introduced Wigner 6j convolution has the potential to be used to speed up a variety of different architectures.\n    - The math behind the Wigner 6j convolution is explained in an intuitive fashion, while also being rigorously derived in the Appendix.\n- Weaknesses\n    - The discussion of baselines in the experiments section is incomplete. The SCN architecture for example is never mentioned in the text and the corresponding paper [1] is not cited.\n    - The number of baselines for the OC22 dataset is very small. It is also not clear why the authors decide to only report metrics on the validation split. Evaluating the performance also on the test split would allow to also compare against the eSCN architecture (like in [2]).\n\n[1] Zitnick, C. L et. al. Spherical channels for modeling atomic interactions. In Advances in Neural Information Processing Systems, 2022.\n\n[2] Yi-Lun Liao et. al. Improved equivariant transformer for scaling to higher-degree representations. arxiv preprint arxiv:2306.12059, 419 2023.\n\nQUESTIONS:\n- At the bottom of page 5 I am not sure why the identity reads $A \\otimes (B \\otimes C) \\cong (A \\otimes B) \\otimes^{6j} C$ shouldn’t the two sides be equal and not just isomorphic according to equation (2.4)?\nIn the following sentence the effective commutativity of the Clebsch-Gordan decomposition is invoked (leading to another isomorphism) but is this really needed here? Since the commutation happens inside a projector onto l-irreps, commutation should be possible without using effective commutativity. \nAs far as I understand the key point of the derivation is that the two sides in Theorem 3.3. are equal and not just isomorphic. So even though both isomorphisms are of course formally correct, I think it would add clarity to emphasize the equality of the expressions.\n- Could the authors include an explicit reference to [1], the original paper of the SCN baseline, which is missing so far?\n- Could the authors also provide results for their model on the OC22 S2EF-Total test split and include the results of eSCN in the table?\n\nMinor comment:\n- On page 4 in theorem 3.2., why is there a ‘k’ index on the spherical harmonic? It seems to me like it should be either an ‘l’ or a ‘u’\n\nI am willing to increase my score if the above points are adequately addressed.\n\nLIMITATIONS:\nThe authors mention limitations in the NeurIPS checklist and state that they were also discussed in section 5. However this discussion is apparently no longer present in section 5. It would be good to reintroduce the discussion of limitations into the main text.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-25"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a novel graph neural network architecture designed to address computational inefficiencies in Equivariant Graph Neural Networks (EGNNs), particularly for applications in molecular modeling. The key innovation is the introduction the concept of Wigner 6j coupling from physics, which shifts the computational focus from edges to nodes in molecular graphs, reducing the complexity from edge-dependent $O(|E|)$ to node-dependent $O(|V|)$.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1.  Methodology: The introduction of Wigner 6j convolution is new for the graph learning community, it shifts computation from edge-based to node-based operations, which is beneficial for handling large molecular systems efficiently.\n\n2. The proposed E2Former shows good experimental results on OC20 and SPICE dataset.\n\nDiscussion on Expressiveness:\nThe paper lacks a thorough discussion regarding the expressiveness of the Wigner 6j convolution. It remains unclear whether the convolution method proposed in E2Former retains the full expressive power compared to traditional full tensor product methods.\nMotivation in Context:\n\nGiven the extensive research focused on accelerating Clebsch-Gordan (CG) tensor products of spherical harmonics, the motivation for introducing E2Former may appear somewhat limited. More clarity on how this approach distinctly advances existing acceleration techniques would be beneficial.\n\nOrganizational Structure:\nThe paper’s organization can be improved for clarity. For instance, the precise definition and implications of \"linear-scaling\" should be prominently highlighted in the introduction to better guide readers through the significance and impact of the proposed approach.\n\nMissing related works on invariant neural networks with invariant polynomials:\n[1] Weisfeiler Leman for Euclidean Equivariant Machine Learning;   ICML 2024\n[2] A new perspective on building efficient and expressive 3D equivariant graph neural networks;    Neurips 2024\n\nQUESTIONS:\nPlease consult the Weaknesses for questions.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-01"}