E2Former: An Efficient and Equivariant Transformer with Linear-Scaling Tensor Products
Authors: Yunyang Li
Lin Huang
Zhihao Ding
Xinran Wei
Chu Wang
Han Yang
Zun Wang
Chang Liu
Yu Shi
Peiran Jin
Tao Qin
Mark Gerstein
Jia Zhang
Venue: NeurIPS 2025 spotlight ()
Keywords: Equivariant Neural network
quantum chemistry
Primary area: machine_learning_for_sciences
TL;DR: We propose  a linear-time  efficient  and equivariant Transformer for scalable molecular modeling.

Abstract: Equivariant Graph Neural Networks (EGNNs) have demonstrated significant success in modeling microscale systems, including those in chemistry, biology and materials science. However, EGNNs face substantial computational challenges due to the high cost of constructing edge features via spherical tensor products, making them almost impractical for large-scale systems. 
To address this limitation, we introduce E2Former, an equivariant and efficient transformer architecture that incorporates a Wigner $6j$ convolution (Wigner $6j$ Conv). By shifting the computational burden from edges to nodes, Wigner $6j$ Conv reduces the complexity from $O(| \mathcal{E}|)$ to  $O(| \mathcal{V}|)$ while preserving both the model's expressive power and rotational equivariance.
We show that this approach achieves a 7x–30x speedup compared to conventional $\mathrm{SO}(3)$ convolutions. Furthermore, our empirical results demonstrate that the derived E2Former mitigates the computational challenges of existing approaches without compromising the ability to capture detailed geometric information. This development could suggest a promising direction for scalable molecular modeling.

E2Former: An Efficient and Equivariant Transformer with Linear-Scaling Tensor Products
Yunyang Li Lin Huang"* 3- Zhihao Ding', Chu Wang" 1 Xinran Wei $1,
Han Yang". Zun Wang"+, Chang Liu Yu Shi Peiran Jin $4
Tao Qin st Mark Gerstein' Jia Zhang?*1
Yale University, *Ubiquant. 'PolyU, HHUST, 'ZGC Academy, "iicosoft Research, 'Shanghai Al Lab https://github. com wttpa/a/ococomyyyy//////ooo
Abstract
Equivariant Graph Neural Networks (EGNNs) have demonstrated significant cess in modeling microscale systems, including those chemistry, biology and materials science. However, EGNNS face substantial computational challenges due to the high cost constructing edge features via spherical tensor products, making them almost impractical for large-scale systems. To address this limitation, we introduce E2Former, an equivariant and efficient transformer architecture that incorporates a Wigner 6j convolution (Wigner 6j Conv). By shifting the com- putational burden from edges to nodes, Wigner 6j Conv reduces the complexity from O(EE) to O(D))) while preserving both the model's expressive power and rotational equivariance. We show that this approach achieves a 7x-30x speedup compared conventional SO(3) convolutions. Furthermore. our empirical results demonstrate that the derived E2Former mitigates the computational challenges of existing approaches without compromising the ability capture detailed geometric information. This development could suggest promising direction for scalable molecular modeling.
Introduction
Molecular simulations underpin critical computational tasks across chemistry 31 36 38
39),
biology (9) and materials science [64), facilitating detailed exploration microscopic processes. Although quantum mechanical approaches such Density Functional Theory (DFT) provide highly accurate predictions 330 37], their computational complexity scales poorly with system size [56). thus limiting practical applicability to small-scale problems. Machine Learning (MI. techniques have emerged as promising alternatives, balancing computational efficiency and accuracy [5] 4 16). ML-based models, particularly Equivariant Graph Neural Networks (EGNNs), significantly reduce simulation times, enabling molecular property predictions and dynamic simulations within practical computational budgets [53] 26
EGNN architectures explicitly encode symmetry constraints such as rotational and reflectional equivariances thtrouug graph based atomic representations. This symmetry-awarenee efficiency. EGNNs have evolved from rotationally invariant embedding methods like SchNet (53 to schemes incorporating bond and dihedral angles [26. 25), scalarization techniques
23
leads strong inductive biases and improved sample
and
spherical tensor-product frameworks such as E(3) and (Ten)))raaforrmmrrs [57
23
Recent
refinements, including Gaunt Tensor Product 44 eSCN convolutions
47 42
primarily focus on
enhancing computational efficiency.
1 CCofirrt authors.
Part of this work was completed while the authors were Microsoft Research.
wtw.steennloo.r.....o...oooo.o.i Corresponding authors. huang_ 6385@out1ook com, pi@gersteinlab orgg jialrs. 20gmail, com Note: JZ and L.H (Ubiquant) thank ScitiX for computing power and training infrastructure. MG is supported by the ALW professorship fund.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

In this work, we specifically focus on leverage spherical harmonics and Gordan tensor products. These models commonly referred to as spherical EGNNs-hard demonstrated state-of-the-art accuracy. especially for periodic systems where symmetry constraints are critical [59 I0] By encoding higher-order geometric corre- lations through irreducible representations (irreps) with angular momentum L > 1, spherical EGNNS offer expressive, data-cfficient models capable of capturing complex geometric interactions [57 55
EGNN architectures [57 23 which
Unfortunately, these gains come at computational cost. The use of spherical tensor products for feature construction incurs complexity driven by two factors: (i) the number of tensor products required, which scales with the number of edges |8| in the molecular graph, and (ii) the computational cost of each tensor product, which grows with the angular momentum cutoff L. Together, these lead to runtime costs of (() (((E when implemented with the sparse eSCN convolution. This scaling presents a significant bottleneck, limiting the use of spherical EGNNS to small- or medium- scale systems, despite their improved performance in principle. While recent ppherical-scalarizzn methods [52] 63 offer efficient alternatives by hypassing tensor products, they sacrifice theoretical completeness 18]. Tensor-product formulations, in contrast, preserve the full space of equivariant functions between irreps. This trade-off motivates our effort to retain the expressive power of tensor products while eliminating their prohibitive complexity.
Here, we introduce the Wigner 6j convolution (Wigner 6j Cony, Figure D. a method that uses Wigner 6j symbols [40] 45 (0), provably reducing tensor product complexity to O(VI) while maintaining the exact expressive power and rotational equivariance.
Contributions. Our contributions can be summarized as follows: (1) We introduce the Wigner 6j convolution, a from O(EE) O(V)). enabling the modeling of larger molecular systems without compromis- ing the network's 's expressive power or symmetry properties. As shown in Figure 2(b), our model demonstrates better scaling behavior than the SO(3) convolution, achieving 7x to 30x speed-up given the sparsity of the molecular graph. (2) We propose E2Former, an equivariant and efficient Transformer architecture specifically designed for scalable molecular modeling. E2Former leverages the Wigner 6j convolution to maintain rotational equivariance while significantly enhancing computational efficiency. (3) Extensive experiments on benchmark datasets like OC20, OC22, and SPICE show that E2Former achieves competitive accuracy in predicting molecular energies and forces, while offering improved efficiency and scalability over existing spherical-equivvriaat methods. (4) Finally, we pre-trained E2Former large- scale dataset and evaluated its performance in molecular dynamics simulations, where it achieves high accuracy with faster speed, outperforming state-of-the-art empirical potential methods and EGNNs. These results suggest its potential to advance large-scale molecular simulations and to serve as a foundational model for machine learning force fields.
technique that reduces the computational complexity
2 Background and Preliminaries
Notation. Throughout this paper, we use and m to denote angular momentum quantum numbers associated with spherical harmonics (() where f 2 0 and -6 < m < L. All spherical harmonics are considered real-valued functions on IR3, Positions of nodes in I are represented as r, while node-level irreducible features are denoted h, E Rexe, where s represents the spherical dimension and e the feature dimension per spherical component (i.e. number of channels). The operation (...((0) is the projection operation which extracts only the ε-th order irreducible component from a representation tensor product. (without any superscripts), and its Wigner 6j counterpart is denoted by g&)
Tensor products of irreps are symbolized by &
In this section, we establish the mathematical foundations necessary for constructing the Wigner-6j Convolution. These include real real-space solid spherical harmonics, tensor products of irreducible representations (irreps), and Wigner 6j recoupling theory. We commence by defining the solid spherical harmonics in real basis, which is commonly used in modern ML applications:
Definition 2.1 (Solid Spherical Harmonics in Real Basis) Let r (x, E R², r= |||| 2 + y² + 2Z, and (r, 0, () be the spherical coordinates with:
arccos
Ø = atan2(y, 2)).

---

(a) Overview of the Framework
(3(3) Conv.
(Fa)
R (n,)
(ry)
Edge Dependent
1= 2
Tensor Product Count
Source Dependent
Target Dependent
Irreducible Representation
Spherical Harmonics
O Edge-Level
H
mi
Solid Sph. Harmonics
xy, / Y, (8)
Σ hy 2
( (r.)
m:
((g) & (e (x)))))) (x))) "1
Binomial Local Expansion
O(EE)
Σ hy
(r:) 2 2 x (r,) + x (r,)) (2)
eG
Edge-Level
'Blononial Theorem" in Tonsor Soace
m.
Rearrange, Factorization and Wigner Recouping
Wigner 6j Conv
O(V))
mi
R2
Σ h;
2 2 (r:)
Σ (h,))(((r)))
(r;))
Lbe hy 2 (x,)
Node-Level Good Scalability
(b)
Wigner Recouping
L=
L=1
Equivalent
L=1
Da
L=1
Figure 1: (a) Overview of the Proposed Approach. Rather than performing tensor products over edges by combining node features and distances, E2Former leverages two key concepts: binomial local expansion and Wigner 6j recoupling. The former represents edge directions in terms of node positions, while the latter reorders the sequence of tensor product operations. Together, the computational complexity of the tensor product is reduced from O(EE) O(IV)). denotes the Clebsch- Gorden tensor product, and @ denotes the CG tensor product where each path is parameterized by a weight governed by the Wigner-6ý coefficients. (b) Illustration of equivalent ways couple the tensor product of three representations: sequentially coupling two tensors before the third (left) or reordering the coupling sequence (right), with equivalence established via the Wigner 6j recoupling.
The (regular) solid spherical harmonics are homogeneous harmonic polynomials of degree defined by:
2 (r) r² (f).
i r=r/r, 820, -1 m se
where y-(O) real spherical harmonics S Equivalently, in (r;
m > 0, ( oe p!l) (cos 0) cos(me),
R (r. 0)
=
( p θ),
m = 0,
(0) [m]] re p(() (cos 0) sin((ml)) m O,
with [m|] the associated Legendre polynomials and 1 normalization constants (chosen per the convention used in this work). For example, in e3nn implementation, the real spherical harmonics
for L 2 take the following form in Cartesian coordinates: R. 2,2) zy, R! (x, v.2)
yz, R. (x, v) 3x (x + 2²), R."" (x, y, 2) z2, R.( (z, y, x² m,m' (g) (()
Under rotation
E (0(3), these functions transform according to: R100 (r) + > ml (()) (
(r), where
()()) mmm (g) are the Wigner D-matrices. This transformation rule ensures that spherical harmonics of
fixed degree l transform properly under the action of SO(3). One useful property of solid spherical
harmonics that will come in useful later is (1)) (r)(() = R RR) (r:) - Rr)(r,)) It is also worth noting that this equality holds universally and only if 8 1.
Next, we introduce the behavior of irreducible representations (irreps). key principle that the tensor product of two irreps is generally reducible, meaning it decomposes into direct sum of other

---

inreps. This decomposition mechanism is precisely what will allow us to relate the general irrep (.) back to (.).
Definition 2.2 (Tensor Products of Irreps). Let (7(1)) and (T(a be irreuucible representations (irreps) of (3).). Their tensor product (7(h) @T7(f)) decomposes into direct sum of irreps: (
Di=li--s U(O) The decomposition is governed by the Clebsch-Gordan coefficients. Specifically, the tensor product, projected onto specific irreducible component ('(s), is denoted as:
(i+t
(Trhi) & U(aa] (ta)
Σ
Clama Ulbi) Eymy mi
(2.1)
fimp f2m2
Commutativity of the Clebsch-Gordan Tensor Product. The tensor product of two irreducible representations (irreps) ((() and ((() of SO(3) is not strictly commutative as bilinear operation on vector spaces: U(a) @ (+)) is not identical to (*h) @ U(a), Nonetheless, this operation effectively commutative at the level of irreducible decompositions. Interchanging the order of the factors does not change the set of irreps that appear, although it permutes the corresponding CG coefficients. In particular, we have:
Uta) @v()) = U 2 Ula)
UO)
(2.2)
|=
Associativity of the Clebsch- Tensor Product. For irreducible representations (irreps) of SO(3), the tensor product is associative up to a canonical isomorphism. Specifically, for any three irreps U(a), (((), and (T(e), the following holds:
(Ula) @ (()) @ Uto) = v(t) @ (U & U(e)).
While the set of resulting irreps is independent of the association order, the CG coefficients that appear in the decomposition do depend on the chosen coupling scheme. Transitions between different coupling orders are governed by Wigner 6j symbols, which express changes of basis without modifying the underlying irreducible content.
Definition 2.3 (Wigner 6j Symbol). For three irreps U(e), U1b) and U(s) of SO(3), one can couple {a d (c js
them either (+)) (UT() U(e) or (Ula) & (+0) ) U(e), The Wigner 6j symbol
relates these two coupling schemes through the identity:
(T()) (r so Trio] (1) (0
b d) [ow s (0
(11jathhe ((2 1)(2j + {e e jj
(2.3)
To simplify notation, we abstract the recoupling process follows:
(r(a) (U()) (()) (V @ (()) 8i u
(2.4)
where S denotes CG tensor product accompanied by a re-indexing via Wigner 6j coefficients.
3 Wigner 6j Convolution
In this section, we introduce the -quiivaaun Node convolution and demonstrate how Wigner 6j recoupling facilitates efficient node-wise computation.
Definition 3.1 (SO(3)-Equivarin)) Node Convolution). Let h; E IR=xc denote the irreducible feature tensor of node i, where s indexes the irreducible representation (irrep) type and indexes the channels within each irrep. Let (r)()) denote the degree-l spherical harmonic evaluated the relative direction. The -qqui--eui node convolution is via the CG tensor product between the source irreps and the spherical harmonics: h; = Σjen() h, @ (r.().
We clarify that our formulation of the SO(3) convolution employs R(-) rather than Y(-). The two formulations are related through a normalization factor. To realize the (b---ba variant, this normalization factor can be absorbed into the attention coefficients, as detailed in Alg. [

---

Wigner 6j convolution. Given the SO(3) convolution, we aim to demonstrate that the operation admits a node-wise factorization via Wigner 6j symbols. In particular, show that the (0 convolution can be expressed as:
h
Σ JEN())
h, @ R (Kw) --1)
R
(r.)
8%
Σ JEN(
hy & (r,))
ij dependent
i-leperdtene
jelependent
The blue boxe- factors R(") aggregate all node-i speciifc terms, whereas the red- ad-boxed factors hy, (F,) isolate the node-j contribution. This separation removes explicit edge dependencies. resulting in the number of tensor products in the network scaling with ((V)). To build further intuition, we draw an analogy to factorization techniques in kernelized attention mechanisms [3] which achieve linear scaling by decoupling query-key interactions.
To establish this result, we introduce the concept of the Binomial Local Expansion. The expansion is based on the key insight that any term (.) of arbitrary order can be expressed through iterative tensor products of the first-order term, (-). This effectively reduces the problem to the first-order case, where we can apply the previously introduced relation (r,) (2) (r,) (()) to factor the edge-dependent expression into node-local terms.
Theorem 3.2 (Bionomial Local Expansion). Let l = uu 2 1. Every É = u spherical harmonic ((())) can be expressed as an irreducible subspace of the u-fold tensor product (r.)) When expanded in terms of node-locee terms, this satisfies:
(rig) = E--))- (:)) [R)((((()) (1)))))) so (1)))
Proof Sketch. This spherical harmonic R R0 ((0())) could be constructed by projecting the f-fold tensor product of the first-order harmonic R.1) (raz) onto the subspace transforming the irreducible representation (irrep) of SO(3). Recall that the projection operator is denoted by [...]()) and using the identity R (r,)) = (r,) (r,) the objective could be rewritten R00 (ray)
(())))).)) - R (r;)) (r))))
We begin by expanding the tensor power (R,))r,,) - R (r,)) s8, which produces sum of 2 tensor products. Each term corresponds to ordered sequence P {i.j}, where each factor is either (2))(r;) or R (r)). Denote the corresponding tensor product as Tp. For example, if P (i. then Tp (r.) & (r,) & R (r:).. Initially, each ordering (*.*,... defines a distinct term. Later, we will show that the projection operator renders the result invariant to the ordering. We write the full expansion as: (r.)-R()((
= ( Σpen. Tp, where 1L, denotes the set of orderings containing exactly u factors of (r.) and l factors of (ry),, Applying the linear projection operator [..]]()) to this sum distributes the operator yields [(R()))) - R (ry))
-'
(-1) 1) Epen. (Tr](0).
The key insight comes from angular momentum coupling theory. Combining l systems with angular momentum yields components with total angular momentum ranging up to f. The subspace associated with the highest possible angular momentum, L e, is unique and corresponds to the fully symmetric combination of the individual factors. The projector [...](4) isolates precisely this unique, symmetric component. As a result, the projected tensor (4)] remains identical for all orderings P E II,, indicating that the projection depends solely on the multiplicities of the factors R (r.(r;) and R ((,) in Tp, rather than their ordering. The inner sum over the (9) identical projected terms simplifies. Let Trep (R() (T.)) @ (R ((e))))((=w))))))))))))))))))))) serve as a representative tensor product for the class 11,. Then: Σpen, [T] = [Teppp = (()()) (r.)) @ (R())r) (r;)) ((t))) )((-)))(0)
Substituting this simplification back into the expression for the projected tensor power yields: (x))))))) -R()) (r)))))) (Q
(R((T)) (()-a))(())
5

---

Theorem 3.3 (Node- NNode-Based Factorization via Wigner 6j). S3(3) convolutions admit a factorization that separates the dependence on the central node from the aggregation over neighbors j. yielding the form:
Σ h, ( JEN())
Σ ==0
(-1))- (1) (r.))
&
Σ hy @ (RU
JEN()
where 8 denotes a CG tensor product where the path weight is parameterized by the corresponding Wigner 6j coefficients.
Proof Sketch. We begin by substituting the spherical harmonic R1 (rey) using the binomial local expansion from into the original SO(3) -equivaraai convolution expression, we obtain:
Σ-1 R- (1) [R)
E
hy
(3.1)
(EN(()
4=0
By linearity of the tensor product, this expression becomes:
Σ JEN(()
h, @ (r.) (r))
(3.2)
To reorganize this expression in terms of node-dependent features, we apply Wigner 6j recoupling.
Letting A hy, B = R(t-w) (ry), C R(w) (r,), The recoupling identity states: A @ (B @ C) = (A B) 80 C. Since CG tensor products commute effectively, we can swap B and C before applying recoupling. This gives:
hy & (r.) R (() (hy @ (((("((r))) 60 (ra).
(3.3)
Applying this recoupling within the sum, we arrive the factorized form as claimed. Note that is safe to apply the recoupling within a projection operator. This constraint can be implemented by fixing one of the intermediate coupling indices the Wigner 6j symbol to
We now formalize the key properties of the resulting Wigner 6j convolution. These results are stated in the following lemmas. Proofs are provided in Appendix and Appendix respectively.
Lemma 3.4 (Equivariance of Wigner 6j Convolution). The Wigner 6j convolution operator (denoted as F), is equivariant under the Euclidean group SE(3). That is, for any rigid transformation g E SE(3), the output satisfies Flg fl = D(g) F[f)..
Lemma 3.5 (Time Complexity of Wigner 6j Convolution). The time complexity of Wigner 6j convolution is O(LI'C + C where L is the degree cutoff and C is the number of channels.
Model Architecture. Based on these, we propose E2Former, a modular architecture that alternates between E2Attention and feed-forward layers (Appendix Fig. 5 a); additional architectural details provided in the Appendix HH). At its core lies a convolution layer based on the Wigner 6j convolution, which serves a backend kernel to efficiently capture rotational symmetries. We highlight two key design considerations underlying E2Former. First, we observe that the attention computation constitutes only a small fraction of the overall runtime in an attention-based SO(3) convolution (Figure 2 (a)). Leveraging this, we integrate the attention mechanism directly into the Wigner 6j convolution (Algorithm DD. As result, while the resulting model not strictly linear, the number of tensor product operations scales linearly with input size, enabling efficient computation. Second, E2Former is not defined solely by its use of Wigner 6j convolutions, Rather, represents a broader architectural principle that combines symmetry- efficiency gains afforded by the Wigner 6ý kernel allow us to reallocate the computational budget toward increased expressivity-e.. by incorporating deeper layers, wider hidden dimensions, more attention heads, and MACE higher-order interactions [6) This trade-off between symietry-driven modeling and architectural scalability is central to the design philosophy of E2Former.
design with practical engineering. The

---

Mereage Gomeh
0611
Lmxx 3 Dense
(6)
Nensage Appregation
0.4
5013) Gonv Time
1.0
S06) Cony Time
Wigner-j Conv Time
Wigner-j Conv Time
0.3
0.4
0.1
0.2
0.0
200
400
600
Size (N)
Order
[bj]
0:10
mmx Sparse
1b)i
Lyna
Sparse
(4)
SO(3) Cony Time
80(3) Conv Time
S0(3) Conv Time
Wigner 6j Conv Time
Wigner-Bj 6j Comv Time
0.20
Wigneniij Conv Time
0.20
Time
0.08
0.15
0.10
0.02
0.05
0.05
0.00
0.00
1000
500
1000
1500
2000
200
Size (N)
Size
Max Neighbors
Figure 2: (a) Breaking down the runtime of attention- based SO(3) convolutions shows that message construction is the slowest step. Calculating attention and combining messages take much less time. (b) We compared the runtime of our Wigner 6j convolution (purple squares) against the standard SO(3) convolution (blue circles). Our method was consistently faster across different graph sizes (N), maximum angular momenta (Lmax), ) and sparsity levels (dense vs. sparse, see subplots b.i-iii). Full experimental details are in Sec. 4.1 (c) Runtime on 1000-node graphs as a function angular
momentum cutoff L (up to Lmax 6). (d) Runtime on 1000-node graphs with fixed Lmax = 3,
varying the maximum number of neighbors from 64 to 512. In (b-d), both methods yield identical outputs.
4 Results
4.1
Scaling Analysis of Wigner 6j Cony and SO(3) Conv
Here, we compare the runtime of Wigner 6j convolution (purple squares) and SO(3) convolution (blue circles). The two implementations are mathematically equivalent and, by construction, produce identical outputs given the same molecular graphs. Specifically, we compare different graph sizes N, maximum angular momenta Lmaxx and connectivity patterns. For dense graphs, defined graphs where every node connected to all other nodes, L (Fig. 2 (b.i)). the quadratic scaling of SO(3) convolution introduces noticeable performance gap. Additionally, for sparse graphs, defined here as graphs with k-nearest neighbor connectivity (k- = 32), at Lmax = 2 and Lemax = (Figs.[2(c) (b.ii) and (b.ii)),, Wigner 6j convolution scales consistently better than the SO(3) convolution. Fig. 2 further shows the impact increasing the angular momentum cutoff up to Lmax = on 1000-node graphs, where
method consistently achieves approximately 7x speed-up over the bascline. Finally, Fig. 21 demonstrates that as the number of neighbors per node increases from 64 to 512 (with Lmax 3), the speed-up from Wigner 6j convolution becomes even more pronounced.
4.2 E2Former Results
We evaluate E2Former which heavily utilizes the Wigner 6j convolution on three standard bench- marks two catalysis datasets (OC20, OC22) and molecular conformer dataset (SPICE) and find that it achieves strong accuracy while maintaining computational efficiency.
4.2.1 Performance on the OC20 Dataset
Dataset Description. The OC20 dataset [10] comprises 1.2 million DFT relaxations computed using the revised Bertew-BBueee--reeeee
(RPBE) functional (2). Eaah system, averaging atoms, represents an adsorbate molecule on a catalyst surface and is designed for the Structure-to-Fnet -Enerry-- and-Forces (S2FF) task. This task involves predicting the system's energy and per-atom forces, with performance evaluated based on the mean absolute eπror (MAE) of these predictions. Following
7

---

we use the 2M subset for training, and evaluate on the validation split. This choice also reflects practical computational constraints, as training on the full dataset requires sigmificant time and resources. All reported results are taken directly from previous publications [27] 41]. We compare two model variants: the 33M-paraeeter version and the 67M-parameter version. A summary of the results is presented in Table | E2Former demonstrates strong performance across all model sizes, with 67M variant achieving results comparable to state-of-the-an methods. Notably, the Small variant (33M parameters) maintains competitive accuracy while offering significant computational advantages.
27 41,,
Table 1: Performance on the OC20-2M dataset. Results are reported in Energy (meV) and Force (meV/A) mean absolute error (MAE). E2Former achieves competitive accuracy and computational efficiency. Approximate training GPU hours are measured on 32G NVIDIA V100 GPUs. The best results are bolded and the second best are highlighted with underline.
Model
Parane (M)
Training GPU Hours
Inference Speed (samplashsc)
Valluatonn
Energy MAE (moV)
Force MAE (meVIA)
GouNet-JT
31
9000
35%
20:50
GenNes-DC
38
1500
2366
25,70
SCN
126
1000
270
21 1.00
cSCN
S1
2200
283
20.50
Equillomer/2
KS
1800
285
20.46
E2Feemerr 33M
800
62
275
21:90
E2Foemer 6751
1500
270
20.50
4.2.2 Performance on the OC22 Dataset
The OC22 dataset 55 is specifically designed for studying oxide electrocatalysis. In contrast to OC20, OC22 features DFT total energies, which could serve as a general and versatile DFT surrogate, enabling investigations beyond adsorption energies. We train on the OC22 S2EF-Total task and measure energy and force MAE on the S2tF-Total validation splits. Table 2 summarizes our results on the OC22 S2EF task. E2Former achieves competitive energy and force MAEs while enabling rapid training and inference. Notably, it converges in just 1,500 GPU only one-third of the runtime required by the SOTA model.
Table 2: Performance on the OC22 S2EF task. Results are reported for Energy MAE (meV) and Force MAE (me) under In Distribution (ID) and Out-Or-Distriuttion (OOD) splits. Approximate training GPU hours are measured on 32G NVIDIA V100 GPUs. The best results are bolded and the second best are highlighted with underline.
Model
# Params (M)
Training GPU Hours
Energy MAE (meV)
Force MAE (meV/A)
ID
OOD
ID
00D
GemNet-OC
39
545
1011
30.00
40.00
EquiformerV2
122
4500
433
629
22.88
30.70
E2Former 67M
67
1500
491
724
25.98
36.45
4.2.3 Performance on the SPICE Dataset
The SPICE dataset [9 comprises small organic molecules and encompasses a diverse array of chemical species with neutral formal charges. The geometries were generated through molecular dynamics simulations using classical force fields, followed by the sampling of various conformations. High-fidelit labeling
achieved at the WB97M-D3(BB / Te2e2D level of calculations.
This dataset includes configurations of up to 50 atoms. It was further augmented with larger molecules, ranging from 50 to 90 atoms, derived from the QMugs dataset (32), well water clusters obtained from simulations of liquid water. Approximately 85% of the SPICE dataset was used for model training, while 15% was allocated for model testing. We evaluate E2Former 33M on the SPICE dataset and a summary of the results is provided in Table 3
Table 3: Performance comparison on the SPICE dataset with actual training time and dataset sizes. Results are reported in Energy (E, me Vatto)) and Force (F, V/À) MAE Approximate training GPU hours are measured on 80G NVIDIA A100 GPUs.
Dutaet Nore Siee Transigg Tine
PabChen (3884))
Miomner (889)
Diases (1389))
Dipeptibs (1025)
SehratedAarisskiiic.c (52)
Water (94) Quoug (144)
Caps bears)
E
Soudl
ElFoner
0.43
4.1
0.51
5.63
192
8

---

E2Former achieved state-of of-the-art art performance in most subsets, particularly in datasets with ample data, such as PubChem and HEShaw330-Dimert Furthermore, compared the MACE-Large model, E2Former achieves approximately a fivefold increase in training speed, thereby further validating its efficiency.
4.3 Meomory and Efficiency Scaling
To further probe efficiency, we evaluated computational performance in even extreme system scale. We benchmarked E2Former against MACE-Large and EquiformerV2 on systems containing up to 6,400 atoms (Table T E2Former consistently achieved the highest throughput, with its performance advantage becoming clearer as system size increased. At 3,200 atoms, E2Former processes data nearly three times faster than MACE-Large Crucially, E2Former was the only model capable of handling simulations at the 6,400-atom scale, size at which both MACE-Large and EquiformerV failed.
Table Comparison of memory cost and efficiency on large-scale systems with up 6,400 atoms.
Atom Number
Memory Cost (GB)
Training Efliciency (Samples/Second)
Equiformer V2-22M
MACE-Large3MM
E2Former-33M
Equiformer V2-22M
MACE-Larg--M
E2F E2Former-3MM
400
26.7
10.7
7.6
1.27
2.1
4.2
800
53
23
16
0.68
1.61
2.36
1600
083
2.46
6400
80
0.59
5 Molecular Dynamies Simulation
In this section, we demonstrate the practical utility E2Former molecular dynamics simulations. While machine learning methods are extensively used predict molecular and material properties, accurately simulating the behavior of systems over extended time periods remains a significant challenge. This task requires only precise predictions at each time step but also long-term stability and performance comparable to established methods such as DFT and empirical potential models. We began by pretraining
E2Former on large in- house dataset derived from
(a)
(b)
Power Spectrum
Efficiency Comparson
E2Fommr
190
DeShaw (2M) [15] and
6FN2XTB
180
GEMS 60) (2.7M), consti-
MADIT
Computational Time
160-
tuting foundational model
Power
140
on machine-learning force
1.0
field.
0.8
0.6
0.4
5.1 Small-scale
0.2
0.0
Amino Acid Systems
1000
E2Fomer
GFN2-xd
MADFT
To evaluate the model, first performed an NVT (T - 300 simulation of an amino acid wrapped by water molecules (The struc- ture is shown in Fig. [a)) totaling 253 atoms. This evaluation was conducted on a system equipped with a single NVIDIA 1100 GPU
Frequency (cm") 1))
Figure 3: (a) Power spectra comparison across computational methods: E2Former (blue), GFN2-XTB (orange), and MADFT (green). The graph corresponds to simulation NVT ensemble, temperature T 300 K, with time step of 1 fs. A structural overlay of the simulated system is displayed for context. (b) Efficiency comparison showing computational time for E2Former, GFN2-xtb, and MADFT. E2Former demonstrates the lowest computational time. The y-axis denotes the computation time for a single frame
and an AMD EPYC 7V13 24-core CPU. Over the course of 10,000 simulation time steps (Ifs per step), we compared the trajectory 's power spectrum obtained from E2Former, CUDA-acceleree DFT 34 namely, the MADFT software and state-of-the-art empirical potential methods GFN2-
xTB 3 Fig. a)) illustrates the results. The evaluation demonstrates that E2Former exhibits long-term stability molecular simulations. The power spectrum shows that F2Former predictions align closely with those of the DFT bascline. In contrast, empirical method shows significant devia-

---

tions in high-freen -reeueeyy regions, particularly near 3,000 and 3,500 frequencies. These high-free -requffu components are critical as they provide insights into bond vibrations [14] and molecular stability [12]. underscoring the ability of E2Former to effectively extrapolate across the molecular potential energy surface. To assess computational efficiency, we compare runtime across methods, which depicted in Fig. b).. E2Former achieves computational speed approximately 1,000 times faster than DFT, and around times faster than GFN2-xTB.
5.2 Large-Scale 6000-atom Water Cluster
100
Water Cluster Power Spectrum
E2Formet
To evaluate accuracy and efficiency at larger scales, we tested our model on 6,000-atom water cluster by analyzing atomic vibration pat- terns. E2Former closely reproduced the ref- erence DFT results, accurately capturing key spectral features: low-frequency modes (be- low 1000 cm cm-), "), the H-O-H bending mode (1650 cm "), and O-H stretching vibrations. In contrast, MACE-F Large exhibited larger devia- tions, particularly for nigh-frequence stretching modes. We further validated our approach on more complex system-a Chignolin peptide solvated in water (approximately 2,000 atoms, Fig 6) suucessfully optimizing its structure
80
MADFT
Power
40
20
500
2000
2500
Frequency (cm "
Figure 4: Power spectra comparison acToss putational methods: E2Former (blue), MACE (or- ange), and MADFT (green).
while maintaining high force prediction accuracy (0.484 kcal/mol/Ā error).
6 Related Work
Invariant GNNs. Invariant geometric GNNs have driven state-of-the-arr performance in predicting and have been instrumental in
molecular and crystalline properties [53 50
26
43 25
advancing protein structure prediction
33
Cartesian Equivariant GNNs. Building on invariance, Cartesian equivariant GNNs explicitly model transformations in RR3 offering greater flexibility. These models have shown strong empirical results and have recently evolved include Cartesian equivariant
in similar domains B3 51 54
2
transformer layers (22)
Spherical Equivariant GNNs. Complementing Cartesian approaches, spherical equivariant GNNs leverage spherical tensors to naturally handle rotational symmetries, relying on the representation theory of (3)).. Recent advancements include SO(3)- and -333ieqquivariant transformer lay- ers 123 41. efficient interatomic potential calculations 77 6 46). and optimizations that reduce convolutions in SO(3) SO(2) [47). These improvements have enabled strong performance in di- verse applications, including geometry, physics, and chemistry (57). dynamic molecular modeling (]. and fluid mechanical modeling (58).
7 Conclusion and Future Work
We introduced E2Former, efficient and scalable Transformer architecture for molecular modeling. By leveraging the Wigner 66 convolution, E2Former shifts computation from edges to nodes, reducing complexity from O((ET) to O(DV) while maintaining rotational equivariance and expressive power. E2Former demonstrated competitive performance across OC20, OC22, and SPICE benchmarks with significantly improved computational efficiency. Its scalability makes it suited for large-scale applications in biology, drug discovery, and materials science.
Future work could focus on optimizing E2Former for hardware accelerators and integrating kernelized Fuclidean attention (2T) Combining Wigner-6ý Conv with SO(2) convolution could further bolster the model's is efficiency. Scaling to the real-world all-atom protein systems will also be investigated.
10

---

References
[1] Brandon Anderson, Truong Son Hy, and Risi Kondor. Cormorant: Covariant molecular neural networks. Advances in neural information processing systems, 32, 2019.
[2] Sarp Aykent and Tian Xia. GotenNet: Rethinking Efficient 3D Equivariant Graph Neural Networks. In The Thirteenth International Conference on LeanningRepresentaiionn. 2025.
[3] Christoph Bannwarth, Sebastian Ehlert, and Stefan Grimme. Gfn2-xtb-ai an accurate and broadly parametrized conssscon tight-binding quantum chemical method with multipole electrostatics and density-dependee dispersion contributions. Journal of chemical theory and computation, 15(3):1652-1671, 2019.
[4] Albert P Bartók, Risi Kondor, and Gábor Csányi On representing chemical environments. Physical Review B--onddeeedd Malter and Materials Physics, 87(18):18411, 2013.
[5] Albert P Bartók, Mike C Payne, Risi Kondor, and Gábor Csányi Gaussian approximation potentials: The accuracy of quantum mechanics, without the electrons. Physical review letters, 104(13):136403, 2010.
[6] Hyes Batatia, David Peter Kovaes, Gregor N. C. Simm, Christoph Ortner, and Gabor Csanyi. MACE: Higher order equivariant message passing neural networks for fast and accurate force fields. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, Advances in Neural Information Processing Systems, 2022.
[7] Simon Batzner, Albert Musaclian, Lixin Sun, Mario Geiger, Jonathan P Mailoa, Mordechai Kornbluth, Nicola Molinari, Tess E Smidt, and Boris Kozinsky. E(3)-equivariant graph neu- ral networks for data-efficient and accurate interatomic potentials. Nature communications, 13(1):2453, 2022.
[8] Lawrence Biedenhar,, James D Louck, and Peter Carruthers. Angular momentum in quantum physics: theory and application. 1984.
[9] Troy Cellmer, Marco Buscaglia, Eric R Henry, James Hofrichter, and William A Eaton. Making connections between ultrafast protein folding kinetics and molccular dynamics simulations. Proceedings the National Academy of Sciences, 108(15):6103-611 61088 2011.
[10] Lowik Chanussot, Abhishek Das, Siddharth Goyal, Thibaut Lavril, Muhammed Shuaibi, Mor- gane Riviere, Kevin Tran, Javier Heras- Domingo, Caleb Ho, Weihua Hu, et al. Open catalyst 2020 (oc20) dataset and community challenges. Acs Catalysis, 11(10):009-6072 2021.
[11] Chi Chen, Weike Ye, Yunxing Zuo, Chen Zheng, and Shyue Ping Ong. Graph networks as a universal machine learning framework for molecules and crystals. Chemistry of Materials,
3 31(9):3564-3577 2019.
[12] Choon-Peng Chng, Annette Dowd, Adam Mechler, and K Jimmy Hsia. Molecular dynamics simulations reliably identify vibrational modes in far-ir spectra of phospholipids. Physical Chemistry Chemical Physics, 2024.
[13] Krzysztof Choromanski, Valerii Likhosherstov, David Dohan, Xingyou Song, Andreea Gane, Tamas Sarlos, Peter Hawkins, Jared Davis, Afroz Mohiuddin, Lukasz Kaiser, et al. Rethinking attention with performers. arXiv preprint arXiv:2009. 14794, 2020.
[14] Edward Ditler and Sandra Luber. Vibrational spectroscopy by means of first-principles molecular dynamies simulations. Wiley Interdisciplinary Reviews: Computational Molecular Science, 12655, 2022.
[15] Alexander G Donchev, Andrew G Taube, Elizabeth Decolvenaere, Cory Hargus. Robert T McGibbon, Ka-Hei Law, Brent A Gregersen, Je-Luen Li, Kim Palmo, Karthik Siva, al. Quantum chemical benchmark databases of gold-standard dimer interaction energies. Scientific data, 8(1):55, 2021.
[16) Ralf Drautz. Atomic cluster expansion for accurate and transferable interatomic potentials. Physical Review B, 99(1):014104 2019.

---

[17] Weitao Du, He Zhang, Yuanqi Du, Qi Meng, Wei Chen, Nanning Zheng, Bin Shao, and Tie- Yan Liu. Se (3) equivariant graph neural networks with complete local frames. In International Conference Machine Learning, pages 5583-5608. PMLR, 2022.
[18] Nadav Dym and Haggai Maron. On the universality of rotation equivariant point cloud networks In International Conference on Learning Representations.
[19] Peter Eastman, Pavan Kumar Behara, David L Dotson, Raimondas Galvelis, John E Herr, Josh T Horton, Yuezhi Mao, John D Chodera, Benjamin P Pritchard, Yuanqing Wang, et al. Spice, dataset of drug-like like molecules and peptides for training machine learning potentials. Scientific Data, 10(1):11, 2023.
[20] Alan Robert Edmonds. Angular momentum in quantum mechanics, volume 4. Princeton university press, 1996.
[21] J Thorben Frank, Stefan Chmiela, Roburt Müller, and Oliver T Unke. Euclidean fast attention: Machine learning global atomic representations at linear cost. arXiv preprint arXiv:2412.08541, 2024.
[22] J Thorben Frank, Oliver T Unke, and Klaus- Müller. So for higher- order geometric interactions on arbitrary length-scales. arXiv preprint 1rXiv:2205.14276 2022.
[23] Fabian Fuchs, Daniel Worrall, Volker Fischer, and Max Welling. Se (3)-transformers: 3d roto-tranee tho-translation equivariant attention networks. Advances in Neural Information Processing Systems, 33:1970- 1981, 2020.
[24] William Fulton and Joe Harris. Representation theory: a first course, volume 129. Springer Science & Business Media, 2013.
[25] Johannes Gasteiger, Florian Becker, and Stephan Günnemann. Gemnet: Universal directional graph neural networks for molecules. Advances in Neural Information Processing Systems, 34:6790-6802, 2021.
[26] Johannes Gasteiger, Janek GroB, and Stephan Günnemann. Directional message passing for molecular graphs. arXiv preprint arXiv:2003.03123. 2020.
[27] Johannes Gasteiger, Muhammed Shuaibi, Anuroop Sriram, Stephan Günnemann, Zachary Ulissi, C Lawrence Zitnick, and Abhishek Das. Gemnet-oc: developing graph neural networks for large and diverse molecular simulation datasets. arXiv preprint arXiv:2204.02785 02782, 2022.
[28] Mario Geiger, Tess Smidt, Alby M.. Benjamin Kurt Miller, Wouter Boomsma, Bradley Dice, Kostiantyn Lapchevskyi, Maurice Weiler, Michal Tyszkiewicz, Simon Batzner, Dylan Madisetti, Martin Uhrin, Jes Frellsen, Nuri Jung, Sophia Sanborn, Mingjian Wen, Josh Rackers, Marcel Rød, and Michael Bailey. Euclidean neural networks: e3nn, April 2022.
[29] BHLB Hammer, Lars Bruno Hansen, and Jens Kehlet Nørskov. Improved adsorption energetics within density-functional theory using revised perdew-burke-erzzeete functionals. Physical review B, (.55991))) 1)1441,,,,, 1999.
[30] Pierre Hohenberg and Walter Kohn. 136(3B):B864, 1964.
Inhomogeneous electron gas.
Physical review,
[31] Gyeong S Hwang, Haley M Stowe, Eunsu Pack, and Dhivya Manogaran. Reaction mecha- nisms of aqueous monoethanolanine with carbon dioxide: combined quantum chemical and molecular dynamics study. Physical Chemistry Chemical Physics, 17(2):831-83) 2015.
[32] Clemens Isert, Kenneth Atz, José Jiménez-Lana, and Gisbert Schneider. Qmugs, quantum mechanical properties of drug-like molecules. Scientific Data, 91):2733 2022.
[33] Bowen Jing, Stephan Eismann, Patricia Suriana, Raphael John Lamarre Townshend, and Ron Dror. Learning from protein structure with geometric vector perceptrons. In International Conference Learning Representations, 2020. 2020.
12

---

[34] Fusong Ju, Xinran Wei, Lin Huang, Andrew J Jenkins, Leo Xia, Jia Zhang, Jianwei Zhu, Han Yang, Bin Shao, Peggy Dai, et al. Acceleration without disruption: Dft software as service. Journal of Chemical Theory and Computation,
2024.
[35] John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ron- neberger, Kathryn Tunyasuvunakool, Russ Bates, Augustin Židek, Anna Potapenko, et al. Highly accurate protein structure prediction with alphafold. Nature, 596(7883):583-558) 2021.
[36] Ferene Karsai, Manuel Engel, Espen Flage-Larsen, and Georg Kresse. Electron-phonon cou- pling in semiconductors within the gw approximation. New Journal of Physics, 20(12): 2018.
[37] Walter Kohn and Lu Jeu Sham. Self-consistent equations including exchange and correlation effects. Physical review, 140(4A):A1133, 1965.
[38] Arpan Kundu, Marco Govoni, Han Yang, Michele Ceriotti, Francois Gygi, and Giulia Galli. Quantum vibronic effects on the electronic properties of solid and molecular carbon. Physical Review Materials, 5(7):0070801, 2021.
[39] Arpan Kundu, Yunxiang Song, and Giulia Galli. Influence of nuclear quantum effects on the electronic properties of amorphous carbon. Proceedings of the National Academy of Sciences, 119631):e2203063113 19, 2022.
[40] Shan-Tao Lai and Ying-Nan Chiu. Exact computation of the 3-j and 6-j symbols. Computer physics communications, 61(3):350-360, 1990.
[41] Yi-Lan Liao and Tess Smidt. Equiformer: Equivariant graph attention transformer for 3d atomistic graphs. In International Conference on Learning Representations, 2023.
[42] Yi-Lun Liao, Brandon Wood, Abhishek Das, and Tess Smidt. Equiformerv2: Improved equiv- ariant transformer for scaling to higher-degree representations. arxiv preprint arxiv:230..12059 2023.
[43] Yi Liu, Limei Wang, Meng Liu, Yuchao Lin, Xuan Zhang, Bora Oztekin, and Shuiwang Ji. Spherical message passing for 3d molecular graphs. In International Conference on Learning Representutions (ICLR), 2022.
[44] Shengjie Luo, Tianlang Chen, and Aditi S. Krishnapriyan. Enabling efficient equivariant operations in the fourier basis via gaunt tensor products. In The Twelfth International Conference on Learning Representations, 2024.
[45] Leonard C Maximon. 3j. 6j. 9j symbols., 2010.
[46] Albert Musaelian, Simon Batzner, Anders Johansson, Lixin Sun, Cameron J Owen, Mordechai Kornbluth, and Boris Kozinsky. Learning local equivariant representations for large-scale atomistic dynamics. Nature Communications, 14(1):579, 2023.
[47] Saro Passaro and C Lawrence Zitnick. Reducing so (3) convolutions to so (2) for efficient equivariant gnns. In International Conference on Machine Learning, pages 27420-27438. PMLR, 2023.
[48] Eric Qu and Aditi Krishnapriyan. The importance of being scalable: Improving the speed and accuracy of neural network interatomic potentials across chemical domains. Advances in Neural Information Processing Systems, 37:139030-139053 2024.
[49] Giulio Racah. Theory of complex spectra. ii Physical Review, 62(9-10):438, 1942.
[50] Soumya Sanyal, Janakiraman Balachandran. Naganand Yadati, Abhishek Kumar, Padmini Rajagopalan, Suchismita Sanyal, and Partha Talukdar. Mt-egenn: Integrating crystal graph convolutional neural network with multitask learning for material property prediction. arXiv preprint
2018
[51] Victor Garcia Satorras, Emiel Hoogeboom, and Max Welling. (n) equivariant graph neural networks. In International conference machine learning, pages 9323-9332. PMLR, 2021.
13

---

[52] Kristof Schütt, Oliver Unke, and Michael Gastegger. Equivariant message passing for the prediction of tensorial properties and molecular spectra. In International Conference on Machine Learning, pages 9377-9388. PMIR, 2021.
[53] Kristof T Schüitt, Huziel E Sauceda, P-J Kindermans, Alexandre Tkatchenko, and K-R Müller. Schnet-a deep learning architecture for molecules and materials. The Journal of Chemical Physics, 148(24))241722,, 2018.
[54] Guillem Simeon and Gianni De Fabritiis. Tensoret: Cartesian tensor representations for efficient learning of molecular potentials. Advances in Neural Information Processing Systems, 36, 2024.
[55] Tess Smidt, Mario Geiger, and Benjamin Kurt Miller. Finding symmetry breaking order parameters with euclidean neural networks. Phys. Rev. 3:L0120200,, Jan 2021.
[56) Attila Szabo and Neil S Ostlund. Modern quantum chemistry: introduction to advanced electronic structure theory. Courier Corporation, 2012.
[57] Nathaniel Thomas, Tess Smidt, Steven Kearnes, Lusann Yang, Li Li, Kai Kohlhoff, and Patrick Riley. Tensor field networks: Rotation-ant clouds. arXiv preprint arXiv:1802.08219, 2018.
neural networks for 3d point
[58] Artur P Toshev, Gianluca Galletti, Johannes Brandstetter, Stefan Adami, and Nikolaus A Adams. (3) equivariant graph neural networks for particle-based fluid mechanies. arXiv preprini arXiv:2304.00150, 2023.
[59] Richard Tran, Janice Lan, Muhammed Shuaibi, Brandon M Wood, Siddharth Goyal, Ab- hishek Das, Javier Heras-Domingo, Adeesh Kolluru, Ammar Rizvi, Nima Shoghi, et The open catalyst 2022 (oc22) dataset and challenges for oxide electrocatalysts. ACS Catalysis, 13(5):3066-3084 2023.
[60] Oliver Unke, Martin Stöhr, Stefan Ganscha, Thomas Unterthiner, Hartmut Maennel, Sergii Kashubin, Daniel Ahlin, Michael Gastegger, Leonardo Medrano Sandonas, Joshua T Berryman, et al. Biomolecular dynamics with muchine-learne quantum- mechanical force fields trained on diverse chemical fragments. Science Advances, 10(14):cadn4397, 2024.
[61] DA Varshalovich, AN Moskalev, and VK Khersonskii. Quantum theory of angular momentum. 1987.
[62] Limei Wang, Yi Liu, Yuchao Lin, Haoran Liu, and Shuiwang Ji. ComENet: Towards complete and efficient message passing for 3d molecular graphs. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, Advances in Neural Information Processing Systems, 2022.
[63] Yusong Wang, Shaoning Li, Xinheng He, Mingyu Li, Zun Wang, Nanning Zheng, Bin Shao, Tong Wang, and Tie- Yan Liu. Visnet: scalable and accurate geometric deep learning potential for molecular dynamics simulation. arXiv preprint arXiv:2210.1633 166118 2022.
[64] Vesselin Yamakov, Dieter Wolf, Simon R Phillpot, Amiya K Mukherjee, and Herbert Gleiter. Dislocation processes in the deformation of nanocrystalline aluminium by molecular-dynamics simulation. Nature materials, 1(1):55 49, 2002.
14

---

Appendix Table of Contents
Glossary of Notations
16
Attention-based Wigner bj Convolution
17
Wigner 3ý and 6ý Symbols
17
C.1 The Wigner 3j Symbol: Definition and Core Properties
18
C.2 The Wigner bj Symbol: Recoupling Coefficients
19
Main Theorem Proofs
20
D.I Binominal Local Expansion
20
D.2 Wigner 6j Recoupling and Node-Ban Base-Based Factorization
23
Proof of Equivariance
25
F Time Complexity
26
Additional Lemmas
27
Technical Details Behind E2Former
28
Addtional Experiments
30
L. QM9 Results
30
1.2 Comparison with Equiformer V2
30
Hy syperparameters
31
15

---

A
Glossary of Notations
Table 5: Glossary of Notations
Symbol
Deseription
V.E
Set of nodes (vertices) 1/ and edges & in molecular graph.
18)
Number nodes and edges the graph, respectively.
N
Oftn used to denote [Vi, the number of atoms (nodes).
Hidden feature dimension (number of channels) at each spherical degree.
Spherical dimension: number irreps (from
I, Lmooe
Maximum angular momentum (highest spherical degree).
2 0, m E (-6....))
Angular momentum quantum numbers for spherical harmonics.
R (x)
Real spherical harmonic of degree f und order evaluated position
An irreducible representation (irrep) of SO(3) at angular momentum
hy Rood
Feature representation of node containing scalar/vector components up to spherical degree
Ry E R
Relative position vector from node node j.
Clebsch-Gordan (CG) tensor product of two irreps.
@%
Wigner Gy-based tensor product (recoupling) of irrps, reorganizing the CG couplings.
Attention coefficient between node and in an equivariant attention layer.
Caamm
GimL Fama f
Clebsch-Gordan coefficient coupling two irreps & f2 to an output irreps is.
F2
Eg mg
Wigner 3j symbol (equivalently related Clebsch-Gordan coefficients).
mh
m2
Ji
J2
Wigner 6j symbol, governing recoupling of three angular momenta in different orders.
Ja
J6
J6
(R)
Wigner D-matrix describing how spherical harmonics degree transform under rotation R.
SO(3)
3D rotation group: E2Former equivariant to transformations 5O(3).
E2Former
The proposed Efficcent and Equivariant Transformer architecture.
Wigner by Conv
The convolution module leveraging Wigner Gj recoupling shift edge-based operations nodes.
16

---

B Attention-based Wigner 6j Convolution
We describe the algorithmic procedure for constructing the Attention- iitention-based Wigner 6j Convolution in- troduced the main text. Assume that attention coefficients a have been precomputed, either through Query-Key inner products via MLP-based attention mechanisms such those in Equiformer v2 [42]. We begin by precomputing the spherical harmonics R(4) up to maximum degree L based on the input positions r.
For each degree k: = 0,..., L, we compute an attention-weighted tensor product between the input features h and the corresponding spherical harmonics R(a), This intermediate representation is then modulated by the attention weights The resulting tensors are subsequently recoupled using a Wigner 6j tensor product, where each recoupling path is parameterized by the Wigner 6j coefficients- offering more flexible alternative the standard Clebsch-Gordee coupling.
Finally, summing over all degrees yields the output irreducible representations (irreps), which are used to update the node embeddings.
Algorithm Wigner 6j-Based Attention
1: Input: Positions IR input features h R Nxdxxa attention weights E RNNNX (H for number for heads, for the number spherical dimension, the number hidden channels), maximum order
2: Output: Output features haw E N
3: Step 1: Precompute Spherical Harmonies
4: for f to L, do
5: R((0) < SH(E,
6: end for
7: Step 2: Compute pairwise distances
8: Dij {- ||r: - -ryll2
(pairwise Euclidean distance)
9: Step Compute Wigner 6ý tp
10: for k = 0 do
11: Compute intermediate tensor product:
Ta + clebsch_gorden_ R (()))
O Gj +- 0ijh (D
(k)
(entrywise power normalization)
13: Apply attention weights:
Tx < einsum jhds -> ihds", (*) Tx)
14: Recouple terms using Wigner 6j symbols:
C Wigmer6jTP(Tx, R ((-)))
15: Update output: how <- hour (f))
16:
end for
Remark B.I. The upshot is that (i) the node-local spherical harmonics R. R(() (Fi) can precomputed once for each node i, and (ii) the partial sum Σ, aiy(h, @ R((-a)(F))) can treated as a single node-based operation. Thus, the number tensor products controlled by | (the number of nodes) rather than |3| (the number of edges). This is precisely the core reason E2Former achieves improved scalability compared to conventional -equivarria Transformers that do edge-level spherical harmonic products.
C Wigner 3j and 6j Symbols
The Wigner 3j and 6j symbols are fundamental constructs in the representation theory of the Lie group SU),), intrinsically linked to the theory of angular momentum in quantum mechanics. These symbols emerge as crucial transformation coefficients when decomposing tensor products of irreducible representations of SU(2). They precisely encode the symmetry properties inherent in such decompositions, thereby providing powerful computational framework for problems involving coupled representations.
17

---

C.1
The Wigner 3j Symbol: Definition and Core Properties
The Wigner 3j symbol is denoted by
j1 m1
J2
j3
(C.1)
M33
where Ji representation labels (angular momenta) and m, their respective components (magnetic quantum numbers). For the symbol to be non-zero, two primary selection rules must be satisfied:
1. Conservation of the m quantum number: my + m2 + mg = 0.
2. Triangle inequalities for the j quantum numbers: li - ja S ja S j1 + J2. and its eyclic permutations. This ensures that the three angular momenta can form closed vector triangle.
The explicit algebraic form the 3j symbol (due to Racah) given by:
j1
j2
sm +ma+ 00 --1-----1---0 1111)1112-10
m7
m2
mg
(11 + j2 ja)((ji j2 ja) +2 in)
x
(11 + j2 + js 1)!
3
x
(je + mx) ma)!
(-1)²
z!(h + j2 - j3 - z)!(íi m1 - z)!(j2 + m2 - =))
(C.2)
(3s d2 m, 2)1(As j1 m2 + =)!'
where the summation over the integer is constrained such that all factorial arguments remain non-negative. The initial ö factor enforces the m-conservation rule.
The 3j symbols possess several important symmetry properties:
ji
j2
ja
Even permutation of columns leaves the symbol unchanged:
m1
m2
m3
j2
ja
j1
m2
ma
m1
Odd permutation of columns introduces a phase factor (-1)i+tatts;
jz
ji
ja
m2
mi
m3
(-1)á++ia+ds
ji ja
j2
mi
m2
m3
Time-reversal symmetry (negation of all m. values):
-mm
ji
m2
j2
ja -ma/
=
(-1)
m
ji
m2
j2
ma
ja
(C.3)
Furthermore, they satisfy crucial orthogonality relations, fundamental for their role transformation coefficients:
Σ (2ja 1)
ji
j2
ja
jz
J's m/y
5. m
(C.4)
m4
m2
ma
m
m2
where the is over all allowed mi, ma for fixed ýı, j2.
Another form is:
Σ (2ja + 1)
j2
js
J1
j2
j3
sm S m
(C.5)
mi
m2
m3
m
mz
ma
Ja.mm
18

---

C.2 The Wigner 6j Symbol: Recoupling Coefficients
The Wigner 6j symbol, denoted f... }. addresses the recoupling of three angular momenta. arises when transforming between different sequential coupling schemes for combining three irreducible representations of (U)... For instance, coupling ji and j2 an intermediate /12, then coupling J12 with ja to final J, versus coupling j2 and j3 to J23, then ji with f23 to J.
It is defined through
over products of four 3j symbols:
j2
j3
E
E
(-1)28-16A-ma)
ji
j2
ja
ji
j5
jo
ji
J5
J6
m2
m3
-m,
m6
j2
J6
J4
Jo
J3
(C.6)
m4
m2
mml)
-m7
mg
ml
where the and m' indices appropriately summed while respecting the 3j symbol selection rules The 6j symbol directly related to the Racah W-coefficient by a phase factor:
j1
j2
ja
=
(C.7)
j1
J5
J6
The Racah W W-coefficient, (aabed; ef), is the transformation coefficient between schemes ((a, b)e, d)e and (a, (b, d'.)c. Thus, the 6j symbol effectively captures the algebraic structure of associativity in tensor products of representations.
Symmetries of the 6j symbol are extensive:
Invariance under any permutation of its columns.
Invariance under the exchange of upper and lower arguments in any columns, e.g.:
Ji
j2
ja
j4
j4
J5
J6
31
J2
jo
These symmetries reflect the tetrahedral symmetry associated with the symbol, often visualized using Yutsis graphs. Each set {2,, jo), (o), and (j4:J5, ja} must satisfy the triangle inequalities for the 6ý symbol be non-zero.
An important orthogonality relation (one of several, including the Biedenharn- identity) is:
Σ (2js {(2ja + 1)(2jo 11 j2
j1 j4
j3
j1
j2
jz
35
J6
34
J5
36
=
6,0.1 A(i,, A(4,, (C.8)
Ja
Here, A(a,b,c) 1 if a, e satisfy the triangle inequalities, and otherwise.
remarkable connection to geometry provided by the Ponzano Donzano-Regge asymptotic formula. For large / values, it relates the 6j symbol to the geometry of tetrahedron whose edge lengths are jk + 2:
ji
j2
j3
COS
Σ (5R + 2) + k=1
(C.9)
j4
Js
J6
12
where V is the volume of the tetrahedron and 0g are the dihedral angles. This formula bridges quantum angular momentum algebra with semi-classical geometric concepts.
19

---

D Main Theorem Proofs
D.1 Binominal Local Expansion
Claim D.1 (Multiplicity- (
61 8
The irreducible representation V(llmas) appears
with multiplicity one in the decomposition V ..... @ Vlt), That is,
Vihi
V(le)
N2V(), with N Limax 1.
Claim D.2 (Schur's Lemma
Let V, be irreducible representations of a group G over an algebraically closed field. If : - W is G-equivariant linear map, then either T 0, or V = W and T a scalar multiple of the identity.
24
Claim D.3 (Recoupling via Wigner 6j Symbols [49). For any two CG coupling trees t and r', the associated contraction operations are related by unitary transformation:
C,' R
where Ry'er is product of Wigner 6j matrices acting intermediate coupling channels.
Oordering-nvarrinnce Lemma D.4 (Ordering- Invariance of Maximally Coupled Representation). Let h ..... h(
(44 & ((t) denote sequence of irreducible features, where each (l.)) is the irreducible representation of SO(3). Let Lmnax = -i 4. Then for any binary CG coupling tree T, the projection of the coupled lensor onto the Lmaxx subspace,
[cr (n (1()) ..... 114 (kmax)
invariant to the choice of T.
Proof. The tensor product (La .... . ((h) admits decomposition into irreducible components of the form D N₁ (L), where N₁. denotes the multiplicity of the spin-I. representation. It is classical fact (Claim D.I. l that the maximal total spin Lmax == E, 4 appears with multiplicity one. Consequently, the subspace ((Tmma) is uniquely defined up to a basis, and any projection onto is one-dimensional in each magnetic subspace.
Each binary CG coupling tree defines a recursive contraction C, via successive applications of the Gordah tensor product. For any two such trees and 7', the corresponding coupled features are related by a transformation C,, Ry',,,,, where Ry' is constructed from a sequence of Wigner 6j symbols. This recoupling matrix unitary (by Claim D.3) and acts the space of intermediate angular momentum.
Since (Lmax) appears with multiplicity one, Schur's lemma Claim D.2 implies that any SO(3)- equivariant transformation, such as R must act as a scalar on this subspace:
B Aptr Id, with [
By adopting fixed phase convention (e.g., the Condon Condon-Shortley convention for CG coefficients), we can choose \ = 1, yielding
[- (1) ..... h ( (kmma) = (CC) ................................ ) (kmax)
be realized the totally symmetric, trace-free subspace of the rank- tensor formed from the inputs (61) ".,"....,b h (1x) Since full symmetrization commutes with all permutations and parenthesizations, the final projected tensor is independent of the coupling order. This concludes the proof.
An equivalent interpretation is that (kmma)
Theorem D.5 (Projection identity for spherical harmonics). Let Letrij == r, ry E IR3 ((0 and ler
R (()( denote the degree-l spherical harmonic. For every integer l 0,
R ((0)))))) -((R((())) 00 (2)
where (.)z is the E-fold tensor product and the [. (00 denotes the orthogonal projector onto the irreducible SO(3) subspace of total angular momentum f.
20

---

Proof. The function R (ri)) transforms according to the irreducible (vector) representation D
of S03.. Hence the tensor power satisfies
(ri)) 0e
X D
DDD
D 60 C"!
t=0
where my is the multiplicity of D in the Clebsch- Gordan decomposition. Applying the projector [-]0 extracts the D summand, yielding rank- tensor that transforms in the same irrep as R
Because D( is irreducible, Schur's lemma (Claim D.2) implies that any two non-zero intertwiners from ( itself differ by scalar. Thus the projected tensor must be proportional to the degree-E spherical harmonic:
(T,)) 20 (0) = =c (((((())))
To determine the constant сg. evaluate both sides the north-pole direction E = 00,0, 1). In the Condon Condon-Shortley convention R()(() = and (Z) (3/(4T). so the only non-vanishing com- ponent of the tensor power corresponds the highest-weight vector [e, E). A direct Clebsch-Gordan calculation (or induction on é) shows that its norm matches that of (x) fixing cg 1. Since both sides transform identically under rotations, equality for one direction implies equality for all directions, completing the proof.
/ /(28 1)/(4))
L
Theorem D.6 (Binomial Local Expansion).
Let F4 F3 E R denote the positions of two nodes i,
and let Fiy = - F4 For each integer 2 L, let (.) denote the real-valued spherical harmonics of order Then,
R (F(g)) (7))))
(F) 00 ) = (-1))
(1)
( R(
(R (0)
(F,) R
4=0
where (.1(0) denotes projection onto the irreducible subspace of total angular momentum via Clebsch
Clebsch-Gordan Gordan decomposition. Equality holds up normalization constant depending the basis choice for R(t) and the CG convention.
Proof. By Theorem D.5 the spherical harmonic (1) (r:y) can be constructed by projecting the f-fold tensor product of first-order harmonics onto the irreducible subspace of total angular momentum
R (T()) (T( - (r:))
86
This expression follows from the identity rıj. = r - r, and the fact that R()) is linear spatial coordinates.
Expanding the tensor power via the multinomial binomial rule yields
(
- -R.
Σ Tp.
==0
PETI.
where each term Tp corresponds to a specific ordering P E II, of the tensor product, containing exactly factors of (r,) and u factors of R() (r.). The total number of such orderings [Ta]] = (.)
Applying the linear projection operator (:)(() to both sides gives:
or (8)
(1))(())) - (r.) ()))))))))))))))))))))))))))))))
Σ (-1
Σ (2) PEN
At this point, we invoke the ordering-invariann invariance lemma (Lemma D.4]. which states that the projection of tensor product onto the highest angular momentum subspace (here, E) is invariant under permuta- tion of the tensor factors. Therefore, the projected tensors [Tp](0) are identical for all P E II,, and depend only the multiplicities of (r,)), and (() (r.) within the product.
21

---

We may thus replace the sum over P PE II with a multiplicity factor times a single representative term. Letting Teep == (R (1,)) @ (R (r:)) @((-u), we obtain:
Σ [T]]0- PEIL.
(C) [T rep
( 11
(r,)) s (r.) (0)
Finally, note that (r) y contains irreducible components of order up to u, and the highest such component R(() (r). Projecting the expression onto total angular momentum thus yields:
R 0 (Tez) = -(-1) ())
(ry)) 60 (((-w (r:))
==0
which concludes the derivation.
22

---

D.2 Wigner 6j Recoupling and Node-Based Factorization
Here, we formalize the proof of the Theorem 3.3 for completeness.
Setup: We consider a typical SO(3) -quuivaana Transformer layer that performs message passing from each node j in the neighborhood of using the tensor product h, @ (Fiy)..). Symbolically,
h
new
Σ JEV(
Osy (hy
@ R (fi)).
Goal:
To show that the expensive edge-based computation over ji can be reorganized so that the tensor product portion (or at least the dominating part of it) depends only on node and a separate node j portion. This achieved by:
(F
Σ
(w( SJ (F))
together with a rearrangement (via Wigner 6j) of hy inside the product.
Theorem D.7 (Node-Based Badde-Based Factorization via Wigner 6ÿ). Let hj RRd the feature of node j. and let Oaj be any scalar weight (e.g. an atlention coefficient). In the (0(3))equivart) layer;
Σ (h, & R (((0))))
JEN(A)
reorganize (F(g into node-i and node-j parts by Theorem Dndand and then apply Wigner 6j recoupling to obtain:
(he & (T)) = Σ(-) (1) R (7.) (F)]
(Σ
[ 2 R (Fi))
JEN(
JEN(()
Proof. We commence with the definition of the (0)33)-quivarial) node convolution for node i.
Substituting the Binomial Local Expansion for (()) (riy) yields:
h
Σ hy &
Σ(-) -==0
R 11) (r.) Rd ( (1)))()
(D.I)
JEN(()
Invoking the linearity of the Clebsch-Gordat tensor product & with respect to its second argument, and subsequently interchanging the order of the finite summations (over j E (i) and E [0, E))
Eq. (D.I) is rewritten as:
h, (-1)--
(9)
Σ
(1 (r,) R( (r))) (0)
(D.2)
==0
JEN(()
We now focus on the term within the summation over j:
Tja h, & (R(()(r)) & R( (((- (r,))
(D.3)
Let UA = hy, Uc R(() (T,), and Up R( (e- (r,) represent the respective irreducible represen- tations. The term T,,, signifies specific coupling scheme: Uc (irrep u) and UB (irrep & u) are first coupled, and their product projected onto the irreducible component transforming as irrep denoted [l' @ (5) Subsequently, UA is coupled with this resulting tensor of irrep f. This scheme corresponds to (UA (Uc & "eB=) where LeB f is the fixed intermediate angular momentum.
The theory of angular momentum recoupling, governed by Wigner 6j symbols (as per Definition 2.3 ). allows for the reordering tensor product operations while preserving the final irreducible content. We seek to transform Ty, into form where UA and Up are coupled first. Specificallyy we apply
23

---

recoupling to achieve the order U (UAU(U) The transformation from (Us@(Uc&Ug) fea=)) (Uc & A Un) (for any resulting total angular momentum) is standard result Wigner- Racah calculus. The operator &3 denotes that the CG tensor product is performed with path weights modified by the appropriate Wigner 6j symbol, which accounts for this change coupling pathway. Note that the intermediate coupling of R(a)(rr (I:) and R( (r₂) results in constrained irrep & (governed by the projection operator); this gives rise to a constrained Wigner 6j coupling, where the constraint (the intermediate angular momentum being inherently managed by the 6j coefficients encapsulated within the @ operation. Thus, we can write:
(U))()))
R (r:) & R (r,)) R (r.)
(r)) 0 (h, 8 (r))
hy X
(D.4)
Substituting Eq. (D4) into Eq.
D.2
h Σ
Σ JEN(()
(r,)
@o (n, 3 (x)))
(D.5)
The term is independent of the summation index j. The operation @ like the standard tensor product @, is linear in its second argument with respect to summation. Thus, R(a) (r:)) can be factored out of the sum over j:
))
h:
Σ(T)J- (T))
Σ JEN(()
hj R (()))(
(D.6)
This expression is the factorized form of the -equivvrra node convolution as stated in the theorem.
24

---

E Proof of Equivariance
We now formally establish that the Wigner 6j convolution is equivariant under the action of the rotation group SO(3). Define the convolutional output as follows:
f(x) = Σ ( 8 (ass (h @ (r)))
We analyze the transformation of this expression under the rotation R SO(3). The transformed output is:
f(R.x) = Σ (R))
Φs (as (D)) @ (H)R
Let us define the intermediate quantity:
Sy A
Σ JEN(1)
Oig by @ (r,)
Under rotation, this transforms as:
Σ ons (D)), @ Die-w) (R)(( = (D) s D(e=a)(R)) Su
JEN((
Substituting into the convolution expression yields:
I(R x) = Σ (1()) (R)R( 03 ((D(o) s ( Sn Sa))
Since the Wigner 6j recoupling tensor @ is SO(3) -qqiivarra (as the Wigner coefficients are invariant under rotation), we may commute the group action:
D(w) (R) (r:) o0 ((D() & ( S. Dont Doot(() (R 88 S..)
where Dout (R) denotes the output representation under SO(3). Therefore:
f(R x) Dout R)) R (r;) @i S.) Dout (x).
This completes the proof of S0(()-cquivaaiance.
25

---

Time Complexity
We analyze the computational complexity of the Wigner-6j convolution, with particular attention to its dependence on the number of nodes V. the number of channels C, and the maximum angular momentum The dominant cost stems from angular momentum coupling via tensor products and Wigner-6j recoupling, while the linear self-mix step introduces an additional quadratic dependence on C. This analysis aligns with the derivations Appendix of the eSCN paper for SO(3) convolution.
Let V denote the total number of nodes (e.g., atoms), and C the number of channels per irreducible representation, The algorithm involves iterating over all valid angular momentum triplets (I1, L2, Ioo) up to order L, subject to triangle inequality constraints.
(1) Tensor Product: Each tensor product between irreps (L1, L2) incurs a cost
O L1 L2)
(2) Wigner-6j and CG Multiplication: The cost of recoupling the tensor via Wigner-6j coefficients is
O L L2 Lo)
(3) Summation Step: The final projection onto Lo includes summation step costing
O I.)
per valid triplet.
The summation step is implemented as follows:
outputs = self._sum__ tensors (oou for ins, out in zip(instructions, outputs) if ins. iuoutt i_out]
The per-node complexity of Wigner-6j convolution is therefore:
ΣΣΣ (Chi2 CL.) O(CL))
11=0 42=0 L.=0
Linear Self-Mix. After the convolution, channel-mixing layer (e.g., MLP) acts independently on each irrep. This operation mixes C' channels across each irrep of order L,, with complexity:
Total Complexity. Summing over all nodes V, the total computational complexity is:
O(VICL + 1))
26

---

G
Additional Lemmas
Lemma G.1 (Many-Body Reduction), Using Wigner 6j convolution, a multi-atomee cluster expan- sion can be evaluated in O((V)) time instead of O(EE)).
Proof. We decompose the MACE architecture into two computational stages and show that each admits nodewise computation at O([V) cost.
Stage 1: (0(3)- (003)-Equvarrint Convolution. each node i E V,
The convolutional component of MACE computes, for
A h
JEN(S)
which has naive complexity O(EE) over all nodes due to edge enumeration.
Stage Self Tensor Product and Projection. The self tensor product (A,) @P and projection [-]()) are purely local operations, independent of neighbors. Since they apply once per node, they require ([Vll) total time.
Key Step: Nodewise Refactorization via Wigner 6ý. identity:
Using Theorem
we invoke the following
Σ hy & (z)) ())
Σ hy (x)) === JEN(s)
R (r,)
u=0
JEN(1)
where T denotes tensor contraction via Wigner 6j recoupling.
Define global moment tensors ((-=u) := Ejev hy & (ry) These are precomputable in O(VV)) time, and reused across all i E V. The refactorized form becomes
d
R ((,((,,)) (r,) & M((-w)
which is independent of the neighborhood structure and thus computable per node in constant time (for fixed t).
The precomputation of {M Yu= costs O((V). The nodewise contractions via @ are constant-time per node for bounded yielding an overall convolutional cost of ((V)). Since the subsequent self tensor product and projection are already node-local, the total complexity of MACE becomes O((V)).
Complexity.
27

---

H
Technical Details Behind E2Former
Input:30 3D Graph
X;
Input 3D Graph
Emberiing
ir
ir2scalar
Linear
Atom Type
7
K
F2j
Layur
MacMul
Linear
RBF
SH
RBF
Linear
Softmax
6j-TP
Layer Norm
MatMul
MatMul
FFN
6j-TP
x
x
Layer Norm
Linear
FFN
Linear
Gate
Head
X2
Linear
(a) E2former architecture
(b) E2Attention
Figure Overview of the E2Former architecture. (a) The main network alternates E2Attention blocks with feedforward layers, repeatedly refining node embeddings from 3D molecular graph. (b) Within each E:2tttention block, scalarized queries/keys (via ir2scalar) are combined with distance- dependent features (RBF) and convolutions (6j-TP), updating the node embeddings equivariantly. (c) The final readout incorporates atomic types and radial/spherical expansions (RBF, SH) into gated projection that produces the per-atom output y/-
In our development, we not only want mathematical integrity but also practical usefulness. Thus, E2Former is not solely characterized by Wigner-6j conv, but rather an architecture that integrates both efficiency and sigmificant engineering components. The benefit of Wigner-6j conv enables us to reallocate the computational budget to increase expressivity elsewhere, such as adding more layers, wider hidden dimensions, or more attention heads.
Architecture Overview. The E2FORMER architecture adheres to the general transformer paradigm, commencing with an embedding layer that generates initial E(3)-equivariant node features. These features are subsequently refined through a stack of E2former blocks, denoted as TransBlock in our implementation. Each TransBlock employs a pre-normalization strategy, structured as: Normalization - Transformer Layer > Residual Connection + Normalization - Equivariant Feed-Forward Network (FFN) - Residual Connection.
Initial Embedding and Feature Representation. The generation of initial equivariant node features, h," for each node i (e.g., atom) with input coordinates and type, is performed by an Initial Equivariant Embedding module. This module constructs features by aggregating information from the local neighborhood. Interatomic distances are encoded using radial basis functions (RBFs), while relative positions, rj r,, are represented using spherical harmonics, R! (7, p.). up to specified maximum degree Imax- The resultant features, h, comprise collection of irreducible representations (irreps) of SO(3). Within each TransBlock, Equivariant Normalization layers are applied before both the attention and FFN sub-layers. These layers operate by normalizing features independently within each irrep channel, which is crucial for stabilizing training and enhancing model performance.
Equivariant Feed-Forward Networks. Subsequent to the attention mechanism, node features are processed by Equivariant feed-Forward Network. E2FORMER accommodates variety of FFN
28

---

types, configurable via the Ifn_typo parameter. These include standard equivariant Multi-Layer Perceptrons (MLPs) that operate on spherical harmonic coefficients (c.g., FeedF www.rrrwwwwwwwrrk..... s3, potentially incorporating grid-based non-linearities inspired by eSCN and EquiformerV2 42 ) as well as explicit many-body interaction modules [6] that integrate equivariant two-body or three-body tensor products. This modular design permits tailored feature processing contingent upon the specific demands of the task.
E2Attention Mechanism. The central component enabling feature interaction in FORMER is the E2Aitention mechanism, which builds upon the Wigner-6ý Convolution to update node embeddings. A key implementation consideration lies in the treatment of positional information. While many spherical EGNNs normalize relative positions, this practice may discard essential directional cues in Wigner-6j Convolution, which is inherently node-centric. To preserve relative geometric information, we retain unnormalized absolute positions the convolution, and instead apply normalization during the attention coefficient computation.
E2Attention explicitly models and adaptively aggregates contributions from multiple spherical harmonic orders. Under the attn_type= aalpe=aallooder" configuration, the mechanism includes zero-order (scalar) path that captures isotropic interactions, a first-order (vector) path using Wigner-6ý Convolution configured for order-1 interactions, and higher-order paths (e.g.. second-order for = 2) to model more complex anisotropic effects.
All-Order Attention Paths.
Computation of Attention Weights. Attention weights azj computed from projected scalar queries and keys derived from h; and hj, enriched by radial basis function (RBF) embeddings of ||rai|| and optionally, learnable embeddings of atomic types 21, 2j. Geometric information be in- corporated into attention in various ways, such as through the which directly integrates spherical harmonics into the attention score computation.
_appaa" configuration,
Gated Aggregation of Orders. A critical component of E2Attention is the Gated Aggregation of
Orders. The contributions from the zero-order (m!)). first-order (m!)), and second-order (m
(2)
pathways are adaptively combined via a learnable gating mechanism. Specifically, the scalar (1 0)
components of the central node's features h, are processed through a small MLP to produce gating
(1) coefficients 9; +9; +9: The aggregated message from neighbor to node i is given by
myj
sj
The updated node feature h is then computed by summing the attentionn messages from all neighbors:
h = Σ anymi mij-
29

---

Orignotn water ansoraa
Figure 6: Force MAE on system of around 2000 atoms, where chignolin peptide is wrapped with water.
I Addtional Experiments
L.1
QM9 Results
We additionally evaluated our method on the QM9 dataset as a quality check. Due to computational constraints, we report results three representative energy metries: Ug. HOMO, and LUMO. E2Former demonstrates competitive performance compared to Equiformer V2 and its predecessor Equiformer. while GotenNet [2] achieves the best overall results. Nonetheless, we emphasize that our model is primarily designed for larger systems, whereas QM9 represents a relatively small-scale benchmark.
Table 6: Performance on QM9 (U,, HOMO, LUMO)
Method
Uo
HOMO
LUMO
E2Former
6.43
14.2
13.6
GotenNet
3.37
13.4
12.2
Equiformer V2
6.17
14.4
13.3
Equiformer
6.59
15.4
14.7
I.2 Comparison with Equiformer V2
To ensure better comparison with the Equiformer V2 model, we aligned the experimental settings between E2Former and Equiformer V2. Specifically, we adopted identical hyperparameters and architecture configurations for both models. including the number of layers, maximum angular momentum orders, and radial basis functions. This alignment guarantees controlled comparison and eliminates confounding factors arising from differing model capacities or training settings.
Table 7 summmrrzzes the performance in error (F), and inference speed (measured in samples per second). The results indicate that E2Former only achieves better accuracy on both energy and force predictions but also delivers significantly faster inference speed under matched conditions.
of energy mean absolute (E), force mean absolute
Table 7: Comparison between E2Former and Equiformer V2 under identical hyperparameter settings.
Method
E
F
Number of Layers
Luxx
M
Inference Speed (samples/sec)
E2Former
20.5
270
12
3
34
Equiformer V2
23.47
296
12
3
22
30

---

Hyperparameters
Table 8: Hyperparameter Configuration for E2Former OC20, OC22, and SPICE
perpeerrncctrr
Deserigtion
Goneral Priiting
optin Ir. snstinl
000015
0,0022
Innial lcening nae for the uptiniecr. Triining bach ste.
optin basch.oize
Rx OC2010222 48 for SPCCB
2odel. backbone, encoder, umbed, dax
256
256
Exheläny dimeraion eact aod.
2:da1 backbooe. hioden sice
256
Hildes sice for intemndiine layes. Iothere lyyers
8:dal hackhone min, layere
aux prighborg
neighboss
message passing.
trrepe_nnde,
Imope
Sead
15x0em 1idee-tSueeeeeeeeee 16
best
s:dal backbous. atta scular head
16
16
Size of sctlar stemic tead pujjections.
2:dal backbone, mon, attn seade
12
52
Number malti-hcl aueaam arna per beprr
a:del backbooe auaver..of of basio
Number rdaal basks fuactions
2:da1 backbone max radius
negtbantued
aapas,
dispoat)
bacthone. droe-peth_rate
Seechati depthrtrop
ssdil backbeos, basis_typp typs
Type nalial cuboddnng (Gassiia sncaring).
sodal hacthone. nome Jagar
Tapprr nomm.th
Tayer, nomm
Noemscostion layer rype syerNim ophoiiaa hree).
sodal hackhone. attn_type
Anamicn mechumion coneting sphrrical
type
alpbu
prodactda
type
lisl-ience
31

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
The claims made in the abstract and introduction directly align with the core contributions: introducing the Wigner 6j convolution, reducing ((&*) complexity to ((VV),, and achieving competitive or superior performance on OC20, OC22, and SPICE. See Sec. (Introduction) and Sec. 4 (Results).
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
It is fine include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Limitations, such as the reliance on spherical harmonics and potential ineffi- ciency for very small graphs where edge-level operations dominate, are discussed in Sec. 5. We also acknowledge assumptions of perfect molecular geometries.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed not penalize honesty concerning limitations.
3. Theory assumptions and proofs
32

---

Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: All theoretical claims are accompanied by clearly stated assumptions (e.g.- irrep decompositions under (3())) and full proofs in Appendix A. Proof sketches are included in the main text (Sec. 3).
Guidelines:
The answer NA means that the paper does not include theoretical results, All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We detail all training protocols, datasets, and evaluation metries in Sec. 4 and Appendix B. Full experimental setup is sufficient to reproduce key results.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to model. In general. releasing code and data often one good way to accomplish this, but reproducibilityy can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that are appropriate to the research performed.
specific model and empirical evaluation, may
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily the architecture clearly and fully.
model architecture, the paper should describe
(c) the contribution is a new model (c.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open- open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibiliity In the case of closed-source models, it may be that access to the model is limited in
33

---

some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We provide anonymized links in the supplemental material the code reposi- tory and data preprocessing scripts. Upon acceptance, we will release these publicly under MIT license.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.cm cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
wttpps.ccpublic/guiiieee guwdes/ CodeeSbmiia ssi onlolicy) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification: Data splits, optimizer (AdamW), hyperparameter selection, and training epochs are all included in Appendix B and Table 2.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification:
This follows the community standards.
Guidelines:
The answer NA means that the paper does not include experiments.
34

---

The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative eπror rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: We report GPU type (A100 40GB), average runtime per epoch, and total compute usage per benchmark task (Appendix C).
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental rums well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics httppe//hharrrpp...cooa////etiaa
Answer: [Yes]
Justification: The study complies with the NeurlPS Code of Ethics. No sensitive data or personally identifiable information used.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive socictal impacts and negative socictal impacts of the work performed?
Answer: [Yes]
35

---

We discuss societal benefits in molecular simulation and drug design in Introduction.
Justification:
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation. generating fake profiles, surveillance), fairness considerations (c.g.. deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (c.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our released models and datasets are low-risk and not subject to misuse concerns. No generative models scraped data involved.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification:
We use OC20, OC22, and SPICE datasets, each cited with license information.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
36

---

The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g.. website), the copyright and terms of service of that source should be provided.
If assets released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided. this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and the documentation provided alongside the assets?
Answer: [NA]
Justification: No new datasets or pretrained models are released. Only architectural code and training recipes are shared.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/mmodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve human participants or crowdsourcing tasks.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: No research involving human subjects conducted.
Guidelines:
37

---

The answer NA means that the paper does not involve crowdsoureing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question:
Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [No]
Justification: LLMs were not used in the development of methods or experimental results. Editing assistance may have been used, but not for scientific reasoning or generation.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to LLM policy httt::///eaarrppcccc////ofeeeeee for what should or should not be described.
38


Reviews:
[{"review_id": "fzvUwi3IoS", "reviewer": "Reviewer_ZFTk", "summary": "This paper introduces a novel graph neural network architecture designed to address computational inefficiencies in Equivariant Graph Neural Networks (EGNNs), particularly for applications in molecular modeling. The key innovation is the introduction the concept of Wigner 6j coupling from physics, which shifts the computational focus from edges to nodes in molecular graphs, reducing the complexity from edge-dependent $O(|E|)$ to node-dependent $O(|V|)$.", "questions": "Please consult the Weaknesses for questions.", "limitations": "yes", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n1.  Methodology: The introduction of Wigner 6j convolution is new for the graph learning community, it shifts computation from edge-based to node-based operations, which is beneficial for handling large molecular systems efficiently.\n\n2. The proposed E2Former shows good experimental results on OC20 and SPICE dataset.\n\nDiscussion on Expressiveness:\nThe paper lacks a thorough discussion regarding the expressiveness of the Wigner 6j convolution. It remains unclear whether the convolution method proposed in E2Former retains the full expressive power compared to traditional full tensor product methods.\nMotivation in Context:\n\nGiven the extensive research focused on accelerating Clebsch-Gordan (CG) tensor products of spherical harmonics, the motivation for introducing E2Former may appear somewhat limited. More clarity on how this approach distinctly advances existing acceleration techniques would be beneficial.\n\nOrganizational Structure:\nThe paper’s organization can be improved for clarity. For instance, the precise definition and implications of \"linear-scaling\" should be prominently highlighted in the introduction to better guide readers through the significance and impact of the proposed approach.\n\nMissing related works on invariant neural networks with invariant polynomials:\n[1] Weisfeiler Leman for Euclidean Equivariant Machine Learning;   ICML 2024\n[2] A new perspective on building efficient and expressive 3D equivariant graph neural networks;    Neurips 2024", "was_revised": true, "final_justification": "I'm generally satisfied with the response of my review and also other reviewer's reviews. Although the practical usage of E2Former still remains, I will raise my score to the positive side of the borderline.I'm generally satisfied with the response. Although the practical usage of E2Former still remains, I will raise my score to the positive side of the borderline.", "rebuttal": "Thank you for your detailed review and helpful feedback. We address your concerns below and describe the improvements we will make in our revision.\n\n**W.1** The reviewer raised an important and insightful question regarding the clarity of expressiveness of the Wigner 6j convolution. We deeply appreciate this thoughtful observation and acknowledge that this fundamental aspect warrants thorough clarification. To address this valuable concern, we emphasize that our Wigner 6j convolution maintains the exact expressive power equivalent to conventional SO(3) convolution methods.\n\n**Theoretical Equivalence**: As demonstrated in Theorems 3.2 and 3.3, the Wigner 6j convolution provides exact representation of the complete angular basis used by traditional CG-based convolutions. The methods are therefore mathematically equivalent by construction. The Wigner 6j recoupling represents a unitary transformation of angular momentum channels, ensuring no loss in representational capability while delivering significant computational benefits.\n\n**Empirical Validation**: The results presented in Figure 2(b–d) provide evidence of this theoretical equivalence, demonstrating that our convolution produces numerically **identical outputs** compared to conventional SO(3) convolution. This empirical validation confirms the expressiveness equivalence while achieving significant efficiency improvements that make the approach practically advantageous.\n\nFollowing the reviewer's feedback, we will add a dedicated subsection to the main text that directly addresses expressiveness equivalence. This addition will clarify the theoretical foundation and demonstrate that representational power is fully preserved.\n\n**W.2** We greatly appreciate the reviewer's insightful perspective on existing CG acceleration methods. Their observation highlights an important distinction that underscores the unique value of our contribution.\n\n\nPrior methods such as eSCN (ICML ’23), Gaunt Tensor Product (ICLR ’24), and SPHNet (ICML ’25) optimize individual tensor product computations, significantly accelerating performance yet remaining constrained by $O(|\\mathcal{E}|)$ scaling, as each edge still necessitates a separate tensor product calculation. E2Former introduces a paradigm shift by drastically reducing the total number of tensor products. Leveraging Wigner 6j recoupling theory, we transform computations from an edge-based approach to a node-based formulation, reducing complexity from $O(|\\mathcal{E}|)$ to $O(|\\mathcal{V}|)$. Importantly, our method is orthogonal to and complements existing Clebsch–Gordan (CG) optimizations; the smaller set of node-based tensor products can further exploit these optimized implementations, offering compounded efficiency gains. We will explicitly highlight this synergistic potential as part of our future work.\n\n\n**W.3** We appreciate the reviewer's valuable feedback regarding the need for clearer organization and more prominent definition of \"linear scaling\" in the introduction. To address this concern, we will add an explicit definition of \"linear scaling\" at the beginning of the introduction, clarifying that this refers to computational complexity that scales linearly with respect to the number of tensor products, which is $O(\\|\\mathcal{V}\\|)$ where $\\|\\mathcal{V}\\|$ represents the number of nodes. We apologize that this fundamental distinction was not made sufficiently clear in the original manuscript.\n\n**W.4** We appreciate the reviewer's suggestion to include recent works on invariant polynomial networks, specifically \"Weisfeiler Leman for Euclidean Equivariant Machine Learning\" (ICML 2024) and \"A New Perspective on Building Efficient and Expressive 3D Equivariant Graph Neural Networks\" (NeurIPS 2024). These works offer complementary perspectives through different invariance structures and polynomial expansions. We will incorporate discussion of these works in Section 6 (Related Work) and identify their integration with our node-centric convolution as a promising future research direction."}, {"review_id": "Hj0g4TzCHc", "reviewer": "Reviewer_t8Wk", "summary": "The paper introduces the E2Former, a new spherically equivariant architecture for molecular modeling. At its core the model uses the novel Wigner 6j convolution, which is mathematically equivalent to conventional SO(3) convolutions, but scales with the number of nodes instead of the number of edges, significantly reducing computational cost. E2Former shows competitive results on several benchmark datasets and is also demonstrated to perform well in molecular dynamics simulations.", "questions": "- At the bottom of page 5 I am not sure why the identity reads $A \\otimes (B \\otimes C) \\cong (A \\otimes B) \\otimes^{6j} C$ shouldn’t the two sides be equal and not just isomorphic according to equation (2.4)?\nIn the following sentence the effective commutativity of the Clebsch-Gordan decomposition is invoked (leading to another isomorphism) but is this really needed here? Since the commutation happens inside a projector onto l-irreps, commutation should be possible without using effective commutativity. \nAs far as I understand the key point of the derivation is that the two sides in Theorem 3.3. are equal and not just isomorphic. So even though both isomorphisms are of course formally correct, I think it would add clarity to emphasize the equality of the expressions.\n- Could the authors include an explicit reference to [1], the original paper of the SCN baseline, which is missing so far?\n- Could the authors also provide results for their model on the OC22 S2EF-Total test split and include the results of eSCN in the table?\n\nMinor comment:\n- On page 4 in theorem 3.2., why is there a ‘k’ index on the spherical harmonic? It seems to me like it should be either an ‘l’ or a ‘u’\n\nI am willing to increase my score if the above points are adequately addressed.", "limitations": "The authors mention limitations in the NeurIPS checklist and state that they were also discussed in section 5. However this discussion is apparently no longer present in section 5. It would be good to reintroduce the discussion of limitations into the main text.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "- Strengths:\n    - The paper tackles a relevant problem, namely the acceleration of equivariant GNN architectures in order to scale them to increasingly large molecular systems. The introduced Wigner 6j convolution has the potential to be used to speed up a variety of different architectures.\n    - The math behind the Wigner 6j convolution is explained in an intuitive fashion, while also being rigorously derived in the Appendix.\n- Weaknesses\n    - The discussion of baselines in the experiments section is incomplete. The SCN architecture for example is never mentioned in the text and the corresponding paper [1] is not cited.\n    - The number of baselines for the OC22 dataset is very small. It is also not clear why the authors decide to only report metrics on the validation split. Evaluating the performance also on the test split would allow to also compare against the eSCN architecture (like in [2]).\n\n[1] Zitnick, C. L et. al. Spherical channels for modeling atomic interactions. In Advances in Neural Information Processing Systems, 2022.\n\n[2] Yi-Lun Liao et. al. Improved equivariant transformer for scaling to higher-degree representations. arxiv preprint arxiv:2306.12059, 419 2023.", "was_revised": true, "final_justification": "The rebuttal addressed my remaining concerns. I think that especially the introduction of the novel Wigner 6j convolution layer is is very relevant for the community, justifying the publication at NeurIPS.", "rebuttal": "Thank you for your detailed review and helpful feedback. We address your concerns below and describe the improvements we will make in our revision.\n\n**W1-2 Q2-3** We agree with the reviewer that a more thorough comparison with relevant baselines strengthens the paper. We apologize for the omission of the **SCN** architecture [1] in our discussion and bibliography. In our revised manuscript, we have:\n\n1. Added a proper citation and discussion of the **SCN** model in the related work and experiments sections.\n2. Expanded our results on the Open Catalyst 2022 (OC22) dataset to include the **S2EF-Total test split**. This allows for a direct comparison with more models, including **eSCN**.\n\nThe new results are presented in the table below and have been integrated into the main paper. **E2Former** demonstrates highly competitive performance, outperforming **eSCN** and **GemNet-OC** on energy predictions and achieving strong results on forces, all while maintaining its significant computational efficiency advantages.\n\n| Model | Energy MAE (meV) ID | Energy MAE (meV) OOD | Force MAE (eV/Å) ID | Force MAE (eV/Å) OOD |\n| --- | --- | --- | --- | --- |\n| **E2Former (ours)** | 345 | 745 | 0.0243 | 0.0352 |\n| eSCN | 350 | 789 | 0.0238 | 0.0357 |\n| EquiformerV2 | 263.7 | 659.8 | 0.0216 | 0.0327 |\n| GemNet-OC | 374 | 829 | 0.0294 | 0.0396 |\n\n**Q1** You are correct. Thank you for this insightful comment and for spotting this inconsistency.\n\nThe relationship should indeed be a strict equality, not an isomorphism. The main point of our derivation, particularly for Theorem 3.3, is that the expressions are strictly equal, and using the isomorphism symbol $\\cong$ is misleading and detracts from this key fact.\n\nWe will revise the manuscript to change the expression on page 5 from $A \\otimes (B \\otimes C) \\cong (A \\otimes B) \\otimes^{\\mathrm{6j}} C$ to $A \\otimes (B \\otimes C) = (A \\otimes B) \\otimes^{\\mathrm{6j}} C$. As you rightly pointed out, this equality follows directly, and the subsequent mention of \"effective commutativity\" is unnecessary in this context. We will remove this justification to avoid confusion and improve the clarity of the argument.\n\n**Minor comment** Thank you for identifying these notational inconsistencies. You are correct—the notation should be $l$. These typographical errors will be corrected promptly. We appreciate your thorough review and attention to detail."}, {"review_id": "876UblugYE", "reviewer": "Reviewer_ejRh", "summary": "This paper introduces E2Former, an SO(3)-equivariant transformer architecture designed for molecular modelling. The primary contribution is a new method which the authors call \"Wigner 6j convolution\", a reformulation of the computationally expensive tensor products that are central to equivariant GNNs based on SO(3) convolutions. By leveraging a binomial expansion of spherical harmonics and the Wigner 6j recoupling of angular momenta, the main computational burden is shifted from being dependent on the number of edges $\\\\lvert \\\\mathcal{E} \\\\rvert$ to being dependent on the number of nodes $\\\\lvert \\\\mathcal{V} \\\\rvert$. Reducing the complexity of the core tensor product operation from $\\\\mathcal{O}(\\\\lvert \\\\mathcal{E} \\\\rvert)$ to $\\\\mathcal{O}(\\\\lvert \\\\mathcal{V} \\\\rvert)$ results in a large computational efficiency gain, because $\\\\lvert \\\\mathcal{V} \\\\rvert$ is typically much smaller than $\\\\lvert \\\\mathcal{E} \\\\rvert$ (even in sparse graphs). The authors build the E2Former architecture around this efficient convolution and demonstrate its effectiveness through extensive experiments on various benchmarks, including OC20, OC22, and SPICE. The results show that E2Former achieves accuracy competitive with or superior to state-of-the-art models while being significantly faster in terms of both training and inference time. The practical utility of the model is further demonstrated through a stable, long-time molecular dynamics simulation that accurately reproduces DFT-level vibrational spectra.", "questions": "1. Can the authors please comment on the point outlined in Weakness 1? In particular, if I am mistaken, I'd appreciate a detailed explanation of what I am missing. If I am right about this, I hope the authors agree with me that this error should be fixed in the manuscript.\n\n2. The paper cites \"Euclidean Fast Attention\" (Frank et al., 2024), which also achieves linear scaling for equivariant attention. Could you elaborate on the conceptual differences between the Wigner 6j convolution and the approach in that work? My understanding is that your method provides a general factorization of the tensor product itself, which could be used in any context (e.g., in a MACE-like model without attention), whereas their method is specific to linearizing the dot-product attention mechanism. Is this interpretation correct? A brief comparison in the related work section could be beneficial.", "limitations": "yes", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 4, "originality": 4, "strengths_and_weaknesses": "**Strengths**\n1. The paper tackles a widely recognised bottleneck in the field of equivariant models for molecular systems, namely the computational cost of SO(3) tensor products. By providing a principled method to achieve $\\\\mathcal{O}(\\\\lvert \\\\mathcal{V} \\\\rvert)$ scaling, this work makes the core operation of many existing (and future) architectures relying on SO(3) convolutions more efficient.\n2. The core technical contribution (Wigner 6j convolution) is highly original. While the mathematical tools used (Wigner 6j symbols and angular momentum theory) are well-established, their application to reformulate and accelerate equivariant convolutions in a GNN is novel and insightful. Leveraging a \"binomial local expansion\" combined with the recoupling scheme is a clever and powerful idea and an elegant mathematical reformulation.\n3. The paper is well-written and organised. Despite the mathematical density of the topic, the authors do an excellent job of building intuition. Figure 1 provides a clear visual overview of the core idea before diving into the formal mathematics. The main text provides proof sketches that convey the key steps, while the appendix contains the full details.\n\n**Weaknesses**\n1. I might be wrong about this (in which case I apologise!), but I believe there is a rather large mistake in the manuscript (which, however, should be relatively easy to fix). The definition of spherical harmonics $Y^{(\\\\ell)}$ (Definition 2.1) seems correct to me, but the construction used to obtain the Wigner 6j convolution relies on a clever trick which seems incompatible with spherical harmonics. Rather, it seems that solid harmonics $R^{(\\\\ell)} = \\\\lVert \\\\vec{r} \\\\rVert^{\\\\ell} Y^{(\\\\ell)}$ are required to make it work. Namely, the identity $Y^{(1)}(\\\\vec{r}_{ij})=Y^{(1)}(\\\\vec{r}_i)-Y^{(1)}(\\\\vec{r}_j)$ (line 122) is crucial for the derivation, but it does not hold for spherical harmonics (it holds only for solid harmonics). The spherical harmonics $Y^{(\\\\ell)}_m$ (using the notation from the paper) for degree 1 are given by (using Definition 2.1 with Racah normalisation): $Y^{(1)}_1 = \\\\frac{x}{r}$, $Y^{(1)}\\_{-1} = \\\\frac{y}{r}$, and $Y^{(1)}_0 = \\\\frac{z}{r}$, where $r=\\\\sqrt{x^2 + y^2 + z^2}$. It is easy to see that the required linearity condition does not hold for these. However, it *does* hold for the solid harmonics $R^{(1)}_1 = x$, $R^{(1)}\\_{-1} = y$, and $R^{(1)}_0 = z$. In fact, the polynomials given for $Y^{(2)}$ in the text (lines 70 and 71) are *actually* expressions for $R^{(2)}$, because they lack the normalisation by $r^2$. I therefore believe that the authors have (unknowingly?) used solid harmonics for their implementation already (otherwise it would likely not work). The empirical results are thus (very likely) unaffected, but the authors should correct the description of their method in the revised manuscript. This also has some practical consequences for architecture design, which the authors may want to discuss, so that readers that want to use Wigner 6j convolutions are aware of them: Typical SO(3) convolutions use \"filters\" of the general form $g\\_{\\\\ell}(r)Y^{(\\\\ell)}(\\\\vec{r})$, where $g\\_{\\\\ell}(r)$ are (learnable) radial functions for the different degrees $\\\\ell$. The fact that Wigner 6j convolutions must instead use filters of the form $g\\_{\\\\ell}(r)R^{(\\\\ell)}(\\\\vec{r})$  is important in so far that the \"weight\" of \"interactions\" between nodes implicitly increases by a factor $r^{\\\\ell}$, which is undesirable for typical applications (usually, the influence of distant nodes should *decrease*). Fortunately, this should not be an issue in practice, because it can be compensated for by scaling down the radial components of filters accordingly (i.e. $g\\_{\\\\ell}(r)Y^{(\\\\ell)}(\\\\vec{r})=r^{-\\\\ell}g\\_{\\\\ell}(r)R^{(\\\\ell)}(\\\\vec{r})$),  but readers should probably be made aware of this fact.\n2. Typically, SO(3) convolutions use filters of the form $g\\_{\\\\ell}(r)Y^{(\\\\ell)}(\\\\vec{r})=r^{-\\\\ell}g\\_{\\\\ell}(r)R^{(\\\\ell)}(\\\\vec{r})$ (see above), because it makes the operation much more expressive (the radial component allows to discriminate between nodes based on distance). When this construction is used, then the whole Wigner 6j convolution *technically* still scales $\\\\mathcal{O}(\\\\lvert \\\\mathcal{E} \\\\rvert)$, because in general, $g\\_{\\\\ell}(r)$ needs to be evaluated for all $r_{ij}$. Of course, the cost of the expensive tensor product is still greatly reduced (so a large speedup is to be expected anyway), but this detail should be discussed in the text. Perhaps it would be more precise to state that only the tensor product component of the convolution is reduced to $\\\\mathcal{O}(\\\\lvert \\\\mathcal{V} \\\\rvert)$.\n3. The methodology is heavily reliant on advanced concepts from representation theory and quantum angular momentum theory (irreps, Clebsch-Gordan coefficients, Wigner 3j/6j symbols). This high barrier to entry may limit the immediate understanding and adoption of the method by members of the broader NeurIPS community who are not specialised in this subfield. This is more of an inherent characteristic of the topic than a flaw in the paper's execution, but it should still be considered. While the authors provide code for their implementation, I imagine an additional standalone script that exemplifies the core idea on a simple \"toy task\" (perhaps showing numerically that Wigner 6j convolution and ordinary SO(3) convolution are equivalent for a small point cloud) would be helpful to many readers. This way, many advanced concepts could be \"demystified\", for example, solid harmonics are just very simple polynomials in Cartesian coordinates, Clebsch-Cordan coefficients are just a tensor of certain numerical constants etc.", "was_revised": true, "final_justification": "I raised my score to 6, because the authors addressed all weaknesses in the original manuscript. \n\nThe theoretical contribution of this paper will be very impactful for the subfield of equivariant neural networks based around irreducible representation of SO(3)/O(3).", "rebuttal": "Thank you for your detailed review and helpful feedback. We address your concerns below and describe the improvements we will make in our revision.\n\n**W1 - 2**  We sincerely thank the reviewer for their exceptionally thoughtful and precise observation. The reviewer is correct to highlight the distinction between spherical and solid harmonics, and we confirm that our model does, in fact, use solid harmonics.\n\nOur implementation realizes this by computing the normalization factor $\\|r\\|^l$ externally and applying it to the spherical harmonic basis functions $Y^l(\\hat{r})$. The empirical correctness of our model, which the reviewer correctly inferred, stems from this choice.\n\nWe agree this is an important detail that was insufficiently clarified. We will revise the manuscript to state our use of solid harmonics explicitly. Furthermore, we will add a clarification on the design of the filter functions, noting the need to adjust the radial component via scaling ($r^{-l}g_l(r)$) to counteract the implicit weighting from the solid harmonic basis.\n\nWe are grateful for this feedback, which significantly sharpens both the theoretical exposition and practical guidance of our paper.\n\n**W3** We sincerely thank the reviewer for their insightful and constructive comments. The reviewer has astutely highlighted the accessibility challenges associated with the advanced mathematical formalism in our paper. While these concepts are integral to the rapidly evolving field of equivariant deep learning at NeurIPS, we fully agree that enhancing accessibility is essential to maximize the impact of our work.\n\nIn response to this valuable feedback, we propose the following steps to improve clarity and accessibility:\n\nFirst, as suggested, we will develop a standalone tutorial script. This script will demonstrate a practical “toy task,” clearly illustrating the mechanics of our method and numerically validating our theoretical claims.\n\nSecond, following the reviewer’s advice, we will introduce a new appendix section explicitly connecting abstract concepts—such as solid harmonics and Clebsch-Gordan coefficients—to straightforward polynomial functions and tangible numerical tensors. For example, we will explicitly present the solid harmonics for $l=1$ and $l=2$ in their Cartesian polynomial forms to effectively bridge abstract mathematical concepts to familiar, concrete functions. Additionally, we will enhance the clarity of these concepts in the main text itself.\n\nWe believe this  approach will improve the transition from theory to practical implementation. We thank the reviewer once again for this excellent guidance, which will substantially enhance both the impact and reach of our work.\n\n**Q2** Thank you for your insightful observation regarding the relationship between our Wigner 6j convolution approach and the Euclidean Fast Attention method by Frank et al. (2024). Your understanding is exactly correct!\n\nThe fundamental distinction lies in the scope and target of optimization. Our method provides a general factorization framework for the tensor product operation itself, which constitutes the computational bottleneck in equivariant neural networks. This factorization is agnostic to the specific architecture in which it is deployed—it can accelerate tensor products whether they appear in attention mechanisms, message-passing layers, or other equivariant operations such as those in MACE-like models. In contrast, the Euclidean Fast Attention method specifically targets the linearization of the dot-product attention mechanism through kernel approximations.\n\nThese two acceleration strategies operate at different levels of the computational hierarchy and are indeed orthogonal to each other. The Euclidean Fast Attention reduces the quadratic complexity of attention mechanisms to linear complexity in sequence length, while our Wigner 6j convolution reduces the complexity of the underlying tensor product operations that appear within each attention computation (or any other equivariant operation). Therefore, the two methods can be naturally combined: one could apply Euclidean Fast Attention to achieve linear scaling in sequence length while simultaneously using our Wigner 6j factorization to accelerate the tensor products within the attention computation.\n\nWe recognize the potential for such combinations and have explicitly identified this as a promising direction in our future work section. The integration of both techniques could potentially yield multiplicative performance improvements, particularly for large-scale equivariant models that employ attention mechanisms. We will clarify this distinction and the complementary nature of these approaches in our revised related work section to better position our contribution within the broader landscape of acceleration methods for equivariant neural networks."}, {"review_id": "5MjeVvlE6h", "reviewer": "Reviewer_7r4S", "summary": "This paper presents the Wigner 6j Convolution (Wigner 6j Conv), a spherical equivariant neural network architecture that leverages Wigner 6-j symbols to shift the computation of tensor products from edges to nodes, enhancing the computational efficiency.\n\nThe authors’ response has fully addressed all of my concerns, resulting in a logical and coherent manuscript. Given the high efficiency and broad application prospects of the proposed method (Wigner6j-conv), I recommend a **Strong Accept**. However, I encourage the authors to carefully revise the manuscript to ensure that the narrative and exposition are fully commensurate with the quality of this excellent work.", "questions": "See weakness.", "limitations": "Yes.", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 2, "significance": 4, "originality": 4, "strengths_and_weaknesses": "### S. 1 Interesting Motivation\nLosslessly reducing the number of tensor product calculations is a valuable research discovery.\n\n### W.1 Ambiguous Presentation\nThe presentation and mathematical writing of this paper have a lot of room for improvement, which has forced me to spend a lot of effort trying to clarify it. Here is my understanding. Please clarify it:\n1. In Def. 2.1, why  normalization factor $k_m^{(l)}$ does not have subscript $m$ when $m\\leq 0$?\n2. In Def 2.2, why does the summarization traverse superscript $l_1, l_2$?\n3. In Line 122, should it be $Y^{(1)}(\\vec r_{ij})\\propto Y^{(1)}(\\vec r_{ic})-Y^{(1)}(\\vec r_{jc})$? There is a lack of decentralization and it is not strict equality.\n4. Fig. 1(a) in this paper is fancy and difficult to understand, and needs to be modified. The orange dashed line is difficult to distinguish, why not use different colors to distinguish. The background color of the formula is not distinguishable, the font is inconsistent, and text \"xx dependent\" is not conducive to understanding.\n\nThe above are some small details that greatly affect the reading experience. If I have misunderstood them and caused errors in the evaluation, please forgive me. Next, I need to point out that there are big problems with the overall presentation logic of this article.\n1. The core of this article is to reduce the number of tensor product calculations, which is somewhat different from reducing complexity. It is recommended to adjust the description.\n2. Simple mathematics takes up a lot of space, including most of Sec. 2, and the proofs of Thms. 3.2 and 3.3, which are trivial and can be placed directly in the appendix. In particular, the authors introduce spherical harmonics in trigonometric form, which is completely unnecessary - for the binomial expansion, spherical harmonics in the form of $x,y,z$ are more helpful.\n3. As an article whose title emphasizes the new model \"E2former\", but the main body of the article hardly discusses the model - the highlight module in Appendix H should be moved forward to the main text (exchanged with the mathematical proof). In addition, the introduction of the model in Appendix H lacks a clear mathematical form, which is inappropriate and should be clearly given.\n\nFrankly speaking, it is very abrupt to give the proof of Thms. 2.2-2.3 in the main text, and the completion of these two paragraphs is out of tune with the completion of other paragraphs. The most important model design part is given in the appendix, making the argument and the statement unable to support each other - it is like an article recycled many times and lost its original coherence and character.\n\n### W.2 Missing Relevant Reference\nI suggest moving \"Related Work\" after \"Introduction\". In addition, it does not reflect the relevance to this article. I think the following changes should be included.\n1. Combine all existing introductions to invariant and equivariant models. Focus on the pros and cons of tensor products, such as strong expressiveness but time-consuming. In addition, the latest spherical-scalarization models, such as SO3KRATES [NC'24], HEGNN[NeurIPS'24] and GotenNet[ICLR'25], should be discussed. They completely abandon tensor products, which is much more efficient than the one in this paper. What are the benefits of retaining tensor products in this paper?\n2. The design of equivariant transformers should be discussed separately, such as SE3-Transformer [NeurIPS'20], SO3KRATES [NeurIPS'22], MEAN [ICLR'23], EquiformerV2 [NeurIPS'23], Geoformer [NeurIPS'23], GotenNet [ICLR'25], DPA2 [NPJ'24], eSEN [arXiv:2502.12147].\n\n### W3. Experimental Design and Results\n1. In the results of OC-22 in the main text, E2former does not have an advantage, which cannot illustrate the potential of this method. In addition, without seeing the ablation experiment, it is difficult to determine which module is effective.\n2. The results in Tab. 5 are inferior to GotenNet, and lack comparisons with other indicators, which makes people worry about the effect of the model on small-scale systems.\n3. Although the authors say that \"our model is primarily designed for larger systems\", the three experiments in the main text are not large enough to be called large-scale. It is recommended to refer to VINE-GATr [ICLR'25 MLMP], Geometric Hyena [ICML'25] for additional experiments.", "was_revised": true, "final_justification": "The authors’ response has fully addressed all of my concerns, resulting in a logical and coherent manuscript. Given the high efficiency and broad application prospects of the proposed method (Wigner6j-conv), I recommend a **Strong Accept**. However, I encourage the authors to carefully revise the manuscript to ensure that the narrative and exposition are fully commensurate with the quality of this excellent work.", "rebuttal": "Thank you for your detailed review and helpful feedback.   We address your concerns below and describe the improvements we will make in our revision.\n\n\n**W1.1** Thanks for spotting those notation errors. You are right—we should use $k_m^{(l)}$ consistently throughout. We will fix these typos immediately. Also, following Reviewer ejRh’s suggestion, we will update our terminology to ‘solid spherical harmonics’ to reflect the unnormalized form we use.\n\n**W1.2** Our original writing attempts to convey that $m$ is tied to $l$. To avoid confusion, we will modify the summation to be  $\\sum_{m_1 = - l_1}^{l_1} \\sum_{m_2 = - l_2}^{l_2}$.\n\n**W1.3**   $Y^{(1)}(\\vec{r}_{ij}) = Y^{(1)}(\\vec{r}_i) - Y^{(1)}(\\vec{r}_j)$ holds exactly for solid spherical harmonics, as clarified with Reviewer ejRh. Here, $Y^{(1)}(\\vec{r}_i)$ follows the standard convention of denoting positions relative to a predefined origin. \n\n**W1.4 Figure 1:** We appreciate your feedback on visual clarity. We have carefully redrawn the figure with distinct colors for different components, improved font consistency, and replaced the \"xx dependent\" labels with more descriptive terminology. We apologize that we cannot provide the revised figure here due to NeurIPS restrictions on sharing updated PDFs or external links, but these improvements will be incorporated in the final version of our paper.\n\n**W1.5 Complexity Clarification:** Thank you for pointing out the difference between reducing tensor product calculations and overall complexity. We apologize for not highlighting this more clearly and will revise our wording to be more precise.\n\n**W1.6 Overall Presentation Logic:** We sincerely thank the reviewer for the detailed and constructive feedback. We appreciate the reviewer's suggestions on the organization and agree that enhancing the clarity and coherence of our manuscript is important.\n\nWe initially chose to emphasize the mathematical derivations in the main text, given that the Wigner 6j convolution is a central contribution enabling the efficiency of our model, as demonstrated by the 7x-30x speedup in Section 4.1. However, we fully acknowledge the reviewer’s concern that a clearer connection between theory and practical architecture would greatly improve readability. Therefore, we will incorporate an **overview and diagram of the E2Former architecture** into the main body, highlighting key design choices like gated aggregation, and will clearly point readers to additional details provided in the appendix.\n\nRegarding the proofs of Theorems 3.2 and 3.3, we respectfully believe they represent meaningful contributions. The introduction of the Wigner 6j convolution specifically addresses the reduction of tensor product calculations from $O(|\\mathcal{E}|)$ to $O(|\\mathcal{V}|)$.  The theoretical justifications for these proofs are crucial: without the proof of Theorem 3.2, the factorization would only work for spherical harmonics of order 1, and it would not be clear why the binomial theorem applies in irreps space. Similarly, without introducing Wigner 6j symbols as done in the proof of Theorem 3.3, the decomposition from edge-level to node-level TPs would not be possible (for example $\\sum_j h_j \\otimes Y(r_i) \\otimes Y(r_j)$ would still require edge-scaling TPs).  The derivation of these results requires careful treatment of angular momentum coupling theory, which we believe will interest the broader ML community. However, we fully acknowledge the reviewer’s point that the proofs may appear abrupt in the current presentation. To address this, we will add an introductory explanation at the beginning of Section 3 to provide clear context and intuition. Additionally, we will greatly **simplify the proof sketches** to improve readability and ensure smoother integration within the main text.\n\nWe greatly appreciate the reviewer’s suggestions and believe these modifications will enhance the accessibility of our manuscript.\n\n\n**W2.1 Section placement:** We agree with this suggestion and will move the “Related Work” section immediately after the introduction to better contextualize our contributions.\n\n**W2.2 Missing comparisons and discussion:** We appreciate your insightful feedback, and to address your valuable suggestions, we will incorporate an expanded and detailed discussion of spherical-scalarization methods. \n\nTo frame our perspective within the broader context of EGNN, we note that scalarization is a special case of the tensor product: its characteristic steerable vector‑to‑scalar interaction is naturally expressed as a TP. For example, when an odd‑type vector interacts with a scalar, the operation can be written as $1\\times L_o \\otimes 1\\times 0_e \\rightarrow 1\\times L_o$ (with similar derivations applicable to other operations). Studying the full tensor‑product setting is therefore still meaningful. Our aim is not to claim that tensor‑product models are superior, but to **optimize** a well‑established framework—one that underlies many state‑of‑the‑art molecular‑modeling systems (e.g., OMat24; ECD, ICLR ’25; QH9, NeurIPS ’23). Moreover, tensor‑product methods remain the preferred approach for systems with periodic boundary conditions (OMat24).\n\n\n**W2.3 Equivariant transformer discussion:** We appreciate this suggestion and will create a dedicated subsection discussing equivariant transformer architectures to provide a more comprehensive overview of the field.\n\n\n**W3** We appreciate the reviewer’s feedback on our experimental design. Our primary goal is to demonstrate that E2Former achieves performance comparable to state-of-the-art spherical equivariant methods (such as Equiformer V2), but with significantly enhanced efficiency. The key contribution of our work is this scalability, enabling large-scale simulations previously impractical for most spherical equivariant models.\n\n\n**W3.1: OC-22 Performance and Ablation Study** While E2Former does not set a new state-of-the-art on the OC-22 benchmark, it achieves competitive results with substantially greater efficiency. E2Former’s training requires only 1,500 GPU hours, a threefold reduction compared to the 4,500 hours needed for the leading Equiformer V2 model. This highlights our method’s core design goal: enabling scalability and efficiency rather than extracting marginal performance gains on existing benchmarks.\n\nAs requested, we conducted an ablation study on the OC20 2M dataset. The table below compares our complete model to several variants, demonstrating that each architectural component contributes to either improved accuracy or enhanced efficiency.\n\n\n|  | Energy MAE | Force MAE | Inference Speed |\n| --- | --- | --- | --- |\n| w/o Gated Aggregation | 270 | 21.2 | 42 |\n|  w/o S2 Activation | 284 | 22.4 | 62 |\n| w/o wigner 6j conv | 273 | 21.8 | 20 |\n| Full | 275 | 21.9 | 62 |\n\n**W3.2: Small-Scale System Performance**  We appreciate the reviewer's observation on small-scale benchmarks. Our model is specifically optimized for efficiency on large-scale systems, where its primary advantages are most evident. The QM9 benchmark was included in the appendix as a quality check rather than a primary evaluation metric. As expected, E2Former's performance on this smaller benchmark is comparable to similar architectures like Equiformer and EquiformerV2, which aligns with our design objectives. We thank the reviewer for raising this point and will work to further enhance performance on small-scale benchmarks in the future.\n\n**W3.3: Real-World Large-Scale Experiments**\nFollowing the reviewer's suggestion, we added new experiments on large-scale systems. We trained the model on a large dataset consist of diverse molecules with DFT force labels and evaluated it using MD simulations of systems containing up to 6,000 atoms. Our results closely match DFT calculations and outperform baseline methods.\n\nWe first tested the efficiency and scalability on those systems. E2Former consistently achieved the highest throughput for systems up to 6,400 atoms, with its performance advantage becoming clearer as system size increased. At 3,200 atoms, E2Former processes data nearly three times faster than MACE. Importantly, E2Former is the only model capable of handling simulations at 6,400 atoms, a scale where other models fail.\n\n| Atom Number | Memory Cost (GB) |  |  | Training Efficiency (Samples/Second)↑ |  |  |\n| --- | --- | --- | --- | --- | --- | --- |\n|  | Equiformer V2-22M | MACE-33M | E2Former-33M | Equiformer V2-22M | MACE-33M | E2Former-33M |\n| 200 | 14.2 | 6 | 3.9 | 1.81 | 5.1 | 4.5 |\n| 400 | 26.7 | 10.7 | 7.6 | 1.27 | 2.91 | 4.2 |\n| 800 | 53 | 23 | 16 | 0.68 | 1.61 | 2.36 |\n| 1600 | - | 38 | 20 | 0.83 | 0.81 | 2.46 |\n| 3200 | - | 74 | 40 | - | 0.41 | 1.2 |\n| 6400 | - | - | 80 | - | - | 0.59 |\n\nTo check both accuracy and speed, we tested our model on a 6,000-atom water cluster and looked at the vibration patterns of the atoms. E2Former matched the reference DFT results very closely, capturing all the important features: low-frequency movements (below 1000 cm⁻¹), the H–O–H bending motion (1650 cm⁻¹), and most importantly, the O–H stretching pattern. MACE-Large, on the other hand, had significant errors, especially for the high-frequency stretching modes. We also tested on a more complex system - a Chignolin peptide in water (~2,000 atoms) - and successfully optimized its structure while keeping force predictions highly accurate (0.484 kcal/mol/A error).\n\n*Note: We apologize that we cannot share the figure due to NeurIPS restrictions. The table below describes the key peaks from our analysis.*\n| Vibrational Mode | Method | Peak Frequency (cm−1) | Relative Error vs. DFT (%) |\n| --- | --- | --- | --- |\n| **H-O-H Bending** | CUDA DFT (Reference) | ~1650 | 0.0% |\n|  | **E2Former-Pretrain (Ours)** | **~1670** | **~1.2%** |\n|  | MACE-large-Pretrain | ~1700 | ~3.0% |\n| **O-H Stretching** | CUDA DFT (Reference) | ~3550 | 0.0% |\n|  | **E2Former-Pretrain (Ours)** | **~3550** | **~0.0%** |\n|  | MACE-large-Pretrain | ~3400 | ~4.2% |"}]

Author rebuttal:

Meta-review: This paper introduces a method to accelerate Clebsch Gordan computations in SO(3) networks via Wigner 6j convolutions. The reviewers agreed on the merit and timeliness of the approach, but initially voiced some concerns regarding technical details, related work and experiments. All these concerns were addressed in the rebuttal and the reviewers anonymously are very supportive of the paper. The reviewers now feel that this paper consists of an important and general contribution which can speed up the many architectures for 3D data which are based on 3D convolutions in Fourier space. Thus I strongly recommend acceptance as a spotlight presentation.
Decision: Accept (spotlight)