Tensor Decomposition Networks for Fast Machine Learning Interatomic Potential Computations
Authors: Yuchao Lin
Cong Fu
Zachary Krueger
Haiyang Yu
Maho Nakata
Jianwen Xie
Emine Kucukbenli
Xiaofeng Qian
Shuiwang Ji
Venue: NeurIPS 2025 poster ()
Keywords: Equivariant Neural Network
Tensor Product
Tensor Decomposition
Primary area: machine_learning_for_sciences

Abstract: SO(3)-equivariant networks are the dominant models for machine learning interatomic potentials (MLIPs). The key operation of such networks is the Clebsch-Gordan (CG) tensor product, which is computationally expensive. To accelerate the computation, we develop tensor decomposition networks (TDNs) as a class of approximately equivariant networks whose CG tensor products are replaced by low-rank tensor decompositions, such as the CANDECOMP/PARAFAC (CP) decomposition. With the CP decomposition, we prove (i) a uniform bound on the induced error of SO(3)-equivariance, and (ii) the universality of approximating any equivariant bilinear map. To further reduce the number of parameters, we propose path-weight sharing that ties all multiplicity-space weights across the O(L^3) CG paths into a single shared parameter set without compromising equivariance, where L is the maximum angular degree. The resulting layer acts as a plug-and-play replacement for tensor products in existing networks, and the computational complexity of tensor products is reduced from O(L^6) to O(L^4). We evaluate TDNs on PubChemQCR, a newly curated molecular relaxation dataset containing 105 million DFT-calculated snapshots. We also use existing datasets, including OC20, and OC22. Results show that TDNs achieve competitive performance with dramatic speedup in computations. Our code is publicly available as part of the AIRS library (https://github.com/divelab/AIRS).

Tensor Decomposition Networks for Fast Machine Learning Interatomic Potential Computations
Yuchao Lin Cong Ful-, Zachary Krueger', Haiyang Yu', Maho Nakata',
Jianwen Xie?, Emine Kucukbenli Xiaofeng Qian', Shuiwang Ji Department of Computer Science and Engineering, Texas A&M University, USA Lambda, Inc., USA
RIKEN Cluster for Pioneering Research, RIKEN, Japan *NVIDIA, USA
DDepartment of Materials Science and Engineering, Texas A&M University, USA Mike Walker '66 Department of Mechanical Engineering, Texas A&M University, USA
Abstract
(0(3)-equivariant networks are the dominant models for machine learning inter- atomic potentials (MLIPs). The key operation such networks is the Clebsch- Gordan (CG) tensor product, which is computationally expensive. To accelerate the computation, we develop tensor decomposition networks (TDNs) as a class of approximately equivariant networks in which CG tensor products are replaced by low-rank tensor decompositions, such as the CANDECOMP/PARAFT decomposition. With the CP decomposition, we prove (i) uniform bound the induced error of 53(3)-equivarianc) and (ii) the universality of approximating any equivariant bilinear map. To further reduce the number of parameters, we propose path-weight sharing that ties all multiplicity-spall weights across the O(L*) CG paths into single shared parameter without compromising equivariance, where L is the maximum angular degree. The resulting layer acts as plug-and-play replacement for tensor products in existing networks, and the computational com- plexity of tensor products is reduced from (LL)) to O(L*). We evaluate TDNs on PubChemQCR, newly curated molecular relaxation dataset containing 105 mil- lion DFT-calculated snapshots. We also use existing datasets, including OC20, and OC22. Results show that TDNs achieve competitive performance with dramatic speedup in computations. Our code is publicly available as part of the AIRS library (tttps///gttubccomm/vvalaAAAAA))
(CP)
Introduction
Symmetry fundamental aspect of molecular and material systems 1. making crucial consideration in developing machine learning interatomic potentials (MLIPs). Equivariant graph neural networks have emerged as dominant frameworks in this domain. Usually, equivariant models incorporate directional features and spherical harmonics maintain equivariance under rotation Among these, tensor product (TP) operations play a central role in fusing equivariant features, providing powerful mechanism for building expressive models that adhere 9..
symmetry [2] 3 4
30(()--quivariance [5
However, the computational cost of tensor product operations grows rapidly with the maximum angular degree L, reaching O(L)) in conventional implementations. Recent efforts to mitigate this
"qqual contribution
"Correpondencee to: Shuiwang Ji <sji@ tamu.edu>
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

cost have focused on accelerating the TP operation or applying frame averaging (FA) to enforce equiv- ariance [T0 12 13. While frame averaging is sechieeture-agnostiie it suffers from discontinuity issues [1]. On the other hand, TP acceleration techniques, such as SO(2)-based convolutions [14 LS
and fast spherical Fourier transformations [16], reduce complexity but are no longer the standard CG tensor product with the same expressivity. Thus, there remains gap in developing method that simultancously reduces computational complexity and parameter count while maintaining similar accuracy and equivariance of CG tensor product.
In this work, we propose Tensor Decomposition Networks (TDNs), a new class of approximately equivariant networks that replace the standard CG tensor product with low-rank tensor decompositions based on CANDECOMP/PARAFT that provides error bounds on equivariance and universality, ensuring consistency under (3(3) transformations. Additionally,
(CP) decomposition. TDNs introduce CP decomposition
weight sharing mechanism consolidates multiplicity-space weights across CG paths, significantly reducing the parameter count from (cT.3) to O(c) with e the parameter count of weight of each path. The resulting layer has expressive power close to conventional TP while lowering computational complexity from (LS) to O(L)). We validate TDNs on a newly curated relaxation dataset with 105 million DFT-calculated molecular snapshots, along with the established OC20 and OC22 datasets, demonstrating competitive accuracy with substantial computational speedup.
Preliminaries and Related Work
Clebsch-Gordan (CG) tensor product is fundamental operation widely used as the backbone for -qqiivaaaaa neural network, enabling the fusion of feature fields at different angular degrees. We discuss the formal definition of CG tensor product with the maximum angular degree L. Consider
two feature fields æ Di, (f1) and y 12=0 (t)), where x(f1), (f2) are type-l1 and type-l2 irreducible representations (irreps) of SO(3). The CG tensor product fuses pair of SO(3)-
representations x (f1) and (f2) into every admissible output type 16 Eg I lz S l + lg via the CG coefficients Cda.ms
f2. defined as:
(f) BoGy (2))
(t))(()))
(fa) ma
fyymi fa.ma
am ylfa)
L3 S m3 S
E
(1)
to
A triple l2,,2, lg) satisfying the CG selection rule 16, lal S S E1 + lg is referred to as a path, and each path constitutes an independent -quu)-equi mapping. Collecting the contributions from all admissible paths, the complete CG tensor product is expressed
min(f +
(2 (c)y)))))(
x OCG y
0
22=
However, the computational complexity of the CG tensor product scales as O(L)) it involves OLL)) distinct paths and O operations per path. This significant cost poses a major bottleneck in practical implementations, especially when dealing with higher angular degrees. To mitigate this computational challenge, inspired by tensor decomposition [17] to low-rank tensors, we propose an efficient approximation based on tensor decomposition techniques, specifically the CP decomposition, to reduce the complexity while preserving approximate equivariance. The detailed formulation and implementation of the CP decomposition are presented in the following section.
Invariant and Equivariant Models. Symmetry has been a widely discussed constraint when developing machine learning methods for predicting chemical properties of molecules. Invariant and cquivariant graph models have been widely used in these cases. Invariant models [18] 2 20 21 aim to consider the rotation invariant features such pairwise distance the input, and make use of these to predict final properties. Equivariant models 3 equivariant features such as pairwise directions and spherical harmonics into the model. These models are built with equivariant blocks to ensure that output features rotate consistently with any rotation applied to the input features, thereby maintaining equivariant symmetry.
23
25 further incorporate
Tensor Product Acceeration. Among these equivariant networks, tensor product 5
26
28 is one of the most important components that fuse two equivariant features into onc. It provides

---

powerful and expressive way 129 to build equivariant networks, while the computational cost of TP usually considerable. Therefore, there are several directions accelerate the equivariant networks. First direction is to accelerate TP. eSCN [1 15, 30
proposes to reduce the SO(3) convolution into SO(2) for TPP when one of the inputs TTP is spherical harmonics. Gaunt tensor product [T6] makes use of fast spherical Fourier transformation to perform the TP. The other direction is to apply frame averaging (FA) [TO [1 [2). which uses group elements from an equivariant set-valued function called frame to transform the input data and subsequently the model's ' output, enabling any models to obtain the desired symmetries. Although is flexible and has no requirement for model architectures, it faces an unsolvable discontinuity problem [Τ]]
3 Tensor Decomposition Networks
This section presents the techniques employed in the proposed Tensor Decomposition Network (TDN). In Section 3.1 we introduce a CP "e-decomposition based approximation for the tensor product, and in Section 3.2 we detail path-wise weight-sharing scheme. These strategies effectively reduce both computational cost and parameter count. In Section 3.3 we analyze the computational complexity of the approximate tensor product, and in Section
3.4
we discuss its error bound and universality.
3.1 Tensor Product and Its Approximation
The tensor product a fundamental operation in equivariant neural networks, enabling the coupling of features across multiple vector spaces. However, direct implementation of the tensor product incurs significant computational cost. To mitigate this, we introduce low-rank approximation using the CANDECOMPPPARAEAT (CP) decomposition to reduce the time complexity.
Tensor Product. Before developing our tensor-product approximation we recall the canonical definition of the tensor product in the simplest non-trivial two-order case. The multi-order coun- terpart and its CP decomposition are introduced Appendix [A] In practice, higher-rank tensors stored flattened vectors via fixed index ordering; this reshaping preserves the vector-space operations. Without loss of generality, we present the following tensor product definition. Let Vi = Rd V/2 ds, and V3 = Ida be finite-dimensional real vector spaces with ordered bases {er) { {gu dd 1=1- The tensor product V1 @ V2 is the space that corepresents bilinear maps: V3 such that m(æ, y) f( @ y). With respect to the chosen bases, m is encoded by a three-way tensor
for every bilinear m: V1 V/2 + V/3 there exists unique linear map m: V1 @ V/2
M = (Mas) E V/3 & V & V2 = dd xd2
and m(e, -b Maij gk. For arbitrary x = Σ 2 and y = Σe= yif, we have
b=
dg d1
da
m( y)
May 34 UJ 9*
M (x (g)
CP Decomposition for Tensor Product Approximation. To reduce time complexity and storage cost of tensor product, we approximate the tensor M via CP decomposition [7 of rank R. The CP decomposition writes the tensor product as sum of R rank-1 tensors by decomposing the three-way tensor M such that
MRij Σ
EA Ai.
where A E R B E RRRiXRR and C E XR are factor matrices capturing the modes of the
tensor. Substituting Eq. into Eq. (2) gives
dg d
d2
n A
m(x, y)
Σ
Bir
Key,gk.
A=1
1=1
Then we rearrange the summation to obtain
Rt
di
de
ds
Bir
Σ
Cjrli
Σ
Acrgk
3

---

21
Z
22
2
y
CG Tensor
A1
Ag
B
R
y
0= 000=1 8=2
CP Decomposition
Figure 1: Illustration of approximating the CG tensor product using CP decomposition. The input features z and both consist of irreps with { = 1, and the CG tensor product produces output feature containing irreps with L 0, 1,2.
For clarity, the above approximation for the order-two tensor product m(x, y) can be expressed in matrix form as
m(a. y) A(B の C²
where @ denotes the Hadamard product. By the universal property that every bilinear map m: V x V/2 V33 factors uniquely linear map m: V1 V22 V3, the tensor product covers important instances used in equivariant learning, notably the CG tensor product (6] and the Gaunt tensor product [16]. Therefore, the tensor product approximation be employed for those specific cases. In this paper, we primarily discuss CG tensor product approximation.
CP Decomposition for CG Tensor Product. Next, introduce the idea to make of CP decomposi- tion to accelerate the CG tensor product calculations. Specifically, the CG coefficient tensor is three-way tensor concatenating CG coefficients Cés G5,6, missible path (f1, Eg, Es) such that
1
L=1
Approximation Error
0.1
L-2
L=3
of all ad-
0.01
0.001
min(t +
0.0001
M
Chi.ka
f1=0 22=0
1E, FP2
The key idea of CP decomposition is to break down the CG coefficient tensor M into low-rank matrices. We demonstrate a simple example of CP decomposition for TP with 6, 1 and ls E {0, 1, 1, as shown in Fig. [ Eq. (4p allows for the simple batch-wise use of the Hadamard product with global hyperparameter rank R. instead of computing tensor product per path. The higher R, the more accurate the tensor product result is. For acceleration, we do not need full rank R and we can small R while keeping low approximation error. We show the rank-error curve Fig.
Rank R
Figure 2: Approximation error for different rank values R across various maximum an- gular degrees L.
3.2 Path- Weight Sharing Tensor Product
In equivariant neural networks, the CG tensor product couples features that transform under irreducible representations (irreps) of SO(3). For maximal degree L, fully connected CG tensor product introduces (LL3) paths, each associated with a distinct weight matrix. This leads to a substantial parameter count, which can hinder model efficiency. To mitigate this, we propose a path-weight sharing mechanism to reduce the parameter count while retaining equivariance.
Concatenating Irreps into a Single Channel. We first unify the multiplicities across all degrees by setting a common multiplicity Letting (1) (26+1) degree f, we concatenate the irrep axes into a single channel:
(26+1) denote the multiplicity-k irrep of
ã concat x E IR (xL+1))

---

All irreps now share common multiplicity index within, so the linear projection and CG contraction over irreps features are executed batched matrix operations whose operands reside contiguously in memory; this yields coalesced memory access on GPUs and improved cache locality on CPUs.
Path-Weight Sharing Tensor Product. We further reduce the parameter count by applying a path-weight sharing scheme. Let W
61,6 denote the multiplicity-space weight matrix associated with the path (f1, f2, 6z), where 0 S {, l2, S L and e is the parameter count of weight of each path. Instead of assigning a unique matrix for each path, we set all such matrices equal a single learnable parameter W, i.. W
éi.d2 W for every admissible path (41, E1,42,45 22, 8z). This collapses the ( distinct weight tensors of the naïve implementation into one, reducing the parameter count from (ccL)) to (c). Because the sharing operates exclusively on multiplicity indices, the irrep content of each block and hence full SO(3) equivariance is retained. The resulting layer therefore retains the classic CG tensor product structure while offering celer-of-magnituee reduction in parameters. We also extend this scheme to equivariant linear layers and use it in our main experiments.
3.3 Complexity Analysis of Approximate Tensor Product
In this section, we analyze the computational complexity the proposed approximate tensor product using CP decomposition. We first investigate the rank of the tensor product. Let ranker (M)) denote the CP rank of the three-way tensor M E IRd xd2 representing the minimal rank R such that there is an equality for Eq. (3, i.c.
ranker (M) = min REN
Mai = Σ A Co. r=1
Determining (M)kcp(M) exactly is NP-hard 17 In practice one specifies rank R as hyper- parameter and optimizes the factor matrices A Rd B R4 x C Rd2x to minimize prescribed loss. A generic upper bound ranker MM S min(d,d2, d1 d2d3) limits the choices of R. Consequently, algorithmic pipelines treat R as external choice, balancing approximation accuracy against computational cost.
By using CP decomposition, the computational cost for evaluating the approximate tensor product in Eq. reduces ( + d2 da)). significant reduction compared to the full tensor contraction cost of Odd, d22g) using Eq. 2 Similarly, the space complexity decreases from (dh to O(R(d, + d2 + d3)), which provides substantial savings when R is small.
For the approximation of the CG tensor product, where d1, dg X O the computational complexity further reduces to ORL2)). In our experiments, we select = 7L". The error curve for the CP decomposition with varying R is discussed in Section 4.1
3.4 Error Bound and Universality Analysis
The error bound and universality analysis provide theoretical guarantees for the CP decomposition of the tensor product in the proposed Tensor Decomposition Network (TDN). This section establishes the error bound for both the approximation and equivariance, demonstrating how the approximation error depends on the spectral tail of the tensor's singular values. Additionally, we formalize the universality property of the CP decomposition, showing that as the rank R increases, the CP-decomposition-baee based tensor product can accurately approximate any -qqiivraan bilinear map, thereby preserving the expressive power of the tensor product while reducing computational complexity.
Error Bound of Approximate Tensor Product. Given a rank R Σ ranker one seeks
CP-decomposition-baesee approximation M by minimizing |M - -M in chosen norm, typically the Frobenius norm. Although the non-convex optimization may admit spurious local minima, the optimization of CP decomposition possesses essentially unique kn approximation under Kruskal's condition [7], and modern alternating least-squares or gradient methods converge to under mild coherence assumptions BI). For error estimation in our setting, we specialize to the CG tensor product, where we let all irreps have the same maximum degree and all dimensions equal such (n) that d = dj d2 d3. Given the singular values ox of the mode-n matricization M(n) of M and
5

---

truncating each Mn)) to rank R = [R priori approximation error bound 32 gives
(n)2 k
1/2
M Muncatte
P
R>Rr
To quantify the loss of equiivariie incurred by CP decomposition, let R SO(3) with representation the Wigner D-matrix (RR). and the -quuiiaaarii error is estimated by
(R, æ. y) (D()) & (()) D(R) y)ll-
The following result bounds this error uniformly over (3) with the proof in Appendix B.1
Theorem 3.1 (Equivariance Error Bound of CP Decomposition): Let CG tensor M E RRdxdxd and M be the rank- R CP CP-decompositions bused approximation obtained by Frobenius minimization. For any rotation E SO(3) and any bounded representations x: y E Rd ||]|| 1lyl C, we have
E( E(R, I 2C
Σ
3
Σ
of
)
1/2
12
where Rr- [ and ok is the k-th singular value of mode-n matricization of M.
Empirical estimates of both the approximation and equivariance errors are provided in Section 4.1
Universality of Approximate Tensor Product. Because the tensor product is universal for bilinear maps, any
bilinear operator can be expressed as composition of tensor product followed by suitable M onto an equivariant subspace. Consequently, the cP-decomposition-bae approximation developed above inherits this universality: for every equivariant tensor product there exists rank- R approximation that converges to it as R >> ranker(M 'he following theorem formalizes the expressivity of our approximation scheme with the proof in Appendix B.2
Theorem 3.2 (Universality of CP Decomposition): Consider repressntttttttooo x V1
Rd V22 = Rd and co-domain V₂ - Idda, For any ( -qqiivaraa bilinear map 4 there exist B RdixR C E Rd2xR A Rd,x such that t can be writlen
④ y)
A(B C y) V3.
with R < dzd2-
4 Experiments
The effectiveness of our method is assessed on two benchmarks: the PubChemQCR dataset, which contains millions of molecular relaxation snapshots, and the established Open Catalyst Project (OCP) datasets. Section 4.1 describes both benchmarks and model configurations. Section 4.2 compares our model with several baselines PubChemQCR and its subset PubChemQCR-S, while Section 4.3 and Section 4.4 report results of our model compared to several tensor- pesoortprodutt baselines on OC20 and OC22 from OCP, respectively. Section 4.5 then analyses the computational efficiency of our approach as the maximum angular degree in the tensor product is varied. Section 4.6 further reports ablations of the proposed components across our model and a second widely used architecture.
4.1 Experimental Setup
Datasets. We use newly curated dataset PubChemQCR [3)] comprising high- sigh-fidelity molecular trajectories derived from the PubChemQC database [37. This dataset encompasses a diverse range of molecular systems, capturing potential energy surfaces and force information critical for understanding molecular interactions. The full dataset consists of 3,471,000 trajectories and 105,494,67 1 DFT-calculated molecular snapshots, with each snapshot containing molecular structure, total energy, and forces. For training efficiency, we use the smaller subset, PubChemQCR-S comprising 40,979 trajectories and 1.504,43 molecular snapshots for model benchmarking. The

---

Table Comparison of model performance on energy and force predictions for the PubChemQCR and the PubChemQCR-S dataset. Our model is trained to compare against several baseline methods on
PubChemQCR-S dataset, including SchNet [18], PaiNN [4, MACE (8, PACE 33, FAENet [T0]
NequlP [2], SevenNet [34], Allegro 35 and Equiformer 77. On the full PubChemQCR dataset, we compare our model with SchNet
18 and PaiNN
The best results are shown in bold.
VALIDATION
TEST
ENERGY MAE
FORCE RMSE
ENERGY MAE
FORCE RMSE
DATASET
MODEL
(me (metm) 4
(meV/A)
(meV/atom) +
(meV/À)
SCHNET
5.30
56.55
5.55
56.22
PAINN
5.13
46.34
5.33
46.92
SMALL SUBSET
NEQUIP
7.37
54.78
8.27
55.59
SEVENNET
8.77
47.63
10.21
48.05
ALLEGRO
10.86
60.71
10.80
61.44
FAENET
7.28
60.24
8.70
60.51
MACE
7.54
51.46
7.47
45.70
PACE
6.24
50.54
6.53
51.43
EQUIFORMER
4.69
34.67
5.38
35.11
TDN
4.46
26.94
5.01
26.43
FULL
SCHNET
7.14
65.22
7.71
67.38
PAINN
3.62
38.30
3.49
39.28
TDN
1.65
19.46
1.50
20.44
100
0.7
bogl+111
1.0
toglL+11
0.6
7L
7
6.11"
11+
Buskime (mms)
CG bascling
0.4
Equivariance
0.6
10
0.2
(a) Approximation
(b) (50333))quivariannee error L
Runtime L (log-scale)
Figure 3: Scaling behaviours of the (P-decomposition--assee tensor product under different maximum angular degree schedules: (a) approximation error, (b) -0(3)--quivariane error, and (c) runtime. The егrоr and runtime of the CP decomposition tensor product depend on the chosen rank, multiplicities. and the maximum angular degree, and are not tied specific dataset.
subset is split into training, validation, and testing sets using 20%--0%%-0% ratio, while the full dataset employs an 80%-10%- 10% split assess generalizability. To prevent data leakage, each trajectory is assigned to only one split in both PubChemQCR and PubChemQCR-S.
The second dataset used in this study is the Open Catalyst Project (OCP) [38), which publishes exten- sive open datasets of DFT relaxations for adsorbate-catalyst surfaces and hosts public Ieaderboard challenges. The flagship releases, OC20 [39] and OC22 (40), encompass thousands of chemical compositions, crystal facets, and adsorbates, serving as comprehensive benchmarks for surrogate models aiming to replace computationally intensive calculations. Each release defines multiple tasks, including Initial-Structurer energies, per-atom forces, and relaxed geometries. We conduct experiments on OC20 IS2RE and OC22 IS2RE, using the data splits and configurations specified in the official OCP repository.
Relaxed- Relaxed-Energy (IS2RE), which require accurate predictions of total
CP Rank Selection. We approximate the fully connected CG tensor product using the rank R decomposition across various configurations of the maximum angular degree L. Four rank schedules are evaluated: R = (L 1)
16 log (L + 1). R 7L, and R 7L2. For each configuration we measure (i) Approximation Error: The relative CP tensor product error, calculated [ & y) M & w) M @ y)ll (ii) Equivariance Error: The expected rqqiivaanve enror, defined as ER R, æ, v)]. where the expectation is averaged over

---

1000 random rotations R and random vectors æ, and (iii) Execution Time:
The runtime for the -P-decomposition- based tensor product under each rank schedule. Among the tested schedules, the quadratic schedule R = 71, consistently achieves the best accuracy-efficien requiring significantly less computation time compared to the CG tensor product baseline. Thus, this schedule provides the best balance between accuracy and computational cost and is adopted for all subsequent experiments. Further rank ablations are described Appendix
trade-off, while also
Model Design. Building on the capabilities of graph transformers, develop the Tensor- Decomposition Network (TDN) by modifying the publicly available Equiformer architecture [7]. Specifically, replace every channel-we wiie linear projection, normalization layer, and activation in Equiformer with batched matrix operations that act simultaneously on the multiplicity dimen- sion of each irrep block, eliminating the need for costly slicing and reshaping between tensor and vector representations. Additionally, the depth-wise tensor product mechanism is removed and the core CG tensor products within the self-attention mechanism are substituted with our rank- R UP-decomposition--aan based tensor product from Section 3.1 integrated with the path-weight-sharing scheme from Section 3.2 This design preserves the expressive power of CG tensor product while significantly improving memory and computational efficiency, Note that for each maximum degree L, we precompute rank-R CP decomposition of the CG coefficient tensor once and cache the factors. These factors are treated as constants during all training runs. We further integrate the equivariant scheme. For the OCP dataset, the same model architecture is
linear layer with the wath-weigt--share applied and a subset of experiments additionally incorporate initial node embeddings following [25]. The detailed model configurations for TDN are described Appendix C
Baseline Implementations. For the PubChemQCR benchmark, we reimplement each baseline model based on its official repository. Hyperparameters are adopted from the best configurations reported in the original papers if unspecified, are tuned using the PubChemQCR-S dataset. All model configurations of baseline models are described Appendix C
4.2 Results on PubChemQCR
We evaluate our method on both PubChemQCR-S and the full PubChemQCR dataset. For small split we compare against nine state-of-the-ard models: SchNet 18]. PaiNN
(4, MACE (8],
PACE 3 FAENet [10), NequlP (9, SevenNet (4. Allegro 35 and Equiformer 7 On the full
PubChemQCR dataset, benchmark against SchNet and PaiNN, the only basclines that scale to its size within our hardware budget. Performance is reported as mean absolute error (MAE) for energies and poot-mean-square еrror (RMSE) for forces over the validation and testing splits. Note that all results are selected based on the lowest validation energy error. The results are shown Table I the proposed TDN model achieves the lowest energy and force prediction errors across PubChemQCR. and the full PubChemQCR dataset, outperforming all baseline methods. In addition, the performance of TDN improves further as the size of the dataset increases.
Training Setup. Across both the PubChemQCR and PubChemQCR-S benchmarks, we adopt a uniform training protocol: a cutoff radius of 4.5 Å: the Adam optimizer with an initial learning rate of 5 x 10 10-4; and a RONPLLAEO scheduler with a patience of 2 epochs. All models are trained for up to 100 epochs on PubChemQCR S and up to 15 epochs on the full PubChemQCR dataset. Unless otherwise noted, experiments are executed on NVIDIA A100-80GB GPUs.
4.3 Results on OC20
(able2)summarizz our performance on the principal OC20 task IS2RE-DIRECT which predicts the relaxed adsorption energy directly from the initial geometry (no noisy-node auxiliary loss). We bench- mark against the widely-used baselines reported to date, including SchNet [18. DimeNet++ [41].
GemNet-dT [19] Seh (20). Equiformer 77] and EquiformerV2 [[5]. Metrics follow the official OC20 protocol: energy mean absolute error (MAE, eV) and energy within threshold (EwT, %) in IS2RE-DIRECH for four validation sub-splits: distribution adsorbates and catalysts (ID), out-of- distribution adsorbates (OOD-Ads), out-of-distribution catalysts (OOD-Cat), and out-of-distribution adsorbates and catalysts (OOD-Both). The IS2RF-DIRECT results are presented in Table 2 demon- strating that our model achieves performance comparable to Equiformer while being more efficient,
8

---

Table 2: Comparison of model performance on energy predictions for OC20 IS2RE- IS2RE-DIRECT
validation set without noisy-node auxiliary loss. Our model is trained to compare against several baseline methods, including SchNet [8]. DimeNNet++ [1], GemNet-dT [19, SphereNet [20),
Equiformer 7 and EquiformerV2 [15. results are shown with underlines.
The best results are shown in bold and the second best
(eV)
(R)
MODEL
113
0OD ADS
0OD CAT
0OD BOTH
AVERAGE
113
(OD ADS
OOD CAT
0OD BOTH
AVERAGE
SCHNET
0.6465
0.7074
0.6475
0.6626
0.6660
3.03
2.65
DIMENET++
0.5636
0.7127
0.5612
0,6492
0.6217
4.25
2.48
4.40
2.56
GEMNET-DT
0.5561
0.7342
0.5659
0.69664
0.6382
4.51
2.24
4.37
2.38
3.38
SPHERENET
0.5632
0.6682
0.5590
0,6190
0.6024
4.56
2.70
4.59
2.70
3.64
EQUIFORMER
0.5051
0.5545
EQUIFORMERV2
0.5161
0.7041
0.5245
0.6365
0.5953
TDN
0.5085
0. 0.6668
0.5104
0.5875
0.5683
5.21
2.54
5.04
2.98
3.94
as detailed in 4.3 This highlights the effectiveness of our architecture maintaining accuracy while significantly reducing computational costs.
Training Setup. For the SS2RE-DRECC task, we follow Equiformer's optimization setup 7 by maximum angular degree L 1, AdamW optimizer with learning rate of 2 x 10 and weight decay of 10 a batch size of 32, and cosine-decay learning-rate schedule. A warm-up is employed for 2 epochs on 122RE-DIRECT with warm-up factor of 0.2; the cosine decay then runs over 30 training epochs. 152RE-DIRECT experiments are run on a single NVIDIA RTX A6000-48GB GPU.
4.4 Results on OC22
Following the OC20 evaluation protocol, evaluate on the OC22 IS2RE-DIRECT test set,
Table Comparison of model performance on en- ergy predictions for OC22 1S2RE-DIRECT testing set. We compare with several baseline methods, in-
which predicts relaxed energy directly from the initial structure and omits the noisy-node aux- iliary loss. We benchmark against strong base- lines up to date as listed in Table 3 and re- port mean absolute error (MAE, eV) for the in- distribution (ID) and out-of-distribution (OOD) splits by averaging across the four standard OC22 sub-splits using the same split scheme as OC20 IS2RE-DIRECT task. As summarized in Table 3 TDN achieves the lowest ID MAE and the best overall average MAE, and attains the second-best OOD MAE, validating the high effectiveness of TDN.
cluding SchNet [8]. DimeNet++ 41 PaiNN [4]. Ge [19], and coGN 42 The best results
are shown in bold and the second best results are
shown with underlines.
MAE (ID) (eV)
MAE (OOD) (eV)
AVERAGE
MODEL
(eV)
SCHNET
2.00
4,85
3.42
DIMENET++
1.96
3.52
2.74
PAINN
1.72
3.68
2.70
GEMNET-DT
1.68
3.08
2.38
COGN
1.62
2.81
2.21
TDN
1.49
2.92
2.20
Training Setup. On OC22 IS2RE, we use the same optimization setup OC20 by maximum angular degree T= 1, AdamW optimizer with a learning rate x 10-4, a weight decay 10-3, a batch size 32, and a cosine learning-rate decay with a 2-epoch warm-up for 1000 epochs on single NVIDIA A100-80GB GPU.
4.5 Efficiency of CP-Decompasition--assd Tensor Product
To evaluate the speed-up achieved by our CP-decomposition-ee based ten- sor product in Section we bench- mark its inference runtime against the fully connected CG tensor prod- uct implementation in e3nn. As shown in Fig. Be the proposed ap- proximation accelerates by factors of 4.0x, 4.8x, 15.0x, 26.7x, 47.6x, and 83.6x maximum degrees I, 1, 2, 3, 4, 5, and 6, respectively.
Table 4: Throughput and parameter count comparison be- tween TDN and Equiformer across different values of maxi- mum degree L.
MODEL
THROUGHPUT (samples/sec)
PARAMETERS
EQUIFORMER (L=1)
311.7 770.8 (x 2.47)
12. 4.5M (x 0.37)
TDN (L=1)
EQUIFORMER (L=2)
71.9
27.9M
TDN (L=2)
312.4 (x 4.34)
4.5M (X 0.16)
EQUIFORMER (L=3)
26.1
54.7M
(L=3)
220.4 (x 8.44)
4.5M (x 0.08)

---

Since TDN is derived from Equiformer by replacing each CG block with CP decomposition and incorporating path- auth-weight sharing, and removing depth-wise tensor product mechanism, we further benchmark end-to-end throughput and parameter count for both networks on a single NVIDIA A100- 80GB GPU and Xeon Gold 6258R processor with a batch size of 128; detailed model configurations are provided in Appendix D The results in ssow show that TDN processes 2.47 to 8.44 x more structures per second and uses 63%% fewer parameters than Equiformer. As the maximum degree L increases, TDN achieves higher throughput and requires fewer parameters, A comprehensive time ablation of CP decomposition and path-weight sharing mechanism is provided in Table 7
4.6 Efficiency Ablation of Tr-Decompositiion-Bssss Tensor Product
To further evaluate the efficiency contributions of our proposed components, we perform ab- lations on the PubChemQCR-S dataset by re- moving the path-weight sharing tensor product and the CP decomposition of TDN while hold- ing all other components fixed. As shown in Section 446) starting from TDN, enabling both mechanisms reduces training time by roughly 78%. TDN is trained with a maximum degree L = 2, four graph transformer layers, a hidden dimension of 64. We also evaluate NequlP [9] by augmenting the base model with path-weight sharing tensor product and CP decomposition. This cuts computational cost nearly 74%. NequlP is trained with maximum degree L. 2, four interaction blocks, and multiplicity 64. All experiments are conducted on a single NVIDIA RTX A6000-48GB GPU. full table of ablation studies including accuracy results is presented in Table
Table 5: Efficiency ablation of the path-weight
sharing tensor product (PS) and CP-decomposition- based tensor product (CP). Results are shown for TDN and NequlP 9 on PubChemQCR-S dataset.
MODEL
TRAINING TIME (min/epoch)
TDN w/o CP PS TDN
19.0 4.2 (x 0.22)
NEQUIP NEQUIP + CP +PS
7.5 2.0 (x 0.26)
5 Limitations
Our approach accelerates (33 tensor products by replacing the full CG tensor product with low-rank CP decomposition. We substantiate both its theoretical speed-up and its empirical efficacy across multiple benchmarks. One limitation of our approach is rank selection: the minimal CP rank needed to attain a given approximation error is unknown in general, and computing exactly is NP-hard [17]. We therefore employ an empirical rank scheduler; however, this scheduler tailored to the CG tensor and must be re-derived when extending other equivariant tensor products. A second limitation is that path-weight sharing mechanism, while reducing parameters and memory, can introduce a small accuracy drop. In future work, we will (i) study broader families of group-equivare -quivariia tensor products, e.g., S2) convolution [14], Gaunt tensor products [16), or more complex tensor products [4), to characterize their optimal rank profiles and develop general rank-selection criteria and adaptive rank selector, and (ii) design adaptive path-selection and grouping strategies that preserve the efficiency benefits of weight sharing while recovering performance.
6 Summary
In this work, we present Tensor Decomposition Networks (TDNs), a novel framework designed to accelerate the computationally intensive Clebsch-Gordan (CG) tensor product in -quuivaaua networks through low-rank tensor decomposition. By leveraging CANDECOMP/PARTEE decomposition and implementing path-weight sharing mechanism, TDNs effectively reduce both parameter count and computational complexity while preserving the expressive power of conventional CG tensor products. We also analyze time complexity, derive approximation error bounds, and establish universality of our approach, providing theoretical guarantees. Extensive evaluations on newly curated PubChemQCR dataset and commonly used OC20 and OC22 benchmarks demonstrate that TDNs achieve comparable predictive accuracy to state-of oftt--ofthh-arr models while significantly reducing runtime. The proposed framework provides a plug-and-pla, alternative to conventional CG tensor products, making it promising approach for scage molecular simulations.
(CP)
10

---

Acknowledgments
SJ acknowledges support from ARPA-H under grant LAY IAX00005, National Institutes of Health under grant U01AG070112, and National Science Foundation under grant IIS-2243850. XQ acknowl- edges support from the Air Force Office of Scientific Research (AFOSR) under grant FA9550-24-1- 0207. We acknowledge Lambda, Inc. and NVIDIA for providing the computational resources for this project.
References
[1] Xuan Zhang, Limei Wang, Jacob Helwig. Youzhi Luo, Cong Fu, Yaochen Xie, and Shuiwang Ji Artificial intelligence science in quantum, atomistic, and continuum systems. Foundations and Trends@ in Machine Learning, 18 18(4):385-912, 2025.
[2] Johannes Gasteiger, Janek GroB, and Stephan Günnemann. Directional message passing for molecular graphs. In International Conference on Learning Representations, 2019.
[3] Victor Garcia Satorras, Emiel Hoogeboom, and Max Welling. E(n) equivariant graph neural networks. In International conference machine learning, pages 9323-9332, PMLR, 2021.
[4] Kristof Schütt, Oliver Unke, and Michael Gastegger. Equívariant message passing for the prediction of tensorial properties and molecular spectra. In International Conference on Machine Learning, pages 9377-9388. PMLR, 2021.
[5] Brandon Anderson, Truong Son Hy, and Risi Kondor. Cormorant: Covariant molecular neural networks. Advances neural information processing systems, 32, 2019.
[6] Nathaniel Thomas, Tess Smidt, Steven Kearne,, Lusann Yang, Li Li, Kai Kohlhoff, and Patrick Riley. Tensor field networks: Rotation-ant translation-equivarian neural networks for 3d point clouds. arXiv preprint arXiv:1802.082299 2018.
[7] Yi-Lun Liao and Tess Smidt. Equiformer: Equivariant graph attention transformer for 3d atomistic graphs. In The Eleventh International Conference on Learning Representations, 2022.
[8] Ilyes Batatia, David Peter Kovacs, Gregor NC Simm, Christoph Ortner, and Gabor Csanyi. Mace: Higher order equivariant message passing neural networks for fast and accurate force fields. In Advances in Neural Information Processing Systems, 2022.
[9] Simon Batzner, Albert Musaelian, Lixin Sun, Mario Geiger, Jonathan P Mailoa, Mordechai Kornbluth, Nicola Molinari, Tess E Smidt, and Boris Kozinsky. E (3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials, Nature communications, 13(1):2453, 2022.
[10] Alexandre Agm Duval, Victor Schmidt, Alex Hernandez-Garcia. Santiago Miret, Fragkiskos D Malliaros, Yoshua Bengio, and David Rolnick. Faenet: Frame averaging equivariant gnn for materials modeling. In International Conference on Machine Learning, pages 9013-9033.
PMLR, 2023.
[11] Nadav Dym, Hannah Lawrence, and Jonathan W Siegel. Equivariant frames and the impossibil- ity of continuous canomicalization. arXiv preprint arXiv:2402.16077 2024.
[12] Yuchao Lin, Jacob Helwig, Shurui Gui, and Shuiwang Equivariance via minimal frame aver- aging for more symmetries and efficiency. In Proceedings of the 41st International Conference on Machine Learning, pages 30042-30079, 2024.
[13] Behrooz Tahmasebi and Stefanie Jegelka. Generalization bounds for canonicalization: A comparative study with group averaging. The Thirteenth International Conference on Learning Representations.
[14] Saro Passaro and Lawrence Zitnick. Reducing so (3) convolutions to so (2) for efficient equivariant gnns, arXiv preprint arXiv:2302.03655, 2023.

---

[15] Yi-Lun Liao. Brandon M Wood, Abhishek Das, and Tess Smidt. Equiformerv2: Improved equivariant transformer for scaling higher-degree representations. The Twelfth International Conference on Learning Representations, 2024.
[16] Shengjie Luo, Tianlang Chen, and Aditi S Krishnapriyan. Enabling efficient equivariant operations in the fourier basis via gaunt tensor products. In The Twelfth International Conference on Learning Representations, 2024.
[17] Tamara G Kolda and Brett W Bader. Tensor decompositions and applications. SIAM review, 51(3):455-500 2009.
[18] Kristof T Schüttt, Huziel Sauceda, P-J Kindermans, Alexandre Tkatchenko, and K-R Müller. Schnet-a deep learning architecture for molecules and materials, The Journal of Chemical Physics, 148(24). 2018.
[19] Johannes Gasteiger, Florian Becker, and Stephan Günnemann. Gemnet: Universal directional graph neural networks for molecules. Advances in Neural Information Processing Systems, 34:6790-6802, 2021.
[20] Yi Liu, Limei Wang, Meng Liu, Yuchao Lin, Xuan Zhang, Bora Oztkkin, and Shuiwang Ji. Spherical message passing for 3d molecular graphs. In International Conference on Learning Representations, 2022.
[21] Limei Wang, Yi Liu, Yuchao Lin, Haoran Liu, and Shuiwang Ji. ComENet: Towards complete and efficient message passing for 3D molecular graphs. In The 36th Annual Conference on Neural Information Processing Systems, pages
2022.
[22] Congyue Deng, Or Litany, Yueqi Duan, Adrien Poulenard, Andrea Tagliasacchi, and Leonidas Guibas. Vector neurons: A general framework for eqqiivaaat networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 1220-11209, 2021.
[23] Bowen Jing, Stephan Eismann, Patricia Suriana, Raphael John Lamarre Townshend, and Ron Dror. Learning from protein structure with geometric vector perceptrons. In International Conference Learning Representations, 2021. 2021.
[24] Philipp Thölke and Gianni De Fabritiis. Equivariant transformers for neural network based molecular potentials. In International Conference on Learning Representations, 2022.
[25] Eric Qu and Aditi S. Krishnapriyan. The importance of being scalable: Improving the speed and accuracy of neural network interatomic potentials across chemical domains. In The Thirty-eighth Annual Conference Neural Information Processing Systems, 2024.
[26] Fabian Fuchs, Daniel Worrall, Volker Fischer, and Max Welling. Se (3)-transformers: 3d roto-translane iro-ttranlltion equivariant attention networks. Advances in neural information processing systems, 33:1970-1955 1981, 2020.
[27] Oliver Unke, Mihail Bogojeski, Michael Gastegger, Mario Geiger, Tess Smidt, and Klaus-Robert Müller. Se (3)-equivariant prediction of molecular wavefunctions and electronic densities. Advances in Neural Information Processing Systems, 34:14434-14447, 2021.
[28] Haiyang Yu, Zhao Xu, Xiaofeng Qian, Xiaoning Qian, and Shuiwang Ji. Efficient and equivari- ant graph networks for predicting quantum Hamiltonian. In Proceedings of the 40th International Conference Machine Learning, pages 40412-40424, 2023.
[29] Nadav Dym and Haggai Maron. On the universality rotation equivariant point cloud networks. arXiv preprint arXiv:2010.024499 2020.
[30] Xiang Fu, Brandon M Wood, Luis Barroso-Luque Daniel S Levine, Meng Gao, Misko Dzamba, and C Lawrence Zitnick. Learning smooth and expressive interatomic potentials for physical property prediction. arXiv preprint 12Xi /47, 2025.
[31] Yuning Yang. On global convergence of alternating least squares for tensor approximation. Computational Optimization and Applications, 84(2):509-52, 2023.
12

---

[32] Lieven De Lathauwer, Bart De Moor, and Joos Vandewalle. A multilinear singular value decomposition. SIAM journal on Matrix Analysis and Applications. 21(44:1253-1278, 2000.
[33] Zhao Xu, Haiyang Yu, Montgomery Bohde, and Shuiwang Ji. Equivariant graph network approximations of high-degree polynomials for force field prediction. Transactions on Machine Learning Research, 2024. Featured Certification.
[34] Yutack Park, Jaesun Kim, Seungwoo Hwang, and Seungwu Han. Scalable parallel algorithm for graph neural network interatomic potentials in molecular dynamics simulations. Journal of chemical theory and computation, 20(1 (0(11):4857-4888 2024.
[35] Albert Musaclian, Simon Batzner, Anders Johansson, Lixin Sun, Cameron J Owen, Mordechai Kornbluth, and Boris Kozinsky. Learning local equivariant representations for large-scale atomistic dynamics. Nature Communications, 14(1):579,202)
[36] Cong Fu, Yuchao Lin, Zachary Krueger, Wendi Yu, Xiaoning Qian, Byung-Jun Yoon, Raymundo Arróyave, Xiaofeng Qian, Toshiyuki Maeda, Maho Nakata, and Shuiwang Ji. A benchmark for quantum chemistry relaxations via machine learning interatomic potentials, 2025.
[37] Maho Nakata and Tomomi Shimazaki. Pubchemge project: large-scale first- cestiprinciples electronic structure database for data-driven chemistry. Journal of chemical information and modeling, 57(6): 1000-13000 2017.
[38] C Lawrence Zitnick, Lowik Chanussot, Abhishek Das, Siddharth Goyal, Javier Heras-Domingo, Caleb Ho, Weihua Hu, Thibaut Lavril, Aini Palizhati, Morgane Riviere, et al. An introduction electrocatalyst design using machine learning for renewable energy storage. arXiv preprint arXiv:2010.09453 2020.
[39] Lowik Chanussot, Abhishek Das, Siddharth Goyal, Thibaut Lavril, Muhammed Shuaibi, Mor- gane Riviere, Kevin Tran, Javier Heras-Domingo Caleb Ho, Weihua Hu, et al. Open catalyst 2020 (oc20) dataset and community challenges. Acs Catalysis, 11(10)6059-6072 2021.
[40] Richard Tran, Janice Lan, Muhammed Shuaibi, Brandon M Wood, Siddharth Goyal, Ab- hishek Das, Javier Deras-Domingo, Adeesh Kolluru, Ammar Rizvi, Nima Shoghi, et al. The open catalyst 2022 (oc22) dataset and challenges for oxide electrocatalysts. ACS Catalysis, 13(5):3066-3084, 2023.
[41] Johannes Gasteiger, Shankari Giri, Johannes T Margraf, and Stephan Günnemann. Fast and uncertainty-aware directional message passing for non-equilibrium molecules. arXiv preprint arXiv:2011.14115. 2020.
[42] R Ruff, P Reiser, J Stühmer, and P Friederich. Connectivity optimized nested graph networks for crystal structures (2023). arXiv preprint 2rXiv:2302.14102
[43] YuQing Xie, Ameya Daigavane, Mit Kotak, and Tess Smidt. The price freedom: Exploring ex- pressivity and runtime tradeoff's in equivariant tensor products. arXiv preprint arXiv:2506.1337 2025.
[44] Thomas Kipf, Ethan Fetaya, Kuan-Chieh Wang, Max Welling, and Richard Zemel. Neural relational inference for interacting systems. In International conference machine learning. pages PMLR, 2018.
13

---

A Multilinear Maps and Tensor Decomposition in Arbitrary Order
Tensor product in arbitrary order. Let N 2 2 and let V = RR4 for = 1,..., N + be
finite-dimensional real vector spaces equipped with fixed ordered bases {er IR=1- A multilinear map
(8)
m: 11 x x VN VN+1
uniquely represented by a linear map
m:
V1 X
VN
VN-1,
m(x(1) ...., ,,.,,,,(M)))) mf @ (1.00.....)))
With respect to the chosen bases, m is encoded by an (N + -wwyy tensor
M E VN+1 & Vi 6
Vy = pdd=+xxd x
defined through
dx
d Min 1i Bi ... dx ix41
m m(x (1)
(N))
Σ
(N) (N+1)
(S)
i
CP decomposition in arbitrary order. rank- R CP decomposition
lower computational cost, we can approximate M by
M Mixtrh
Σ
Ainzi ( B Exrt
where A E RRdIIIR and B(d E R4XB (i (i= 1 1,...,,,, N). Substituting into Eq. the summation the obtain
5 and rearranging
dN
dv-1 Σ
m (1) ())) =
Σ
Σ
B (1)
Σ lx=1
B NJ
(M)
A C
(N+1)
r=1
fw11=l
which can be expressed in matrix form
m(x (1) x()) se A B(UT x1) ... ((N) B(NTxx x()) 2(M)
where @ denotes the Hadamard product.
B CP-Based Tensor Product
B.1 Error Bound CP-Based Tensor Product
Theorem B.1 (Equivariance Error Bound of CP Decomposition): Let CG tensor M Rdxaxd and M be the rank- R. CP-decomposition based approximation obtained by Frobenius mininization. For any rotation R E SO(3) and any bounded representations x, y E Rd [+]|, ||yll S C, we have
e(R, v) S 2C
Σ
(nj2) k
1/2
K>
where RT = [R|/3] and of the kith singular value of mode-n matricization of M.
Proof.
Given any rotation R and represettttssstnnnoonnnnss x,
[D(R)=]| = V T D(R)
Ia V
14

---

and
[D( @ (Z & u)ll= / Tr((( () @ D(R) (D 8 (z)) v))
Tr((( v) (D D(R) y))
Tr((( (I [) 8 0
TT @ (x @y) >| @ yll
(6)
Since the tensor product Frobenius norm is equal to multiplication of Frobenius norms, have
8 R, v) = ()M @ ((R))) (( @ v)
( ( D(Ryy)
+D(R)M(= - &
D(R)y)
= =IIM - M
+|ID(R)(M - (=
(7)
=II(M - ()) ((R)) y)ll
+ ( (= 8 uyll
S |M (D(() (x 8 9)1rr
+ || M| 1
2 M M yll
2 M |
Let Mirincated be the truncated Tucker approximation of M with multilinear ranks (Rr, Rr, RT). A priori approximation error bound 32 gives
(n)2) 1/2
M Minucatee
Σ KSRR
ok
CP- De-Decomposing the Tucker core with size of (Ry, Rr, RT) yields CP rank at most R Therefore, the truncated Tucker tensor Munncated can be written as a CP tensor M with rank at most R$: in other words, there exists CP tensor M such that ranker (M) 1 R2 and
M
M Mr F < |
M
M Mununctee|| || <
3
(n))2
1/2
ok
Ry
Inserting this Eq. yields
3
(m)2) 1/2
E R. x. v) < 20
Σ
%k
nm
k> Rr
15

---

B.2 Universality of CP-Based Tensor Product
Theorem B.2 (Universality of CP Decomposition): Consider represttttsss æ E Vi R E V22 = 2 and co-domain Va = IR For any -quuvaaanaa bilinear map D there exist B RdixR, RR A E Rdd X such that can be written as
( y) A(B x о C' y) V/3
with R S d2d2.
Proof. A bilinear map is uniquely encoded by third-order tensor T E V33 Vi & V via (x. y) T(x [ @ y) and all equivariant tensors T form the subspace H (V3 @ V @ V22) (/23) Hom (V1 @ 22 Va). For simplicity, we consider the multiplicity-fres case V, O H and the general case replaces the scalars below by linear maps on multiplicity spaces. Decomposing H. into irreps gives
H
Homsora (H 2 H(t) H(ta)).
By the Clebsch-Gordar decomposition, (o (H (((a), H(E)) one-dimensional when S li + & and zero otherwise. For each path (2,, E3), there is a unique map Ch H(a) (f2)) + (taa) with matrix elements the Clebsch-Gordae coefficients Cy fa.ma.
|e, l2 S lg
as presented by Eq. I Therefore, any equivariant bilinear map & admits the expansion
Σ P1.t
C t1,2 E2,l2"
for some scalars d, fs Now we index coordinates of V1, V2 V3 by i (4,, m1), (f2, m2). and k = (43, ms). Let the third-order tensor Thi = ari Ei Taij x; Vj- To write & in the CP decomposition form, we take R = did2 and index the column dimension by pairs (i,j). Define B RdixR E RR and R by
t C fg
f1.m1..2.mg and then (<x, y) =
B Br, = 1[' = i]. Cý.(s)) = 11j = jl. with 1[:] the indicator function. Then ())))s)) = x, and (C ()()) y₁ TB C y)(s)) a; and therefore
A = Tkij-
4)),
Σ 4,1
Tkis x; y ((x,y))
In practice, we approximate T by lower-rank CP decomposition with computed A, B, C, and R d2 x (L + 1)4, corresponding to Fig.
C Model and Training Configurations
PubChemQCR baseline model configuration. Table 6 summarizes the configurations for all other baseline models. SchNet [ and PaiNN 4
used as implemented in the FAIRChem repository v1. FAENet follows their OC20 release with an ((3) stochastic frame and the "simple" message-passing variant. For MACE 8 we include the real-agnostic residual interaction block. For PACE (33,, we retain its interaction block and set the edge-booster dimension to 256. For NequIP D MACE, Allegro [35]. SevenNet [34], and PACE, we adapt the official repositories to PyTorch Geometric and use Bessel basis with polynomial cutoff smoothing, keeping all numerical settings their defaults except the irrep settings. For these models, we set identical irrep-channel dimensions according to Table acacos irrep blocks. For Equiformer (7]. each graph-transforma layer uses 4 attention heads, and the irreps embedding comprises 128 scalans and 64 vectors. All tensor-prode cetor--roduct-baee brase methods, including NequlP, MACE, Allegro, SevenNet, PACE, and Equiformer, only retain even-parity irreps and use T.maz except Equiformer for fast training. In addition, all baseline models are trained with gradient-based foree prediction except Equiformer for fast training.
16

---

Table Configurations including layer counts, hidden (maximum irrep-channel) dimensions, and batch sizes of baseline models including SchNet [8]. PaiNN 14, MACE [8], Equiformer 7
PACE
FAENet 10,, NequlP [ and Allegro 35 for PubChemQCR experiments.
MODEL
LAYERS
HIDDEN DIMENSION
BATCH SIZF
SCHNET
128
128
PAINN
4
128
32
FAENET
4
128
64
NEQUIP
5
64
16
SEVENNET
5
128
16
MACE
128
PACE
2
128
8
ALLEGRO
2
128
EQUIFORMER
4
128
32
TDN model configuration. For PubChemQCR and PubChemQCR-S datasets, TDN employs six graph-transformer layers with MLP attention, an trreps-channel dimension of 256, maximum angular degree L = 1, and a graph coph-trranffrmer layer for the force output head. For the OC20 IS2RE-DIRECT task. TDN adopts the same model configuration and builds the radius graph on the fly with cutoff of 5.0 and 500 neighbors. For the OC22 IS2RE task, TDN also uses similar configuration with six graph-transforne degree-9 BOO feature
layers, cutoff of 12.0 with 20 neighbors, and additional 25 adding to the initial node embedding.
D Additional Ablation Studies
Time ablation of path-weight sharing and CP decomposition. Table 7 presents the time ablation study of path-weight sharing and CP decomposition over the TDN model. As shown the table, CP decomposition significantly increases the GPU and CPU throughput while path-weight sharing mechanism substantially reduces the parameter count the model. Note that because TDN removes the depth-wise tensor- prosuut operator, TDN without CP decomposition and without path- sharing in the tensor-pres pussortproductt and equivariant -inear layers is not identical to the vanilla Equiformer. single NVIDIA A A100-80 80-B GPU and Xeon Gold 6258R processor with a batch size of 128. All experiments are conducted under identical irrep configurations across varying L with 256 irrep-chaer channel dimension, six graph transformer layers, and 8 attention heads of each layer.
All experiments
Table 7: GPU Throughput and parameter count for Equiformer and TDN variants with or without decomposition (CP), path-weight sharing tensor product (PS), path-weight sharing equivariant linear layer (PS-Linear) across maximum degree L. Values parentheses indicate CPU throughput.
MODEL/ VARIANT
THROUGHPUT (samples/sec)
PARAMS
EQUIFORMER TDN w/o CP TDN w/o CP + PS TDN who CP PS + PS-Linear
311.7 (7.5)
12.1
770.8 (20.2)
4.5
328.1
320.6
317.8
9.1
EQUIFORMER
71.9 (2.4)
27.9
TDN TDN w/o CP TDN w/o CP + PS TDN w/o CP + PS + PS-Linear
312.4 (8.7)
4.5
2
83.7
4.5
82,5
6.3
82.1
14.6
EQUIFORMER TDN TDN w/o CP TDN w/o CP PS TDN w/o CP + PS PS-Linear
26.1 (0.6)
54.7
220.4 (5.8)
4.5
3
26.2
4.5
25,6
8.9
25.6
21.3
17

---

Ablation study of CP decomposition rank. To further demonstrate the practical implication of CP-decompositon-biee based tensor product and the adopted scheduler, we conduct the ablation study of ranks over n-body system dataset [44] In Table 8 of the n-body system experiment, the TDN is trained with a maximum angular degree
2, three interaction blocks, and hidden dimension of 72 in 5000 epochs over single NVIDIA RTX 2080Ti-1
GPU. As shown in the table, our adopted schedule uses much smaller ranks yet matches the accuracy obtained with the largest ranks, corresponding to the results of rank schedule selection in. Note that the largest rank is deduced from the upper bound Section 3.3
Table TDN Rank-sweep table of the n-body system experiment. The last line TDN without CP decomposition (CP) and path-weight sharing tensor product (PS).
7t R-BODY TESTING MSE
EQUIVARIANCE ERROR
0.0081
0.60
0,0064
0.42
20 (OUR SCHEDULER)
0.0051
0,21
0.0040
0.02
(HIGHEST RANK)
0.0039
<.001
TDN w/o CP PS
0.0038
Ablation of CP-Decomposition-Ban -aseed Tensor Product for TDN and NequlP. Under the settings described in Section [4.6 Table shows that for TDN trained for 60 epochs, enabling both mechanisms yields slightly higher validation energy MAE and force RMSE while reducing training time by roughly 78%, suggesting only minor effect on predictive accuracy. We also evaluate NequlP [9] trained for 100 epochs. Adding the path-weight sharing tensor product and CP decomposition preserves downstream accuracy while cutting computational cost nearly 74%. Note that accuracy metrics are reported from abbreviated training runs intended only to assess relative trends; the fully tuned and converged results are reported in Table [
Table 9: Ablation study of the weight sharing tensor product (PS) and CP-deccoposittion-bseeee tensor product (CP). Results are shown for TDN and NequlP on PubChemQCR S dataset.
VALIDATION
MODEL
ENERGY MAE
FORCR RMSE
TRAINING TIME
(me V/ato)
(meV/A) 4
(min/epoch) |
TDN we CP PS
12.74
92.74
19.0 4.2 (x 0.22)
TDN
12.95
96.53
NEQUIP
11.22
90.09
7.5 (x 0.26)
NEQUIP CP PS
11.15
92.49
18

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
The main claims made in the abstract and introduction detailed in the results.
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitation in Section 5
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
19

---

Justification: We present the complete proofs of our theoretical results in Appendix B.] and Appendix B.2
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof's can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification:
We provide all the details of our method for the reproduction.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
20

---

Answer: [Yes]
Justification:
We include the code repository link in the abstract.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.co cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https guwdes/dde/eeee Cooeeeoodeesssseeeeee //nips.com htc/://bbb///gddde///eeeeeueisss ssiniioiscyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Question:
Answer: [Yes]
Justification: We have specified all the training and test details including data splits, opti- mization hyperparameters, and model hyperparameters.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer:
[NA]
Justification:
All the datasets we employ are large enough for statistical significance.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
21

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We state the computational resources in Appendix
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code of ethics
Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Question:
Answer:
[Yes]
Justification:
The research conducted in the paper conform with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We briefly describe the societal impacts in Section |
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no socictal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
22

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
This is not relevant to this work.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We have cited all relevant datasets we used.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
23

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: We the existing datasets train and evaluate our method.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/cooe/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
We do not use crowdsourcing in this paper.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification: We use LLM only for grammar editing.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs any important, original, non-standard components.
Please refer to our LLM policy fattps://neur//: httpp/////////ooeeeeeee///////// for what should or should not be described.
25


Reviews:
[{"review_id": "TW3X5mxLLB", "reviewer": "Reviewer_7Mvd", "summary": "This paper introduces tensor decomposition networks (TDNs) as a class of approximately equivariant networks whose CG tensor products are replaced by low-rank tensor decompositions, such as the CANDECOMP/PARAFAC (CP) decomposition. With the CP decomposition, this paper proves (i) a uniform bound on the induced error of SO(3)-equivariance, and (ii) the universality of approximating any equivariant bilinear map. To further reduce the number of parameters, this paper proposes path-weight sharing that ties all multiplicity-space weights across the O(L^3) CG paths into a single path without compromising equivariance, where L is the maximum angular degree. The resulting layer acts as a plug-and-play replacement for tensor products in existing networks, and the computational complexity of tensor products is reduced\nfrom O(L^6) to O(L^4). Experiment results show that TDNs achieve competitive performance with dramatic speedup in computations.", "questions": "Please see the weakness above.", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n- Introducing tensor decomposition in MLFF and prove its effectiveness is significant for the field.\n\n- This paper provides solid theoretical guarantees (error bounds, universality) for the proposed CP-based tensor approximations.\n\n- Strong empirical results demonstrating TDN's performance improvements on multiple datasets over multiple models.\n\nWeaknesses:\n\n- In line 233, what is the motivation of the proposed rank schedules?\n\n- Lack of ablation study on CP decomposition and path-weight sharing.\n\n- While CP decomposition improves computational efficiency, it also introduces abstraction: the learned rank-1 components may lack clear physical or geometric interpretations.", "was_revised": true, "final_justification": "My questions have been addressed. I keep my score as it was.", "rebuttal": "We appreciate that Reviewer 7Mvd finds our method significant. To further strengthen our contribution, we add *n*-body experiments [1,2] as an additional benchmark.\n\n> In line 233, what is the motivation of the proposed rank schedules?\n\nThe rank schedules are empirical heuristics and are corresponded to different complexity growth of the CP decomposition. The adopted one gives a relatively small SO(3)-equivariance error with low computational cost across every schedule in Figure 3 and Section 4.1. Further ablations clarify that this schedule gives a good trade-off between efficiency and approximation error:\n\n- **Ablations of ranks at $L=2$** show that using this schedule leaves downstream metrics with negligible impact. In the below rank‑sweep table of the *n*-body system experiment, the TDN is trained with a maximum angular degree $L=2$, 4 interaction blocks, and a hidden dimension of 64 in 5000 epochs. In that table, our schedule uses much smaller ranks yet matches the accuracy obtained with the largest rank to approximate the Clebsch-Gordan tensor product. Note that the largest rank is deduced from the upper bound in line 164; see [3].\n\n- Likewise, in the **ablations of CP-decomposition and path‑weight‑sharing** (in the next reply), turning CP decomposition on and off under our schedule has negligible effect on final performance but markedly improves efficiency.\n\n| R                          | *n*-Body Testing MSE | Equivariance Error |\n| -------------------------- | ----------------------- | ------------------ |\n| 10                         | 0.00910                 | 0.60               |\n| 15                         | 0.00825                 | 0.42               |\n| 20                         | 0.00648                 | 0.21               |\n| **28 (Our schdule)**      | 0.00438                 | 0.02               |\n| 81 (Highest possible rank) | 0.00437                 | <1e-3              |\n\n\n> Lack of ablation study on CP decomposition and path-weight sharing.\n\nBelow we present the ablation study of CP decomposition and path-weight sharing for both the ***n*-body experiment dataset** and the **subset of PubchemQC-Traj-Small dataset**. For the *n*-body experiment, the testing results of TDN are presented in the table, where the TDN is trained with a maximum angular degree $L=2$, 4 interaction blocks, a hidden dimension of 72, and a weight decay of $5\\times 10^{-6}$ in 2000 epochs. And for the subset of PubchemQC-Traj-Small dataset, we randomly select one conformer for each trajectory from PubchemQC-Traj-Small to build the training, validation, and testing set. The testing results of TDN over this subset are presented in the table, where the TDN is trained with a maximum angular degree $L=2$, 4 interaction blocks, and a hidden dimension of 64 in 60 epochs. Note that in the table, the *n*-Body experiment appears under the label \"*n*-Body,\" while the subset of the PubChemQC-Traj-Small dataset is labeled simply as \"Subset.\"\n\n|                                          | *n*-Body MSE | Subset Testing Energy MAE (eV) | Subset Testing Force RMSE (eV/Å) | Subset Training Time Cost (min/epoch) |\n| ---------------------------------------- | ------------ | ------------------------------ | -------------------------------- | ------------------------------------- |\n| Plain                                    | 0.00414      | 0.01274                        | 0.09274                          | 19.0                                  |\n| + Path-weight sharing                    | 0.00439      | 0.01503                        | 0.09321                          | 9.8                                   |\n| + Path-weight sharing & CP decomposition | 0.00429      | 0.01295                        | 0.09653                          | 4.2                                   |\n\nAs shown in the above table, introducing both path-weight sharing and CP decomposition does not affect the performance a lot while significantly boosting the efficiency, although we notice that introducing path-weight sharing alone will hurt the performance a bit. We believe this is because the Clebsch Gordan map is replaced by the low-rank surrogate with rank-1 factors, which act as an implicit regularizer as they filter out high-variance directions that weight sharing alone cannot suppress. We will include this table in the appendix of the camera‑ready version.\n\n> While CP decomposition improves computational efficiency, it also introduces abstraction: the learned rank-1 components may lack clear physical or geometric interpretations.\n\nThank you for raising this point. Although the rank‑1 components from the CP decomposition are indeed harder to interpret physically, they do not strip away information contained in the Clebsch Gordan tensor if the rank is sufficiently large, as shown in the empirical ablation study above and Theorem 3.2.\n\nConceptually, Clebsch Gordan coefficients are coordinates of the SO(3) tensor product in the standard coupled-angular-momentum basis. Switching to a CP basis simply means **re-expressing the same sparse linear map in a different compact rank‑1 factor basis**; the transformation is analogous to choosing a new set of axes in linear algebra, and information is re‑packaged.\n\n\n[1] Kipf, T., Fetaya, E., Wang, K.-C., Welling, M., and Zemel, R. Neural relational inference for interacting systems. In International conference on machine learning, pp. 26882697. PMLR, 2018.\n\n[2] Satorras, V. G., Hoogeboom, E., and Welling, M. E(n) equivariant graph neural networks. In International conference on machine learning, pp. 9323–9332. PMLR, 2021.\n\n[3] Kolda, T. and Bader, B. Tensor decompositions and applications. SIAM review, 51(3):455–500, 2009."}, {"review_id": "esfzBmUEIm", "reviewer": "Reviewer_cmnz", "summary": "The work proposes to use tensor decomposition, namely the CANDECOMP/PARAFAC (CP) decomposition, to accelerate the Clebsch-Gordan (CG) tensor product machine learning force field (MLFF) models. Moreover, to achieve further reduction, the authors propose a weight-sharing scheme.", "questions": "- Why is EQUIFORMER selected as the main method for acceleration? It would be nice to see if the CPD applicability and performance are not only for EQUIFORMER but also for other tensor-product-based methods.\n- Is the GPU runtime reported in Table 2? Can the authors show the CPU runtime as well?\n- Would the proposed method benefit from other tensor network (decomposition) structures? What is the author's opinion?", "limitations": "Yes, the limitations are discussed in the Appendix.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n- The addressed problem of compression and acceleration of MLFF models is under-explored in the community but is still very important.\n- An error bound analysis is provided.\n- A rank selection scheme suitable for the method is presented.\n\n**Weaknesses**\nPlease address the questions section in the relevant section.", "was_revised": true, "final_justification": "I will raise my original score, as this work is highly significant for both AI and computational material science communities.", "rebuttal": "We appreciate that Reviewer cmnz finds our method significant.\n\n> Why is EQUIFORMER selected as the main method for acceleration? It would be nice to see if the CPD applicability and performance are not only for EQUIFORMER but also for other tensor-product-based methods.\n\nWe focused on Equiformer [1] because Equiformer uses the fully-connected Clebsch Gordan (CG) tensor product as the basic equivariant operation. This is the canonical formulation of an SO(3)‑equivariant network and is significant for the machine learning force field prediction task [1-3]. Also, the fully-connected CG tensor product is a cost‑dominating kernel so the acceleration is easy to measure. Furthermore, Equiformer outperforms many other baselines with fully-connected CG tensor products in several molecular tasks [1]. Considering all the above, Equiformer provides a standard and widely recognized baseline.\n\n**The CP‑decomposition method itself is not tied to Equiformer**. Any architecture whose core step is a bilinear operation, e.g., Tensor Field Networks (TFN) [2], NequIP [3], or other e3nn “TensorProduct” blocks, can be approximated in the same way. We further provide the ablation study over the NequIP architecture [3] with and without our techniques in the below table. The employed dataset is the subset of PubchemQC-Traj-Small dataset, where we randomly select one conformer for each trajectory from PubchemQC-Traj-Small to build the training, validation, and testing set. And the NequIP is trained with a maximum angular degree $L=2$, 4 interaction blocks, and a multiplicity of 64 for 100 epochs. As shown in the below table, the energy and force validation results are similar, but the training and inference time are accelerated by 2 times with our techniques.\n\n|  NequIP                                        | Validation Energy MAE (eV) | Validation Force RMSE (eV/Å) | Training Time  (min/epoch) | Inference Time (min/epoch) |\n| ---------------------------------------- | -------------------------- | ----------------------------- | -------------------------- | -------------------------- |\n| Plain                                    | 0.01122                    | 0.09009                       | 7.5                        | 1.5                        |\n| + Path-weight sharing & CP decomposition | 0.01115                    | 0.09249                       | 2.0                        | 0.5                        |\n\n\n\n> Is the GPU runtime reported in Table 2? Can the authors show the CPU runtime as well?\n\n\nThe time reported in Table 2 is the inference time over a single A100-80GB GPU, and the runtime includes both GPU and CPU overhead. The pure CPU implementation runtime over AMD EPYC 9684X 96-Core Processor is detailed in the below table.\n\n|                  | Throughput (samples/sec) |\n| ---------------- | ------------------------ |\n| Equiformer (L=1) | 7.3                      |\n| TDN (L=1)        | 17.0                     |\n| Equiformer (L=2) | 1.8                      |\n| TDN (L=2)        | 7.9                      |\n| Equiformer (L=3) | 0.6                      |\n| TDN (L=3)        | 4.7                      |\n\n\nWe will include this CPU table in the appendix of the camera‑ready version.\n\n\n> Would the proposed method benefit from other tensor network (decomposition) structures? What is the author's opinion?\n\n\nThe current study uses the CP decomposition because it is the simplest: it requires only rank‑1 components, is easy to fit once, and introduces no new hyperparameters beyond the rank schedule. Additionally, the error bound and the uniqueness of CP decomposition are well studied in [4]. Other tensor decomposition methods are promising as well, such as **Tucker decomposition** [4] and **tensor‑train decomposition** [5]. These alternatives offer a different trade‑off between parameter count, arithmetic complexity, and the cache locality. We view a systematic comparison of these alternatives, and their effect on training stability and accuracy, as valuable future work.\n\n\n> It would be nice to add relative numbers on parameter counts and processing speed directly to Table 2.\n\nWe present the updated Table 2 below, where we include the multiplicative ratio with respect to the Equiformer baseline with the same $L$. We will include this table in the camera‑ready version.\n\n\n|                  | Throughput (samples/sec) | Parameter       |\n| ---------------- | ------------------------ | --------------- |\n| Equiformer (L=1) | 470.9                    | 8.86 M          |\n| TDN (L=1)        | 583.1 (x 1.24)           | 4.42 M (x 0.50) |\n| Equiformer (L=2) | 122.5                    | 18.19 M         |\n| TDN (L=2)        | 199.7 (x 1.63)           | 4.42 M (x 0.24) |\n| Equiformer (L=3) | 48.4                     | 33.43 M         |\n| TDN (L=3)        | 142.9 (x 2.95)           | 4.42 M (x 0.13) |\n\n\n\n[1] Liao, Y. and Smidt, T. Equiformer: Equivariant graph attention transformer for 3d atomistic graphs. In The Eleventh International Conference on Learning Representations, 2022.\n\n[2] Thomas, N., Smidt, T., Kearnes, S., Yang, L., Li, L., Kohlhoff, K., and Riley, P. Tensor field networks: Rotation-and translation-equivariant neural networks for 3d point clouds. arXiv preprint arXiv:1802.08219, 2018.\n\n[3] Batzner, S., Musaelian, A., Sun, L., Geiger, M., Mailoa, J. P., Kornbluth, M., Molinari, N., Smidt, T. E., and Kozinsky, B. \"E(3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials.\" Nature communications 13, no. 1 (2022): 2453\n\n[4] Kolda, T. and Bader, B. Tensor decompositions and applications. SIAM review, 51(3):455–500, 2009.\n\n[5] Oseledets, I. V. Tensor-train decomposition. SIAM Journal on Scientific Computing, 33(5), 2295-2317."}, {"review_id": "ckJeJHlG2P", "reviewer": "Reviewer_oXft", "summary": "This manuscript presents an interesting and timely contribution to the development of efficient SO(3)-equivariant neural networks. It addresses a known computational bottleneck in machine learning force fields: the expensive Clebsch–Gordan (CG) tensor product. The key innovation lies in approximating spherical tensors via low-rank decompositions; specifically, using CANDECOMP/PARAFAC (CP) decomposition to express the CG tensor as a sum of rank-1 tensors.", "questions": "Missing some important citation:\n\nBrandon Anderson, Truong-Son Hy and Risi Kondor, Cormorant: Covariant molecular neural networks, NeurIPS 2019.\n\nhttps://dl.acm.org/doi/10.5555/3454287.3455589", "limitations": "- Approximate Equivariance: The method only maintains approximate SO(3)-equivariance. While the authors provide theoretical bounds on the equivariance error, the practical implications of this approximation, especially in sensitive downstream tasks, are not deeply analyzed.\n\n- CP Rank Selection is Manual: The required CP rank R is treated as a fixed hyperparameter. Since determining the optimal CP rank is NP-hard, this introduces potential inefficiency. A more adaptive or data-driven approach to rank selection would improve scalability and generality.\n\n- Limited Scope of Decomposition: The proposed CP decomposition is demonstrated only for the Clebsch–Gordan tensor product. It's unclear how well the approach generalizes to other types of equivariant operations (for example, permutation-equivariance: please see \"Predicting molecular properties with covariant compositional networks\" at Journal of Chemical Physics https://pubs.aip.org/aip/jcp/article/148/24/241745/964420/Predicting-molecular-properties-with-covariant).\n\n- The evaluation is mostly limited to molecular force field tasks (energy and force prediction). It would strengthen the paper to show broader utility in other domains where SO(3)-equivariance is important (e.g., protein modeling, 3D object recognition).\n\n- No Ablation on Universality vs. Accuracy Trade-off: Although the paper proves universality theoretically, it lacks an ablation showing how the approximation degrades or improves as R increases. Practical guidance for choosing R would be valuable.\n\n- The path-weight sharing mechanism drastically reduces parameter count by using a single weight matrix across all paths. While this helps with efficiency, it may limit the expressiveness or fine-grained control across angular degrees, especially at high L.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- Novelty: The idea of replacing the CG product with a CP decomposition is elegant and reduces computational complexity from O(L^6) to O(L^4), a substantial improvement.\n\n- Theory: The authors provide theoretical guarantees, including equivariance error bounds and a universality theorem for their approximation.\n\n- Efficiency: They combine CP decomposition with a path-weight sharing scheme to reduce parameters significantly (from O(c^2 L^3) to O(c^2)).\n\n- Empirical results: Strong performance is demonstrated on PubChemQC-Traj, OC20, and OC22, with impressive speed-ups and competitive accuracy.\n\nWeaknesses:\n- While the approximation method is justified theoretically, the required CP rank R is treated as a hyperparameter. Estimating or learning optimal ranks adaptively could improve generality.\n\n- The approximation is \"approximately equivariant\". The trade-off between error and efficiency for approximate equivariance has been well-explored in the literature. I think this work could benefit from more discussion in real-world generalization contexts.", "was_revised": true, "final_justification": "I have read the authors' rebuttal as well as other reviewers' comments. The authors have added more experimental results. I decided to keep my original rating as borderline accept.", "rebuttal": "We appreciate that Reviewer oXft finds our method significant.\n\n> While the approximation method is justified theoretically, the required CP rank R is treated as a hyperparameter. Estimating or learning optimal ranks adaptively could improve generality.\n\n> CP Rank Selection is Manual: The required CP rank R is treated as a fixed hyperparameter. Since determining the optimal CP rank is NP-hard, this introduces potential inefficiency. A more adaptive or data-driven approach to rank selection would improve scalability and generality.\n\nSelecting the CP rank $R$ is itself an NP‑hard problem as mentioned in the limitation section, so in practice **the rank schedules are empirical heuristics**. The adopted one gives a relatively small SO(3)-equivariance error with low computational cost across every schedule in Figure 3 and Section 4.1, and ablation studies in the below table of NequIP confirm that this schedule suffices for downstream tasks while cutting time cost substantially. Note that the NequIP is trained with a maximum angular degree $L=2$, 4 interaction blocks, and a multiplicity of 64 for 100 epochs with our proposed schedule.\n\n|  NequIP                                        | Validation Energy MAE (eV) | Validation Force RMSE (eV/Å) | Training Time  (min/epoch) | Inference Time (min/epoch) |\n| ---------------------------------------- | -------------------------- | ----------------------------- | -------------------------- | -------------------------- |\n| Plain                                    | 0.01122                    | 0.09009                       | 7.5                        | 1.5                        |\n| + Path-weight sharing & CP decomposition | 0.01115                    | 0.09249                       | 2.0                        | 0.5                        |\n\nAlthough this heuristic meets the needs of our current force‑field tasks, we agree that a data‑driven approach, such as learning a rank predictor or applying Bayesian optimization over ranks, could generalize the method and further reduce redundancy; we identify this as an important direction for future work.\n\n\n> The approximation is \"approximately equivariant\". The trade-off between error and efficiency for approximate equivariance has been well-explored in the literature. I think this work could benefit from more discussion in real-world generalization contexts.\n\n\nWe agree that the value of approximate equivariance ultimately rests on how well the model generalizes. We share the concern and therefore measured our CP-decomposition-based approximation on two large-scale, experiment-derived benchmarks: Open Catalyst Project and PubChemQC-Traj. Both datasets contain millions of chemically diverse geometries and are labelled with high-fidelity DFT energies and forces. We believe the performance on these two datasets is a strong indicator of generalization ability.\n\nTo push the generalization study further, we plan to train on OMol [1], a newly released molecular dataset that spans even broader compositional space. These additional experiments may help clarify how the approximation behaves on systems that differ more sharply from the training distribution.\n\n> Missing some important citation: Brandon Anderson, Truong-Son Hy and Risi Kondor, Cormorant: Covariant molecular neural networks, NeurIPS 2019.\n\n\nThank you for raising this issue. We will update the citation at line 25 and line 72 in the camera-ready version.\n\n\n> Approximate Equivariance: The method only maintains approximate SO(3)-equivariance. While the authors provide theoretical bounds on the equivariance error, the practical implications of this approximation, especially in sensitive downstream tasks, are not deeply analyzed.\n\n> No Ablation on Universality vs. Accuracy Trade-off: Although the paper proves universality theoretically, it lacks an ablation showing how the approximation degrades or improves as R increases. Practical guidance for choosing R would be valuable.\n\n\nThank you for raising these points. To further demonstrate the practical implication of this approximation and strengthen the contribution, we conduct the **ablation study of ranks over *n*-body system dataset** [2]. In the below rank‑sweep table for the *n*-body system experiment, the TDN is trained with a maximum angular degree $L=2$, 4 interaction blocks, and a hidden dimension of 64 in 5000 epochs. In that table, our adopted schedule uses much smaller ranks yet matches the accuracy obtained with the largest ranks, corresponding to the results of rank schedule selection in Section 4.1. Note that the largest rank is deduced from the upper bound in line 164; see [3].\n\n\n| R                          | *n*-Body Testing MSE | Equivariance Error |\n| -------------------------- | ----------------------- | ------------------ |\n| 10                         | 0.00910                 | 0.60               |\n| 15                         | 0.00825                 | 0.42               |\n| 20                         | 0.00648                 | 0.21               |\n| **28 (Our schdule)**      | 0.00438                 | 0.02               |\n| 81 (Highest possible rank) | 0.00437                 | <1e-3              |\n\n\n\n\n> Limited Scope of Decomposition: The proposed CP decomposition is demonstrated only for the Clebsch–Gordan tensor product. It's unclear how well the approach generalizes to other types of equivariant operations (for example, permutation-equivariance).\n\n\nTensor product is widely used in SO(3)-equivariant neural networks. Particularly, the **Clebsch-Gordan tensor product is the canonical formulation of an SO(3)-equivariant neural network**. For permutation equivariance, to our knowledge, it is unknown that tensor product will play a role in it. The permutation-equivariant application is usually built by deep sets or graph neural networks, where bilinear operations are not necessarily used. For other types of equivariant operations, as long as they can be written in bilinear forms, our CP decomposition can be applied to approximate the operations.\n\n\n> The evaluation is mostly limited to molecular force field tasks (energy and force prediction). It would strengthen the paper to show broader utility in other domains where SO(3)-equivariance is important (e.g., protein modeling, 3D object recognition).\n\n\nWe appreciate this suggestion. Our study deliberately targets machine-learning interatomic potentials since accelerating the Clebsch Gordan map yields a great practical benefit there, and the benchmarks (Open Catalyst Project, PubChemQC-Traj) are well established. Extending the approach to other domains, e.g., protein folding and 3-D object recognition, is indeed an exciting next step, and we leave it as future explorations.\n\n\n\n> The path-weight sharing mechanism drastically reduces parameter count by using a single weight matrix across all paths. While this helps with efficiency, it may limit the expressiveness or fine-grained control across angular degrees, especially at high L.\n\n\nWe agree that path-weight sharing will limit the expressiveness of the tensor product. However, **coupling with CP decomposition, the result can match the full Clebsch Gordan (CG) map**. To demonstrate this, we further conduct the ablations of CP decomposition and path-weight sharing for both the ***n*-body experiment dataset** and a **subset of PubchemQC-Traj-Small dataset** in the below table. For the *n*-body experiment, the testing results of TDN are presented in the table, where the TDN is trained with a maximum angular degree $L=2$, 4 interaction blocks, a hidden dimension of 72, and a weight decay of $5\\times 10^{-6}$ in 2000 epochs. And for the subset of PubchemQC-Traj-Small dataset, we randomly select one conformer for each trajectory from PubchemQC-Traj-Small to build the training, validation, and testing set. The testing results of TDN over this subset are presented in the table, where the TDN is trained with a maximum angular degree $L=2$, 4 interaction blocks, and a hidden dimension of 64 in 60 epochs. Note that in the table, the *n*-Body experiment appears under the label \"*n*-Body,\" while the subset of the PubChemQC-Traj-Small dataset is labeled simply as \"Subset.\"\n\n\n|                                          | *n*-Body MSE | Subset Testing Energy MAE (eV) | Subset Testing Force RMSE (eV/Å) | Subset Training Time Cost (min/epoch) |\n| ---------------------------------------- | ------------ | ------------------------------ | -------------------------------- | ------------------------------------- |\n| Plain                                    | 0.00414      | 0.01274                        | 0.09274                          | 19.0                                  |\n| + Path-weight sharing                    | 0.00439      | 0.01503                        | 0.09321                          | 9.8                                   |\n| + Path-weight sharing & CP decomposition | 0.00429      | 0.01295                        | 0.09653                          | 4.2                                   |\n\nAs shown in the above table, introducing path-weight sharing alone will indeed incur a small accuracy drop. However, introducing both path-weight sharing and CP decomposition produces comparable results to the full CG tensor product while significantly boosting the efficiency. We believe this is because the Clebsch Gordan map is replaced by a low-rank surrogate with rank-1 factors, which act as an implicit regularizer as they filter out high-variance directions that weight sharing alone cannot suppress. We will include this table in the appendix of the camera‑ready version.\n\n[1] Levine, D. S., et al. The open molecules 2025 (omol25) dataset, evaluations, and models.\n\n[2] Kipf, T., et al. Neural relational inference for interacting systems. In International conference on machine learning, pp. 26882697. PMLR, 2018.\n\n[3] Kolda, T. and Bader, B. Tensor decompositions and applications. SIAM review, 51(3):455–500, 2009."}, {"review_id": "j0afJr0oOE", "reviewer": "Reviewer_S6y1", "summary": "In this paper, the authors propose the CANDECOMP/PARAFAC (CP) low-rank tensor decomposition to approximate the tensor involved in the Clebsch-Gordan (CG) tensor product, commonly used in dominant machine learning force field (MLFF) models. These models, based on equivariant graph neural networks, incorporate rotational symmetry — a fundamental property of molecular and material systems. In addition to the CP approximation, a path-weight sharing strategy is proposed that further reduces the number of learnable parameters, leading to a total computational complexity reduction of the CG tensor product from O(L6) to O(L4). The impact of the proposed method is evaluated on the computation time, approximation accuracy, and equivariance preservation using two benchmarks: the PubChemQC-Traj dataset, which contains millions of molecular dynamics snapshots, and the Open Catalyst Project (OCP), a large-scale dataset of DFT relaxations for adsorbate–catalyst systems.", "questions": "1-\tThe method for selecting the optimal rank R is not clearly explained. Is R tuned via cross-validation?\n\n2-\tWhich dataset Figure 3 refers to?\n\n3-\tHow do you define the “wall-clock runtime” that you evaluated in the experiments? Is it related to computation time during training, testing or both?", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths\n\n•\tThe results presented in this paper are relevant to the development of force field models that incorporate rotational invariance, an important property for molecular and material systems.\n\n•\tThe paper provides theoretical bounds on both approximation and equivariance errors, derived from the singular values of the matrix representations of the approximated tensor MM.\n\nWeaknesses\n\n•\tThe low-rank structure of the tensor M deserves further discussion. While the paper relates the rank R to the maximum angular degree L, a theoretical justification or empirical analysis of the inherent low-rank nature of M is lacking.\n\n•\tThe method for selecting the optimal rank R is not clearly explained. Is R tuned via cross-validation? Clarifying whether this is a learned or predefined hyperparameter is needed.\n\n•\tThe theoretical analysis focuses on the forward computation of the tensor product. However, during training, the gradients with respect to the CP decomposition parameters must also be computed. Therefore, an analysis of the computational complexity of gradient backpropagation should be included.\n\n•\tFigure 3: It is unclear which dataset this figure refers to. This should be explicitly stated in the caption or text.\n\n•\tThe evaluation of computation time is described in terms of “wall-clock runtime” but it is ambiguous whether this refers to training time, inference time, or both. A more precise description of the measurement setup is needed.\n\n•\tIn Line 285, a reference to a table is made, but the table number is missing.", "was_revised": true, "final_justification": "After reading the author's responses and other reviews, I decided to keep my rating. However, I thank the authors for clarifying my misunderstanding of Fig. 3 and the term \"wall-clock runtime\" used in the paper.", "rebuttal": "We appreciate that Reviewer S6y1 finds our method significant.\n\n\n> The low-rank structure of the tensor M deserves further discussion. While the paper relates the rank R to the maximum angular degree L, a theoretical justification or empirical analysis of the inherent low-rank nature of M is lacking.\n\nOur work mainly approximates the Clebsch Gordan (CG) tensor product via CP decomposition. The CG coefficient tensor is intrinsically low-rank because of the sparsity that most index triples $(\\ell_1, \\ell_2, \\ell_3)$ in the CG tensor violate the selection rules $ |\\ell_1 - \\ell_2| \\le \\ell_3 \\le \\ell_1 + \\ell_2 $ and $m_3 = m_1 + m_2$; their entries are therefore identically zero.\n\nEmpirical evidence supports this view. By the upper bound of the tensor rank [1], the maximum rank of a CG tensor is $(L+1)^4 \\propto O(L^4)$ while we successfully employ only an $O(L^2)$ rank schedule to approximate the CG tensor. Below we present the ablation study of ranks over *n*-body system dataset [2], where the TDN is trained with a maximum angular degree $L=2$, 4 interaction blocks, and a hidden dimension of 64 in 5000 epochs. In the table, our adopted schedule uses much smaller ranks yet matches the accuracy obtained with the largest ranks, corresponding to the results of rank schedule selection in Section 4.1 and showing the low rank nature of the CG tensor.\n\n| R                          | *n*-Body Testing MSE | Equivariance Error |\n| -------------------------- | ----------------------- | ------------------ |\n| 10                         | 0.00910                 | 0.60               |\n| 15                         | 0.00825                 | 0.42               |\n| 20                         | 0.00648                 | 0.21               |\n| **28 (Our schdule)**      | 0.00438                 | 0.02               |\n| 81 (Highest possible rank) | 0.00437                 | <1e-3              |\n\n\n\n> The method for selecting the optimal rank R is not clearly explained. Is R tuned via cross-validation? Clarifying whether this is a learned or predefined hyperparameter is needed.\n\n> The method for selecting the optimal rank R is not clearly explained. Is R tuned via cross-validation?\n\nRank $R$ is a **predefined hyperparameter**, not a learned quantity. Because computing the exact CP rank of the CG tensor is NP-hard, we instead explore several simple growth rules before network training (Section 4.1) to guide the selection of rank $R$. Our adopted schedule gives a relatively small SO(3)-equivariance errors with low computational cost as demonstrated in Figure 3, and rank-sweep ablations of $L=2$ in the above table confirm that this schedule suffices for downstream tasks. \n\nAlthough this heuristic meets the needs of our current force‑field tasks, a data‑driven approach, such as learning a rank predictor or applying Bayesian optimization over ranks, could generalize the method and further reduce redundancy; we identify this as an important direction for future work.\n\n\n> The theoretical analysis focuses on the forward computation of the tensor product. However, during training, the gradients with respect to the CP decomposition parameters must also be computed. Therefore, an analysis of the computational complexity of gradient backpropagation should be included.\n\n\nWe present both the **theoretical and empirical time complexity analysis** of the CP decomposition for the tensor product. Let $A\\in \\mathbb{R}^{d_3\\times R}, B\\in \\mathbb{R}^{d_1\\times R}, C\\in \\mathbb{R}^{d_2\\times R}$ and input $x\\in \\mathbb{R}^{d_1}, y\\in \\mathbb{R}^{d_2}$. By Equation (3), the forward map is \n$$\nz = A(B^\\top x\\odot C^\\top y ).\n$$\nGiven loss gradient $g_z = \\frac{\\partial \\mathcal L}{\\partial z}\\in \\mathbb{R}^{d_3}$, the backward gradient to $x$ is\n$$\ng_x = g_z\\frac{\\partial{A(B^\\top x\\odot C^\\top y )}}{\\partial x} = g_z\\frac{\\partial A \\text{diag}(C^\\top y) B^\\top x}{\\partial x} = g_z A \\text{diag}(C^\\top y) B^\\top = (g_z A \\odot C^\\top y) B^\\top\n$$\nand similarly the backward gradient to $y$ is\n$$\ng_y = (g_z A \\odot B^\\top x) C^\\top.\n$$\nEach expression requires a time complexity of $O(R(d_1+d_2+d_3))$. Therefore, the total time complexity to compute the backward gradient is $O(R(d_1+d_2+d_3))$, identical to the forward pass. \n\nFurthermore, the computation time of both forward and backward is presented in the below table. Each pass is measured with a batch size of 1000 and a multiplicity of 128. The backward step is consistently $\\approx 2-3×$ the forward cost, independent of L, confirming that the same time complexity of forward and backward pass.\n\n\n|                                            | Forward Computation Time | Backward Computation Time |\n| ------------------------------------------ | ------- | -------- |\n| L = 1 | 0.42 ms | 1.29 ms  |\n| L = 4 | 0.57 ms | 1.53 ms  |\n| L = 6 | 0.83 ms | 2.24 ms  |\n\n\n\n> Figure 3: It is unclear which dataset this figure refers to. This should be explicitly stated in the caption or text.\n\n> Which dataset Figure 3 refers to?\n\n**Figure 3 is not tied to any dataset**. The curves in Figure 3 report the approximation error and runtime of the CP decomposition for the CG tensor product itself, which only depends on the chosen rank, multiplicities, and maximum angular degree. The low-rank reconstruction can be learned and evaluated without any molecular data. To learn the CP factor matrices in CP decomposition, we minimize the Frobenius norm between the exact and approximate CG tensor product on batches of standard normal inputs; this guards against data bias and is done once before any network training. We will amend the caption to state this explicitly in Figure 3 in the camera‑ready version.\n\n\n> The evaluation of computation time is described in terms of “wall-clock runtime” but it is ambiguous whether this refers to training time, inference time, or both. A more precise description of the measurement setup is needed.\n\n> How do you define the “wall-clock runtime” that you evaluated in the experiments? Is it related to computation time during training, testing or both?\n\nThe \"wall-clock runtime\" refers to the inference time over the PubchemQC-Traj-Small dataset. We will revise line 260 to state this explicitly in the camera‑ready version.\n\n\n> In Line 285, a reference to a table is made, but the table number is missing.\n\nThe table reference should be \"Table 3\". We will revise line 285 and update this in our camera-ready version.\n\n\n[1] Kolda, T. and Bader, B. Tensor decompositions and applications. SIAM review, 51(3):455–500, 2009.\n\n\n[2] Kipf, T., Fetaya, E., Wang, K.-C., Welling, M., and Zemel, R. Neural relational inference for interacting systems. In International conference on machine learning, pp. 26882697. PMLR, 2018."}]

Author rebuttal:

Meta-review: This paper proposes tensor decomposition networks (TDNs) that replace Clebsch-Gordan tensor products in SO(3)-equivariant neural networks with low-rank CP decompositions and a path-weight sharing strategy. The contribution is motivated by the high computational cost of equivariant tensor products in machine learning force fields, and the authors provide both theoretical guarantees (error bounds and universality) and strong empirical validation. Reviewers agree that the problem is important, the technical development is sound, and the experiments convincingly demonstrate significant efficiency gains with little to no accuracy loss.

Concerns were raised regarding the manual choice of CP rank, the approximate nature of equivariance, and the scope beyond Clebsch-Gordan tensor products. The authors responded with additional ablation studies, backward pass analysis, CPU runtime benchmarks, and a broader discussion of possible extensions. While some reviewers maintained borderline scores, they acknowledged that their questions were satisfactorily addressed. One reviewer raised their rating to clear accept, emphasizing the significance of the work for both AI and computational material science communities.

Overall, this is a technically solid paper with a well-motivated contribution and substantial potential impact. The methodological limitations are acknowledged and do not diminish the paper’s core value.
Decision: Accept (poster)