Scaling Laws and Symmetry, Evidence from Neural Force Fields
Authors: Khang Ngo
Siamak Ravanbakhsh
Venue: ICLR 2026 Poster (2026)
Keywords: compute-optimal scaling laws
geometric deep learning
interatomic potentials
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)
TL;DR: We found “architecture-dependent” scaling exponents across architectures with increasing levels of symmetry expressivity in the area of learning interatomic potentials.

Abstract: We present an empirical study in the geometric task of learning interatomic potentials, which shows equivariance matters even more at larger scales; we show a
clear power-law scaling behaviour with respect to data, parameters and compute
with “architecture-dependent exponents”. In particular, we observe that equivariant
architectures, which leverage task symmetry, scale better than non-equivariant
models. Moreover, among equivariant architectures, higher-order representations
translate to better scaling exponents. Our analysis also suggests that for computeoptimal
training, the data and model sizes should scale in tandem regardless of the
architecture. At a high level, these results suggest that, contrary to common belief,
we should not leave it to the model to discover fundamental inductive biases such
as symmetry, especially as we scale, because they change the inherent difficulty
of the task and its scaling laws.

Published as a conference paper at ICLR 2026
SCALING LAWS AND SYMMETRY, EVIDENCE FROM NEURAL FORCE FIELDS
Khang Ngo 12, Siamak Ravanbakhsh
1 Mila Quebec Al Institute, 2 School of Computer Science, McGill University (khang. ngo, siamak. ravanbakhsh, @mi quebec
ABSTRACT
We present an empirical study in the geometric task of learning interatomic po- tentials, which shows equivariance matters even more larger scales; we show clear power-law scaling behaviour with respect to data, parameters and compute with "architecture-deeendee exponents". In particular, ant architectures, which leverage task symmetry, scale better than non-equivariant models. Moreover, among equivariant architectures, higher-order representations translate to better scaling exponents. Our analysis also suggests that for compute- optimal training, the data and model sizes should scale tandem regardless of the architecture. At high level, these results suggest that, contrary to common belief. we should not leave to the model to discover fundamental inductive biases such as symmetry, especially we scale, because they change the inherent difficulty the task and its scaling laws.
observe that equívari-
Unconstrained MPNN
EGNN
GemNet-OC
eSEN
Bony
Imax
10
10
1nxx
lmax
Val Loss (log)
Val Loss (log)
L(C) 0.930-9.14
L(C) = 0.75H
10
L(C) 0.78C 0.17
10
L(C) = 0.69H
L(C) 0.49C-225
L(C) 0.36
L(C) 0.700-046
L(C) 0.36H-045
1
10' PFLOPs (C)
10
10
10'
GPU Hours (H)
Figure 1: Performance of neural network interatomic potentials follows power law (linear in log- log space) in training compute (PFLOPs, GPU-hours). The scaling behaviour varies with archi- tectural complexity: the slope of the performance curve improves the architecture changes from unconstrained to low-order to high-order, implying that performance gaps widen with increasing compute. Body order number of nodes whose states define message within a layer. Tensor order f: order of geometric features processed by the models. Left: Empirical scaling laws along the FooPsalptimal frontier. Right: Empirical scaling laws along the train-time-optimal frontier.
INTRODUCTION
Recent years have witnessed extensive study of neural scaling laws across various machine learning domains, including natural language and vision. The general observation supported by the theory is that test errors exhibit power-law relationship with the scale of training data points, model
Code available https::/gitthb..coo
iiaalin---- inn- 19/cca1 laws aara-and-ssae and-symmet ry

---

Published as a conference paper at ICLR 2026
parameters, and the amount of compute in floating- point operations (FLOPs). These laws identify the optimal scaling of model size with dataset size for a given compute budget, enabling an optimal use of resources at scale.
A common view is that the scaling behaviour is consistent across various expressive architectures for a task-i.e., the choice of architecture can only increase or decrease the loss by multiplica- tive factor that remains constant across sufficiently large scales. This belief is supported by both theoretical results (Sharma & Kaplan, 2022; Bahri et al., 2024) and empirical studies in different domains (Ahmad & Tesauro, 1988; Hestness et al., 2017), including language (Kaplan et al., 2020; Hoffmann et al.. 2022) and vision (Zhai al., 2022). and it is further reinforced by Sutton's 's bitter lesson (Sutton, 2019), which highlights that attempts to encode inductive biases, such as symmetry, explicitly are often outperformed in the long run, since models can learn these structures on their own when scaled.
The specific inductive bias of symmetry, and in particular Euclidean and rotational symmetry have been successfully leveraged in many domains, including for molecular force fields. The success of these networks often attributed to their improved generalization and robustness to out-of- distribution data (Batatia et al., 2022; Petrache & Trivedi, 2023). However, one may argue that equivariant networks are harder to scale as their specialized operations, such as tensor products, spherical harmonics (Thomas et al., 2018; Anderson et al., 2019; Liao & Smidt, 2023), or high-order message passing (Gasteiger et al., 2020b; Klicpera et al., 2021), are complex and computationally expensive. At the same time, several works in protein folding (Abramson et al., 2024), molecule con- former generation (Wang et al., 2024), and Neural Network Interatomic Potentials (NNIPs) (Deng al., 2023; Qu & Krishnapriyan, 2024; Rhodes et al., 2025) demonstrate that non-equivariant net- works can perform well in geometric tasks. Brehmer et al. (2025) also show that non-equivariant architectures trained with data augmentation can perform on par with their equivariant counterparts when given sufficient compute. All of this paints picture in favour of forgoing equivariance and scaling simpler non-equivariant models.
This paper presents a careful empirical study that questions this growing mindset and shows that equivariance matters even more as we scale. We report clear architecture-dependen scaling expo- nent in model size, data size, and compute, for several widely used scalable NNIPs architectures that encode rotational and permutation symmetry to varying degrees. This translates to performance gap that grows with scale, favouring scalable models with higher-order symmetry bias at larger scales; see fig. 1.
Our target domain for this study has witnessed growing number of deep learning techniques for predicting quantum properties of atomistic systems recent years, where neural models approxi- mate computationally demanding ab initio calculations, such density functional theory. The most promising progress being made on NNIPs, which map molecular systems to their energies and forces. NNIPs' foundation models are unlocking new possibilities through efficient and accurate molecular dynamics, and our findings in this domain identify the most promising direction for the design of models that are trained scale.
1.1 KEY FINDINGS AND CONTRIBUTIONS
In this work, we conduct comprehensive scaling-law experiments, drawing on best practices and insights from prior work on the expressive power of (geometric) message passing neural networks (MPNNs) (Loukas, 2020; Joshi et al., 2023), maximal update parametrization (nP) (Yang et al., 2021), and compute-optimal scaling (Hoffmann et al., 2022). Our key findings are:
Clear power law scaling. desssage-passing NNIPs obey power-law scaling with respect to com- pute, data, and model size. For compute, unlike prior studies that report only FLOPs within chitectures, we characterize scaling with both FLOPPs and wall-clock training time (GPU-hours). Given that equivariant networks can be less GPU-friendly, this approach provides more com- plete view for practical purposes. While prior work in geometric domains has shown architecture- dependent scaling with respect only the dataset size (Batzner al., 2022), to our knowledge, none of them provide complete and comparable picture to ours.
Architecture-dependeee exponents. Power-law exponents increase the "degree" of equivari- ance grows, from non-equivariant (unconstrained) models to lower- to higher-order equivariant designs.
2

---

Published as a conference paper at ICLR 2026
Compute-optinal scaling. We find that the power-law exponents for dataset and model size in compute-optima scaling are similar across non-equivarian and equivariant architectures of differ- ent representation degrees. This means that compute-optimge scaling should increase the model and dataset size in tandem; this mirrors the findings of Hoffmann et al. (2022) in natural language.
Multi-epoch training and data-augmentation. While our main results consider a single epoch regime, we show that the same scaling laws hold across tens of epochs in multi-epoch setting. This is because, at scale - even with 1% of our training set- - the effect of overfitting is negligible. For non-equivariant models, data augmentation is required to avoid overfitting and maintain the scaling coefficients. We also consider inference-time augmentation for the unconstrained model, and show that it only changes the multiplicative coefficient (rather than the exponent) in the scaling law, and its benefit saturates quickly with the number of augmentations for this task.
Scaling effect of symmetry loss. Enforcing symmetry through loss does NOT seem to provide the same benefits as having an equivariant architecture.
Trend in optimal depth. For a fixed parameter and compute budget, the optimal depth of the network is correlated with the "degree" of equivariance among the architectures we studied; with equivariant networks, the benefit of depth saturates at higher values, and for higher rotation order networks this value grows larger; this corroborates the claims of Joshi et al. (2023); Jia et al. (2020).
Organization. The rest of the paper is organized follows. Section 2 outlines problem setup and symmetry constraints through architectures and loss. Section 3 discusses our experiments, includ- ing key hyperparameters and scaling strategies. Section 4 presents our results and their analysis. Section 5 concludes with an emphasis on limitations of our work, and important directions for near- future work. Details on related works, background, experimental setup, and additional results are included in the Appendix.
2 SETUP
An atomistic system can be represented as a point cloud X = (().,,)....., In)}, where 2ạ E and x; E Rd are the the atomic number and the position respectively. The potential energy e(X) is a scalar that is invariant to global translations and rotations, while forces (X) = (e(X) are vectors that are translation invarrian and rotation-egue -auttion-quivariant. The task of our NNIPs is train neural network de N x R + R R1+3xn that takes X as input and predicts the potential energy (scalar) and atom-level forces, one for each atom - i.e.. the + (X), (X).... So.n While it is sufficient to learn the energy for predicting conservative forces, direct force prediction significantly more scalable. Using this approach enables one to benefit from the dense signal during pre-training. In post-training. the force prediction can be removed and the model can be fine-tuned to predict conservative forces through backpropagation via the predicted energies, ensuring a good balance between computational cost and accuracy (Bigi al., 2025; Fu et al., 2025). We minimize the per-atom mean absolute error (MAE) and mean squared error of forces (Fu et al., 2025; Wood al., 2025):
(X))).
L(o9, X) Ae [ e(x)) + AS -x())))
de
(1)
with Xe, X5 > 0 are the coefficients that control the relative importance of energy and force predic- tions; we use de- Ay.
2.1 ARCHITECTURES
Since we observed instability issues when scaling vanilla transformers for this task, we focused on cessage-passing architectures. Here, in addition to basic unconstrained MPNN, following the classification of MPNNs in Joshi et al. (2023), we considered three widely adopted equivariant architectures that cover various body and tensor orders. The body order corresponds to S,, repre- sentations, and refers to the number of nodes participating in a message function. The tensor order f corresponding to SO(3) representations, and denotes the order of the geometric tensor embed- dings processed by each model. Below we briefly enumerate these; for more background on these architectures, appendix C:
3

---

Published as a conference paper at ICLR 2026
1. unconstrained: vanilla MPNN that directly processes geometric features, i.e., relative position vectors, without any symmetry constraints.
invariant sealars:
geometric message passing neural network (GemNet-OOC (Gasteiger et al., 2022) is a variation of GemNet (Klicpera et al., 2021) adapted for large and diverse molecular dataset. Although it uses invariants such as interatomic distances and angles, and therefore has a tensor order e = O, it can approximate equívariant functions from edge-based invariant features, because it performs geometric message passing with two-hop and edge-directional embeddings; see Theorem 3 in Klicpera et al. (2021). Because GemNet-OC incorporates dihedral-angle infor- mation, its two-hop messages depend simultancously on the states four nodes, and thus it is classified four-body.
3. Cartesian vectors:
E(n)-qquivariant graph neural network (EGNN) (Satorras et al., 2021); in particular, the extension of Levy et al. (2023), which allows for more than one equivariant vector channel. We use a specific μP informed scaling, in which scalar channels scale quadratically wrt number vector channels, see appendix C for details.
high-order spherical tensors: equivariant Smooth Energy Network (eSEN) (Fu al., 2025), which uses higher-order irreducible representaions of rotation group (E 2 2). Unlike other ar- chitectures the same category (e.g.. Thomas et al., 2018; Batzner al., 2022; Liao & Smidt, 2023). we found eSEN more scalable because it uses frame alignment to sparsify the tensor product, allowing it to eliminate Seescch-Gordan coefficients and to directly parameterize ker- nels with linear layers (Passaro & Zitnick, 2023).
2.2 SYMMETRY LOSS
symmetry-bassed losses have been used in different settings from self-supervised learning (Dan- govski et al., 2021; Bai al., 2025), to physics-informed settings (Akhound-Sadeehh et al., 2023; Yang et al., 2024), generative modelling (Tong et al., 2025) and symmetry discovery (Escriche & Jegelka, 2025). A canonical choice is loss that penalizes deviations from equivariance constraints for randomly sampled global transformations (c.g., Kim et al., 2023a; Elhag et al., 2025; Bai et al., 2025):
1 M Σ
Laym (00; x,
[(vo(Pin x), (gi u):
where ~ NG is sampled from the Haar measure, Pin and Pom define linear actions on inputs x and targets y, respectively, and C is the task loss. The symmetry-augmentt term is added to the base objective in eq. (1) when training fa.
For our task, the translation part of the special Euclidean group SE(3) SO(3) xT3) is accounted for by centring the molecule at its center of mass, and the loss is only for the rotation group.
EXPERIMENTS
3.1 DATASET
We conduct our experiments the OpenMol neutral- subset (Levine et al., 2025), with 34M training samples and 27K held-out validation samples." Treating atom nodes as tokens, the training corresponds approximately D 9.2 10 tokens. Following sealing studies in ILMs, we consider a single-epoch training regime, where each sample is observed exactly once. While multi-epoch setting can be more practical for A14Science due smaller datasets compared to language, our goal was to stay faithful to existing methodologies and avoid possible confounding effects (Muennighoff al., 2023).
3.2 OPTIMIZATION
Following Choshen et al. (2025), which shows that estimating scaling laws from intermediate checkpoints yields more robust results, we track validation losses throughout training and these
We also tried using regularization that measures invariance by diffcrentiating along infinitesimal genera- tors, similar to Rhodes et al. (2025), but we could not achieve stable training. We use the neutral subset rather than the full 100M-molecule corpus due main-memory constraints.
4

---

Published as a conference paper at ICLR 2026
points excluding the first 1% - 10% of steps- to a standard scaling-law functional form (Kaplan et al., 2020; Hoffmann et al., 2022). A well-known caveat in scaling-law analyses is the sensitiv- ity to learning-rate schedules, particularly when predefined decay steps are used (Hoffmann et al., 2022: Hu et al., 2024). To address this, we adopt scheduler-free Adam W-style optimizers (Defazio et al., 2024), which not only remove the need for tuning decay schedules but also allow us cap- ture model training dynamics within single run-without retraining from scratch at each data ratio or relying on checkpoint restarts (Hu et al., 2024). Crucially, this approach enables more accurate measurement of training time by mitigating kardware-related artifacts and helps us derive scaling laws directly with respect training time, measured in GPU-hours.
3.3
TUNING
Investigating scaling behaviours of neural networks necessitates evaluations under optimal condi- tions for both efficiency and performance. We, therefore, perform systematic experiments to deter- mine critical hyperparameters affecting the scaling behaviours of those architectures. Our analysis include non- equivariant MPNN, EGNN, and GemNet-OC. Due the higher computational cost of eSEN, we adopt the optimal hyperparameters from Passaro & Zitnick (2023); Fu et al. (2025).
Learning Rates and Batch Sizes. When fine-tuning the model, we swept over 12 configurations for each architecture by testing three learning rates, {le - 4, 5e 4, 1e he--3 3). and four batch sizes, (64, 128, 256, 512). We performed the tuning with approximately one million parameters. As shown in fig. 10, smaller batch sizes and larger learning rates resulted in lower validation losses. This finding about small batch sizes aligns with observations in (Gasteiger et al., 2020a; Frey et al., 2022).
Saturation Depth. Depth d and width (embedding dimension) govern the parameter count N. While Kaplan et al. (2020) found Transformer performance depends mainly on and is independent of architectural factors such as d and w, recent MPNNs work shows that depth choice can undermine power-law scaling behaviour (Liu et al., 2024; Sypetkowski et al., 2024). We therefore probe the depth-saturation point on 3D geometric graphs the depth beyond which validation error no longer improves for fixed capacity (Loukas, 2020). To isolate depth/width from model size, we fix N z 10 and sweep (d, w²) As shown in fig. 10, non-equivariant MPNNS degrade with increasing depth (Topping et al., 2022)), whereas equivariant models continue to improve, with validation losses plateauing at depths T. E (12..... 16), consistent with prior reports (Joshi et al., 2023; Passaro & Zitnick, 2023; Pengmei et al., 2025).
wnver-smothhing/over-sso
Infinite-Widd Scaling. Our depth-saturation experiments are motivated by the universality con- dition for passage-passing-- width; see Corollary 3.1 in Loukas (2020). For each architecture type, we train series of mod- els with an increasing number of channels (width) along a scaling ladder. We fix the optimal hy- perparameters innsssding depth, learning rate 17" le 3, and batch size-foo for x: MM--aaameter models with base width Wbase. and use μP (Yang et al., 2021) to transfer 17 to other widths w via n(w) = 11" Whee We increase model size until the memory of single NVIDIA A100 (40 GB) w
architectures, which requires "sufficient depth" and unbounded
GPU is exhausted. In other words, we keep depth and batch size constant across model sizes and scale the width as high as our hardware allows.
SCALING LAWS
4.1 SCALING COMPUTE
Nominal FLOPs are hardware-agnostic, yet equivariant models often have lower GPU utilization. so FLOPs may understate practical cost. We therefore fit scaling laws in both theoretical FLOPs C and wall-clock training hours H, training all models on identical hardware; see Appendix D.
Counting FLOPs. Following Kaplan et al. (2020); Hoffmann et al. (2022), define the compute as theoretical FLOPs counting C' incurred from training a model of N parameters on D training tokens:
Cx D.
(3)
5

---

Published as a conference paper at ICLR 2026
Unconstralned MPNN
EGNN
GemNet-OG
6SEN
16
10"
edf-spritit
12a
10
10"
16'
CIN, 2.33ND
29.05ND
35,18DD
1o"
10"
15"
16"
10"
10'
10"
Model Parametem (N) Takers (D)
Model Parameters (N) Tokens (D)
Model Parameters (N) Tokens (D)
Model Parametors (N) x Tokens (D)
Figure 2: Estimation of for architectures used in our study.
untonetrained MPNN
EGNN
GemnNtt OC
10
Val Loos (log)
10
10
10
10"
Youtier
10"
10"
10'
10"
10
10"
10"
10' PFLOPs
10
PFLOPs
PFLOPs
PFLOPs
Unconstrainad MPNN
EGNN
GomNet-OC
+SEN
10
10"
Vall Loss (log)
10
Torter
10
10
10
10
10"
10"
10
10
10"
10
GPU Hours
GPU Houm
GPU Hours
GPU Hours
Figure 3: Pareto frontiers of training compute in log-log spaces. Top: Efficient loss-FL.OPs frontier.
Bottom: Efficient loss-train-time frontier. Across architectures, the log- frontiers are approxi- mately linear. Line color encodes model size (small, large).
Here, K is an archtecture-dependen constant representing the number of FLOPs required for a single forward pass over one imput token. During training. each input incurs both forward and a backward pass, with the latter approximately doubling the FLOP cost. Consequently, the total training cost per token is 3k. For architectures dominated by dense linear layers, ss 2, yielding the widely used expression C' z 6ND for transformer-basee language models.
Estimating K. We empirically estimate k by varying N and D, recording FLOPs for pass over D. and fitting C' vs. N.. Figure 2 shows a clear linear trend with distinct x per architecture: uncon- strained MPNNs with mostly linear layers give x z 2, whereas equivariant architectures incur larger K.
Val Loss (log)
Compute-Optine Frontier. For compute scaling, we follow ap- proach 1 in Hoffmann et al. (2022). For each compute budget, we select the minimum validation loss achieved across runs, yielding the loss-compute Pareto frontiers. As shown in Figure 4, loss- compute frontiers across architectures follows in linear relation- ships in log-log space. We then fit these frontier points to the power law:
10"
fast 2, (UU 0.51C-1
10"
10'
10"
10"
PFLOPs (C)
Figure 4: Using higher orders of feature tensors in eSEN leads to better scaling expo- nents compute.
= = + FAC L(H)= = Lox + PA
(4)
where Tex is the irreducible loss for the given architecture and dataset, and Fo,Ye, FR. and in fit parameters. Unlike language modeling with cross-entropy, force-field tasks do not admit a clear theoretical baseline for Loo (Brehmer et al., 2025; Wood et al., 2025); therefore, set L z unless noted otherwise. This choice of Loe gives exponents that are consistent with the alternative
OG exhibits noisier learning curves because it relies on empirically estimated pey-layer scaling factors-approstt approximate from few random hatches-rather than explicit normalization (e.g., LayerNorm) to control activation variance.

---

Published as a conference paper at ICLR 2026
derivation in section 4.3. Figure/ summarizes main results, which indicate different exponents for architectures with increasing levels of symmetry expressivity. We also find the argument holds within the same architecture; particularly, fig. 4 shows improvement in compute-scaling expo- nents as the max order mmx increases from 2 to in eSEN. Finally, it is worth noting that we do not study the effects of denoising pretraining, as done for Orb, a non-equivariant model, by Neumann et al. (2024). Consequently, our compute-scaling results are not directly comparable to this line of work, since their compute budget allocated differently between pretraining and downstream fine-tuning.
Unconssrained MPNN
EGNN
GamNat-OC
oSEN
Val Loss (log)
-1 11411000-------
A.D1 11x110
201
2012
10
10
10
10
10
10
10"
10"
Training Takens (lag)
Training Tokans (log)
Training Tokens (log)
Traning Tokans (log)
Unconstrained MPNN
EGNN
Go
VVal Loss (log)
10
L000 2 2,3
2D0 - 3.
L900 - 10W
Di soa
-1000
10"
10"
10"
10"
10"
10"
10"
Model Parameters (log)
Model Parameters (log)
Model P Parameters (log)
Model Parameters (log)
Figure 5: Top: Scaling number of training tokens. Bottom: Scaling number of parameters
4.2 SCALING PARAMETER AND DATASET
Law-Powwr--aw. To analyze scaling in model size N and dataset size D, we follow approach of (Hoffmann et al., 2022) and fit the triplets (N, D, L) to the separable power-law model as:
L(N, L.x + x N + BB
LNN, D) is the validation loss represented as function of (N, D). Loc A, A, 3 are parameters that we fit. Notably, we found Lx to be in all architecture families. For each N, we measured validation loss at training set fractions r E {0.1, 0.2, ... 1.0), that is D, =r. D
Scaling Analysis. Figure 5 presents our fit for four architectures under study. The top row shows the power-law fit in validation loss when the number of training tokens, D. is the limiting factor. Power-law exponents B, are .31, .39,..0 .50 and .75 from left right. The bottom row shows this relationship when the number of model parameters, N, is the bottleneck. Here, the exponent from left to right is 28, .9,,.9... .2, and .82 These results highlight three phenomena:
Data Efficiency: In data- lama-eimited scenarios, equivariant models demonstrate superior scaling be- haviours compared to unconstrained models, demonstrated by their larger scaling exponents 3. Moreover, equivariance of higher orders translates to larger exponents.
When bottlenecked by model size, equivariant models exhibit higher scaling ex- ponents with respect to N. This occurs because explicit symmetry constraints enable greater expressivity with fewer parameters. Furthermore, the scaling exponent gap between high-order architectures (i.e., eSEN, GemNet-OC) and lower-order ones (i.e., EGNN) considerable. While higher order representations are known to result in better expressivity (Joshi et al., 2023), the fact that the benefit of such representations grows with scale is novel finding.
Expressivity:
3: The exponents remain close across architectures. We discuss this finding in section 4.3.
In brief, observe larger data-scaling exponents 3 for equivariant networks, consistent with prior reports (Batzner et al., 2022: Brehmer et al., 2025; Wood et al., 2025). Meanwhile, our parameter- scaling exponents a are larger for equivariant networks, and this differs from (Brehmer al., 2025),
Becauus the GemNet-OC loss curve is high-variance, smooth it using exponential moving average with a smoothing factor of 0.9.
7

---

Published as a conference paper at ICLR 2026
which report larger for unconstrained models; note that the tasks are not directly comparable. Together, the increases in both 9 and a for equivariant models change the slope of the compute- optimal frontier under C' ND, which is one of our main findings.
4.3 COMPUTE- OPTIMAL ALLOCATION
We have presented two scaling laws far: (1) power law with respect to the compute-optinal frontier in section 4.1, and (2) a sun-power-law with respect to parameter count and the number of training tokens in section 4.2. In this section, discuss the connection between them. Given fixed compute budget C' (FLOPs), we seek the optimal allocation between model size N and training tokens D. We pose this constrained optimization problem that combines eq. (5) with eq. (3); in particular, we have:
N' (C), (C) = argmin L(N, D), 3RND =
(6)
Recall that L(N,D) D) = Lo + AN + BD-8 with Loo z O. Let (C) and (C) denote, respectively, the compute-optimal model size and data size for fixed compute budget C. Solving eq. (6) yields (C) = G (C( = G-1g- where 5 = 3x.
ad a 33
a+B b a+3 (Hoffmann et al., 2022; Brehmer et al., 2025). Furthermore, plugging back the results to L(N, D), we get back the loss-compute frontier power law similar to eq. (4):
L(C)= = L(N) = = F,O
a8 Table I presents the values of F and %o obtained where F. = AG + BG and % o+B
from two methods. The results show good agreement between them, indicating the consistency of our power laws. We further visualize the compute-optimal allocation between model size N and data size D in section 4.3. Across architectures, find b Rs 0.5, indicating that parameters and tokens should be scaled in roughly equal proportions, consistent with the Chinchilla allocation for transformer language modelling (Hoffmann al., 2022).
Table I: Compute-optimal scaling law parameters with 95% confidence intervals; see appendix E. Compute is scaled to PFLOPs.
Architecture
Param
Fit Method
Compute-Optimal Frontier
Sum-Power-Law eq. (7)
eq. (4)
Fo
0.928 [0.925-0,930]
0.934 [0.863-0.952]
MPNN
Y
0.142 [0.141-0.143]
0.146 [0.142- -22-0.10 159]
Fo
0.775 [0.761-0.792]
0.811 [0.784-0.832]
MC-EGNN
7
0.173 10.169-0.17)
0.195 [0.188-0.204]
Fe
0.488 [0.485-0.491]
0.479 [0.424-0.542]
GemNer-OC
7
0.255 [0.252-0.257]
0.256 [0.232-0.282]
Fe
0.703 [0.696-0.712
0.669 11577--0.799
cSEN
7
0.403 [0.401-0.406]
0.392 [0.342-0.451]
MPNN
EGNN
WBEN
Model Paramelers
Training Tokens
10*
Figure 6: Optimal model size (left) and training size (right) at fixed level of compute (log-log)
10
NIC) C30
((C) 22
10'
0CC) ca
.
(CC)
c340
( c
10
10"
Compute (FLOPs)
Compute (FLOPs)
8

---

Published as a conference paper at ICLR 2026
Sym
N 79M
929M
7010
$..0"
Val (log)
10"
$+10
1-50
I(C) - 1.18C-110
((D)
(09.500
-340
L(N) - 18.5N
L(C) 0.93
L(D) - 156.360
0000 22.77
10"
10'
10"
10"
10"
10' Model Parameters
PFLOPs
Training Tokens
Figure? 7: Comparison of scaling exponents for unconstrained MPNNS with/without symmetry reg- ularization. Left: Rightward shift of the log-log cosspcomptte frontier. Middle: Symmetry loss increases the data-scaling exponent (3), indicating improved data efficiency. Right: The parameter- scaling exponent (ax) decreases, suggesting that the regularization benefits larger models more. Val- idation loss excludes the regularization term (task loss only).
4.4 EFFECT OF SYMMETRY LOSS IN SCALING LAWS
In this experiment, we train an unconstrained model augmented with a symmetry-loss term. The loss is L Lobj + X sym- where Lobj is the task loss in eq. (1) and Laym is the symmetry loss in eq. (2) wherein we set M = 5. We use unit coefficients for both terms (i.e., X = 1), as smaller weights (A < 1) on Leym are reported have negligible effect (Elhag et al., 2025). For validation, we track Lesj. ensuring direct comparison with models trained without the symmetry penalty. We fit the learning-curve trajectories to the functional forms in eq. (4) and eq. (5). Figure 7 shows the resulting fits. Compared with models trained without Layms we observe:
Opposite Changes in Slopes D and N: Under the scaling form L(N, D) = L.o + AN-a + BD-3 when N is sufficiently large (N - oo), adding symmetry-constraint loss slightly in- creases the data exponent 3, indicating improved sumple-efficienyy: the model leverages the regularizer to infer approximate symmetries from data. Conversely, in the infinite-data regime (D + 30), a smaller model-size exponent a implies that increasing the parameter count more effectively reduces loss.
We hypothesize that because the N- and D-slopes change in opposite directions, the induced exponent y with respect to compute C ND is preserved. Furthermore, the sampling-bass regularizer in eq. (2) functions as data augmentation: in addition to each original sample, we also evaluate group-transformed inputs and predict the correspond-
Unchanged Compute-Optimet Slope:
ingly transformed targets. As result, the training FLOPs scale Csym (M + 1) Cunconstrs shifting the compute-optimat frontier to the right by x ? log(M + 1) z 0.14 log (6) in log-log coordinates as shown in fig. 7. Our fits indicate that approximate symmetry enforeed via sampling-based augmentation may be unnecessary for compute-optimal scaling, as the relevant scaling exponents remain unchanged.
4.5 EFFECT OF MULTI-EPOCH TRAINING IN SCALING LAWS
In the previous sections, we present clear scaling-law trends across architectures in the one-pass training regime, keeping our empirical setup aligned with insights from rigorous theoretical works that focus on this setting (Paquette et al., 2024; Bordelon et al., 2024). Unfortunately. scaling laws for scenarios in which data repeated for multiple epochs remain under-explored. both in practice and in theory. To our knowledge. Muennighoff et al. (2023) is among the few works investigating this area, showing that under fixed compute, training models for small number of epochs on re- peated data has negligible effects on the loss, behaving almost as if the models are trained on fresh data. We examine this hypothesis in our study by simulating a scenario where data extremely lim- ited by sampling only 1% of the full dataset, to train the models. The data is repeated over 100 epochs in each training run, which naturally enables the use data augmentation to improve the data effi- ciency of unconstrained models. Also, validation losses recorded after every 1000 gradient steps. Figure shows the resulting fits this regime. We observe that for unconstrained models trained without augmentation, the loss-compute frontier follows power-law scaling in early epochs, similar

---

Published as a conference paper at ICLR 2026
Data Augmentation
Compute-optinal Power Laws
No Data Augmeniation
2-30'
2x20
Val Loss (log)
10
100
10a
no augmertation
6x36
UG) 0,97C
sugmentation 10G 0.85-022-22
frontior
frontier
10-2
10-1
100
10a
10a
10
10 1
100
10 PFLOPs
10a
10
2
10
100
10a
PFLOPS
PFLOPs
Figure 8: Loss-compute Pareto frontiers for unconstrained models trained on 1% of the data for 100 epochs, shown without data augmentation (Left) and with data augmentation (Middle). Right: The linear trend in log-log space is broken at late training when data augmentation omitted, and is recovered when data augmentation is utilized.
to findings of Muennighoff et al. (2023), However, this power law breaks down when the number of passes exceeds a certain threshold, as heavy data repetition induces overfitting, In contrast, data augmentation substantially stabilizes the learning curves and ultimately recovers the same power law (% = 0.14, F. 0.96) with respect to the compute-optimet frontier as in the one-epoch training regime, indicating that the effect of data augmentation under low-data regime is the same adding fresh data one-pass training over larger datasets. Furthermore, we also examine the power laws of cSEN under this regime, and observe that its compute-optimal power-law holds as the same in one-pass training. Importantly, fig. reveals that the gap between data augmentation and equivariant networks continues to grow as compute increases through multi-epoch training.
CONCLUSION, FUTURE WORKS AND LIMITATIONS
Our empirical study of scaling laws in the geometric task of interatomic potentials shows that the degree to which an architecture encodes domain symmetries is correlated with the exponent in its power-law scaling behaviour. The empirical change in exponent is dramatic, suggesting that the role of symmetry potentially extends beyond simply reducing data dimensionality (Sharma & Kaplan, 2022).
Compute-optimal Power Laws
Val Loss (log)
100
This is because the degrees of freedom in the input and output are as 3n for n atoms, while the rotation group is only tbree-dimensiona an important future research direction in developing a the- ory that explains this scaling behaviour. On the practical side, our work provides a recipe for scaling the model and data size in geometric tasks, such as force fields, and it motivates the development of more scalable models that utilize higher-order representations.
Data augmentation (IC) 0. 0.96C
Our findings, therefore, suggest
1
Equlvariant (eSEN) ((C) 0.730
10 1
10 PFLOPS
10
Figure Scaling with training compute for unconstrained models trained with data augmentation and eSEN. The mod- els are trained on 1% of training dataset for 100 epochs.
Other directions for future work are, in part, motivated by the limitations of this work: (I) Our scaling-law analy- sis focuses on single-epoch, academic-scale settings for NNIPs. Extending it to multi-epoch training and larger models, as well as more diverse models and datasets, is natural next step. (2) Our study of symmetry losses was confined to one simple choice; it is possible that training with alternative definitions, if scalable, could provide a different scaling behaviour. (3) Our work completely ignores the family of architettur--gnostic equivariant models, such as frame averaging and canonicalization (e.g.. Puny et al., 2022; Kaba et al., 2023; Kim et al., 2023b). We plan to study their scaling laws in the future. (4) Finally, leave to future work sys- tematic, large-scale evaluation of denoising pretraining for both unconstrained models (Neumann al., 2024; Rhodes et al., 2025) and equivariant networks (Liao et al., 2024a).
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
This research is supported by the Canada CIFAR AI Chairs program, Samsung Al Labs, IVADO, and the NSERC Discovery Grant. Computational resources for experiments are provided by the Digital Research Alliance of Canada (Compute Canada) and Mila.
REFERENCES
Josh Abramson, Jonas Adler, Jack Dunger, Richard Evans, Tim Green, Alexander Pritzel, Olaf Ronneberger, Lindsay Willmore, Andrew J. Ballard, Joshua Bambrick, Sebastian W. Boden- stein, David A. Evans, Chia-Chun Hung, Michael O'Neill, David Reiman, Kathryn Tunyasuvu- nakool, Zachary Wu, Akvilé Zemgulyté, Eirini Arvaniti, Charles Beattie, Ottavia Bertolli, Alex Bridgland, Alexey Cherepanov, Miles Congreve, Alexander I. Cowen-Rivers, Andrew Cowie, Michael Figurnov, Fabian B. Fuchs, Hannah Gladman, Rishub Jain, Yousuf A. Khan, Caro- line M. R. Low, Kuba Perlin, Anna Potapenko, Pascal Savy, Sukhdeep Singh, Adrian Stecula, Ashok Thillaisundaram, Catherine Tong, Sergei Yakneen, Ellen D. Zhong, Michal Zielinski, Augustin Zidek, Victor Bapst, Pushmeet Kohli, Max Jaderberg, Demis Hassabis, and John M. Jumper. Accurate structure prediction of biomolecular interactions with alphafold 3. Nature, 63008016);493-5000, Jun 2024. ISSN 1476-4687. doi: 10. 10.1038/4411866-444-04777--- URL https://doi.org/
1038/ /418//411866-2 024 07487-w.
Subutai Ahmad and Gerald Tesauro. Scaling and generalization in neural networks: A case study. In D. Touretzky (ed.), Advances in Neural Information Processing Systems, volume 1. Morgan- Kaufmann, 1988. htttt:::://cccceciiinttt//o///ee neurips wwwpappr_filliicceeeeeeeeeeaaaaa files/paper/ 1988/file//t d1 f (1t.1/11111001000400000433333330 Pap.err.d..
Tara Akhound-Sadegh, Laurence Perreault-Levasseurr Johannes Brandstetter, Max Welling, and Siamak Ravanbakhsh. Lie point symmetry and physics-informed networks. In A. Oh, T. Naumann, A. Globerson, K. Saenko. M. Hardt, and S. Levine (eds.), Advances Neu- ral Information Processing Systems, volume 36, pp. 42468-42481. Curran Associates, Inc., 2023. URL
http::acecccc...eeeoooooooeaeeae pdf.
Ibrahim M Alabdulmohsin, Behnam Neyshabur, and Xiaohua Zhai. Revisiting neu- ral scaling laws language and vision. S. Koyejo, Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Information Process- ing Systems, volume 35, pp. 22300-22312. Curran Associates, Inc., 2022. URL neurips cc/ http://ppppeeeeeeeeeeeeeeeee/eee 2ttp///222////e///// påf.
Brandon Anderson, Truong Son Hy, and Risi Kondor. Cormorant: Covariant molecular neu- ral networks. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Gareett (eds.), Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019. URL https: httpsss://ocooeddepeee htpppp/aareerrrrrroopppeepppp__e wttppcpp_ffpeess/////peppppppspe Paps---aprrpppp. pappssas..22aaaaaa.pp........
Yasaman Bahri, Ethan Dyer, Jared Kaplan, Jachoon Lee, and Utkarsh Sharma. Explaining neural scaling laws. Proceedings of the National Academy of Sciences, 121(27):c23118881111
Yulu Bai, Jiahong Fu, Qi Xie, and Deyu Meng. A regularization-guiide equivariant approach for image restoration. In Proceedings the Computer Vision and Pattern Recognition Conference, pp. 2300-2310, 2025.
Ilyes Batatia, David Kovacs, Gregor Simm, Christoph Ortner, and Gabor Csanyi. Mace: Higher order equivariant message passing neural networks for fast and accurate force fields. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Information Processing Systems, volume 35, pp. 14422-11436 Curran Associates, Inc., 2022. URL https: https::/prroceedigg.... nww.siss........................ ceerssss.....sss...ppppeees htl__fffleesppppee ppppp//22err// at1e:///ccccrriaaa333aaaaaaaaaae

---

Published as a conference paper at ICLR 2026
Simon Batzner, Albert Musaclian, Lixin Sun, Mario Geiger, Jonathan P. Mailoa, Mordechai Ko- mbluth, Nicola Molinari, Tess E. Smidt, and Boris Kozinsky. E())-qquivariant graph neural networks for data-efficient and accurate interatomic potentials. Nature Communications, 13 (1), May 2022. doi: 10.1038/s41400 419-8/441-4---0222299933-- URL https: ptaps://ddii aeriioogg/100 341167-000 02777-022-00 29939--.
Filippo Bigi, Marcel F. Langer, and Michele Ceriotti. The dark side of the forces: assessing non-conservative force models for atomistic machine learning. In Forty-second International Conference on Machine Learning, 2025. URL https: /ttps/////peeeeeeeee net forum? d= 0E13LBosas.
Blake Bordelon, Alexander Atanasov, and Cengiz Pehlevan. A dynamical model of neural scal- ing laws. Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp (eds.), Proceedings of the 41 International Confer- ence on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pp. 4382. PMLR, 21-27 Jul 2024. URL https::/proceed// .ttp.edd/5 bardel . html.
mlr
Johann Brehmer, Sönke Behrends, Pim De Haan, and Taco Cohen. Does equivariance matter scale?, 2025. URL http::s:////peeeiiieeee/ net. wtt.riirmmeet/oommmm d=i IWey fGeyffGof.
Ethan Caballero, Kshitij Gupta, Irina Rish, and David Krueger. Broken neural scaling laws. In The Eleventh International Conference on Learning Represemations, 2023. URL https://
openreview. n forum? fesprummid=sscjjeeo jveqlekijjeecc cz.
Leshem Choshen, Yang Zhang, and Jacob Andreas. A hitchhiker's guide to scaling law estimation. In International Conference on Machine Learning, 2025.
Corinna Cortes, Lawrence D Jackel, Sara Solla, Vladimir Vapnik, and John Denker. Learning curves: Asymptotic values and rate of convergence. Advances in neural information process- ing systems, 6, 1993.
Rumen Dangovski, Li Jing, Charlotte Loh, Seungwook Han, Akash Srivastava, Brian Cheung, Pulkit Agrawal, and Marin Soljačić. Equivariant contrastive learning, arXiv preprint arXiv:2111.00S: 2021.
Aaron Defazio, Xingyu Yang, Harsh Mehta, Konstantin Mishchenko, Ahmed Khaled, and Ashok Cutkosky. The road less scheduled. In A. Globerson, L. Mackey, D. Bel- grave, A. Fan, U. Paquet, Tomezak, and C. Zhang (eds.), Advances Neural In- formation Processing Systems, volume 37, pp. 9974-10007. Curran Associates, Inc., 2024. URL https: https::/proceeed: http:///poooeococceeeeeo hot.aeecceeccc-pppeeooo-eeeeeee. pdf.
wwwpeeewwnnnnnnndndeeeeeeeeeeeee neurips cc
Bowen Deng. Peichen Zhong, KyuJung Jun, Janosh Ricbesell, Kevin Han, Christopher Bartel, and Gerbrand Ceder. Chgnet pretrained universal neural network potential for charge-informe atomistic modelling. Nature Machine Intelligence, 103)1103, 2023.
Alexandre Agm Duval, Victor Schmidt, Alex Hernández-Garcfa. Santiago Miret, Fragkiskos D. Malliaros, Yoshua Bengio, and David Rolnick. FAENet: Frame averaging equivariant GNN for materials modeling. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 9013-9033. PMLR, 23-29 Jul 2023. URL https: http://ooooeeccccce//////oe .aw.eediiinsmm.........mmmm.mm.e www.mmrer.sss.....mmmmmm.mmmmm.e v202/220////// duval23a. html.
Ahmed A. A. Elhag, T. Konstantin Rusch, Francesco Di Giovanni, and Michael M. Bronstein. Re- laxed equivariance via multitask learning. In ICLR 2025 Workshop on Machine Learning for Ge- nomics Explorations, 2025. http: wpw.revieeeeeeeeeeeeeeeee. net
Eduardo Santos Escriche and Stefanie Jegelka. Learning equivariant models by discovering sym- metries with learnalle augmentations. arXiv preprint arXiv:2506.039/4, 2025.
12

---

Published as a conference paper at ICLR 2026
Nathan Frey, Ryan Soklaski, Simon Axelrod, Siddharth Samsi, Rafael Hoomez-Bombarelli Connor Coley, and Vijay Gadepally. Neural scaling of deep chemical models. ChemRxiv, 2022. doi:
1.1264344ceemxxi2222--5551222 2022-3s512.
Nathan C Frey, Ryan Soklaski, Simon Axelrod, Siddharth Samsi, Rafael Gomez-Bombarelli, Con- nor W Coley, and Vijay Gadepally, Neural scaling of deep chemical models. Nature Machine Intelligence. 5(1 1(11):129753355 2023.
Xiang Fu, Brandon M Wood, Luis Barroso-Luque Daniel S. Levine, Meng Gao, Misko Dzamba, and C. Lawrence Zitnick. Learning smooth and expressive interatomic potentials for physical property prediction. Forty-second International Conference on Machine Learning, 2025. URL https:/ http:///ppppeeeeeeee/eee net wwwp///////rr////e forum? (errrummddRRPPjxxxXT
Johannes Gasteiger, Shankari Giri, Johannes T. Margraf, and Stephan Günnemann. Fast and uncertainty-aware directional message passing for non-equilibrium molecules. In Machine Learn- ing for Molecules Workshop, NeurIPS, 2020a.
Johannes Gasteiger, Janek GroB, and Stephan Günnemann. Directional message passing for molecular graphs In International Conference on Learning Representations, 2020b. URL htttps://oe http///pppeeeeeeeeeeeeeeeeeeeeer net
httpr//wwww.eetooooooeo/eeeeee/e forum?1d-BleWbbxtcPP
Johannes Gasteiger, Muhammed Shuaibi, Anuroop Sriram, Stephan Günnemann, Zachary Ulissi, C. Lawrence Zitnick, and Abhishek Das. How do graph networks generalize to large and diverse molecular systems? arxiv preprint arxiv:2204.0277 2022.
Tom Henighan, Jared Kaplan, Mor Katz, Mark Chen, Christopher Hesse, Jacob Jackson, Heewoo Jun, Tom B Brown, Prafulla Dhariwal, Scott Gray, et al. Scaling laws for autoregressive generative modeling. arXiv preprint arXiv:2010. 14701, 2020.
Joel Hestness, Sharan Narang, Newsha Ardalani, Gregory Diamos, Heewoo Jun, Hassan Kianinejad, Md Mostofa Ali Patwary, Yang Yang, and Yanqi Zhou. Deep learning scaling is predictable, empirically. arXiv preprint arXiv:1712.00409, 2017.
Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, al. Train- ing compute-optimal large language models. arXiv preprint arXiv:2203. 15556, 2022.
Shengding Hu, Yuge Tu, Xu Han, Chaoqun He, Ganqu Cui, Xiang Long, Zhi Zheng, Yewei Fang, Yuxiang Huang, Weilin Zhao, al. Minicpm: Unveiling the potential of small language models with scalable training strategies. arXiv preprint arXiv:2404. 06395. 2024.
Weile Jia, Han Wang, Mohan Chen, Denghui Lu, Lin Lin, Roberto Car, Weinan E, and Linfeng Zhang. Pushing the limit of molecular dynamics with ab initio accuracy to 100 million atoms with machine learning. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, SC '20. IEEE Press, 2020.
Chaitanya K. Joshi, Cristian Bodnar, Simon V Mathis, Taco Cohen, and Pietro Lio. On the expres- sive power of geometric graph neural networks. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), Proceedings of the 40th In- ternational Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 15330-15355. PMLR, 23-29 Jul 2023. URL httttsss//ppseeeeeeeeeeeee proceedings mmrr. press/v200 /wrp222/jsshiai joshisssshaa...... html.
Sékou- Oumaa Kaba, Arnab Kumar Mondal, Yan Zhang, Yoshua Bengio, and Siamak Ravan- bakhsh. Equivariance with learned canonicalization functions. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), Proceedings of the 40th International Conference Machine Learning, volume 202 of Pro- ceedings of Machine Learning Research, pp. 15546-15566. PMLR, 23-29 Jul 2023. URL https: wtppppeeednngsssp mlr www.presssseeeeee / v22 / aa.akaaaaaaaaaaa..aa.....aaaaa.. html.
Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. arXiv preprint arXiv:2001.0836/ 2020.
13

---

Published as a conference paper at ICLR 2026
Hyunsu Kim, Hyungi Lee, Hongseok Yang, and Juho Lee. Regularizing towards soft equivari- ance under mixed symmetries. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 16712-16727. PMLR, 23-29 Jul 2023a. URL https: CRL.httts://prceeee http://oooeeoiceccooooooo ngs . ml .resss/ w202/kim23p html.
Jinwoo Kim, Dat Nguyen, Ayhan Suleymanzade, Hyeokjun An, and Seunghoon Hong. Learn- ing probabilistic symmetrization for architecture agnostic equivariance. In A. Oh, T. Nau- mann, A. Globerson, K. Saenko, M. Hardt, and Levine (eds.), Advances in Neural Information Processing Systems, volume 36, pp- 18582-18612. Curran Associates, Inc., ngg www.:/ceeeennnnneeeeeeeee.eeee.e neurips cc/ cwtpaper_eeeeeeeeeeeeeeee www.e/feeesspee/aeeeeeeee 2023b. URL http://00occcccca77//aaaaaaaaaaa Paaper Conferonneeceeeeecececeeeeoe pdf.
Johannes Klicpera, Florian Becker, and Stephan Günnemann. Gemnet: Universal directional graph neural networks for molecules. In Conference on Neural Information Processing (NeurIPS), 2021.
Daniel Levine, Muhammed Shuaibi, Evan Walter Clark Spotte-Smine Michael Taylor, Muham- mad R Hasyim, Kyle Michel, Ilyes Batatia, Gábor Csányi, Misko Dzamba, Peter Eastman, et al. The open molecules 2025 (omol25) dataset, evaluations, and models. arXiv preprint arXiv:2505.08762. 2025.
Daniel Levy, Sékou-Ouma Kaba, Carmelo Gonzales, Santiago Miret, and Siamak Ravanbakhsh. Using multiple vector channels improves e (n)-equivariant graph neural networks. arXiv preprint arXiv:2309.03139 2023.
Chaojian Li, Zhifan Ye, Massimiliano Lupo Pasini, Jong Youl Choi, Cheng Wan, Yingyan Celine Lin, and Prasanna Balaprakash. Scaling laws of graph neural networks for atomistic materials modeling. arXiv preprint arXiv:2504.088 00.55040.8/// 12, 2025.
Yi-Lun Liao and Tess Smidt. Equiformer: Equivariant graph attention transformer for 3d atomistic graphs. In The Eleventh International Conference Learning Representations, 2023. URL FARqOID.
https: / oppnreviewww wwwriiewneee./eee wtwpee/wwwweeettoumeeeeeeeeeeee
Yi-Lun Liao. Tess Smidt, Muhammed Shuaibi, and Abhishek Das. Generalizing denoising to non-equilbbrium structures improves equivariant force fields. Transactions Machine Learn- ing Research, 2024a. ISSN 2835-8856. URL https: http::s///ppnreviee..o whGZYUbIWA.
forum?id-
Yi-Lun Liao, Brandon M Wood, Abhishek Das, and Tess Smidt. Equiformerv2: Improved equivari- transformer for scaling higher-degree representations. In The Twelfth International Confer- ence on Learning Representations, 2024b. URL https: http:////pneeeeeweecc/e net wwwpeevwwwwweee.ruu.nnee. forum?ie d - mCOBKZmrZD.
/
Jingzhe Liu, Haitao Mao, Zhikai Chen, Tong Zhao, Neil Shah, and Jiliang Tang. Neural scaling laws on graphs. arXiv e-prints, pp. arXiv-2402, 2024.
Andreas Loukas. What graph neural networks cannot learn: depth vs width. In International Confer- ence Learning Representations, 2020. URL https: http::////ereeee..reee hwtpsriwwwwweeeeeeeeeeeeeeeeeeee B1 B112bp4YW5.
Alexander Maloney, Daniel Roberts, and James Sully. solvable model of neural scaling laws. arXiv preprint arXiv:2210. 16859, 2022.
Niklas Muennighoff, Alexander Rush, Boaz Barak, Teven Le Scao, Nouamane Tazi, Aleksandra Pik- tus, Sampo Pyysalo, Thomas Wolf, and Colin Raffel. Scaling data-constrained language mod- els. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (eds.), Advances in Neural Information Processing Systems, volume 36, pp. 50358-50376. Curran Associates, Inc., cc/ pawer lttpeeerrfiieee/pppeeeeee
2023. URL https://proceeeings..eeett file/ 9d8 94 8b /tt////8888880lee222888222222133 Paper
httpr//ceeeccceaaaaeeo--rrrreeee www.erenooeeeccoooooee pdf.
Mark Neumann, James Gin, Benjamin Rhodes, Steven Bennett, Zhiyi Li, Hitarth Choubisa, Arthur Hussey, and Jonathan Godwin. Orb: fast, scalable neural network potential, 2024. URL
https: http///xvvvvv/..///ev///.ee...at wttp///ww.vvogg//////////////// 2255 22270..
14

---

Published as a conference paper at ICLR 2026
Elliot Paquette, Courtney Paquette, Lechao Xiao, and Jeffrey Pennington. 4+3 phases of compute-optimal neural scaling laws. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang (eds.), Advances in Neural Information Processing Systems, volume 37, pp. 16459-16537. Curran Associates, Inc., 2024. doi: 10.52202/ 079017-0526. URL https::tt//..... httpps:/ppooceeeiiiiiee neurips htcc//aper_file//pppppee Peaper-Conference.pde
Saro Passaro and C. Lawrence Zitnick. Reducing SO(3) convolutions to SO(2) for efficient equivariant GNNs. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engel- hardt, Sivan Sabato, and Jonathan Scarlett (eds.), Proceedings of the 40th International Con- ference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 27420-27433 PMLR, 23-29 Jul 2023. URL https: http:///poceeedeeeeeee /ttts://pppo 2202/passaro23a... html.
mlr .tt.ssessr....s////mmmmm
Zihan Pengmei, Zhengyuan Shen, Zichen Wang, Marcus D. Collins, and Huzefa Rangwala. Push- ing the limits of all-atom geometric graph neural networks: Pre-training, scaling, and zero-shot transfer. In The Thirteenth International Conference on Learning Representations, 2025. URL
https:// /pppr////eeeeeeeewe.. net/ forum?id=4521:: InIX..
trade-offs un- Mircea Petrache and Shubhendu Trivedi. In A. Oh, T. Naumann, A. Globerson, der (approximate) group equivariance.
K. Saenko, M. Hardt, and S. Levine (eds.), Advances in Neural Information Pro- cessing Systems, volume 36, pp. 61936-61959. Curran Associates, Inc., 2023. URL
hew. ccppppperrpcpppppppppppppp. c355000cGG11110001066fffaaaaa222
Omri Puny, Matan Atzmon, Edward Smith, Ishan Misra, Aditya Grover, Heli Ben-Hamu, and Yaron Lipman. Frame averaging for invariant and equivariant network design. In International Conference on Learning Representations, 2022. httpt http::::s///ppeeeeiieeee net/forum? forum?
id= 55R
Eric Qu and Aditi Krishnapriyan. The importance of being scalable: Improving the speed and accuracy of neural network interatomic potentials across chemical domains. In A. Globerson, L.. Mackey, D. Belgrave, A. Fan, U. Paquet, Tomezak, and C. Zhang (eds.), Advances in Neu- ral Information Processing Systems, volume 37, pp. 139030-339053 Curran Associates, Inc., 2024. URL https::/proceeedingg neurips oc wc/ppppprra.. fulee/facbellllfff111llellllllld
http://aapeeeeeeeeeeeeeeeeeeeeee wwweefillappeeee /ailes/paper/202// Paper Conference .dff.
Benjamin Rhodes, Sander Vandenhaute, Vaidotas Šimkus, James Gin, Jonathan Godwin, Tim Duignan, and Mark Neumann. Orb-v3: atomistic simulation at scale. arXiv preprint 062302,504.06211, 2025.
Víctor Garcia Satorras, Emiel Hoogeboom, and Max Welling. E(n) equivariant graph neural net- works. In International Conference Machine Learning (ICML), 2021.
Utkarsh Sharma and Jared Kaplan. Scaling laws from the data manifold dimension. Journal of Machine Learning Research, 23(9):1-34, 2022. URL http: http:///ww...oogppaps/se/////
20-1111. mt...
Charlie Victor Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling LLM test-time compute optimally can be more effective than scaling parameters for reasoning. In The Thirteenth Interna- tional Conference on Learning Representations, 2025. URL https: /tpprr/eee.//eeee......m .ne//
Richard Sutton. The bitter lesson. Incomplete Ideas (blog), 13(1):38, 2019.
Maciej Sypetkowski, Frederik Wenkel, Farimah Poursafaei, Nia Dickson, Karush Suri, Philip Frad- kin, and Dominique Beaini. On the scalability of gnns for molecular graphs. In A. Glober- son, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang (eds.), Advances in Neural Information Processing Systems, volume 37, pp. 10870-19906 Curran Associates, Inc., 2024. URL https: https: http:://sss/p.peeeeeeeeeeeceeoee wttp:/oocceennnnnnonnoiieeeeeeee neurips httpp//eeccaapppaaeeeaaeeaaeaaee Ei
15

---

Published as a conference paper at ICLR 2026
Nathaniel Thomas, Tess Smidt, Steven Kearnes, Lusann Yang, Li Li, Kai Kohlhoff, and Patrick Riley. Tensor field networks: Rotation-and -equiaaiioneuuvviiiant neural networks for 3d point clouds. arXiv preprint arXiv:/802.082)) 2018.
Vinh Tong, Trung-Dung Hoang, Anji Liu, Guy Van den Broeck, and Mathias Niepert. Rao-blackwell gradient estimators for equivariant denoising diffusion. arXiv preprini arXiv:2502.09890, 2025.
Jake Topping, Francesco Di Giovanni, Benjamin Paul Chamberlain, Xiaowen Dong. and Michael M. Bronstein. Understanding over-squashing and bottlenecks on graphs via curvature. Interna- tional Conference Learning Representations, 2022. URL https::/p///.mm htttts://openreeettee /ttp//ppeeeeeeeeeeeeeeee net/
Corum?id=7UmjjRGar A.
Yuyang Wang, Ahmed A. A. Elhag, Navdeep Jaitly, Joshua M. Susskind, and Miguel Ángel Bautista. Swallowing the bitter pill: Simplified scalable conformer generation. In Ruslan Salakhutdi- nov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp (eds.), Proceedings of the 41st International Conference Machine Leurning, vol- ume 235 of Proceedings of Machine Learning Research, pp. 50400-50418. PMLR, 21-27 Jul 2024. URL https::///..m... /ttp://///pccceiigggg.... mlr wwwpppeesssveeee
wang2 4g html.
Brandon Wood, Misko Dzamba, Xiang Fu, Meng Gao, Muhammed Shuaibi, Luis Barroso-Luque, Kareem Abdelmaqsoud, Vahe Gharakhanyan, John R Kitchin, Daniel S Levine, et al. Uma: A family of universal models for atoms. arXiv preprint arXiv:2506.33971, 2025.
Ge Yang, Edward Hu, Igor Babuschkin, Szymon Sidor, Xiaodong Liu, David Farhi, Nick Ryder, Jakub Pachocki, Weizhu Chen, and Jianfeng Gao. Tuning large neu- ral networks via zero-shot hyperparameter transfer. In M. Ranzato, A. Beygelzimer, Y. Dauphin. P.S. Liang, and Wortman Vaughan (eds.), Advances in Neural In- formation Processing Systems, volume 34, pp. 17084-17097. Curran Associates, Inc., neurips. oc fttprrrpsccppppee...eeeeeee///ee files/papee/2221/
2021. URL https::/proceeddiii http://aecasccccooeeeeeeeeeeeeee
Jianke Yang, Wang Rao, Nima Dehmamy, Robin Walters, and Rose Yu. Symmetry- informed governing equation discovery. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomezak, and C. Zhang (eds.), Advances in Neural In- formation Processing Systems, volume 37, pp. 65297-65327. Curran Associates, Inc., 2024. URL https://proceeddnggg www.rris...ppppeeee.eeeeeeeeeeee seraaprrfiileepppeeeeeeee file #ile/77Eaerddo6671115eeee111s6 Paper--app----e Conference .dff.
Xiaohua Zhai, Alexander Kolesnikov, Neil Houlsby, and Lucas Beyer. Scaling vision transformers. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp- 1110-12113, 2022.
16

---

Published as a conference paper at ICLR 2026
A RELATED WORKS
Neural Scaling Laws. Numerous studies investigating the scaling behavior of neural networks (Ahmad & Tesauro, 1988; Henighan et al., 2020; Hoffmann et al., 2022; Kaplan et al., 2020; Sharma & Kaplan, 2022) demonstrate a predictable relationship: performance improves model size N, dataset size D, and computational budget C' increase. Various functional forms have been proposed to model these scaling laws. Using test error e as the evaluation metric, Cortes al. (1993) and Hestness et al. (2017) proposed the functional form L = oz - + Los. where b > 0, Loe 2 0 denotes the irreducible error, and x can be N, D, or C. While useful, this form may result in infinite error as the scaling variable x approaches zero (e.g., for models performing random guessing in classification tasks). To address this limitation, Zhai et al. (2022) introduced a more general form, L = a(x + c)-b + Loo. where the parameter represents an effective offset, indicating the scale at which performance significantly surpasses random guessing. The coefficients of these power laws have been empirically explored across various research domains and tasks, including autoregressive generative modeling (Henighan et al.. 2020; Kaplan et al., 2020; Hoffmann et al., 2022) and computer vision (Zhai et al., 2022; Henighan et al., 2020; Alabdulmohsin et al., 2022). Further research by Hoffmann et al. (2022) and Snell et al. (2025) has improved methodologies for studying scaling laws, allowing for the determination of optimal scaling strategies to achieve the best performance on specific tasks under given constraints. Additionally, Caballero et al. (2023) introduced "broken" scaling laws to better model and extrapolate neural network scaling behaviours, particularly when the scaling functions exhibit non-monotonic transitions.
Scaling laws for MPNNS on Molecular Graphs. Learning accurate molecular representations is a fundamental challenge in drug discovery and computational chemistry. Numerous studies have investigated the scalability of graph neural networks (GNNs), particularly message-passing neural networks (MPNNs), for predicting molecular properties. For 2D molecular graphs, prior research al. (2024); Li et al. (2025) has demonstrated the promising
by Liu al. (2024); Sypetkowski scalability of MPNNs, showing that network performance follows a power-law scaling behaviour with increases in both dataset and model sizes. Similar scaling trends have also been observed by Frey et al. (2023); Li et al. (2025); Wood et al. (2025) for E(3)/SE(3) equivariant MPNNs trained on 3D atomistic systems. In contrast to these observations, Pengmei et al. (2025) demonstrate that scaling behaviours of geometric GNNs deviate from conventional power laws across different settings, including self-supervised, supervised, and unsupervised learning.
Our work extends scaling-law analysis across NNIP architectures. Unlike Wood et al. (2025), which derives compute-optimal scaling for equivariant models on mixed moleeie datasets with periodic coordinates and other auxiliaries, we focus on molecules using only atomic 3D coordinates and atomic numbers. Despite using the eSEN same backbone, Wood et al. (2025) report that, for dense models 6, the compute-optima strategy scales model size N faster than data size D, whereas in our setting we observe nearly equal scaling between N and D; though the tasks are different. Relative to Frey et al. (2023), our study uses substantially larger force-field field dataset, enabling more robust scaling estimates that consider different architectures.
B SUPPORTING FIGURES
C DESIGN PRINCIPLES FOR GEOMETRIC MESSAGE PASSING
Table 2: Architectures and their expressivity.
Architectures
Characteristic
Tensor Order (0)
Body Order (v)
Unconstrained MPNN
unconstrained invariant equivariant
2
GemNet-OC
4
EGNN
2
eSEN
cquivariant
2
2
2
'The authors also study the effect of linear mixture-of-experrs,, while don't consider this in our work.
17

---

Published as a conference paper at ICLR 2026
Unsonstrained MPNN
EGNN
GemNettDC
Bazze 122
Darzy
5.75
bizze +250
5150
Birce vibe +512
610
6200
Val
6179
6140
6128
Learning rate
rale
168
302
16,
88
12, 32
175
12,
B.
8,
238
125
d= 6,
Lo9ss
5, W 301
di 5,
158
4,
115
612
4032
4001
100
Training
Training
Training
Figure 10: A sweep over batch size-learming rate (top row), and depth-width (bottom row) for three architectures with numbers parameters are approximately equal to 1M.
Following Joshi et al. (2023), we categorize the models and their expressivity in Table The tensor order l denotes the order of the geometric tensor embeddings processed by each model. The body order refers to the number of nodes participating in a message function. Because GemNet-OC incorporates dihedral-angee -angle information, its two-hop messages depend simultaneously on the states of four nodes, and thus it is classified as four-bot four-body. By contrast, the remaining architectures use one-hop message passing that depends only on the source and target nodes, i.e., two-body.
Let [M] (1) ....,,iy, ( denote the node embeddings at layer I, with 1: initialized from input features zt. For distance cutoff 0, define the neighborhood
((e) {u # || z.ll cc..
We update node by aggregating messages from its neighbors using permutation-invarian (mean) aggregator, making the overall layer permutation-cquit -qquiiaiiiinnnuuuiiian:t
(++1
(0)
(0) m aI++
(.)))))
(8)
aEA(()
Let Tao Iu To be the relative position vector and dim h symmetry properties (e.g., E(3)/S((3) invariance or equivariance) are determined by architectural choices in (t) in particular, how it uses Tav (c.g., through rotational invariants such ||rwell via equivariant tensor constructions). We discuss specific message constructions in the next section.
be the embedding width. The
C.I UNCONSTRAINEH MESSAGE PASSING
Following Duval et al. (2023), geometry is injected directly:
m00 6ml (t) (t) Tuv: ||/uoll2),
with h A E R and dm p2w+4 + R is an MLP. Because raw vectors Two are processed without symmetry constraints, rotational equivariance is not enforced, and thus the node embeddings (t)
le not rotationally invariant.
C.2 DIRECTIONAL MESSAGE PASSING
GemNer-T (Klicpera et al., 2021) and GemNer-OC (Gasteiger et al., 2022) construct messages from multi-bodi E(3)-in -ivaainna geometric features, including pairwise distances, bond (three-body) an- gles, and dihedral (four-body) angles. Define the bond angle PPunk (row: Fok) and the dihedral
18

---

Published as a conference paper at ICLR 2026
angle Wuekj as the angle between the planes (u, v, k) and (u, j. (e.g., via normals nı Teu XPok and mg x Tky x Tku). Then the message from u to u is
(t) m u
Σ (EN(U) (EN(
om h (()) || Punk, Wusks)
(10)
where ( h E R" are scalar node embeddings and om is a learnable function operating on E(3) ((3)-invariant inputs (distance, angles) together with scalar features. Because the geometrie inputs are E(3) innvariant (rotation/translation invariant) and h's are scalar channels, the resulting message (33)-invariant well. Linear and bilinear interactions inside on do not affect this invariance so long as they act on invariant/scalar quantities.
C.3 CARTESIAN- VECTOR MESSAGE PASSING
The original EGNN uses single vector channel (node coordinates) (Satorras et al., 2021), which limits expressive power (Joshi et al., 2023). To address this, we use a multi-channel extension, MC-EGNN (Levy et al., 2023). Each atom carries both scalar features n,4 RR and E vector channels x E 3 F with the relative vector x!0 x= - E 3
MC-EGNN maintains invarianceeoquiivaiaaee by updating the invariant node embeddings h, and the multi-channel equivariant vectors Xa using invariant messages. In particular, messages depend only rotation-translattin invariants:
(2) om (h(), AS9, (1x)) Om IR 2w-E
(11)
where ||Xuolle E R denotes the channel-wise Euclidean norm (applied over the spatial com- ponents). The invariant message ) from Equation (11) is then used in Equation (8) to update (t+1), In addition, MC-EGNN updates the vectors via a channel mixer:
((++1) = x(( + [N(')
Σ
((0) D (m(m( ue
(12)
E
(EN(()
where E.. is linear map with weights W IR - R EXEE followed by reshape. This channel mixing preserves equivariance because E3) actions (and permutations) act on the 3D indices but on the channel index.
Varianee Scaling. Our empirical analyses suggest that increasing E significantly improves performance; see appendix This, in turn, suggests that studying scaling laws requires scaling both the invariant dimension w and the equivariant channels E. However, very large E can be computationally expensive, and scaling arbitrarily may cause exploding gradients due to the matrix- valued function Φ and the matrix product X Pm To scale properly, one should ensure that each layer's output scales as é(1) and gradient update scales as ((1) (Yang al., 2021), ie., are invariant across widths. Let W, ~ NO(0, with o = @ ////mmi,, EE))/w?). Setting (11//w). Under μPP we assume the entries of mu-ro and X..
E = E' vw yields have variance O(1); then W W also has ((1)-entries, and since reshape has no parameters, the entries of (m) remain (()). For the matrix product, because both Xav and Φ (Mn-)u) have -varinne entries, (Xuv (m which sums over E, scales OEE). To keep gradient updates stable across widths, we scale (m by 1/F zs 1/Vui instead of 1 //E, which has the same effect as scaling the logits of dot-product attention by 1/emb_die _im rather than 1/V Vembed_dii as discussed by Yang et al. (2021).
C.4 HIGH-ORDER TENSOR MESSAGE PASSING
Irreducible Representations. High-order equivariant models (c.g., Thomas et al. (2018); Ander- son et al. (2019); Batzner et al. (2022); Batatia et al. (2022); Liao et al. (2024b); Wood et al. (2025)) use 50(3) irreducible representations (irreps) as node embeddings:
Emax
(1)
h E RC (vo dim v( = 28 + 1,
(13)
0
19

---

Published as a conference paper at ICLR 2026
with + denotes concatenation of multiple order-E tensors that expanded by C₃ channels, and thus total dimension is d := Et=u C2(2E + 1). Assume all orders have the same number of channels C, the atom embedding has size of C(emax+1)²
SO(3) Convolution. Let Puv Fuv/ hau///raoll: monics:
Messages couple source irreps with spherical har-
Whil2ts
(0)) C u, (4,, m1), (fyma) mii
Σ
(lams)
V (Fuv)
(14)
f1
Eggnng where Eyma, Pamn are Clebsch- Gordan coefficients, 62,6 are learnable weights, and Y is the order-f spherical harmonics of the unit displacement vector fuvs 181 - Lal S lz S [e + Egl and my [-(4...., 4r.
Efficient Convolution. eSCN/<SEN (Passaro & Zitnick, 2023: Fu et al., 2025) sparsify eq. (14) by rotating vector Fuv into an edge-aligned frame. Let Ree E TR3x3 such that Rucfur = 1,0). Then y (RuvPuv)) O unless m2 0. Therefore,
simplify eq. (14) as:
m u-utt = D
we 62,6a
e my mi
Σ
R(00 (1 my) ((2,0))
(lfama)
(15)
here (1)4) (1) DE, h (f..m)) where we denote Dr, == Di, (Rav) and Di, == Deg (Rwe) denote Wigner-D matrices of order & and fg, respectively. The output rotated back by De, to ensure equivariance, and without loss of generality, we re-scale y (Rwofwe) to 1. Given that m2 0, (laamma (f.m1)) (f,.0)
non-zero only when mı +m3. This further simplifies the computation to:
(1) m mi-ros
- ta
Wr,
fa -
(I(())
clla.ms) + +1((0) mm(l)mms))
(t)..)) v,(f1, 1amm-mmmy my) (Es, (ss,(t)))((2.0)))) (16)
mts.
E1.82
ma
Rearranging eq. (16), we obtain:
m auttl = - Σ θ (1:0)
(6)
mg)) Σ Ee 82.88 (fa 0.(4 ma) Σ Ei 82 (t.-mm))((ta0))
(1
(la.ms))
(17)
Passaro & Zitnick (2023) propose to replace the Clesbh-Gordon coefficients with parameterized weights as:
(a) Wng
ms) Σ We, la (tr,ms).
ms)
Σ
ms) ms) &2,8s Cle
(t.- for m 2 0,
(18)
(s) ma
ms) Σ We, f2.82 (a)-
Σ WF. 82. C (t). for < 0.
ms)
(19)
Plugging back this into eq. (16), we obtain:
m um
D
Σ
yig
(61)
(20)
where:
Vi..0 WW (R,,0))))
[61]
((li,ts) (1) (1)
(21)
(61) y m))))
Wmm
(t),)a)
W
(fi)ta)
(alle.ms) ((())
(6))
=
ii (a)
(f,
for m3 > 0.
(22)
((ss mm))
Wing
(a((1.mms))
Therefore, the overall computation is reduced to equivalent SO(2) linear operation with a param- eterized kernel as in eq. (22). In eSEN, SO(2) blocks are applied only for values |myl S mmaa S bmaxi set M.naz = 2 as similar to the default setting.
20

---

Published as a conference paper at ICLR 2026
Table Scaling ladder for unconstrained MPNN.
Depth
Width
# Params
128
222404
5
256
838020
320
1293284
5
375
1763133
441
2422704
5
517
3311714
607
4543769
5
712
6226800
5
835
8535023
1150
16101729
5
1349
22109506
1583
30389712
5 1857
41755644
2179
57415632
5 2557
78974296
Table Scaling ladder for EGNN. We scale the number of channels for equivariant vectors as ww, ensuring stable update and -var-iari entries when scaling
Depth d
Width
Params
12
32
156358
12
48
306199
12
56
403992
12
64
516553
12
81
786925
12
102
1195553
12
120
1590131
12
160
2745453
12
200
4231015
12
300
9214218
12
500
24959524
12
600
35673024
12
721
51092972
12
800
63035228
D EXPERIMENTAL SETUP
Graph Construction. Except for GemNet-OC, geometric graphs are built with radial cutoff of 6 Á and a maximum of 30 neighbors per atom. For GemNet-OC, using the same cutoff occasionally failed on molecules where high-order message passing requires a larger candidate neighborhood. 10 Å cutoff and cap the neighborhood at 50 neighbors for this architecture.
Accordingly,
Direct-Force Prediction. Although the pre-training performance of direct-force models does not always transfer seamlessly certain downstream physical tasks, these models provide substantial efficiency gains during pre-training. We focus on the scaling behaviour of direct-force models in the pre-training regime, i.e., training across large collections of molecular systems, and therefore employ them consistently throughout our experiments.
Energy Reference and Normalization. Following Wood et al. (2025)², we apply the same ref- erence scheme the energies. Then, we normalize those referenced energies as é' (e p)/a, where μ and o are the sample mean and standard deviation estimated from the training set. Atomic
Phttps://oo thub. com/ wtt.c//acccccoeeeeerrrreooo Fai www.e//tteeeeeeeeeeeee
21

---

Published as a conference paper at ICLR 2026
forces for atom f -de 0z,, are invariant to constant energy shifts; accordingly, rescale them by the same factor: f. = fi/o.
Table 5:
Scaling ladder for eSEN. Since eSEN use high-order tensors expanded into C' channels each, the embedding width (emax + C)²C.
Depth
max
Mmmx
# Channels C
# Hidden Channels
# Params
12
4
2
32
441778
4
2
32
878434
12
4
2
10
32
1100746
4
2
12
32
1325714 14
12
4
2
16
32
1783618 8
12
4
2
20
32
2252146
4
2
32
32
3721474
Table 6: Scaling ladder for GemNer-OC.
Depth
Width
# Params
4
32
312992
48
575472
4
64
937280
80
1398416
4
96
1958880
112
2618672
4
128
3377792
144
4236240
4
160
5194016
176
6251120
4
192
7407552
204
8340060
Training Hardware. We trained all models across the different architectural families under iden- tical hardware conditions on NVIDIA 40GB A100 GPUs of the same compute cluster each run within single unit. This setup naturally incorporates overheads such as data loading, CPU bot- tlenecks, and metrics logging. During training, we continuously monitored validation losses along with the corresponding wall-clock training times. The reported training times thus include forward and backward passes over training samples, as well forward passes over validation samples during intermediate evaluation checkpoints. We chose batch sizes to strike balance between performance and computational efficiency, as using extremely small batch sizes impractical. Based on the empirical results in Figure 10, set the batch size to 128 for EGNN, and 64 for the remaining architectures These batch sizes were fixed across model sizes within each model family.
Scaling Ladder. Tables 3, 4, 5, 6 detail model sizes of all architectures used in this study.
Table 7: Scaling parameters for sum-power-lav in eq. (5) with 95% confidence intervals.
Architecture
log (A)
log
Unconstrained MPNN
1.356 [1.307-1.371]
2.194 [2.14722.608]
0.276 [0.266- 0.278]
0.311 [0.301-0.368]
EGNN
1.582 [1.494.1.692]
2.750 (2.660-2.863)
0.387 [0.370-0.408]
0.394 [0.382-0.401]
GemNer-OC
2.109 [1.901-2.422]
3,261 (2.842-3.607)
0.524 [0.484-0.584]
0.499 (0.444-0.544)
eSEN
3.760 [3.119-4.333]
5.129 [4.348-6.224]
0.817 [0.706-0,918]
0.753 (0.662-0.883
Foor fair comparison, douhle EGNN's reported training time.
22

---

Published as a conference paper at ICLR 2026
E UNCERTAINTY IN SCALING LAWS.
Due to the cost of training NNIPs, we perform scaling study within range of compute and model sizes and do not consider tuning other 95% confidence intervals on the fit parameters of eq. (4) and eq. (5) from 1000 non parammeetrii boot- straps. Table 7 demonstrates the values of parameters of eq. (5) along with confidence intervals shown in parenthesis.
such as weight decay. Thus, we construct
F EFFECT OF SCALING VECTOR CHANNELS
We evaluate the effect of multi-channel vectors in EGNN by plotting validation loss against the number vector channels E in fig. 11. Loss consistently decreases as E grows.
18
2=10"
10"
Vedtor Charmels
Val Locs
3x 10"
200M
400M
800M
300M
Traning
Figure 11: Effect of scaling number of equivariant channels in EGNN
G SYMMETRY ADVANTAGES ON DATASETS OF DIVERSE MOLECULAR TYPES
In this experiment, we investigate whether the advantages of symmetry diminish when models are trained on datasets containing more diverse molecular types, e.g., electrolytes, metal complexes, and biomolecules rather than only neutral species. We train an unconstrained MPNN and eSEN on the 4M split of the OpenMol dataset, which is sampled uniformly across the diverse molecular types mentioned above. For validation, we use held-out subset of 79K samples from the entire 4M split, using the remainder for training. As shown in fig. 12, observe that the benefits gained by symmetry-aware design still persist even with this highly diverse dataset. Specifically, the difference scaling exponents between the unconstrained MPNN and eSEN remains significant, a result sim- ilar to our findings for the neutral species split. It is important to note that the scaling exponents for each architecture itself may differ from those observed in the neutral split. This is expected because the coefficients of neural scaling laws also dependent the training dataset (Maloney et al., 2022; Bahri et al., 2024; Bordelon et al., 2024).
H EFFECT OF TEST-TIME AUGMENTATION IN SCALING LAWS
An unconstrained model de can be made equivariant at test time via group averaging (GA) with no additional training cost.
M
f0(x) = M Σ Ponlle Doo (p))
(23)
where 9i ~ Me is a sample from the Haar measure, Pin and Paur denote linear actions on input and output of Φo. respectively. In this section, explore the effect of GA on the performance of
23

---

Published as a conference paper at ICLR 2026
eSEN
Compute-optimal Laws
Unconstrained
100
6x10-1
Val Loss (log)
100
1x
1x101
eSEN ((C) - 0.49C-33
2x30-1
Uncenstrairec KINN
frontier
frontier
LIGI -LC)==0050 0.65C
10a
10a
10
10
10:
10"
10
10a
PFLOPs
PFLOPs
PFLOPs
Figure 12: Loss-compute Pareto frontiers for unconstrained models (Left) and eSEN (Middle) trained on OMol-4M split. Right: The compute-optimal power laws reveal that the benefits of symmetry persist at scale, even when models are trained datasets containing high diversity of molecular types.
unconstrained models at scale, especially when the model's parameter count and number of group elements increase.
Figure 13 shows that group averaging yields only a minimal improvement in the performance of de across all model sizes. Furthermore, this improvement saturates the number of rotations, M, increases beyond certain threshold. To better understand this behavior, we fit scaling laws with respect the number parameters N for both the baseline and the GA models:
L L- -
(24)
We use the parameter scaling relationship to analyze the impact of group averaging (GA). LD represents the lowest achievable loss for Φø at fixed data size D; eq. (7). For this analysis, we utilize models trained on the largest dataset, D Dmax se 9.2 x 10 (atoms). Based on the results in section 4, the data-limited loss is L.D R 1.6 x 10 x D performance at the onset of saturation, specifically at M = 32. The fit results in fig. 13 show that group averaging preserves the power-law exponent that governs the scaling of Φe. As the parameter count increases, the performance shows only a slight downward shift in the log-log scaling relationship, due to a minor change in the multiplying constant A.
0.223. We consider the GA
0.50
Baseine - -9
0.48
4x10
Averagig 23.99N-922
Val Loss
0.46
3-10"
2x10"
0.40
0338
2 2
2
2
2
2
10'
Number of Rotations
Number of Parameters (log)
Figure 13: Scaling analysis for group averaging (GA) at test time. Left: The benefit derived from increasing the number of rotations saturates beyond certain threshold. Right: Scaling performance with respect parameter count shows that utilizing group averaging results in a slight downward shift of the linear trend in log-log space, while the critical scaling exponent remains unchanged.
24

---

Published as a conference paper at ICLR 2026
1 DECOMPOSING ENERGY AND FORCE LOSS
Figure 14 shows the decomposition of the total loss in eq. (1) into its energy and force components. We observe that the energy learning curves are relatively noisy, whereas the force losses remain smooth across model sizes for each architecture.
EGNN
GemNet
eSEN
Unconstrained
Vill Enwry Lows (bg)
10
10"
10"
10'
Number of Cradiont Stage (log)
Number Gradient Steps (log)
Number Gradient Steps (log)
Number Gradient Steps (log]
EGNN
GemNet
eSEN
Unconstrained MPNN
Force (bgg)
T
t0"
10
10.
10"
10" Number of Gradient Steps (log)
10
10
10"
10"
10'
10"
10
10"
Number Gradient Steps (log)
Number of Gradient Stops (log)
Number of Gradient Stops (log)
Figure 14: Learning curves of all models, including Energy Loss (Top) and Force Loss (Bottom). Line color encodes model size (small, large)
J EFFECT OF TRANSLATION INVARIANCE
non-translation invariant translation invariant
2 10
100
Val
6x
100000
200000
400000
500000
Training Steps
Figure 15: MPNNs exhibit a performance drop when translation invariance is not maintained.
K TRAINING INSTABILITIES OF VANILLA TRANSFORMERS
We further push the limit of lacking inductive biases by training a vanilla transformer for force-field tasks. In particular, we train a GPT-style encoder which atomic coordinates are fed directly into the model by concatenating them with embeddings computed from atomic numbers. As result, E(3) equivariance is completely ignored in this setup. Moreover, because the transformer lears global attention over all atom nodes, the inductive bias of local neighborhoods. which is critical in NNIPs, is also abandoned. As shown in Figure 16, vanilla transformers fail to exhibit meaningful learning, as the learning curves saturate rapidly and remain unchanged for the remainder of training.
25

---

Published as a conference paper at ICLR 2026
2,500
N 30M
10M
2.475
2,450
2.425
Val
2.375
2.350
2.325
100000
200000
300000
400000
500000
Training Steps
Figure 16: Training instabilities of vanilla transformers force-field tasks.
L USAGE OF LARGE LANGUAGE MODELS
We used LLMSs assist with the writing of the paper (mainly for polishing) and also for coding.
26


Reviews:
[{"review_id": "JSspl5K6RO", "reviewer": "Reviewer_58Kv", "summary": "The authors study how machine learning interatomic potentials (MLIPs) scale with data, parameters, and compute, comparing architectures with increasing degrees of equivariance. They train on the OMol neutral-molecule subset, treat atoms as “tokens,” and adopt a single-epoch training regime to mirror LLM practice. They evaluate scaling against both FLOPs and GPU-hours, arguing this better reflects practical costs for (often less GPU-friendly) equivariant models. Empirically, they report clear power-law behavior and that scaling exponents grow with architectural equivariance, so performance gaps widen at larger scales; they also observe that compute-optimal training should scale model size and dataset size in tandem. A symmetry loss regularizer improves sample efficiency but does not match the benefits of fully equivariant architectures.", "questions": "1. Why restrict training to a single epoch?  Have you tried multi-epoch training (e.g. 10 or 80 epochs) on a smaller scale to see if exponents change?  How do you justify that one pass through data captures the scaling behavior of fully trained models?\n2. How do you expect inclusion of the charged/large molecules (omitted in the neutral split) to affect your conclusions?  Could the symmetry advantages reverse or diminish in chemically diverse subsets?\n3. How sensitive are your results to the learning-rate and batch-size choices? You tuned these at 1M parameters and then scaled. Is it possible that some models (e.g. the largest eSEN) were suboptimally trained? \n4. The introduction cites Sutton’s “bitter lesson” about scale overtaking bias. Given your modest scaling regime, how confident are you that “we should not leave symmetry to be discovered by the model”? Could larger-scale experiments eventually reverse the trend you observe?\n5. Nit: where is Figure 6?", "limitations": "", "rating": 2, "confidence": 5, "soundness": 1, "presentation": 3, "contribution": 2, "strengths": "1. The paper applies the modern neural-scaling methodology to neural interatomic potentials, an important area in AI-driven chemistry.  This fills a gap for the field.\n2. The paper is well written and easy to follow.", "weaknesses": "As the first paper (that I am aware of) to investigate the neural scaling law in the MLIP domain, I think the standard / stakes are high, and incorrect claims could lead to consequences in following model design. In this paper, the authors draw a lot of methodology directly from NLP, but I think these two fields have some fundamental differences. I found the following critical flaws in the three aspect of scaling law: Compute (1, 2, 3), Data (3, 4), and Parameters (5), and additional issues (6,7)\n1. Insufficient training / compute to claim asymptotic scaling: The scaling-law fits are based on small training budgets.  From Figure 1 and Section 4, the largest models run correspond to only \\~10^1 GPU-hours (per run) or \\~10^3 PFLOPs of compute.  This is tiny compared to typical scaling-law studies, which span orders-of-magnitude more compute (e.g. billions of tokens or thousands of TPU/GPU years). Even for MLIP training, this is too small: the eSEN small direct model is trained for around 10^5 PFLOP (80 Epochs) in OMol 4M, where the authors is training 10^3 PFLOPs on OMol 34M. That is **100x less FLOPs on 8x more data**. Moreover, each model is trained for only a single epoch through 34M samples (no repeated passes).  For neural potentials, one epoch training is extremely light: standard MLIP practice often trains tens of epochs to convergence.  The plots (e.g. Figure 1) do not show curves flattening -- the losses keep decreasing.  This suggests models are not converged even at the largest scale tried.  Fitting power laws in such a sub-convergent regime is problematic: the “effective” exponent can vary dramatically in early vs late training. In short, scaling laws are **asymptotic** statements, and here the training scale is too small to robustly infer asymptotic behavior.  This undermines the claim that explicit symmetry consistently improves scaling: with more compute, any performance gap might shrink or change.\n2. Single-epoch training regime. In Section 3.1, the authors intentionally use **only one epoch** (each sample seen once) to mirror LLM practice. However, this contradicts common MLIP training, where models typically see the data many times (e.g. \\~**80 epochs** in the OMol 4M baselines). Single-epoch training likely means none of the models fully fit the data distributions; indeed the validation losses are still dropping at the end of training. This choice can distort scaling-law estimates. For instance, early in training, increasing model size might seem more beneficial (higher \\alpha) simply because larger models learn faster per data pass, but with multi-epoch training a smaller model could catch up. By never allowing converged fits, the authors effectively inflate the “scaling advantage” of larger/equivariant models. They justify this by wanting to “avoid confounding effects”, but do not analyze how one vs multiple epochs alters the conclusions. Without at least one multi-epoch comparison, we cannot be sure the reported power-law behaviors would hold under standard MLIP training. This is a **major methodological gap**: the conclusions about scaling hinge on a very nonstandard (for this domain) training regime.\n3. Global target vs. token-wise analogy. The paper frequently draws analogies to language scaling laws, but **the supervised targets here are fundamentally different**. In language models, the loss (cross-entropy) is accumulated over **each token prediction**. Here, the primary target is the global energy of a molecule (plus per-atom forces, calculated by the gradient of the energy, thus is heavily correlated with the total energy). Each system yields **only one energy scalar despite many atoms**. The authors do treat atoms as “tokens” in counting dataset size, but this is a loose analogy: the model must capture a global property that depends on all atoms jointly. This could be a much harder task than NLP cross entropy target. Thus, as we discussed in weakness 2, seeing each atom once may not be analogous with LLM training, and the model requires much more training time to converge. As a result, statements like “we follow LLM scaling methodology” may be overreaching without addressing that molecular systems are global-structure tasks.\n4. Limited dataset regime (neutral split only). The OMol dataset is a high quality and diverse dataset, but all experiments use only the **neutral-molecule subset** of OMol (\\~34M samples), despite the full dataset being ~100M with diverse splits. This choice (claims to be taken due to memory limits, which if that's because of the system size, the authors could always decrease the batch size, since the batch size they chose was quite large (64) ) of using the neutral split, rather than **random sample**, could bias the domain. As described in the OMol paper, **non-equivariant models, such as GemNet, performs much better than equivariant models in EF error** when trained on all splits, such as biomolecules and electrolytes. This dataset selection clearly favors the equivariant baselines, and leads to a misleading or even incorrect results. I.e. the conclusions about “symmetry matters more at scale” could be dataset-specific. The authors should justify why neutral-only results would hold in the full OMol or other datasets (none of which is provided).\n5. Unfair comparison by raw parameter count. In NLP, comparing models by parameter count is largely meaningful because mainstream LLMs use a homogeneous Transformer architecture: blocks are architecturally identical, most weights live in token-wise MLPs and attention projections, and the compute per parameter and weight reuse pattern are effectively uniform across models. As a result, plotting loss vs. parameters is reasonably apples-to-apples. In contrast, **for GNN-based MLIPs the notion of “one parameter” does not carry across architectures**: (i) parameters are shared and reused across all nodes/edges; (ii) different designs incur very different work per parameter (e.g., dihedral terms in GemNet-OC, high-order tensor ops in eSEN); (iii) body order and tensor order change effective capacity and FLOP intensity; and (iv) throughput at the same N can differ drastically. This is largely why the kappa number varies between architectures. Consequently, the Fig. 5 style plots of loss vs. N conflate capacity, compute, and inductive bias, and can be misleading. A fair comparison should be iso-compute (FLOPs or GPU-hours), or at least normalized by an architecture-dependent cost.\n6. Exponent reliability and fitting issues. The fitted power-law exponents (α, β, γ) are central to the paper’s claims, but their reliability is questionable. First, as noted, the data range is relatively narrow (datasets from 10%-100%, models \\~10^6-10^7 parameters): typically one requires several orders of magnitude variation to robustly determine an exponent.  Here both N and D spans are \\~1-2 orders of magnitude at best. Second, the assumption $L_\\inf \\approx 0$ (irreducible loss) is ad hoc and may bias exponents high.  In real molecular data there should be some finite noise/error floor (analogous to the entropy in NLP); assuming zero means the model is “expected” to achieve perfect prediction eventually.  Small positive L_\\inf can drastically change a fitted β or γ in a power law fit (as studied in Hoffmann et al. 2022).  Third, Figure captions reveal instability: GemNet-OC needed smoothed loss, and the authors exclude the first 1–10% of training from fits.  These steps suggest the raw curves were not clean power laws. With such variance, the numeric exponents conflict across FLOPs vs wall-time (Fig.1).  Without error bars or fit-statistics on α,β (only γ had CIs in Table 1), it’s hard to trust these values.  In summary, the power-law behavior is claimed too strongly: given the limited and noisy regime, the reported exponents may not reflect true asymptotic trends. The results could be artifacts of the experimental choices (one-shot training, hyperparams, smoothing).\n7. Additional concerns: (i) The study considers only four architectures; it is not clear if these are representative. For example, EGNN and GemNet are relatively simple and old models -- how would a more modern one (PaiNN, DPA-1/2, EScAIP, even NequIP) behave? (ii) The paper also mixes different body-order and tensor-order notions without clarity. (iii) Hyperparameters (depth, widths) are tuned only at ~1M params and then scaled; it’s possible that later models were not fully optimized. (iv) The authors do not report if they repeated experiments to measure variability (confidence intervals are absent for N,D scaling). (v) Finally, the broad claim that one should “not leave symmetry to be discovered by scaling” may overstate the results. The bitter-lesson cited in the intro suggests that even biased models can eventually be outperformed by larger unconstrained ones; these experiments do not go far enough to test that (no extremely large unconstrained model was trained). In its current form, the paper’s strong conclusion about avoiding bitter lesson is not fully justified by the limited empirical data.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "OQ6cMgb2pZ", "reviewer": "Reviewer_AQ3A", "summary": "This work introduces large-scale experimental results illustrating scaling laws for validation loss across equivariant message passing architectures and for NNIP tasks. As a result, several practical observations are made, including optimal architecture choices based on available compute.", "questions": "**Q1. Generalization.** The claim on line 80\n\n> While our study is limited in scope to Special Euclidean symmetry of neural interatomic potentials and force fields, as well as a few representative architectures, there is no reason to believe the results should remain confined to this particular domain and the choice of efficient equivariant models.\n\nappears at odds with the line 843 on related works stating\n\n>Despite using the eSEN same backbone, Wood et al. (2025) report that, for dense models 5, the compute-optimal strategy scales model size N faster than data size D, whereas in our setting we observe nearly equal scaling between N and D; though the tasks are different\n\nI suggest that the authors reconsider the claims of generalization of these scaling curves beyond the scope of the present work.\n\n**Q2. Undefined acronyms.** Please be sure to introduce acronyms like NNIPs that may be familiar to those working on these specific tasks, but not to a more general audience.\n\n**Q3. Energy vs. direct-force training.** Can the authors please substantiate the claim in line 138\n\n> While it is sufficient to learn the energy for predicting conservative forces, direct force prediction is significantly more scalable.\n\n**Q4. Transformer instability.** Can the authors provide evidence for the line 149 where they observed\n\n> instability issues when scaling vanilla transformers for this task, we focused on message-passing architectures.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "The paper tackles an important and timely problem, studying the effect of higher-order equivariant features on accuracy/compute tradeoffs. It presents large-scale experiments and reports empirical scaling behavior with clear ablations. The approaches used in the study are well motivated and clearly written.", "weaknesses": "Some claims appear stronger than what the experiments directly support.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "tX6cPH5frX", "reviewer": "Reviewer_SZDj", "summary": "The paper presents an empirical study on the scalability of equivariant models vs non-equivariant models (w.r.t.  SO(3) symmetry group) for the molecular force fields task (predicting energy and forces of molecules). More specifically, the authors show how models like GemNet, eSEN, and EGNN tend to scale better than unconstrained GNN architectures on the OpenMol dataset. They study the scalability of these models across multiple access, including: model parameters, PFLOPs, and GPU hours, against validation loss of the molecular force fields task.", "questions": "See above.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "* I think the problem being studied is interesting and quite relevant to the current ongoing discussion on equivariant vs non-equivariant models design, in the area of geometric deep learning.\n* Interesting to see that different equivariant models have different scalability behavior, for example, eSEN has lower validation loss with more GPU hours compared to the GemNet architecture.", "weaknesses": "* I think the evidence is not sufficient to support the claims; some rewrites might be required to not make it a general claim or show more results on other tasks. \n* The study is limited to a single-epoch training regime, and studying the GPU hours in the range of less than 100 hours is limited. It would be beneficial to see how this could be extended for longer training times, and if the same trend holds or not. \n* Also, comparing recent models like eSEN to vanilla GNN might limit the claims of the paper, as GNN/ MPNN is an old baseline now. More unconstrianed architectures should be included (e.g., how this is applied to Graph Transformers).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "oiGIywyvFq", "reviewer": "Reviewer_RXkz", "summary": "This paper presents an empirical investigation of scaling laws for neural force fields used in learning interatomic potentials. The authors systematically study how equivariant and non-equivariant architectures scale with respect to data size, model parameters, and compute budget. The key finding is that equivariant architectures exhibit superior scaling exponents compared to non-equivariant models, with higher-order equivariant representations showing the best scaling behavior. The study provides power-law fits demonstrating \"architecture-dependent exponents\" and suggests that compute-optimal training requires scaling data and model size together. The authors argue that fundamental inductive biases like symmetry should be built into architectures rather than left for models to discover, especially at scale.", "questions": "- How do the resulting models compare to state-of-the-art models on the OpenMol dataset? That would be good to know how well the results represent real-life results on such datasets.\n- Have the authors considered taking a model like eSEN and manipulating just its equivariant convolution layers to be unconstrained? That would remove some of the impact of architectural design on the comparison.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- Rigorous empirical analysis: The paper provides tight power-law fits across the tested regime, demonstrating clear architecture-dependent scaling exponents with convincing statistical evidence.\n- Clear presentation of core results: The main findings about equivariant architectures exhibiting better scaling behavior than non-equivariant models are presented clearly.\n- Practical insights: The finding that data and model size should scale in tandem for compute-optimal training is actionable regardless of architecture choice.\n- Important research question: Investigating whether fundamental inductive biases matter at scale is highly relevant to the broader machine learning community.", "weaknesses": "- Severely limited training regime: Training on only a single epoch is a poor experimental choice that prevents data augmentation and fails to test the most interesting hypothesis from Brehmer 2025—that equivariance benefits may disappear with sufficient augmentation over multiple epochs. While the authors may draw inspiration from language model training, molecular datasets are orders of magnitude smaller, making this analogy weak.\n- Insufficient scale: The maximum compute budget of ~30 GPU hours is quite small for claiming insights about \"large-scale\" behavior on molecular datasets. True scaling studies typically involve orders of magnitude more compute.\n- Limited symmetry analysis: What the paper calls \"unconstrained MLP\" is actually translation-invariant (operating on relative positions). The study only assesses rotational equivariance on top of translational invariance, not the full contribution of geometric symmetries. This should be explicitly clarified as it affects the interpretation of results.\n- Unexplained contradictions with recent work: The conclusions don't adequately address the success of models like Orb, which achieve near state-of-the-art performance on MatBench Discovery despite being only translational equivariant (like the unconstraint MPNN in this paper). This weakens the paper's strong claims about the necessity of equivariance at scale. Could it be that the non-rotationally-equivariant model chosen by the authors is a worse choice than Orb?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)