Learning 3D Anisotropic Noise Distributions Improves Molecular Force Fields
Authors: Xixian Liu
Rui Jiao
Zhiyuan Liu
Yurou Liu
Yang Liu
Ziheng Lu
Wenbing Huang
Yang Zhang
Yixin Cao
Venue: NeurIPS 2025 poster ()
Keywords: 3D molecular pre-training
molecular force field
coordinate denoising
equivariant neural networks
Primary area: machine_learning_for_sciences

Abstract: Coordinate denoising has emerged as a promising method for 3D molecular pretraining due to its theoretical connection to learning molecular force field. However, existing denoising methods rely on oversimplied molecular dynamics that assume atomic motions to be isotropic and homoscedastic. To address these limitations, we propose a novel denoising framework AniDS: Anisotropic Variational Autoencoder for 3D Molecular Denoising. AniDS introduces a structure-aware anisotropic noise generator that can produce atom-specific, full covariance matrices for Gaussian noise distributions to better reflect directional and structural variability in molecular systems. These covariances are derived from pairwise atomic interactions as anisotropic corrections to an isotropic base. Our design ensures that the resulting covariance matrices are symmetric, positive semi-definite, and SO(3)-equivariant, while providing greater capacity to model complex molecular dynamics. Extensive experiments show that AniDS outperforms prior isotropic and homoscedastic denoising models and other leading methods on the MD17 and OC22 benchmarks, achieving average relative improvements of 8.9% and 6.2% in force prediction accuracy. Our case study on a crystal and molecule structure shows that AniDS adaptively suppresses noise along the bonding direction, consistent with physicochemical principles. Our code is available at https://github.com/ZeroKnighting/AniDS.

Learning 3D Anisotropic Noise Distributions Improves Molecular Force Field Modeling
Xixian Liu';
Rui Jiao"; Zhiyuan Liu Yurou Liu', Yang Liu",
Ziheng Lu,, Wenbing Huang", Yang Zhang Yixin Cao
Fudan University, PTsinghua University, "National University of Singapore *Renmin University of China, 5 Microsoof Research
Institute of Trustworthy Embodied AI, Fudan University xixian. liu@mila..... queebme,,.uuuee,c jiaor21 Amails.... tsinghua. edu. cn zhi yuanenu. edu. sg, hwenbingeruc edu.
Abstract
Coordinate denoising has emerged as promising method for 3D molecular pre- training due its theoretical connection to learning a molecular force field. How- ever, existing denoising methods rely on oversimplified molecular dynamics that assume atomic motions be isotropic and homoscedastic. To address these limita- tions, we propose a novel denoising framework AniDS: Anisotropic Variational Autoencoder for 3D Molecular Denoising. AniDS introduces structure-awes anisotropic noise generator that can produce atom-specific, full covariance matrices for Gaussian noise distributions better reflect directional and structural variabil- ity in molecular systems. These covariances are derived from pairwise atomic interactions as anisotropic corrections to an isotropic base. Our design ensures that the resulting covariance matrices are symmetric, positive semi-definite, and SO(3)-equivariant while providing greater capacity to model complex molecular dynamics. Extensive experiments show that AniDS outperforms prior isotropic and homoscedastic denoising models and other leading methods on the MD17 and OC22 benchmarks, achieving average relative improvements of 8.9 % and 6.2% in force prediction accuracy. Our case study on a crystal and molecule structure shows that AniDS adaptively suppresses noise along the bonding di- rection, consistent with physicochemical principles. Our code is available at https://githube www www./eeowoiggtttt...oo AniDS
Introduction
Accurately and efficiently predicting atomic properties of molecules and materials is fundamental to a wide range of downstream applications, including drug discovery [ 2]. material design (
catalyst design 4 [.), and molecular dynamics simulations K 7 Ab initio methods such Density Functional Theory (DFT) [ are widely regarded as the standard for atomic property prediction, but their high computational cost significantly limits scalability to large systems or datasets. To overcome
this challenge, machine learning potentials 9 have been developed to accelerate atomic simulations by orders of magnitude, with recent advances of graph neural networks (GNNs) TO
| further
improving prediction accuracy. Among these methods, coordinate denoising [3 has emerged as a promising training objective, due its theoretical grounding in learning molecular force fields.
Classical coordinate denoising the perturbed version X + e, where the noise ~ (0 TT) is drawn from isotropic Gaussian. This objective corresponds to learning the molecular force fields when the data distribution p(X)
12 involves recovering the original atomic coordinates from
"Equal contribution. Correspondence.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

(a) Classical Denaising
(b) DenoiseVAE
(c) AniDS (Ours)
Fixed Noise
Isotropic Generator
Anisotropic Noise Generator
Σ; = an
Σ, azl
Σ, a
E
a,70
[ry]
Fi
|rull
Fgy
JEM(() Anisotropic
Anisotropic
Anisotropic
Ieput Structure
Hateroscedastic
Heteroscedostic
Heterossedestic
Figure 1: Comparison between different denoising approaches. The opaque spheres represent noise distributions. Our approach captures noise distribution that is both anisotropic and heteroscedastic.
approximated mixture of isotropic Gaussians centered each data point |B i.e., p(X) PC
WX o A closer examination of this formulation reveals two implicit assumptions: (1) atomic motions assumed to be isotropic, with identical variance along all axes (o = oÿ of), ignoring direction-dependeet stiffness such as bond stretching [I]] and (2) atomic motions are assumed to be homoscedastic [15]. with all atoms sharing the same noise scale o², ignoring variability in energy potentials across distinct structures. While these assumptions make the implementation easier, they oversimplify the molecular dynamics, potentially leading to inaccurate force field learning.
Relaxing these assumptions demands a new noise distribution that is theoretically consistent with the assumed atomic motion. Denoise [ learns adaptive noise scale o² for different atoms, lifting the assumption of homoscedasticity. although the isotropic assumption remains less explored. On the other hand, earlier works [177 [8] move beyond isotropic Gaussian noises by introducing handcrafted noises, e.g. perturbing bonds and dihedrals. However, these noises are applied independently at fixed scales, thus failing address the assumption of homoscedasticity.
In this work, we propose a novel denoising framework: Anisotropic Variational Autoencoder for 3D Molecular Denoising (AniDS), aiming improve molecular force field learning by adaptively generating anisotropic Gaussian noises for coordinate denoising, thereby lifting all three assumptions discussed above. AniDS achieves this by adaptively generating "full covariance matrix" (i.e., without the diagonal constraint) for each atom's 's Gaussian noise distribution, enabling it to capture directional variability and different molecular structures. However, generating such full covariance matrix introduces two major challenges that have hindered prior efforts: (1) Ensuring the covariance matrices are symmetric, positive semi-definite, and equivariant, which are fundamental properies of covariance matrices and molecules. While these properties are trivially satisfied in isotropic settings, they require explieit modeling and constraint in the anisotropic case. (2) Guiding the covariance learning process with proper physicochemical priors. The space of full covariance matrices is high- dimensional and susceptible to trivial or degenerated solutions if not properly regularized, which often leads to unstable or failed pretraining.
To address these challenges, AniDS introduces structure-aware anisotropic noise generator that leverages 3D molecular structures to produce aton-specific full covariance matrices for anisotropic noise distributions. Specifically, the generator first constructs isotropic base term and then applies anisotropic corrections derived from the learned pairwise atomic interactions. These anisotropic cor- rections enable the model capture local rigidity and directional variability across different molecular structures, acting as a physicochemical prior. Additionally, our design ensures the covariance matrices symmetric, positive semi-defnnite and -quii--quiyaariant Equipped with this expressive noise generator, AniDS supervises molecular denoising autoencoder using our theoretically grounded denoising loss that approximates learning the molecular force field.
AniDS achieves an average relative improvement of 8.9% on MD17 and 6.2% on OC22 in force pre- diction. Ablation studies confirm the effectiveness of each component. Our case study further shows that AniDS adaptively adjusts the direction and scale of the noise according to the atomic interactions' strength in 2 OugPd12 and struuuss,s aligning with physicochemical principles.
2 Preliminary: Coordinate Denoising and Denoise VAE
We begin by introducing the coordinate denoising VAE []6] that addresses the homoscedastic assumption.
1K objective and the subsequent study Denoise-

---

Notation of Molecules. In this work, develop a framework to learn force fields for both small molecules and crystal materials, For clarity, we present the methodology using the notation of small molecules; the extension crystals straightforward and involves revising the denoising autoencoder to reflect crystals' periodicity. Details of this adaptation are provided in Appendix B
A 3D molecule M (Z, E) is represented by: (I) Z E N with Z, E N denotes the atomic number of the i-th atom: (2) X E RNx3 where X, E R specifies the 3D coordinates the i-th atom; and (3) E Nxxxd where Ey E R denotes the bond existence and bond type between atoms and j. Here, N denotes the number of atoms in the molecule.
Coordinate Denoising. Given a molecule M in equilibrium, in which the atom-wise forces are near zero, a corrupted version is generated as M = (2, X, O)) where the atomic coordinates are perturbed by Gaussian noise: X = X + ae with
(0, I). Here is hyperparameter controlling the added noise scale, which is usually around 0.1 to generate a small perturbation. Then, a denoising autoencoder 21
(-) is trained to predict the added noise by minimizing the following loss:
MM( (M) (X X| Theoretically, this denoising objective is shown approximate learning molecular force field therefore improving its performance.
Denoise [6]. The standard coordinate denoising operates under the homoscedasticity assumption, treating all atoms equally. Specifically, a fixed noise scale a is used to simulate minor thermal fluctuations for all atoms, but fails to capture the energy variations across different molecular structures. For example, in a rigid structure like benzene ring, minor coordinate perturbation can lead to a disproportionately large energy change, whereas similar perturbations in more flexible regions may have negligible energetic impact. Resolving this issue, Denoise VAE trains noise gener- ator (-) to adaptively assign a distinct noise scale o; to each atom based on the molecular structure:
fo, E R lie M] (M).
(1)
X, = X + o,E,
E -=,
(2)
We then obtain the perturbed molecule M (Z, X, L), and perform denoise training as follows:
(X X)
Deenosee
M
IE
x(xx)
Σ
of
iEM
CKL
1
Σ
DEL (N o.)))
M
iEM
where p; is the prior distribution, set to oZ) and op is hyperparameter. The KL divergence term LKI. acts regularizer prevent the noise generator vb(-) from collapsing trivial solutions where a; + 0. This ensures meaningful denoise training. The final training objective combines both losses VAn Denoise Denoise AKL with balancing coefficients Denoise and ARL-
3 Methodology
In this section, we start by overviewing the AniDS framework. We then present the motivations and detailed formulations of its key components in Sections 522 and Section 3. respectively. Finally, describe how AniDS can be adapted to different training schemes in Section 3.4
3.1 Overview of the AniDS Framework
AniDS consists of three key components: (1) a structure- anisotropic noise generator v-(-) that generates per-atom noise distribution conditioned on the molecule structure M; (2) denoising autoencoder <6-) that is trained for coordinate denoising; and (3) denoising objective LAuDDS that approximates learning molecular force field.
Structure-aware Anisotropic Noise Generator. Given a molecule M. AniDS employs a noise generator (+)) to generate full covariance matrix Σ, E 3 for each atom E M, defining an anisotropic Gaussian noise distribution (0, E,). Unlike prior works that assume diagonal E; for theoretical and computational convenience 22 [3 AniDS models Σ, as a dense matrix to capture directional variability. To ensure that the generated noise reflects the underlying molecular structure, ()()) should be implemented with structure-aware molecular encoders 23 24]. This enables the generated covariance matrices to reflect local and global molecular structural contexts. Given this generator (-),, the perturbed molecule M can be obtained as:

---

Structure oncre
setropic Naiee
Sampling
berciiing Autsencoder
SAmiDS
AAniDSWAB
Input Structure M (c) The Workflow AnIDS
Coveriance Matrixt,
Perturbred Siructune
Predicted Soore
Pre Structure Genercter !
benelsing Autoencoder $
Node: wise Head
Esptropic base tern
Scere
Molezale Enester
Edge-wite
Weight R,
Anisotropic Factor Vu
Inil
T-il
Nodevwiis Head
Reguleter
Fine fune
Strutters-warrs Artootrspic Notse Geraraton +
Deneising Autoencoder
Score
(Ferce)
D, a
octu
Fiy
Tiy Igg
(c) Pre troining Auxilary Fine tuning
(Shrae0
2f Neighborr))
Anisstropic Noise Generater
Autoencader +
Energy
(Ferce) Gaasrstor %
Frozen Parumetiers
+ Parameters
Optional Irput/Ourput
(4) Ausiliory Triining from Scretch
(5tner))
Figure 2: Overview of the AniDS framework.
[E E R3x3 M) (M),
(5)
L Choleskyy (E,);
X; = X; ~
(7)
M = M X, E).
(8)
where X; is the perturbed coordinate of the i-th atom, sampled via Cholesky decomposition 25 of the learned covariance Σ. While the noise genrator can be any neural network capable of producing symmetric, positive semi-definite, and 3D-equivariant covariance matrices, we introduce
our implementation in Section 3.2
Denoising Autoencoder. The denoising autoencoder & takes the perturbed structure M as input and predicts atom-wiss vectors intended to recover the original coordinates: ((M) R Nx3 AniDS is model-agnostic framework that can be integrated into a wide range molecular backbones. In this work, we implement á with EquiformerV? V2 [24] and Geoformer 126], which have demonstrated strong performance on molecular benchmarks. To enable coordinate denoising on non-equilibrium structures, we follow 27 to include force encoding as the additional features for the corrupted atoms. Unlike equilibrium structures, which are uniquely defined by local energy minima, multiple non-equílibrium structures may share the same energy level, introducing ambiguity into the denoising targets. Including force information helps disambiguate these cases by anchoring the target specific molecular state. Our detailed implementations of the denoising autoencoders and the force encoding are introduced in Appendix [..T
AniDS's Denoising Objective. AniDS's denoising loss jointly trains the noise generator and the denoising autoencoder recover the noise vector scaled by the inverse covariance:
Σ ( - E x,)
LAMDS
M (X.x)
FEM
We show in Section 3.3 that the objective above approximates learning the molecular force field. Additionally, we compute KI. divergence loss between the learned anisotropic noise distribution and an isotropic Gaussian prior o(
1
CKL 2
Σ iEM
tx(o,",,)
(10)
Σ,
which prevents the learned covariance from collapsing into trivial solutions like zeros. Recall that the normalized anisotropic weights {ny} }o  on Eq. [5 satisfy Σ, < degenerate covariances that become overly isotropic in flexible regions, we regularize the total anisotropic mass I; ; The towards a target level e (0, 1) via one-sided hinge-squared penalty:
by construction. To avoid
C,
Σ M
[max(() (--x--)) -.)]
(11)
RE.M
Intuitively, Eq. discourages vanishing anisotropic corrections (i.e., I, too small), while preserving the PSD guarantee I, < 1 from [S The final loss LANDSVVE is a weighted sum of the denoising and regularization terms:
L LAUIDS-VA = AADDDS CAuDD + AKLCKL + NyG-
(12)

---

3.2
Structure-Aware Anistropic Noise Generation
The noise generator ())) constructs symmetric, positive semi-definite, and -quii-equvvariant covari- ance matrices {E E MJ. These matrices encode both isotropic and anisotropic uncertainty in atomic positions, reflecting each atom's 's structural context. The generation consists of main steps:
Atom-wise Structural Representation. A molecular structure encoder p(.) is used to extract struc- tural representations, to guide the covariance generation process. This work shares the architecture as the denoising autoencoder. Formally, we have:
(h; E R M) p(M),
(13)
where h; encodes the structural and chemical contexts atom
Covariance Matrix. Given the structural representation above, we parameterize the covariance matrix Σ, for atom the sum of a isotropic base term and an anisotropic term, motivated by our physicochemical prior of pairwise atomic interactions:
Fúj Fig
Fúj
Σ
a,I
agY
(14)
ríj
Isotropic Base
jeiighoorr
Anisatropic Corrections
exp(bis)
Mi Σ EleNcighbor(s) (xp(ba) +
(Normalized anisotropic weight),
(15)
where Faj X; - X, R is the relative position vector. The parameters a,, byj. and o are adaptively derived from the atomic structual representations {h.li E M) using MLPs (i.e., wo, Wi, and wy) and radial basis function
v7()), respectively:
Isotropic base term a; = exp (wo (h:)) E R It controls the baseline noise level for atom i, reflecting its intrinsic flexibility. Larger a, values indicate greater isotropic flexibility.
Anisotropic weight bij wi ([h;; hy; (((r,)))) corrections, which determines the directional stiffness between atoms and j. For example, along covalent bonds, high by produces high Kr that suppresses motion along rij. Conversely, bij < for distant atom pairs eliminates the corresponding directional influence of stiffness.
R. It controls the weights of anisotropic
Anisotropic regulator c; exp (b:)) RR+, regulates the influence of anisotropic cor- rections. In symmetric or rigid environments, such as aromatic rings, larger e; values suppress individual directional components, resulting in a more isotropic noise distribution. contrast, smaller allows stronger anisotropic corrections to reflect local uncertainty in flexible regions.
Our noise generator is designed ensure three essential properties of the covariance matrix:
Symmetry: Σ; Σ is satisfied due to the symmetry of the isotropic base and the outer product.
Positive Semi-Definiteness: The covariance matrix is positive definite (E, > 0) if E, Tis <
enforeed via the softmax in Equation [S]. This condition guarantees the anisotropic correction term does not overpower the isotropic base I.I. A detailed proof is in Appendix B
Equivaique and Inv)-invaeeeece: Given SE(3) transformation = (R., t) of rotation R E SO(3) and translation t E I applied on the molecule M, the transformed covariance matrix satisfies g Σ, = RE,R T. the covariance matrix is equivariant to the rotation but invariant the translation. This is because the translation is canceled out when computing rij X; X,.
3.3 AniDS Learns Molecular Force Fields
Here, we show that AniDS's denoising objective approximates learning molecular force field.
Step 1: Boltzmann Distribution and Gaussian Mixture Approximation. The distribution of
E(X) molecular structures follows the Boltzmann distribution: Pphysical (X) x exp(- R _(X)), ). where EX)
we approximate Pphysical as mixture of Gaussians centered at the known structures, typically selected as the equilibrium conformations [ Subsequent study [17 demonstrates that learning this Gaussian mixture over non-equilibrium struc- tures is theoretically equivalent to modeling a hybrid noise distribution over equilibrium ones (Sec
the potential energy. Following prior works [ 29
5

---

Proposition 3.4 in )7 In their work, non-equilibrium structures generated by applying small tor- sional perturbations to equilibrium conformations. Further studies show that intermediate states from molecular dynamics simulations can also serve as effective non-equilibrium samples for denoising
22 27 which are also adopted in our work. Specfically, we have:
Pphysical X) qr
Σ (x())) D
(16)
where ((k is the k-th structure in dataset D. We define q: (x:(X I WX x
(8)
Here (k) E Es denotes the coordinate of the i-th atom in the k-th structure, and ب E R2x3 is
the covariance matrix of the i-th atom in the k-th structure.
Step 2: Score Function of the Gaussian Mixture. The score (gradient of the log-density is:
Σ - 98
[XX (b)
V X lo (X)X(1))
V log (X)
(17)
ΣP qE (a)
Step 3: Link to Molecular Force Field. Under the Boltzmann distribution, the force field is proportional the score:
F F(X) = - = KBT Vx log Pphysical X):
(18)
For small perturbation (X ~ X ) ) the Gaussian mixture is dominated by the nearest structure X X
simplifying the score (ef. Equation 17 to:
N
Vx log t X
log q3 (x X
=
-
(19)
/=1
Substituting this into the force field expression (ef. Equation (8) ) we identify:
N F(X) x > (k) (X.
(20)
Step 4: Denoising as Learning Force Field. AniDS trains denoise autoencoder ((M) predict the noise scaled by the inverse covariance (ef. Equation ( By Vincent's theorem 30 this is equivalent to score matching:
AADDS o E qu (x)] (M) V log QE (E(())))))) (X)
(21)
At convergence, o² (M) vx log qz 6" (M) x -X)). Prior works of Coordinate Denoising [27] and Ven [6 can be seen as special cases of our approach, with detailed derivations provided in Appendix B
Vx log Pphysical ()), recovering the force field:
3.4 Adapting AniDS Different Training Schemes
Consistent with prior works, we adopt AniDS under two training schemes: (1) as a pretraining objective followed by task-specific fine-tuning [ 31] and (2) as an auxiliary task jointly optimized
with supervised force field learning when training from scratch 27 12]
Bl we can apply AniDS (cf. Equation [2] to pretrain the denoising autoencoder é and the structure-aware noise generator vi altogether large pretraining dataset. The pretrained encoder øi is then fine-tuned on downstream datasets for supervised force field learning, with the AniDS objective retained the following paragraph. During fine-tuning, the parameters of the noise generator yi are kept frozen.
Pre-training and Fine-tuning, Following
auxiliary loss, as described in
Supervised Learning with Partial Corruption and Auxiliary Denoising. We use AniDS as an auxiliary loss for supervised force field learning. Inspired by (27]. we adopt a partially corrupted denoising strategy. Specifically, only subset of molecule's atom coordinates is corrupted using the noise generator. The model is trained with the weighted sum of supervised force field loss on the uncorrupted atoms and AniDS loss (cf. Equation (2) on the corrupted atoms. This design ensures that the model learns from clean ground truth coordinates for supervised force field learning, while still benefiting from denoising regularization. Compared corrupting all the atoms, this method mitigates the mismatch between the the ground truth force field label and the perturbed structure. Implementation details are provided in Appendix B.3

---

Table I: MAE for force prerdiction on MD17's test sets. Forces are reported in units of kcal/mol.
Bold numbers indicate the best performance. Green denotes relative improvement to the baseline.
Model
Aspárin
Benzeme
Ethanol
Malunaldehylle Naphthalene Salicylic Acid Toluene
Uracil
Avg
SchNet
1.35
0.31
0.39
0.66
0.58
0.85
0.57
0.56
0.66
DimeNet
0.499
0.187
0.230
0.383
0.215
0.374
0.300
PaiNN
0.319
ReeT
501
0.253
0.196
0. 1.109
0.169
0.061
0.129
0.067
0.095
0.135
NequiP (mmax
10
0.184
0.071
0.129
0.099
0.088
0.046
0.076
SRG)-DDM
0.288
0.129
Coond [8]
0.211
0.169
0.095
0.139
0.083
0.109
0.058
0.074
0.114
Hod [
0.209
0.199
0.001
0.1415
0053
108
0.054
0.076
0.116
Slide
18
0.174
0.169
0,088
0.153
0.48
100
0.014
DeNS
(L
0.131
0.141
0.060
0.101
0.039
0.085
0.044
0.076
0.085
DeNS
27
(Lmax=33
0.120
0.141
0.055
0.095
0.037
0.074
0.042
0.067
0.079
(Lmax-2)
10:05
0.139
11435
19:1%
0.084 111
0.036 9.07%
110 055
17:00
Table 2: Performance comparison on OC22's S2EF-Total validation set.
Model
Params
Energy E-MAE (meV)
Force F-MAE (mÀ)
OOD
Avg
OOD
Avg
Gem-et-OC
22
39M
545
1011
778
30.0
40.0
35.0
GemNet-OC (0C20+0C22)
39M
464
859
661.5
27.0
34.0
30.5
E2former
4
67M
491
724
607.5
25,98
36.45
31.22
Equiformer V2
A
122M
433.0
629.0
531
22.88
30.70
26.79
EquiformerV2 V2 + DeNS
27
127M
391.6
533.0
462.3
20.66
27.11
23.89
EquifonnerV2 AmiDS (ours)
129M
370.0
525.4
447.7
19.53
25.27
22.4
4
Experiments
Datasets. We briefly describe the datasets in our experiments, and leave additional setup details in Our experiments involve four datasets: (1) PCQM4M22 R2 contains 3,746,619
Appendix C.2
molecules along with their Density Functional Theory (DFT) calculated 3D equilibrium structures.
(2) OC22 33 large-scale dataset of DFT calculated structures designed advance machine
learning for oxide electrocatalysts. (3) MD17 344 provides molecular dynamics trajectories of small organic molecules, with both energy and force labels. (4) MPtrj [35] includes 1.58 million structures obtained from DFT relaxation trajectories of over 146,000 materials in the Materials Project [36]
4.1 Results of pre-training and Fine-tuning
Setup. We pre-train our model on PCQM4Mv2 and subsequently fine-tune it exclusively on MD17. We adopt the Equiformer-V V2 backbone 244 Note, we use only the 3D structures without the property values PCQM4Mv2. During the fine-tuning, given that the pre-trained AniDS has already learned decent distribution for adding noise, we freeze the noise generator and only train the denoising autoencoder, using AniDS as an auxiliary task along with supervised learning. Following 277 we use 950 molecules for training and 50 for testing. No noise is added during validation and testing.
MD17. Table D presents the results on MD17. Our method achieves the best across all evaluated tasks. Compared to the previous results, we observe an average improvement of approximately 8.9 % across the tasks. We attribute this significant improvement our structure-aware noise generator, which produces anisotropic noise, enabling our model to learn the molecular potential energy surface more effectively. Furthermore, while DeNS (Lmax 3) enhances input features by increasing the degree of irreducible representations in EquiformerV2, AniDS (Lmez = 2) outperforms it while using a lower dimensional irreducible representation. This demonstrates the efficacy of our adaptive structure-drive noise sampling strategy, suggesting that well-informed noise learning can lead to more generalizable force field prediction.
4.2 Results of Supervised Learning with Partial Corruption and Auxiliary Denoising
Setup. Here AniDS is used an auxiliary task when conducting supervised learning on OC22. Due to resource constraints, we adopt the pre-trained models from 27 This pre-trained model utilizes the EquiformerV2 architecture, and its training process incorporated DeNS as an auxiliary task. Given its architectural similarity to the model used for MD17, we can easily extend its training by integrating our AniDS framework. Following (27]. we conduct training on OC22's 2ss S2EF-Total task, which contains 8.2 million structures. We further evaluate performance on the provided validation split K3 Additionally, our results on MPTrj provided in Appendix C
7

---

Table 3: Ablation Studies on the Aspirin dataset of MD17. (a) Comparison of different denoising methods under the me-train-fine-tune scheme. (b) Comparison of different denoising methods under the pre-train-fine-tune fune-tune scheme. Standard supervised fine-tuning is used without AniDS as an auxiliary task. (c) Hyperparameter analysis under the pre-train and fine-tune scheme.
(a) Pre-train and Fine-tune
(b) w/o Auxiliary Fime-tuning Model
(c) Hyperparameters
Model
F(MAE)
F(MAE)
Eq
F(MAE)
Op = 0.5
0.13018
Eq pre-train
0.1929
Eq pre-train
0.1929
Op 0.1
0.11180
BqV2-DeNS
0.1178
EqV2-DeeNS
0.1424
Op = 0.05
0.11594
Bq VeE
0.1143
0.1400
AKL 1.5
0.11714
Eq add
0,1128
Eq add
0,1389
AKL 1.0
0.11180
EqV2-AniDS
0.1118
EqV2-AniDS
0.1369
AKL = 0.5
0.11461
OC22. Table 2 presents the results. Compared to baselines, AniDS achieves state-of-the-arr perfor- mance across all tasks. This is because AniDS can more accurately generate noise that aligns with the atomic vibration modes, thereby better assisting the model learn molecular force fields. Notably, our model demonstrates an improvement of ~3.2% % in energy prediction for both the in-distribution (ID) and -fitt-riiitiiuttionn (OOD) datasets, and an improvement of -6.2% in force prediction for both ID and OOD datasets. These superior results underscore the effectiveness of our method.
4.3 Ablation Study and Hyperparameter Analysis
Impact of Different Denoising Methods. We evaluate various denoising strategies with the same
Equiformer--v2 backbone (24). pre-trained on the PCQM4Mv2 dataset. The methods include: (1) no
pre-training: (2) denoise training with fixed-scale isotropic Gaussian Noise VEqVD-DeNS (77); (3)
VAE []]; (4) AniDS with additive anisotropic correction instead of subtractive correction
(ef. Equation 4
EqV2-AniDS-addl, and (5) our method AniDS.
Table compares two fine-tuning settings following denoising pre-training: (a) fine-tuning with denoising as an auxiliary task, and (b) fine-tuning without denoising, In both settings, our model con- sistently outperforms all baselines. All denoising-basee pre-training methods substantially outperform the baseline without pre-training, supporting the benefit of learning molecular noise distributions. No- tably, models using anisotropic noise (2qV2-AniDS) and Eq E2Q2-Anm VVV-AiDDS-ada add) consistently outperform confirming that isotropic assump-
those using isotropic noise (EqV2-DeNS and BgV2-Denoise tions oversimplify molecular dynamics. Moreover, the subtractive correction in V- yields slightly better results than the additive correction in Eq V2-AniDS- suggesting that incorporating directional stiffness from physicochemical priors improves the model's structural understanding.
Impact of Different Tp and AKL. The choice of prior distribution Tp and the strength of the KL loss AKL both affect model performance. As shown in Table c) (top). varying о, influences how effectively the model learns from the noise prior, with the best performance achieved at = 0.1. In addition, T (bottom) shows the effect of different λKL values. The KI. loss helps prevent trivial solutions (e.g., collapsing to zero noise) by regularizing the learned noise distribution. Our results indicate that setting λKL = 1.0 yields the best performance. Larger values overly bias the model toward the prior, ignoring structure-specific features, while smaller values lead to under-regularized, uninformative noise distributions. Together, these results highlight the importance of carefully balancing prior assumptions and regularization strength when modeling physically meaningful noise.
4.4
Case Study: Validating Anisotropic Noise Behavior on Crystal and Molecular Structures
To verify whether the learned noise captures the physical anisotropy of the energy landscape, we de- sign an experiment to probe how directional perturbations influence energy changes. To quantitatively evaluate this effect, first perform eigenvalue decomposition on the learned covariance matrices and apply small random perturbations along the resulting eigenvectors. The corresponding changes in energy are then measured using the Symmetric Mean Absolute Percentage Error (sMAPE).
Crystal Validation. The resulting energy changes, along with the associated eigenvectors and eigen- values, are visualized in Figure ()..
8

---

(6) Gianucty 0024
(b)dnn along Eigenvectoes
(c) Prodicted Eigenvalass Groundrruhh Eneegy Change
ISSMAF esgeovalue
MANI S2009
-00000 -0.37
3.861
0.002
LaMAPE
WOLD)
ABDE 14679
6J89
0292
0.57
Pd In H 10
Auee 52000
Auxa 21PM
30031
Figure 3: Visualization of the 2 Pd12 crystal. We select oxygen atoms at indices (32, and palladium atom at index 21. Figure (b) presents the eigenvectors of the oxygens. Figure (c) shows the relationship between the structural energy and applied noise.
(b) predicted Eigenvalues Ground-truth Energy Changes
Jamoria
1330
4000
0.1000
5000
(a) Local Geometry SNPH,
4030
3000
00005
3000
3300
HIS
H14
NIO
S12
P11
atom-wise Eigervalues Energy Changes
0.21005
IuMAMC
HI6
0.01000
5000
0.00015
4333
Figure 4: Analysis the SNPH, molecule. (a) Local geometry showing atom indices. (b) Per-atom directional alignment between predicted eigenvalues and ground-truth energy sensitivities (1/sMAPE). (c) Atom-wise comparison of averaged eigenvalues and energy sensitivities.
For example, atoms {32, 36) are spatially close. identical oxygen atoms with strong repulsive interaction. The eigenvector corresponding to the smallest eigenvalue for both atoms aligns with the bond direction between them (Figure B)). Due to the short distance and strong repulsion, perturbations along this axis lead to significant energy changes, and AniDS accordingly reduces noise in this direction. In contrast, atom 21 resides in a more symmetric environment. The model thus produces larger and more isotropic noise for this atom. This strongly supports our main idea: AniDS allows for structure-aware noise generation, reducing excessive disruption.
Molecular Validation. To further confirm the generality of the anisotropic noise modeling, we conduct an additional analysis on the SNPH, molecule.
Figure b) shows that the predicted eigenvalues exhibit a consistent trend with the ground-trun -truth energy changes measured by 1 1/sMAPE, demonstrating that AniDS effectively captures direction- dependent energy stiffness. Notably, for atoms H6-1 and 11- the smallest eigenvalues correspond to eigenvectors aligned with the P-H bonds, matching the physically stiff directions of the molecule.
Figure D c) further summarizes the per-atom averaged behavior. Hydrogen atoms display larger eigenvalues and higher 1/ 1/sMAPE values, indicating that their displacements have relatively minor effects on total energy. In contrast, the N, P. and S atoms situated at the core of the molecular structure and show smaller eigenvalues, signifying greater energy sensitivity to positional perturbations. The results show that perturbations along different directions affect the potential energy surface to varying degrees, validating the rationale behind the anisotropic noise design: AniDS suppresses perturbations in energy -eergyssensitive directions while allowing more noise along flexible axes, closely aligning with the ground-truth energy changes.
Related Works
Coordinate Denoising for 3D Atomistic Systems. Coordinate denoising has been widely explored for pretraining on 3D atomistic systems 212 22 showing strong performance in predicting

---

While the method was originally proposed to equilibrium molecular structures, it is later extended to non-equilibrium systems by incorporating atomic force information into the model [27]] Coordinate denoising's 's effectiveness stems from its theoretical connection to learning molecular force fields [3] under the assumption that the noise distribution is isotropic Gaussian and the data distribution
quantum chemical properties, force fields, and energies A4 15 18
mixture of such Gaussians.
To overcome the limitations of the isotropic assumption, Frad 11 combines torsional and coordinate noise, while Slide [8] applies independent Gaussian noise on bond lengths, angles, and torsion angles, However, these methods use fixed noise scales, ignoring the variability of energy potentials across distinct structures, Addressing this, Denoise [T6] leverages a Variational Autoencoder 44 to learn atom-wise adaptive noise variances. In contrast, AniDS learns full covariance matrix for each atom's 's noise distribution, enabling the modeling of anisotropic and structure-aware noise.
Other 3D Molecular Pretraining Methods. A representative approach involves directly pretraining on large-scale molecular dynamics simulation datasets. For example, MatterSim [47] and JMP 48 are trained on large-scale molecular dynamics trajectories and achieve strong performance on down- stream property prediction tasks. However, these methods rely heavily on high-quality large-scale simulation datasets, which are often expensive to obtain and limited in scalability. Another line of work focuses on integrating multiple structural representations For instance, MoleculeSDE 44 and Transformer-M M [50] combine 2D topological graphs with 3D geometric point clouds, using complementary perspectives to enhance molecular representations. UniCor [51] unifies three mainstream self- seffssupervied strategies into multi-view contrastive learn- ing framework to construct more comprehensive molecular representations. MoleBlend (22) fuses 21) and 3D structural information at the atomic relation level, enabling fine-grained structural modeling. Recent advances further explore cross-mode moddl and aanguage-groudder representations for molecular systems. 3D-MoLM [53) and MolCA (54) align molecular graphs and textual descriptions through multimodal pretraining. while NEXT-MOL 55 and ReactXT [56) integrate 3D diffusion or reaction-
improve model expressiveness.
context modeling with language understanding. SIMSGT 57 revisits tokenization and decoding for masked graph modeling, and ProtT3 [58] extends text- based pretraining to proteins. Beyond molecule-text alignment, UAE-3D [59] propose unified latent space for 3D molecular diffusion, and leverage language models for single-cell representation learning. At a broader scale, M 61 envisions a general language of natural sciences, and deep-learning frameworks such as 62 highlight the potential of Al to uncover physical principles in crystalline materials.
scMMGPT [60)
6 Conclusion and Future Work
We present AniDS, denoising framework that learns anisotropic, structure-aware noise distributions to enhance molecular force field modeling. By generating atom-specific full covariance matrices conditioned on molecular geometry, AniDS lifts the isotropic and homoscedastic assumptions inherent in prior denoising methods. Grounded in theoretical connections to force field learning, AniDS supports both pretraining and auxiliary fine-tuning. Extensive experiments on MD17 and OC22 show that AniDS achieves leading performance. Looking forward, we aim to extend AniDS to larger and more complex systems, such as proteins and RNAs, and apply it for molecular simulations.
7 Acknowledgement
This work was supported in part by the Ministry of Education (MOE T1251RES2309 and MOE T2EP20125-0039) and the Agency for Science, Technology and Research (A*STAR H25J6a0034).
References
[1] Alexandre Blanco-Gonzalez, Alfonso Cabezon, Alejandro Torres, Paula Antelo-Riveing Angel Pineiro, and Rebeca Garcia-Fandin The role of ai drug discovery: challenges, opportunities, and strategies. Pharmaceuticals, 16(6):891, 2023.
Daniel Conde-
[2] Kit-Kay Mak, Yi-Hang Wong, and Mallikarjuna Rao Pichika. Artificial intelligence in drug discovery and development. Drug discovery and evaluation: safety and pharmacokinetic assays, pages 1461-1498, 2024.
[3] Dana Bishara, Yuxi Xie, Wing Kam Liu, and Shaofan Li. A state-of-the-art review on ma- chine learning-based multiscale modeling, simulation, homogenization and design of materials. Archives of computational methods in engineering. 1911)2191-222, 2023.
10

---

[4] Lowik Chanussot, Abhishek Das, Siddharth Goyal, Thibaut Lavril, Muhammed Shuaibi, Mor- gane Riviere, Kevin Tran, Javier Hieras-Domingg Caleb Ho, Weihua Hu, et al. Open catalyst 2020 (oc20) dataset and community challenges. Acs Catalysis, 11(10)6059-6072, 2021.
[5] Janice Lan, Aini Palizhati, Muhammed Shuaibi, Brandon M Wood, Brook Wander, Abhishek Das, Matt Uyttendaele, C Lawrence Zitnick, and Zachary W Ulissi. Adsorbml: a leap in efficiency for adsorption energy calculations using generalizable machine learning potentials. upj Computational Materials, 9(1):17,, 2023.
[6] Boris Kozinsky, Albert Musaclian, Anders Johansson, and Simon Batzner. Scaling the leading accuracy of deep equivariant models to biomolecular simulations of realistic size. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, pages 1-12, 2023.
[7] Stefan Chmiela, Alexandre Tkatchenko, Huziel E Sauceda, Igor Poltavsky, Kristof T Schütt, and Klaus-Robert Müller. Machine learning of accurate energy-conserving molecular force fields. Science advances, 3(5):e1633015 2017.
[8] Robert G. Parr and Weitao Yang. University Press, USA, 1994.
Theory of Atoms and Molecules. Oxford
[9] Jörg Behler. Perspective: Machine learning potentials for atomistic simulations. The Journal of chemical physics, 145(17), 2016.
[10] Simon Batzner, Albert Musaelian, Lixin Sun, Mario Geiger, Jonathan P Mailoa, Mordechai Kornbluth, Nicola Molinari, Tess E Smidt, and Boris Kozinsky. E(3)-equivariant graph neu- ral networks for data-efficient and accurate interatomic potentials. Nature communications, 13(1):2453, 2022.
[11] Oliver Unke, Mihail Bogojeski, Michael Gastegger, Mario Geiger, Tess Smidt, and Klaus- Robert Müller. Se(3)--quivariaat prediction of molecular wavefunctions and electronic densities. Advances in Neural Information Processing Systems, 34:1 44344-4444447 2021.
[12] Jonathan Godwin. Michael Schaarschmidt, Alexander Gaunt, Alvaro Sanchez-Gonzalez, Yulia Rubanova, Petar Veličković, James Kirkpatrick, and Peter Battaglia. Simple gnn regularisation for 3d molecular property prediction & beyond. arXiv preprint orXiv:2106.0797/ 2021.
[13] Sheheryar Zaidi, Michael Schaarschmidt, James Martens, Hyunjik Kim, Yee Whye Teh, Alvaro Sanchez-Gonzale Peter Battaglia, Razvan Pascanu, and Jonathan Godwin. Pre-training via denoising for molecular property prediction. arXiv preprint arXiv:2206.00 00030,,,33,,,.. 2022.
[14] Andrew Leach. Molecular modelling: principles and applications. Pearson education, 2001.
[15] Trevor Hastie, Robert Tibshirani, Jerome H Friedman, and Jerome H Friedman. The elements of statistical learning: data mining, inference, and prediction, volume 2. Springer, 2009.
[16] Yurou Liu, Jiahao Chen, Rui Jiao, Jiangmeng Li, Wenbing Huang, and Bing Su. Denoisevae: Learning molecule-adaptive noise distributions for denoising baased 3d molecular pre-training. In The Thirteenth International Conference on Learning Representations, 2025.
[17] Shikun Feng, Yuyan Ni, Yanyan Lan, Zhi-Ming Ma, and Wei- Ying Ma. Fractional denoising for 3d molecular pre-training. In International Conference on Machine Learning, pages PMLR, 2023.
[18] Yuyan Ni, Shikun Feng, Ying Ma, Zhi-Ming Ma, and Yanyan Lan. Sliced denoising: A physics-informed molecular pre-training method. arXiv preprint arXiv:2311.02124, 2023.
[19] Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEFF/CVE conference on computer vision and pattern recognition, pages 16000-16009, 2022.

---

[20] Jacob Devlin, Ming-Wei Wing-Weii Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers), pages 4171-4186, 2019.
[21] Zhiyuan Liu, Yaorui Shi, An Zhang, Enzhi Zhang, Kenji Kawaguchi, Xiang Wang, and Tat- Seng Chua. Rethinking tokenizer and decoder in masked graph modeling for molecules. Advances in Neural Information Processing Systems, 0--25544258777 2023.
[22] Yuyang Wang, Changwen Xu, Zijie Li, and Amir Barati Farimani. Denoise pretraining on nonequilibrium molecules for accurate and transferable neural potentials. Journal of Chemical Theory and Computation, 19(15):0077-507, 2023.
[23] Tianlang Chen, Shengjie Luo, Di He, Shuxin Zheng, Tie- Yan Liu, and Liwei Wang. Geom- former: A general architecture for geometric molecular representation learning. arXiv preprini arXiv:2406. 16853, 2024.
[24] Yi-Lun Liao, Brandon Wood, Abhishek Das, and Tess Smidt. Equiformerv2: Improved equivariant transformer for scaling to higher-degree representations. ICI.R, 2024.
[25] Gene H Golub and Charles F Van Loan. Matrix computations. JHU press, 2013.
[26] Yusong Wang, Shaoning Li, Tong Wang, Bin Shao, Nanning Zheng, and Tie-Yan Liu. Geometric transformer with interatomic positional encoding. Advances in Neural Information Processing Systems, 36:55981-5558 2023.
[27] Yi-Lun Liao, Tess Smidt, Muhammed Shuaibi, and Abhishek Das. Generalizing denoising to non-equilibrium structures improves equivariant force fields. arXiv preprint 4rXiv:2403.09549, 2024.
[28] Johannes Gasteiger, Janek GroB, and Stephan Günnemann. Directional message passing for molecular graphs. arXiv preprint XXiv:2003.03122 2020.
[29] Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. Advances in neural information processing systems, 32, 2019.
[30] Pascal Vincent. A connection between score matching and denoising autoencoders. Neural computation, 2367):1661-1674 2011.
[31] Shengehao Liu, Hongyu Guo, and Jian Tang. Molecular geometry pretraining with se (3)- invariant denoising distance matching. arXiv preprint arXiv:2206. 13602, 2022.
[32] Maho Nakata and Tomomi Shimazaki. Pubchemqe project: a large-scale first-principles electronic structure database for data-driven chemistry. Journal of chemical information and modeling, 3706))1300013000 300--108, 2017
[33] Richard Tran, Janice Lan, Muhammed Shuaibi, Brandon M Wood, Siddharth Goyal, Ab- hishek Das, Javier Heras-Domingo, Adeesh Kolluru, Ammar Rizvi, Nima Shoghi, et al. The open catalyst 2022 (oc22) dataset and challenges for oxide electrocatalysts. ACS Catalysis, 13(5):3066-3084, 2023.
[34] Stefan Chmiela, Alexandre Tkatchenko, Huziel E Sauceda, Igor Poltavsky, Kristof T Schütt, and Robert Müller. Machine learning of accurate energy-conserving molecular force fields. Science advances, 3(5)ee160300) 2017.
[35] Bowen Deng, Peichen Zhong. KyuJung Jun, Janosh Riebesell, Kevin Han, Christopher J Bartel, and Gerbrand Ceder. Chgnet as a pretrained universal neural network potential for charge-informed atomistic modelling. Nature Machine Intelligence, 509): 2023.
[36] Anubhay Jain, Shyue Ping Ong, Geoffroy Hautier, Wei Chen, William Davidson Richards, Stephen Dacek, Shreyas Cholia, Dan Gunter, David Skinner, Gerbrand Ceder, and Kristin A. Persson. Commentary: The materials project: A materials genome approach to accelerating materials innovation. APL Materials, 1(1):011002, 07 2013.
12

---

[37] Kristof 'T Schütt, Huziel E Sauceda, P-J Kindermans, Alexandre Tkatchenko, and K-R Müller. Schnet-a deep learning architecture for molecules and materials. The Journal of Chemical Physics, 148(24), 2018.
[38] Johannes Gasteiger, Shankari Giri, Johannes T Margraf, and Stephan Günnemann. Fast and uncertainty-aware directional message passing for non-equilibrium molecules. arXiv preprint arXiv:2011.14115. 2020.
[39] Kristof Schütt, Oliver Unke, and Michael Gastegger. Equivariant message passing for the prediction tensorial properties and molecular spectra. International Conference on Machine Learning, pages 9377-9388. PMLR, 2021.
[40] Philipp Thölke and Gianni De Fabritiis. Torchmd-net. equivariant transformers for neural network based molecular potentials. arXiv preprint N2Ki::2002.05541, 2022.
[41] Shengchao Liu, Hongyu Guo, and Jian Tang. Molecular geometry pretraining with SE(3)- invariant denoising distance matching. In The Eleventh International Conference Learning Representations, 2023.
[42] Johannes Gasteiger, Muhammed Shuaibi, Anuroop Sriram, Stephan Günnemann, Zachary Ulissi, C Lawrence Zitnick, and Abhishek Das. Gemnet-oc: developing graph neural networks large and diverse molecular simulation datasets. arXiv preprint arXiv:2204.022 02782, 2022.
[43] Yunyang Li, Lin Huang, Zhihao Ding, Chu Wang, Xinran Wei, Han Yang, Zun Wang. Chang Liu, Yu Shi, Peiran Jin, et al. E2former: A linear-time efficient and equivariant transformer for scalable molecular modeling. arXiv e-prints, pages arXiv-2501, 2025.
[44] Rui Jiao, Jiaqi Han, Wenbing Huang, Yu Rong, and Yang Liu. Energy-motivated equivariant pretraining for 3d molecular graphs. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 8096-8118 8104, 2023.
[45] Gengmo Zhou, Zhifeng Gao, Qiankun Ding, Hang Zheng, Hongteng Xu, Zhewei Wei, Linfeng Zhang, and Guolin Ke. Uni-mol: A universal 3d molecular representation learning framework. 2023.
[46] Diederik Kingma and Max Welling. Auto-encoding variational bayes. In ICLR, 2014.
[47] Han Yang, Chenxi Hu, Yichi Zhou, Xixian Liu, Yu Shi, Jielan Li, Guanzhi Li, Zekun Chen, Shuizhou Chen, Claudio Zeni, et al. Mattersim: A deep learning atomistic model across elements, temperatures and pressures. arXiv preprint arXiv:2405. 2405029000. 2024.
[48] Nima Shoghi, Adeesh Kolluru, John R Kitchin, Zachary W Ulissi, C Lawrence Zitnick, and Brandon M Wood. From molecules to materials: Pre- Pre-training large generalizable models for atomic property prediction. arXiv preprint arXiv:2310. 16802, 2023.
[49] Shengchao Liu, Weitao Du, Zhi-Ming Ma, Hongyu Guo, and Jian Tang. A group symmetric stochastic differential equation model for molecule multi-modal pretraining. In International Conference Machine Learning, pages 21497-21526. PMLR, 2023.
[50] Shengjie Luo, Tianlang Chen, Yixian Xu, Shuxin Zheng, Tie- Yan Liu, Liwei Wang, and Di He. One transformer can understand both & 3d molecular data. arXiv preprint a:X1::2210001755, 2022.
[51] Shikun Feng, Yuyan Ni, Minghao Li, Yanwen Huang, Zhi-Ming Ma, Ying Ma. and Yanyan Lan. Unicorn: A unified contrastive learning approach for multi-view molecular representation learning. arXiv preprint arXiv:2405.103: 2024.
[52] Qiying Yu, Yudi Zhang, Yuyan Ni, Shikun Feng, Yanyan Lan, Hao Zhou, and Jingjing Liu. Multimodal molecular pretraining via modality blending. arXiv preprint arXiv:2307.06235, 2023.
13

---

[53] Sihang Li, Zhiyuan Liu, Yanchen Luo, Xiang Wang, Xiangnan He, Kenji Kawaguchi, Tat-Seng Chua, and Qi Tian. 3d-molm: Towards 3d molecule-text interpretation in language models. In ICLR, 2024.
[54] Zhiyuan Liu, Sihang Li, Yanchen Luo, Hao Fei, Yixin Cao, Kenji Kawaguchi, Xiang Wang, and Tat-Seng Chua. Molea: Molecular graph-language modeling with cross-modal projector and uni-modal adapter. In EMNLP, 2023.
[55] Zhiyuan Liu, Yanchen Luo, Han Huang, Enzhi Zhang, Sihang Li, Junfeng Fang, Yaorui Shi, Xiang Wang. Kenji Kawaguchi, and Tat-Seng Chua. NEXT-MOL: 3d diffusion meets Id language modeling for 3d molecule generation. In The Thirteenth International Conference on Learning Representations, 2025.
[56] Zhiyuan Liu, Yaorui Shi, An Zhang, Sihang Li, Enzhi Zhang, Xiang Wang, Kenji Kawaguchi, and Tat-Seng Chua. Reactx:: Understanding molecular "reaction-siee shipppponssspp" via reaction- contextualized molecule-text pretraining. In Findings of the Association for Computational Linguistics: ACL 2024, Association for Computational Linguistics, 2024.
[57] Zhiyuan Liu, Yaorui Shi, An Zhang, Enzhi Zhang, Kenji Kawaguchi, Xiang Wang, and Tat-Seng Chua. Rethinking tokenizer and decoder in masked graph modeling for molecules. In NeurIPS, 2023.
[58] Zhiyuan Liu, An Zhang, Hao Fei, Enzhi Zhang, Xiang Wang, Kenji Kawaguchi, and Tat- Seng Chua. Prott3: Protein-to-text generation for text-based protein understanding. In ACL. Association for Computational Linguistics, 2024.
[59] Yanchen Luo, Zhiyuan Liu, Yi Zhao, Sihang Li, Hengxing Cai, Kenji Kawaguchi, Tat-Seng Chua, Yang Zhang, and Xiang Wang. Towards unified and lossless latent space for 3d molecular latent diffusion modeling. arXiv preprint S.ti::2503.156660 2025.
[60] Yaorui Shi, Jiaqi Yang. Changhao Nai, Sihang Li, Junfeng Fang. Xiang Wang, Zhiyuan Liu, and Yang Zhang. Language-enbanced representation learning for single-cell transcriptomics. arXiv preprint arXiv:2503.094 094427, 2025.
[61] Yingce Xia, Peiran Jin, Shufang Xie, Liang He, Chuan Cao, Renqian Luo, Guoqing Liu, Yue Wang, Zequn Liu, Yuan-Jyue Chen, al. Naturelm: Deciphering the language of nature for scientific discovery, arXiv e-prints, pages urXiv-2502, 2025.
[62] Jielan Li, Zekun Chen, Qian Wang, Han Yang, Ziheng Lu, Guanzhi Li, Shuizhou Chen, Yu Zhu, Xixian Liu, Junfu Tan, et al. Probing the limit of heat transfer in inorganic crystals with deep learning. arXiv preprint arXiv:2503.1 / 2025.
[63] Jonathan Schmidt, Tiago FT Cerqueira, Aldo H Romero, Antoine Loew, Fabian Jäger, Hai-Cher Wang, Silvana Botti, and Miguel AL Marques. Improving machine-learning models in materials science through large datasets. Materials Today Physics, 48: 101560, 2024.
[64] Jinzhe Zeng, Duo Zhang, Anyang Peng, Xiangyu Zhang, Sensen He, Yan Wang, Xinzijian Liu, Hangrui Bi, Yifan Li, Chun Cai, et al. Deepmd-kit kit v3: A multiple- backend framework for machine learning potentials. Journal of Chemical Theory and Computation, 2025.
[65] Ilyes Batatia, David P Kovacs, Gregor Simm, Christoph Ortner, and Gābor Csányi. Mace: Higher order equivariant message passing neural networks for fast and accurate force fields. Advances in neural information processing systems, 35:1 1423-12211111 1436, 2022.
[66] Luis Barroso-Lugue Muhammed Shuaibi, Xiang Fu, Brandon M Wood, Misko Dzamba, Meng Gao, Ammar Rizvi, Lawrence Zitnick, and Zachary W Ulissi. Open materials 2024 (omat24) inorganic materials dataset and models. arXiv preprint mxXiv:2410.127711 2024.
[67] Viktor Zaverkin, Francesco Alesiani, Takashi Maruyama, Federico Errica, Henrik Christiansen, Makoto Takamoto, Nicolas Weber, and Mathias Niepert. Higher-rank irreducible cartesian tensors for equivariant message passing. Advances in Neural Information Processing Systems, 37:124025-1222 124068, 2024.
14

---

[68] Anders S Christensen and O Anatole Von Lilienfeld. On the role of gradients for machine learning of molecular energies and forces. Machine Learning: Science and Technology, 1(4):045018, 2020.
[69] Stefan Chmiela, Valentin Vassilev-Galindo, Oliver T Unke, Adil Kabylda, Huziel E Sauceda, Alexandre Tkatchenko, and Klaus- Müller. Accurate global machine learning force fields for molecules with hundreds atoms. Science Advances, 9())eadd08833 2023.
[70] Yunyang Li, Yusong Wang, Lin Huang, Han Yang, Xinran Wei, Jia Zhang, Tong Wang, Zun Wang, Bin Shao, and Tie- Yan Liu. Long-short-range message-passing: physics-informed framework capture non-local interaction for scalable molecular dynamics simulation. arXiv preprint arXiv:2304. 13542, 2023.
15

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We assure that the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
The limitations are in Appendix
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should refleet the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [Yes]
16

---

Justification:
We include the full assumptions and complete proof in Section S and Ap-
pendix B
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems. The proof' can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification:
We provide details for reproduction in Section and Appendix
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
17

---

Answer: [Yes]
Justification:
Our code is released httpt:tt////// h/wp:////nnnnmuuusuooomm het.://uuuu.pppee.eciccc////////
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please the NeurlPS code and data submission guidelines (httpss:/nnips.cc)
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https for more details.
y //nips.co
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
The details are provided in Section and Appendix
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary appreciate the results and make sense of them.
The full details can provided either with the code, appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: Our experiments mostly computational expensive. Therefore, we run only one group of experiment for each setting, following previous works.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
18

---

It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful figures symmetric error bars that would yield results that are out of range (e.g. negative enror rates).
show in tables or
error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: These details are provided in Appendix
C2
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make it into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification:
We respect the NeurIPS code of ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: This work is about small molecules and materials, which does not involve societal impact.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
19

---

The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our research about small molecules and materials that dose not have such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: The discussion in Appendix 222.2
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, papprssii thcode. cow/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
20

---

If this information is available online, the authors encouraged to reach out the asset's ss creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: Our released code at lhttps://anonynnnn .open. science/r/AniDS is accompanied with README the documentation.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mooel part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
We do not include these resources.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
We do not include such risks.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such the institution conducting the review.
21

---

16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods in this research? Note that the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification: The core method development in this research does not involve LLMS any important, original, or non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non- components.
Please refer to our LLM policy hhttps://neurpp::: http://errri..cccooooeeeeeee//// L for what should or should not be described.
22


Reviews:
[{"review_id": "0g6mhyUeyW", "reviewer": "Reviewer_Y5LU", "summary": "This work proposes a novel anisotropic denoising framework with a structure-aware generator for molecular force field learning. The framework contains a structure-aware noise generator that predicts a covariance matrix from the structure, a denoising autoencoder and a covariance-scaled reconstruction loss. A KL divergence regularizer is also implemented to avoid trivial solution. The noise generator is designed to control the anisotropic level based on the atomic context, and to ensure the covariance properties. The author proves the objective approximates the force field.\n\nThe framework is tested on several datasets and tasks, including MD17 force prediction and energy prediction in OC22, and demonstrated better performance than the benchmarks. Ablation study shows the advantage of pre-training on PCQM4Mv2 and the slight advantage of using subtractive correction than additive. The eigenvector of the learned covariance matrix is shown to correspond to the bond direction.", "questions": "1. How to interpret the off-diagonal parts of the covariate matrix? How would they relate to atomic interactions?\n2. The relationship between the numbers in Fig 3c and the conclusions in 4.4 could be more clearly explained. Does small sMAPE (and large eigenvalue) mean the perturbation has less impact on the energy, and similar values on v1~3 mean more isotropic noise?\n3. Could the authors discuss briefly about how the method could be scaled to larger systems? Especially for pre-training and fine-tuning?", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. The work has a sound theoretical foundation. Relaxing the homoscedastic and isotropic constraints could potentially enhance the model capacity, and the design of the proposed noise generator captures the anisotropic property of atomic-specific and context-dependent dynamics. \n2. The authors provide comprehensive results on several datasets and tasks, ablation study, and case study and visualization, which offer strong evidences for the superiority of the proposed method. \n\nWeaknesses:\nI have only minor questions about the results. See Questions", "was_revised": true, "final_justification": "I appreciate the authors' responses and will maintain my current rating.", "rebuttal": "We sincerely thank you for your valuable comments and the positive feedback! They have helped us to enhance the related work of our paper to better summarize the current status of the research. To address your concerns, we present the following point-by-point responses.\n\n> Q1: How to interpret the off-diagonal parts of the covariate matrix? How would they relate to atomic interactions?\n\n**Response:**\n**Explanation.** Thank you for the thoughtful question. The off-diagonal components of the covariance matrix capture the correlations between different spatial directions in the noise distribution for each atom. Unlike diagonal matrices—which assume that noise in each direction (x, y, z) is independent—the off-diagonal entries encode directional dependencies, effectively modeling how motion in one axis correlates with motion in another.\n\n**Eigenvector Analysis.** To better interpret these terms, we perform eigenvalue decomposition on the covariance matrix. The resulting eigenvectors indicate principal directions of noise, while the corresponding eigenvalues determine the magnitude of the noise along each direction. **These principal directions are not necessarily aligned with the coordinate axes, reflecting the fact that atomic interactions are inherently anisotropic—forces such as bond stretching or angle bending exhibit direction-specific stiffness.**\n\n\nBy learning full covariance matrices that include off-diagonal terms, our model can capture these directional preferences induced by local chemical environments and interatomic forces. This allows the learned noise distributions to reflect the underlying anisotropic energy landscape, ultimately leading to more accurate force field modeling. We believe this is one of the key advantages of our structure-aware anisotropic noise generator.\n\n>Q2: The relationship between the numbers in Fig 3c and the conclusions in 4.4 could be more clearly explained. Does small sMAPE (and large eigenvalue) mean the perturbation has less impact on the energy, and similar values on v1~3 mean more isotropic noise?\n\n**Response:** \nWe appreciate the reviewer’s question and the opportunity to clarify the interpretation of Figure 3c and Section 4.4. In Figure 3c, the three directions v1, v2, and v3 correspond to the eigenvectors of the learned covariance matrix for a given atom. For each direction, we apply a small perturbation and measure its impact on the total energy using sMAPE (Symmetric Mean Absolute Percentage Error).\n\nA larger sMAPE indicates that a perturbation in that direction causes a greater change in energy, i.e., the system is more sensitive along that direction. Conversely, a smaller eigenvalue (i.e., less noise injected by our model in that direction) reflects the model’s decision to suppress noise along directions where perturbations are energetically costly. This behavior is consistent with our goal of aligning the noise distribution with the underlying energy landscape.\n\nRegarding isotropy: if the eigenvalues across all three directions are similar, it suggests that the model considers the local environment approximately isotropic, and thus applies similar noise levels in all directions. In contrast, discrepancies among eigenvalues indicate anisotropic noise, where the model adapts the magnitude of noise based on the directional stiffness of the local atomic environment.\n\nThis coupling between energy sensitivity (sMAPE) and learned noise scale (eigenvalue) highlights the structure-aware nature of our anisotropic noise generator, which aims to respect physical constraints and improve force field learning.\n\n>Q3: Could the authors discuss briefly about how the method could be scaled to larger systems? Especially for pre-training and fine-tuning?\n\n**Response:** \nThank you for raising this important point. Our method is designed to be scalable to larger systems, both during pre-training and fine-tuning. The key reason is that our anisotropic noise generator and denoising autoencoder are both implemented as neural networks with local and structure-aware message passing, which naturally generalize to systems with more atoms.\n\nIn particular, the noise generator predicts per-atom full covariance matrices based on local structural contexts, allowing the computation to scale linearly with the number of atoms. This design ensures that even for large molecules or materials, the model can make predictions in a parallel and efficient manner. Furthermore, since our framework is model-agnostic, it can be paired with scalable backbones such as EquiformerV2 or other graph-based architectures tailored for large-scale molecular systems.\n\nDuring pre-training, we can leverage existing large-scale datasets (e.g., PCQM4Mv2, OC22, or MPTrj) that already contain millions of structures. Fine-tuning on specific tasks or systems involves freezing the noise generator and only training the denoising autoencoder, which further reduces computational cost and improves scalability.\n\nIn future work, we also plan to explore hierarchical representations and domain decomposition to further enhance scalability for very large systems such as proteins or crystalline solids."}, {"review_id": "iTNFgQDTGH", "reviewer": "Reviewer_gHi8", "summary": "This paper presents AniDS, an anisotropic variational autoencoder for 3D molecular denoising, designed to overcome key limitations in previous denoising approaches, namely the assumptions of isotropic atomic motion and a uniform noise scale across all atoms. AniDS learns atom-specific, full covariance matrices for Gaussian noise distributions, incorporating geometric priors to capture directional dependencies in atomic perturbations. The authors establish a connection between their denoising approach and learning molecular force fields. The method is evaluated using multiple datasets, including MD17 (with PCQM4Mv2 pre-training), OC22, and MPTrj, showing improved accuracy in force prediction compared to baseline methods.", "questions": "My questions would relate to the issues raised above, so addressing them would suffice to change my evaluation of the paper.", "limitations": "Yes.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "This paper has several strengths:\n\n1. The proposed method introduces structure-aware noise generation and learns per-atom full covariance matrices, which is a non-trivial extension of previous work.\n2. The connection to force field learning is established and well-motivated.\n3. The analysis comparing eigenvalue magnitudes of learned covariance matrices with energy changes along corresponding perturbations provides valuable insight.\n\nHowever, before acceptance, several weaknesses should be addressed:\n\n1. The manuscript lacks a clear positioning of the proposed approach within the broader scope of related work. For example, a related strategy is to estimate energy response to perturbations using reference or predicted forces (e.g., https://www.nature.com/articles/s41524-020-0323-8, https://doi.org/10.48550/arXiv.2408.05215). The authors mayconsider discussing whether learning the perturbation itself is more advantageous than modeling the energy response to a random perturbation.\n2. The accuracy of the resulting models is primarily evaluated on the MD17 and OC22 datasets. While MD17 is widely used, state-of-the-art models typically achieve very low errors on this dataset, which makes any comparison less meaningful. Therefore, recent works such as NequIP, MACE, and ICTP have used a subset of 50 samples drawn from rMD17 to benchmark accuracy under more challenging conditions (i.e., https://www.nature.com/articles/s41467-022-29939-5, https://doi.org/10.48550/arXiv.2206.07697, and https://doi.org/10.48550/arXiv.2405.14253). To make stronger claims about the potential improvements offered by AniDS, the authors may consider evaluating it on MD17 (or rMD17) in a similar way. Furthermore, the authors may include results for the baseline model trained with AniDS from scratch, i.e., without pre-training.\n3. The OC22 dataset is an important benchmark; however, it appears to be approaching a saturation point similar to that of QM9, which may reduce its reliability for assessing model performance. Setting this aside, the reported improvements in force predictions are rather modest, reflecting limitations similar to those discussed regarding the MD17 dataset. Furthermore, the energy errors are reported in meV, complicating their assessment. The authors may consider presenting errors in meV/atom instead, as this aligns better with the established standards in computational materials science and enables a more straightforward assessment of machine-learned force field accuracy.\n4. The improvements over DeNS are generally modest. Given the increased model complexity (i.e., 129M vs. 127M parameters), it is unclear whether these gains are practically significant. The authors might consider applying their approach to smaller but challenging datasets (e.g., the HEA dataset in https://doi.org/10.48550/arXiv.2405.14253, but any other dataset can be used instead) using smaller models. For molecular systems, applying the approach to ANI datasets or MD22 (again, reducing the training set size, if necessary) could provide further insights.\n5. Since the primary goal of improving machine-learned force fields is their robust application in atomistic simulations, the authors should consider including results that demonstrate this, for example, by performing molecular dynamics simulations in an NVT ensemble and evaluating long-term energy conservation.\n6. The comparison of eigenvalue magnitudes of learned covariance matrices with energy changes is currently limited to a few selected atoms. Including more general correlation plots would better demonstrate the broader validity of these results.\n7. Finally, the manuscript does not address the applicability of the proposed approach to tasks where reference forces are unavailable. Discussion on this point would clarify the method’s scope and potential limitations.", "was_revised": true, "final_justification": "I thank the authors for their detailed responses and will raise my score accordingly.", "rebuttal": "Thank you for the extensive efforts on reviewing our submission and the valuable suggestion. Here we present a detailed response to address your concern.\n\nDue to space limitations, we do not include the full reviewer question here. Thank you.\n\n\n>Q1:\n\n**Response:** Thanks for the insightful comment. While modeling the energy response to perturbations is a promising strategy, our work follows the research line of learning denoising force fields, which offers several advantages. First, denoising objectives are theoretically equivalent to learning the score function, which further corresponds to the negative force field under the Boltzmann assumption. This allows the model to explicitly approximate a pseudo harmonic force field. Second, according to this theoretical connection, we can specifically design the noise distribution to reflect properties of the underlying force field. In AniDS, we introduce structure-aware noise covariances to allow the model capture the anisotropic flexibility of the local structures, which is nontrivial to explicitly design for energy-response-based methods. Last, the denoising objective can be applied in purely unsupervised pretraining on equilibrium structures like PCQM4Mv2, without requiring energy or force labels, while energy-response-based methods typically rely on zeroth-order labels (i.e. energies) for training. We agree that energy-response-based methods offer an important perspective and we will extend a discussion of such approaches in the revised related work section.\n\n\n>Q2: \n\n**Response:** \nWe thank the reviewer for this insightful suggestion. To better evaluate the effectiveness of AniDS under more challenging conditions, we followed the protocol introduced in recent works such as NequIP, MACE, and ICTP, and conducted experiments on four subsets of the revised MD17 (rMD17) dataset. For each molecule, we used 50 training samples, following the ICTP setup. The results are summarized in the table below.\n\n| Dataset | NequIP | MACE | ICTP|  EquiformerV2| EquiformerV2 + AniDS (Ours) |\n|:-----:| :-----:|:-----:|:-----:|:-----:|:-----:|\n| Aspirin | 52.0 |43.9 | **40.19** |71.4 |**40.21**|\n| Benzene | 2.9 |2.7 |2.45|-|**2.31**|\n| toluene | 15.1 |12.1 |11.24|19.28|**10.7**|\n| uracil | 40.1 |**25.9** |**25.97** |50.45 |28.8|\n\n(*Unit: meV/Å; values are force MAE. Full experimental details and hyperparameter settings will be included in the updated manuscript.“–” indicates results not completed due to time constraints.*)\n\n\nAs shown above, AniDS achieves competitive performance on most datasets, and demonstrates notable improvements over the EquiformerV2 baseline trained from scratch. This confirms that the structure-aware anisotropic noise modeling introduced by AniDS leads to improved learning efficiency and accuracy.\n\nRegarding the second point, we appreciate the interest in assessing the performance of AniDS when training from scratch. Our method is designed to learn structure-dependent, atom-wise noise distributions through pretraining on diverse molecular configurations. As such, training AniDS on a single molecule (or limited structural diversity) may restrict its ability to learn meaningful noise patterns, potentially diminishing its performance—a limitation that has also been observed in other VAE-based methods. Nevertheless, for completeness, we include a from-scratch variant of our model trained without pretraining in the ablation study (Table 3(b)), where it still provides notable improvements over non-pretrained baselines. These results further suggest that AniDS is flexible and effective, even without large-scale pretraining.\n\nWe hope these additions address the reviewer’s concerns and strengthen the empirical claims of our work.\n\n\n\n>Q3: \n\n\n**Response:** \nThank you for this insightful feedback. We agree that OC22, while still widely used, is beginning to show signs of performance saturation, and we appreciate the reviewer’s perspective on the limitations this may introduce in evaluating model improvements.\n\nRegarding the unit of measurement: we reported energy errors in meV, consistent with previous works on OC22 such as the original benchmark paper[1]  and recent baselines including E2Former[2] and DeNS[3], to ensure direct comparability.\n\nThank you again for pointing this out.\n\n**Reference:**\n[1]Tran, Richard, et al. \"The Open Catalyst 2022 (OC22) dataset and challenges for oxide electrocatalysts.\" ACS Catalysis 13.5 (2023): 3066-3084.\n[2]Li, Yunyang, et al. \"E2Former: A Linear-time Efficient and Equivariant Transformer for Scalable Molecular Modeling.\" arXiv preprint arXiv:2501.19216 (2025).\n[3]Liao, Yi-Lun, et al. \"Generalizing denoising to non-equilibrium structures improves equivariant force fields.\" arXiv preprint arXiv:2403.09549 (2024).\n\n\n\n\n>Q4: \n\n**Response:**\nWe thank the reviewer for the constructive suggestion. We agree that evaluating AniDS on smaller but challenging datasets with reduced model sizes would help demonstrate its practical utility and generalizability beyond OC22 and MD17.\n\nDue to time and computational resource constraints, we conducted a preliminary evaluation on the AT–AT–CG–CG subset of the MD22 dataset, which is known for its chemical complexity and force field modeling difficulty. The results are summarized below:\n\n| Dataset | ICTP| ViSNet-LSRM|MACE|Allegro|TorchMD-Net| sGDML|  Equiformer| Equiformer + AniDS (Ours) |\n| ------- | --------------- | ------------- | ----- | ---------|------ |---|---|---|\n|AT–AT–CG–CG|3.37| 4.61 |5.0| 5.55| 14.13| 30.36|5.43|**3.22**|\n\n(*Unit: meV/Å; values are force MAE. Full experimental details and hyperparameter settings will be included in the updated manuscript.*)\n\nDespite the smaller scale of the dataset and model, **AniDS continues to provide consistent improvements**, confirming that the benefits of anisotropic, structure-aware noise modeling are not limited to large-scale settings. \n\nWe appreciate the reviewer’s helpful guidance, which has helped us broaden the scope of our evaluation.\n\n\n\n>Q5:\n\n\n**Response:** \nWe thank the reviewer for highlighting the importance of assessing model robustness in realistic atomistic simulation settings. We conducted a geometry optimization test using our model on a subset of the Aspirin molecule from MD17.\n\nSpecifically, we randomly selected 15 equilibrium structures and used our force predictor (EquiformerV2 + AniDS) to optimize the geometries. We then compared the optimized structures to those obtained using DFT geometry optimization and evaluated the RMSD between atomic positions.\n\nDue to licensing restrictions with FHI-aims (used in the original dataset with the PBE-vdW-TS functional and tight settings), we performed DFT-based geometry optimization using PySCF at the PBE-D3BJ/def2-SVP level, which provides comparable accuracy in molecular geometries.\n\nThe results are summarized below:\n\n| Structure Index | RMSD Before (Å) | RMSD After (Å) | Improvement (Å) |\n| :-: | :-: | :-: | :-: |\n| 1               | 0.5692          | 0.4863         | 0.0829          |\n| 2               | 0.6278          | 0.5628         | 0.0650          |\n| ...             | ...        | ...       | ...   |\n**Average**     | **0.5019**      | **0.4162**     | **0.0857**      |\n\n\n\nThe significant reduction in RMSD after optimization indicates that the learned force field can guide the system toward physically meaningful minima, showing promise for its application in downstream atomistic simulations. \n\n\n> Q6: \n\n\n**Response:** \nWe appreciate the reviewer’s thoughtful feedback. The case study in Section 4.4 focused on a few representative atoms in a periodic crystal to provide a clear and interpretable illustration of the relationship between learned covariance eigenvalues and energy sensitivity. However, we agree that presenting a more comprehensive, quantitative evaluation would strengthen our conclusions.\n\nIn the revised manuscript, we plan to include an additional analysis based on a single-molecule system, where the number of atoms is more manageable and not affected by periodic boundary conditions. Specifically, we will provide correlation plots between the eigenvalues of the learned covariance matrices and the measured energy sensitivity (e.g., sMAPE) along the corresponding eigenvectors, computed across all atoms in the molecule. This will allow us to more broadly validate the model’s ability to suppress noise in energy-sensitive directions.\n\n>Q7: \n\n\n**Response:** \nWe appreciate the reviewer’s question.We would like to emphasize that AniDS is designed to operate in both supervised and unsupervised settings. In particular, our model can be pre-trained purely on equilibrium structures without any force supervision, such as those found in PCQM4Mv2. In this setting, AniDS learns from coordinate perturbations alone using the denoising objective (Eq. 9), which is theoretically grounded in approximating the underlying force field via score matching.\n\nThis unsupervised pretraining can then be followed by fine-tuning on smaller force-labeled datasets (e.g., MD17), or directly applied to downstream tasks where only structural information is available. Moreover, AniDS can also serve as an auxiliary regularization objective when reference forces are partially available, helping improve model robustness and generalization.\n\nWe will clarify these use cases in the revised manuscript to better highlight the flexibility and broad applicability of AniDS across different data regimes."}, {"review_id": "8hgMWEhs7K", "reviewer": "Reviewer_Mv5z", "summary": "This paper proposes AniDS, a novel denoising-based training framework to assist molecular force field prediction. Building upon prior denoising frameworks, the method introduces atom-specific full covariance matrices and designs the neural network to ensure that the learned covariance matrices are symmetric, positive definite, and equivariant. The approach is evaluated on multiple benchmarks including MD17, OC22, and MPtrj.", "questions": "1. Without of prior knowledge, why is the model able to learn reasonable anisotropic correction terms? Could the authors provide an intuitive explanation? Also, since the KL term aligns the learned noise with an isotropic prior, does it suppress the learning of anisotropic components?\n\n2. While a case study is provided to illustrate the quality of the learned force field, I believe this is not sufficient. Could the authors compare the learned force field (i.e., the regression target in Eq. (9)) with DFT-computed force fields, for instance by computing correlation coefficients as in the SliDe paper?\n\n3. Could you provide a comparison of training efficiency with existing baselines?", "limitations": "Yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n1. Denoising pre-training to enhance force field prediction is a hot direction, and refining noise design to make it consistent with physics is promising.\n\n2. The method is sound, and the experimental results are comprehensive, clearly demonstrating the effectiveness of the proposed approach.\n\n3. The paper is well written, and the figures are clear and informative.\n\nWeaknesses:\n\nMore evidence is needed to justify that the learned anisotropic noise is physically meaningful (see Questions 1–2 below).", "was_revised": true, "final_justification": "This paper proposes a data-driven approach to learning noise distributions for denoising pretraining. The idea is sound and empirically validated through experiments. During the rebuttal phase, the authors provided additional explanations and experimental evidence that further show the physical rationality of the learned noise distribution. Based on these strengths, I stand by my recommendation for acceptance.", "rebuttal": "Thank you for your feedback. Your suggestions have enhanced the reliability of our work and the readability of the paper. Below are responses to your questions one by one.\n\n>Q1: Without of prior knowledge, why is the model able to learn reasonable anisotropic correction terms? Could the authors provide an intuitive explanation? Also, since the KL term aligns the learned noise with an isotropic prior, does it suppress the learning of anisotropic components?\n\n\n**Response:** \nThank you for the insightful question. While the model does not explicitly encode prior knowledge about anisotropy, it learns reasonable anisotropic correction terms through supervised signals from the denoising objective, which is theoretically grounded in force field learning. This objective guides the model to assign higher noise along directions that are energetically less sensitive, and lower noise in directions where small perturbations lead to large energy changes. Over time, this enables the model to discover direction-specific stiffness patterns purely from data.\n\nThe key intuition is that our anisotropic noise generator is conditioned on molecular structure via a graph neural network encoder. As the model observes repeated patterns of atomic environments and their response to perturbations (e.g., rigid bonds vs. flexible chains), it learns to modulate the noise distribution accordingly. The learned anisotropic correction terms are not handcrafted but emerge from optimizing the denoising loss, which penalizes inaccurate reconstructions—particularly in directions where force gradients are steep.\n\nRegarding the KL term: although it encourages the learned noise distribution to remain close to an isotropic Gaussian prior, its role is primarily regularization, not constraint. We carefully balance the KL loss with the denoising loss using a tunable coefficient (λ_KL), ensuring that the model retains flexibility to learn meaningful anisotropic components while avoiding degenerate or trivial solutions (e.g., collapsing to zero noise). Our ablation study (Table 3c) shows that setting λ_KL appropriately (e.g., λ_KL = 1.0) allows the model to learn informative, structure-aware anisotropic noise while maintaining stability during training.\n\n\n> Q2: While a case study is provided to illustrate the quality of the learned force field, I believe this is not sufficient. Could the authors compare the learned force field (i.e., the regression target in Eq. (9)) with DFT-computed force fields, for instance by computing correlation coefficients as in the SliDe paper?\n\n\n**Response:** \nThank you for this valuable suggestion. We agree that directly comparing the learned force field with DFT-computed reference forces provides a stronger validation of the model’s physical fidelity.\n\nFollowing the approach in the SliDe paper, we conducted a same analysis between the predicted forces and DFT-calculated forces. Due to computational constraints, we randomly selected 1,000 structures from PCQM4Mv2 with fewer than 32 atoms, and computed their DFT ground-truth forces. We then pre-trained models using supervised force regression on these structures and fine-tuned them on MD17 using only 50 training samples per molecule.\n\nThe table below reports the Force MAE(kcal/mol/Å), comparing AniDS with DeNS and models trained from scratch:\n| Dataset | train from scratch | DeNS | AniDS|  DFT label supervised|\n| ------- | --------------- | ------------- | ----- | ---|\n| Aspirin | 1.689 |1.390 |1.346 | 1.626 |\n\nInterestingly, we observe that pretraining with DFT-supervised labels does not lead to significant improvement over the unsupervised methods. This may be due to the fact that PCQM4Mv2 primarily consists of equilibrium structures, which provide limited force variation, or inconsistencies in the level of DFT theory used during force computation.\n\nNevertheless, even under such conditions and with a limited number of training samples, AniDS still achieves the best performance, demonstrating its strong capability to learn accurate force fields from small datasets.\n\n\n> Q3: Could you provide a comparison of training efficiency with existing baselines?\n\n\n**Response:** \nThank you for raising this important point. To assess the additional computational cost introduced by AniDS, we conducted a comparison of training efficiency and model complexity across different noise modeling approaches using the rMD17 dataset (Aspirin molecule, 50 train data). The results are summarized below:\n\n| Method                   | #Parameters | Training Time per Epoch (s) |\n| -----------------------  | ----------- | --------------------------- |\n| EqV2 (No Noise)          |   6.35M    |       6.2              |\n| EqV2 + DeNS              |   6.35M    |       6.9              |\n| EqV2 + DenoiseVAE        |   7.2M     |       7.3              |\n| **EqV2 + AniDS (Ours)**  |   7.36M    |       7.3              |\n\nAs shown, AniDS introduces a **moderate increase in training time and model size**, primarily due to the more expressive structure-aware noise generator that outputs full covariance matrices. However, this cost is justified by the **significant improvement in performance** (lowest MAE among all baselines)."}, {"review_id": "gjevkrs9Sa", "reviewer": "Reviewer_Hzfs", "summary": "This paper proposed a new denoising framework called AniDS for machine learning based molecular force fields. The proposed method addressed the limitations of previous noise injection design in terms of isotropic and homoscedastic. It is more physics informed than random noise. AniDS is claimed to be SO(3) equivariant, and the experimental results on MD17 and OC22 S2EF show that it is better than previous noise distributions.", "questions": "As listed above in Weaknesses.", "limitations": "Limitations are not provided in the main paper but in the appendix. This might not be a good practice.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "## Strengths:\n1. This paper proposed a new, better, and physics informed noise distribution for MLFF pretraining.\n2. The proposed noise distribution satisfies key physical restrictions including SO(3) equivariance. \n3. The experimental results show that the proposed AniDS is better than DeNS, and can be used to boost EquiformerV2's performance. \n\n## Weaknesses:\n1. Clarity suggestions: AniDS is a general framework that can apply to many models, but the base model name is missing in table 1. It is not very clear what base models have been used by DeNS by looking at the table. \n2. Although energy and force prediction task is included, but dynamic simulation tasks (e.g., using predicted energies and forces to relax structures) are missing. \n3. Analysis about additional computational cost of using AniDS beyond previous noise distributions is currently missing.", "was_revised": true, "final_justification": "A good paper with good insights. The quality could be further enhanced by including additional evaluations with other baseline methods for the dynamics simulation tasks.", "rebuttal": "Thank you for your feedback. Your suggestions have enhanced the reliability of our work and the readability of the paper. Below are responses to your questions one by one.\n\n>Q1: Clarity suggestions: AniDS is a general framework that can apply to many models, but the base model name is missing in table 1. It is not very clear what base models have been used by DeNS by looking at the table.\n\n\n**Response:**\nThank you for pointing this out. We agree that Table 1 could be clearer in indicating the backbone models used. In our experiments, EquiformerV2 was used as the base model for all variants, including AniDS and the compared DeNS results. We will update the table caption and column headings in Table 1 to explicitly reflect this and avoid ambiguity.\n\n> Q2: Although energy and force prediction task is included, but dynamic simulation tasks (e.g., using predicted energies and forces to relax structures) are missing.\n\n\n**Response:** \nWe thank the reviewer for highlighting the importance of assessing model robustness in realistic atomistic simulation settings. We conducted a geometry optimization test using our model on a subset of the Aspirin molecule from MD17.\n\nSpecifically, we randomly selected 15 equilibrium structures and used our **force predictor (EquiformerV2 + AniDS)** to optimize the geometries. We then compared the optimized structures to those obtained using **DFT geometry optimization** and evaluated the **RMSD** between atomic positions.\n\nDue to licensing restrictions with FHI-aims (used in the original dataset with the PBE-vdW-TS functional and tight settings), we performed DFT-based geometry optimization using **PySCF** at the **PBE-D3BJ/def2-SVP** level, which provides comparable accuracy in molecular geometries.\n\nThe results are summarized below:\n\n| Structure Index | RMSD Before (Å) | RMSD After (Å) | Improvement (Å) |\n| :-: | :-: | :-: | :-: |\n| 1               | 0.5692          | 0.4863         | 0.0829          |\n| 2               | 0.6278          | 0.5628         | 0.0650          |\n| 3               | 0.6069          | 0.3159         | 0.2911          |\n| 4               | 0.6123          | 0.4240         | 0.1883          |\n| 5               | 0.5517          | 0.3687         | 0.1830          |\n| 6               | 0.6717          | 0.6050         | 0.0668          |\n| 7               | 0.7171          | 0.6088         | 0.1083          |\n| 8               | 0.2936          | 0.2011         | 0.0925          |\n| 9               | 0.7087          | 0.4609         | 0.2478          |\n| 10              | 0.2067          | 0.1326         | 0.0741          |\n| 11              | 0.3159          | 0.2272         | 0.0887          |\n| 12              | 0.3194          | 0.2558         | 0.0637          |\n| 13              | 0.6505          | 0.5308         | 0.1197          |\n| 14              | 0.4138          | 0.2696         | 0.1441          |\n| 15              | 0.7162          | 0.6286         | 0.0876          |\n| **Average**     | **0.5019**      | **0.4162**     | **0.0857**      |\n\n\nThe significant reduction in RMSD after optimization indicates that the learned force field can guide the system toward physically meaningful minima, showing promise for its application in downstream atomistic simulations. \n\nWe appreciate the reviewer’s suggestion and agree this direction is valuable for future extensions.\n\n>Q3: Analysis about additional computational cost of using AniDS beyond previous noise distributions is currently missing.\n\n\n**Response:** \n\nThank you for raising this important point. To assess the additional computational cost introduced by AniDS, we conducted a comparison of training efficiency and model complexity across different noise modeling approaches using the rMD17 dataset (Aspirin molecule, 50 train data). The results are summarized below:\n\n| Method                   | #Parameters | Training Time per Epoch (s) |\n| -----------------------  | ----------- | --------------------------- |\n| EqV2 (No Noise)          |   6.35M    |       6.2              |\n| EqV2 + DeNS              |   6.35M    |       6.9              |\n| EqV2 + DenoiseVAE        |   7.2M     |       7.3              |\n| **EqV2 + AniDS (Ours)**  |   7.36M    |       7.3              |\n\nAs shown, AniDS introduces a **moderate increase in training time and model size**, primarily due to the more expressive structure-aware noise generator that outputs full covariance matrices. However, this cost is justified by the **significant improvement in performance** (lowest MAE among all baselines)."}]

Author rebuttal:

Meta-review: This paper proposes AniDS, an anisotropic and heteroscedastic denoising pretraining framework that learns a full SO(3)-equivariant covariance structure for atomic noise. The method is theoretically motivated and empirically shown to capture anisotropic aspects of atomic interactions, improving performance across molecular and material benchmarks.

**Strengths:** Reviewers agreed the problem is well-motivated and the anisotropic treatment of noise is novel and physically meaningful. Results are strong across multiple backbones and datasets. The rebuttal further strengthened the case with experiments on rMD17/MD22, geometry optimization, and analysis of noise–force correlations. Computational overhead appears modest relative to performance gains.

**Weaknesses:** Some reviewers noted incremental aspects relative to prior denoising pretraining frameworks, and raised questions about clarity and cost. These concerns were largely addressed in rebuttal and discussion.

**AC Recommendation:** There is strong consensus among reviewers (three 5s, one 4). Given the novelty, empirical strength, and rebuttal clarifications, I recommend Accept.
Decision: Accept (poster)