Physics-Informed Weakly Supervised Learning For Interatomic Potentials
Authors: Makoto Takamoto
Viktor Zaverkin
Mathias Niepert
Venue: ICML 2025 poster (2025)
Keywords: interatomic potential
physics informed method
weakly supervised method
Primary area: applications->chemistry_physics_and_earth_sciences
TL;DR: A new physics informed weakly supervised learning method for interatomic potential machine learning model

Abstract: Machine learning is playing an increasingly important role in computational chemistry and materials science, complementing expensive ab initio and first-principles methods. However, machine-learned interatomic potentials (MLIPs) often struggle with generalization and robustness, leading to unphysical energy and force predictions in atomistic simulations. To address this, we propose a physics-informed, weakly supervised training framework for MLIPs. Our method introduces two novel loss functions: one based on Taylor expansions of the potential energy and another enforcing conservative force constraints. This approach enhances accuracy, particularly in low-data regimes, and reduces the reliance on large, expensive training datasets. Extensive experiments across benchmark datasets show up to 2× reductions in energy and force errors for multiple baseline models. Additionally, our method improves the stability of molecular dynamics simulations and facilitates effective fine-tuning of ML foundation models on sparse, high-accuracy ab initio data. An implementation of our method and scripts for executing experiments are available at \url{https://github.com/nec-research/PICPS-ML4Sci}.

Physics-Informie
Weakly Supervised Learning for Interatomic Potentials
Makoto Takamoto'
Viktor Zaverkin Mathias Niepert
Abstract
As a result, they are typically limited to small- or medium- sized systems and short simulation times, which constrains their applicability and the accuracy of predicted proper- ties. Classical force fields can extend these length and time scales, providing a computationally efficient alternative to first-principles approaches but often lack accuracy. Machine- learned interatomic potentials (MLIPs) offer promising compromise between the accuracy of first-principles meth- ods and the efficiency of classical force fields (Smith et al., 2017; Chanussot et al., 2021; Unke et al., 2021; Merchant et al., 2023; Kovács et al.. 2023; Batatia et al., 2023).
Machine learning plays an increasingly important role in computational chemistry and materials sci- ence, complementing expensive ab initio and first- principles methods. However, machine-leamee interatomic potentials (MLIPs) often struggle with generalization and robustness, leading to un- physical energy and force predictions in atom- istic simulations. To address this, we propose physics-informed, weakly supervised training framework for MLIPs. Our method introduces two novel loss functions: one based on Taylor expansions of the potential energy and another enforcing conservative force constraints. This approach enhances accuracy, particularly in low- data regimes, and reduces the reliance on large, expensive training datasets. Extensive experi- ments across benchmark datasets show up to 2x reductions in energy and force errors for multi- ple baseline models. Additionally, our method improves the stability of molecular dynamics sim- ulations and facilitates effective fine-tuning of ML foundation models sparse, high-aceuracy ab initio data. Code and scripts to reproduce the ex- periments are available at https:://// //qi thub.. com/nece http://ccrreeearcc/ppprrrcs
Despite their promise, MLIPs face significant challenges. In particular, they require training datasets that compre- hensively cover both configurational (atomic positions) and compositional (atomic types) spaces, typically generated via molecular dynamics (MD) simulations based on ab initio first-princpples methods. Due to the high computational cost of these simulations, the resulting datasets are often sparse, limiting the ability of MLIPs to generalize to new molecular and material systems.
Active learning has been used address the challenge of data sparsity (Li al., 2015; Vandermause et al., 2020; Za- verkin & Kästner, 2021; Zaverkin et al., 2022; van der Oord et al., 2023; Zaverkin et al., 2024b), but it still relies on the generation of a substantial amount of first-principles data, typically using DFT, to train an initial model capable of guiding phase space exploration through extended MD sim- ulations. This motivates the development of methods that can complement active learning by reducing the reliance on expensive data acquisition. Moreover, MLIPs often suffer from limited generalization and robustness during MD simu- lations, exhibiting sensitivity to outliers and local structural perturbations. These issues stem largely from the insuffi- cient coverage configurational and compositional spaces in existing datasets and data generation workflows.
1. Introduction
Ab initio and first-principles methods are essential for the computational exploration of molecular and material prop- erties in the chemical sciences and engineering (Parrinello, 1997; Carloni et al., 2002; Iftimie et al., 2005). How- ever, commonly employed ab initio and first-principles ap- proaches such as coupled cluster (CC) (Purvis & Bartlett, 1982; Bartlett& & Musial, 2007) and density functional theory (DFT) (Hohenberg & Kohn, 1964; Kohn & Sham, 1965), respectively, require substantial computational resources.
Contributions. This paper addresses the challenges of train- ing machine-learned interatomic potentials (MLIPs) with limited data by proposing a physics-infomma weakly super- vised learning (PIWSL) approach. The method enables an accurate prediction of potential energies and atomic forces in systems subjected to local perturbations. The main con- tributions are: (i) We introduce PIWSL based on fundamen- tal physical principles, such as the conservative nature of
'NEC Laboratories Europe, Heidelberg, Germany Universiity of Stuttgart, Stuttgart, Germany, Correspondence to: Makoto Takamoto smakoto.takamotoo neclab.eu>.
Proceedings of the 42nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Physics-Informed Weakly Supervised Learning for Interatomic Potentials
No.
(a)
DFT
E
E
or;, i.S)
B
ör
WL Loss
MLIP
E ESS; 0)
B(Str: 0)
MLIP
B(S+; 0)
$ PISC Loss
No frr, F. (S:
år
B
8r"
E (S 0)
> 1-1
(or F, (Str 0)
(b)
MLIP E (S;
(S;
B
ōr
PITC Loss
MLIP
E (Sée' 0)
MLIP
0)
Figure 1: Schematic illustration of physics-informed weakly supervised losses used in this work. (a) Taylor-expansion based weak label (WL) loss with approximate labels obtained from reference energies and atomic forces (Cooper et al., 2020). (b) Physics-inspired atomic forces predicted by an MLIP. (c) Physics-inspirdd spatial consistency (PISC) loss with approximate labels obtained from energies and atomic forces predicted by an MLIP. Here, E(S; 0) and F, (S; 0) denote the potential energy and atomic forces predicted by an MLIP parametrized by 0, S and Sar define the original atomic structure and the one perturbed by ór.
myyllo-expansion-comee
(PITC) loss with approximate labels obtained from energies and
forces. By leveraging a Taylor expansion of the potential en- ergy. derive two novel thysics-informed loss functions, physics-informed Taylor consistency (PTTC) and physics- informed spatial consistency (PISC), which form the core of the PIWSL framework (illustrated in figure 1b and c). (ii) Through extensive experiments, we demonstrate that PIWSL enables effective training of MLIPs without access to large datasets and enhances robustness during molecular dynamies (MD) simulations. (iii) We show that PIWSL. improves the accuracy of total energy and atomic force pre- dictions, even when force labels unavailable. This is par- ticularly relevant for fine-tuning MIIP foundation models with reference ab initio energies, where force calculations (e.g., at the CCSD(T)/CBS level) are often computationally prohibitive (Smith et al., 2019; 2020; Zaverkin et al., 2023; Hobza & Sponer, 2002; Feller et al., 2006). (iv) Finally, PI- reduces sensitivity issues in MD simulations caused by small training datasets by explicitly modeling the response of the potential energy to local atomic perturbations.
higher-body-order representations (Shapeev, 2016; Drautz, 2019; Zaverkin & Kästner, 2020; Zaverkin et al., 2021) and the emergence of equivariant message-passing neural net- works (MPNNs) (Thomas et al., 2018; Schütt et al., 2021; Shuaibi et al., 2021a; Passaro & Zitnick, 2023; Liao et al., 2023; Batzner et al., 2022; Musaelian et al., 2023; Batatia et al., 2022; Zaverkin et al., 2024a) significantly advanced the field. These methods enable the cost-efficient generation of accurate MLIPss for modeling interactions in many-body atomic systems and account for crucial inductive biases, such the invariance of the potential energy under rota- tion. The evaluation of MLIPs for MD simulations has been addressed in previous work (Fu et al., 2023; Bihani et al., 2024).
Physics-Informnn Machine Learning, Physics-informed machine learning aims to embed known physical princi- ples into the machine learning process. In the context of MLIPs, models based equivariant message passing neu- ral networks (MPNNs) enforce rotational invariance of the predicted potential energy by design. These models lever- age equivariant features to capture and construct many-body interactions in a manner consistent with the symmetries of the underlying physical system (Thomas et al., 201 8; Batzner et al., 2022; Batatia et al., 2022; Musaelian et al., 2023; Liao al., 2023; Zaverkin al., 2024a). Further- more, physics constraints can be integrated via auxiliary loss functions, prompting ML models to learn important phys- ical relationships, as demonstrated for physics-informed neural networks (PINNs) (Raissi et al., 2019; Cai et al., 2022), which learn to model solutions of partial differen- tial equations by minimizing residuals during training. The application of physics-informed machine learning to molec- ular modeling has garnered increasing interest in both the
2. Related Work
Machine-Learned Interatomic Potentials. The field of machine-learne interatomic potentials (MLIPs) emerged over two decades ago (Blank al., 1995) and has been one of the most active research directions since then (Behler & Parrinello, 2007; Artrith et al., 2011; Artrith & Urban, 2016; Smith et al., 2017; Shapeev, 2016; Schütt et al., 2017: Thomas et al., 2018; Unke & Meuwly, 2019; Drautz, 2019; Zaverkin & Kästner, 2020; Zaverkin et al., 2021: Thomas et al., 2018; Schütt et al., 2021; Shuaibi al., 2021a; Pas- saro & Zitnick, 2023; Liao al., 2023; Batatia al., 2022: Batzner et al., 2022; Musaelian al., 2023; Duval et al., 2023; Zaverkin et al., 2024a). The development of local
2

---

Physics-Informed Weakly Supervised Learning for Interatomic Potentials
machine learning and computational chemistry communi- ties (Godwin al., 2022; Shui et al., 2022; Ni et al., 2024; Liao et al., 2024). In particular, DeNS (Liao et al., 2024), an extension of NoisyNode (Godwin et al., 2022) designed for non-equilibrium structures, shares several similarities with our approach. PIWSL differs from DeNS in two key aspeets: (1) PIWSL does not require force labels, allow- ing it to fine-tune models using only energy data, which is useful for high-accuracy methods like CCSD(T)/BBS; and (2) PIWSL operates without modifying the underlying model architecture, whereas DeNS introduces architectural changes via an additional force module. The potential to combine these two methods to enhance performance re- mains an open and promising direction for future research. Previous work by Cooper et al. (2020) has provided key motivation for our current research and is discussed in more detail in subsequent sections.
Here, denotes a point-wise loss function such as the abso- lute and squared error between the predicted and reference total energies and atomic forces. Typically, reference en- ergies E and atomic forces F are computed with ab initio or first-principles methods such as CC or DFT, respec- tively. The relative contributions of energies and forces in the loss function of Eq. (1) are balanced by the weighting coefficients C. and Cr.
Weakly Supervised Learning. Generating many reference labels with first-principles approach is challenging due to the high computational cost. Furthermore, the calculation of atomic forces can be infeasible for some nigh-accuracy ab initio methods, e.g., for CCSD(T)/CBS. In this work, we focus on weakly supervised learning methods to improve the performance of MLIPs in scenarios where only a limited amount of data is available. These involve the generation of approximate but physically motivated total energies for atomic structures generated by small perturbations of their atomic positions, i.e., Ser = {r; + òr; 3:) Nat with a per- turbation vector òr, where òr, is the perturbation vector for atom i. Approximate labels are computed with MLIPs during their training.
3. Background and Problem Definition
Machine-Learned Interatomic Potentials. An atomic con- figuration, denoted as S (r., Z, | comprises Nat atoms characterized by their positions r, E R and atomic numbers Z, E N. We consider machine-learned interatomic potentials (MLIPs) that map atomic configurations to scalar energies, i.e., fe S + E E I. where 0 denotes train- able parameters. Let F(S; 0) be the energy predicted by an MLIP for configuration S. In most MLIPs, atomic forces are obtained as the negative gradient of the potential energy with respect to atomic positions:
4. Physics-informed Weakly Supervised Learning
Generating approximate labels for weakly supervised loss functions for MLIPs is particularly challenging. Small perturbations in atomic structures can lead to significant changes in energies and atomic forces, making standard weak supervision techniques, effective in many ML do- mains (Yang et al., 2022), inapplicable in this context. To overcome this, we propose a physics-informed weakly su- pervised learning (PIWSL) approach that leverages known physical principles. Specifically, our method incorporates (i) Taylor expansion of the potential energy to model the system's 's local response to atomic displacements, and (ii) a spatial consistency loss that encourages consistent energy differences along two distinct spatial paths, composed of three perturbation vectors in total, from a common refer- ence configuration to the same target structure. These two components are combined into the PIWSI. loss, which aug- ments the supervised objective with physically grounded weak supervision, improving both robustness and accuracy.
F; (8:0) V., (S:00.
Hence, these MLIPs ensure that the resulting forces are conservative (curl-free) and the total energy is conserved during a dynamic simulation. However, some models are designed to predict atomic forces directly (Hu et al., 2021: Passaro & Zitnick, 2023; Liao et al., 2023; Chanussot et al., 2021). While this approach avoids expensive gradient com- putations, it does not enforce energy conservation (Chmiela et al., 2017).
Trainable parameters 0 are optimized by minimizing loss functions on training data 'D comprising a total of Nirain atomic configurations (((() J. Narain as well as their energies {ES YEED and atomic forces (FFg)iissse
4.1. Physics-Informed Taylor Aisloo--ppaasionnBBaae Consistency Loss
(D: = Σ LSS:))
SED
This section introduces the physics-informed Taylor- expansion-based consistency (PITC) loss. Particularly, we relate the energy predicted directly for a displaced atomic configuration with the energy obtained by the Taylor expan- sion from the original configuratio;; see figure (b). We estimate the energy for an atomic structure S drawn from
= Σ
Col [E (S:0), E
SED
Na
e 0, ]. (1)
+ C
3

---

Physics-Informed Weakly Supervised Learning for Interatomic Potentials
the training dataset with atomic positions displaced by a vector dr: Sar = (r. + or; 7 =- For this atomic config- uration, we expand the energy predicted by an MLIP in its second-order Taylor series around the atomic perturbation vector ōr; and obtain
energy at r-+ór using Eq. (3) through the PITC loss. For the second path, we directly compute the energy E (Sar: 0) for atomic positions displaced by ór', and use it to approximate E (Sór; 0) after applying the second perturbation vector 5r" = or - 6r'. The physics-informed spatial consistency (PISC) loss is formally defined as
Nat Kór; F. (S:
E' (Ssr; 0) ~ E (S:0) Σ
Lrisc (S; 0) = { (Sor: 0) Errrc (Sar', "err,,r":::::: 0) (4)
tkzna(rr; F; (Str: 0) - F; 0))] + O (11511"), (2)
where Eprrc (Sár, 5r'; 0) is the potential energy estimated via PITC formula in Eq. (2) from the configuration Sor perturbed by 6r'. After jointly training with the PITC and PISC losses, the three distinct estimations at Ser become spatially consistent. Importantly, our approach, grounded in the conservative nature of interatomic forces, is not re- stricted to configurations involving two perturbation paths or three perturbation vectors, We explore several alternative consistency configurations in section D.2.
where (+) denotes the inner product, and we used the fol- lowing relation between forces and gradients of the energy F; = -VE i.e., defined as the negative gradients of the potential energy. The parameter kand controls the contribu- tion of the second-order term. Setting k2nd = 1/2 recovers the exact second-order Taylor expansion, while kzud = leads to the first-order approximation' Using approximate labels E' (Sse; 0), we define the PITC loss as
Lerre (S; 0) {( (Sár; 0) E ()
4.3. Combined Physics-Informe Weakly Supervised Loss
Nat
(rr:: (1 - kznd) F. (S;0) + knnd F; (Sor; 0) (3)
Together with the usual MLIP loss function defined in Eq. (1), the overall objective, which we refer to as the PI- WSL loss, can be written as
where& l denotes a point-wise loss for regression problems and ör is either randomly sampled or determined adversari- ally; see section 4.4 for more details. Hence, whenever we encounter structure S in a batch during training, new år is computed for each S. Empirically, second-order terms become important when MLIP prediction errors approach the accuracy limit of first-order approximations, i.c., when first-order terms alone no longer suffice to improve accu- racy, or when the dataset predominantly consists of relaxed structures.
arg min 2 (D; - arg min (8(80)
SED
+CerrcLPIre (S: + CPSSC (S; (5)
where CPITC and CPISC are the weights of the weakly supervised PITC and PISC losses.
4.4. Perturbation Directions and Magnitudes
The effectiveness of the proposed approach depends on appropriate choices of the perturbation vectors őr. We in- troduce and justify various strategies for generating the perturbations used in Eq. (3) and Eq. (4). Any vector ór can be written as ör = eg/llgl /l/gglll. where e is the magnitude of ór and g/llg||a is its direction. Physical constraints can limit e. Specifically, we can obtain the maximum perturbation length from the validity of the Taylor expansion in Eq. (2), which, as discussed in section 5.3, is typically given as at most 30% of the original bond length whose shortest exam- ple is the bond between carbon and hydrogen atoms, about 1.09 A; see also figure 2 (c) and (d). The specific values of chosen for our experiments are provided in section B.1.
4.2. Physics-Informi Opatial-Consistengy Loss
We propose a physics-informed approach for generating weak labels, grounded in the principle of conservative forces. Specifically, we leverage the property that the energy dif- ference between two points on a potential energy surface is independent of the path taken. To construct the weak super- vision signal, we consider two distinct paths from a common reference point to the same target point, each composed of a sequence of perturbation vectors. An example of two such paths is illustrated in (c)). The proposed loss compares the energy obtained by directly displacing the atomic positions of the original configuration S (denoted as A) by ör (from A to C). with the energy obtained through two consecutive perturbations: ăr' (from A to B) followed by 5r" (from B to C).
To determine g/ g/ /llgl|a we explore two strategies. First, we compute it as the unit vector of perturbation vector sam- pled from the uniform distribution on the interval (-1,1) for each direction
For the first path, we directly predict the energy with an MLIP, i.e., E (Str: O), which is related to the approximated
8r = lggl2
(6)
Second, we compute an adversarial direction following the approach proposed by Goodfellow et al. (2014); Miyato
'A more detailed derivation is provided in section D.2.2.
4

---

Physics-Informed Weakly Supervised Learning for Interatomic Potentials
al. (2018). This direction is defined as the gradient of the loss with respect to the atomic coordinates r, indicating the direction in which the prediction error increases most rapidly for the current predicted energy. Assuming an Lg-norm constraint on the perturbation magnitude, the adversarial direction can be approximated as in Miyato et al. (2018)
To evaluate the effect and dependency of the physics- informed weakly supervised approach in detail, we per- formed the training various datasets: ANI-1x as hetero- geneous molecular dataset (Smith et al., 2020), TiO2 as a dataset for inorganic materials (Artrith & Urhan, 2016)*, the revised MD17 (rMD17) dataset containing small molecules with sampled configurational spaces for each (Chmiela et al., 2017; 2018; Christensen & von Lilienfeld, 2020), the MD22 dataset containing larger molecules (Chmiela et al., 2023), and LMNTO as another material dataset (Cooper et al., 2020); the benchmark results for rMD17, MD22, and LM- NTO are provided in section D.I. The detailed description of each dataset is provided in section B.3.
gradv = eg/l| where g- = dist (ypred, ytel), (7)
where Las denotes distance- based objective function that is maximized by introducing the adversarial perturbation orad with yPrdd and yte representing the model prediction and reference values, respectively. For computational effi- ciency, we primarily employ the random perturbation strat- egy defined in Eq. (6) in our experiments. quantitative comparison between random and adversarial perturbation directions is presented in section E.
5.2. Benchmark Results
We compare models trained using the PIWSL loss (see Eq. (5)) with baseline models trained using the standard supervised loss only (see Eq. (1)). We also compare our approach recently proposed data augmentation method that incorporates the task of denoising random perturba- tions of the atomic coordinates into the learning objective (NoisyNode) (Godwin et al., 2022). A comparison with weak label method (Cooper et al., 2020) is provided in sec- tion E. More details on the setup are provided in section B. In the following, all evaluation metrics are computed for the test dataset.
5. Experiments
We evaluate our method through extensive experiments de- signed to address the following objectives: (1) compare PIWSL with established baselines, (2) analyze the effect of PIWSL using the aspirin molecule, including molecular dy- namies (MD) simulations, and (3) assess PIWSL's ability to enhance foundation model finetuning sparse datasets, par- ticularly for energy and force prediction tasks where force labels are unavailable.? 2 Unless otherwise noted, our experi- ments use the first-order PITC. For the MD17 and MD22 datasets, however, the second-order term is incorporated to meet target accuracy requirements." 3
Heterogeneous Molecular Dataset ANI-1x. The re- sults provided in Table 1 show that our approach improves the baseline models' performance in almost all cases. In particular, the error reduction for the predicted energies is often between 10 % and more than 50 %. Interestingly, we observe an improved accuracy for potential energies and atomic forces because we include force prediction in PITC and PISC losses, different from the previous work (Cooper et al., 2020). In most cases, except for SchNet, the data aug- mentation method (NoisyNode) deteriorates the accuracy of the MLIPs because it does not incorporate the proper response of the energy and atomic forces to the perturbation of atomic positions.
5.1. Models and datasets
We trained the following representative models that are provided in the Open Catalyst code base (Chanussot et al., 2021): SchNet (Schütt et al., 2017), PaiNN (Schütt et al., 2021), SpinConv (Shuaibi et al., 2021a), eSCN (Passaro & Zitnick, 2023), and Equiformer v2 (Liao et al., 2023), covering MLIPs with a smaller (SchNet, SpinConv, PaiNN) and larger number of parameters (eSCN, Equiformer v2). Moreover, we also considered the MACE model (Batatia et al., 2022), a state-of-the-ard model that we use to evaluate the impact of PIWSL on the (DCC(C)) and MD22 dataset. Unless otherwise mentioned and except for SchNet, forces are directly predicted and not computed through the negative gradient of the energy. The results where forces are computed as negative energy gradients are analyzed in section E. I and section D.9.
Dependence on Size of Training Data ANI-1x. We train MLIPs with training set sizes of [50, 102, 10 10", 10 10 5 x 10%]. The results for training data sizes of 105. 10 and 5 x 10t are provided in section D.1. The results are plotted in figure 2 (a) and (b). Although the observed error reduction depends strongly on the type of MLIP used, the benefit of the weakly supervised losses often decreases slightly with the number of training samples. This result can be expected as the area covered by the weakly supervised losses is also gradually covered by the reference data the number of training samples increases. Moreover, the gain in
? Additional results are presented in section E, including: (4) comparison with a prior weakly supervised method, (5) an ablation study, and (6) an analysis of random versus adversarial perturbation vectors.
'The impact of the second-order term is discussed in sec- tion F.5.
'A review of solid-state materials datasets for MLIP training can be found in (Lee et al., 2023), for example.
5

---

Physics-Informed Weakly Supervised Learning for Interatomic Potentials
Table 1: Energy (E) and force (F) root-mean-square errors (RMSEs) for the ANI-Ix dataset. The results are obtained by averaging over three independent runs. Energy RMSE is given in kcal/mol, while force RMSE is in kcal/mol/A.
Nusin = 100
Nusis 1000 Noisy Nodes
Baseline
Noisy Nodes
PIWSL
Bascline
PIWSL
E
65.09 2.42
57.39 + 0.05
60.30 + 1.77
31.49 = 0.01
31.10 + 0.00
31,50 + 0.00
SchNet
F
29.06 0.19
25.62 + 0.01
28.20 + 0.60
18.94 + 0.01
18.10 + 0.00
18.93 + 0.00
E
168.01
1.22
464.55 = 6.91
109,89 + 11.46
56,62 = 2.80
305.76 + 33.93
24.53 0,48
PaiNN
F
21.33
0.10
20.82 + 0.03
18.76 + 0.30
12,96 = 0.06
14.25 0.18
11.43 0,05
SpinCouv
162.14
7.55
147.73 + 2.23
1.30.7 + 8.58
43.59 + 1.71
299.33 + 419.10 10
39.44 1.31
F
21.22 0.43
21.08 = 0.43
21.61 + 0.44
14.51 = 1,07
15.83 + 0.75
13.59 0.20
214.52 7.55
521.92
12.05
183,70 + 9.79
59.59 = 8.92
241.34 +: 20.16
21.03 0,56
cSCN
20.07 0.27
23.68 + 0.11
19.69 + 0.05
12.50 + 0.78
14.42 0.84
11.83 0.12
398.71 13.69
632.38
154.98 8.83
54,52 = 4.52
854.33 317.7
20.89 0.50
Equiformer
F
20.71 0.05
21.82 0.01
20.55 + 0.05
10.10 = 0.00
24.79 2.05
9.68 0.03
Table 2: Energy (F) and force (F) root-mean-square errors (RMSEs) for the Tio, dataset. The results are obtained by averaging over three independent runs. Energy RMSE is given in kcal/mol, while force RMSE is in kcal/mol//.
Ntrain 100 Noisy Nodes
Nerain 2000 Noisy Nodes
Baseline
PIWSL
Baseline
PIWSL
E
17.21 + 0.00
19.68 1 0.00
17.08
0.00
10.16 1 0.00
44,44 1 0.00
10.14 1 0.00
SchNer"
F
2.84 = 0.00
2.70 + 0.00
2.83
0.00
1.87 + 0.00
7.45 + 0.00
1.85 = 0.00
E
14.41 +: 0.16
n/a²
13.95
0.09
2.44 1: 0.03
n/a"
2.30 + 0.10
PaiNN
F
1.59 + 0.01
n/a
0.01 1.56
0.27 + 0.02
n/a"
0.24 + 0.00
E
20.00 = 0,42
18.76 + 0.74
16.98
0.99
2.78 + 0.67
2.76
0.42
2.05
0.39
SpinConv
F
1.58 + 0.03
1.53 + 0.03
1.59
0.03
0.67 +: 0.15
0.61 +: 0.09
0.61 0.05
E
16.41 1.10
20.92 + 0.00
12.63
0.78
1.78 + 0.07
20.90
0.00
0.90 0.09
eSCN
F
1.57 = 0.04
1.66 + 0.00
1.44
0.03
0,42 + 0.17
1.66 + 0.00
0.16 0.01
E
18.21 + 0.02
19.06 + 0.02
13.93
0.09
1.73 + 0.05
18.54 + 0.10
1.27 + 0.02
Equiformer
F
1.56 0.01
1.64 0.00
1.51
0.19
0.13 + 0.01
1.59 0.00
0.13 0.00
" We used larger batch size of 32 for SchNet since we obtained extremely high errors for the batch size of 4. A more detailed discussion of the experimental results for SchNet is provided in section D.I.
à Because of a numerical instability of PaiNN when perturbing atomic coordinates, the cutoff' radius is reduced from 12 to 5 in this experiment. Predicted values become n/a when atomic configurations are perturbed.
accuracy of energy predictions is more significant than that for forces trained only indirectly through the consistency constraint in PITC: see Eq. (3). Finally, it is shown that the improvement is more significant for highly parameterized MLIPs, which benefit the most from increasing the training data size through PIWSI.
values, close to even less than kcal/mol in predicted energy, the PIWSI. still reduces the error. This observation indicates strong evidence of the effectiveness of PIWSL applied to bulk materials.
5.3. Qualitative Impact of PIWSL
Inorganic Bulk Materials- - 2. Titanium dioxide (TiO2) is a highly relevant metal oxide for industrial applications, featuring several high-pressure phases. Thus, ML models should be able to predict total energies and atomic forces for various high-pressune phases of TiO2, considering periodic boundaries (relevant when aggregating over the local atomic neighborhood). The results for trained models are provided in Table 2. Similar to the ANI-Ix dataset, our approach improves the accuracy of predicted energies and atomic forces. Interestingly, even when the error in the potential energy for 2000 training configurations reaches small RMSE
We evaluate the prediction variance and robustness of an MLIP model trained with PIWSL using the aspirin molecule, focusing on the potential energy's dependence on the C-H bond length. In this work, robustness refers to the prediction robustness of an MLIP perturbations in atomic coordi- nates. In the literature, the robustness of MLIPs also means their stability during MD simulations. We train PaiNN on the rMD17 17 aspirin dataset using 100 and 200 configurations with and without the PIWSL loss. The detailed training setup and errors of the used MLIPs are summarized in sec-
6

---

Physics-Informed Weakly Supervised Learning for Interatomic Potentials
(8)
(b)
(c)
0.0
Rel. energy in kcal/mol
Rel. energy in kcal/mo)
al (E-(MEE))))))
Alog1o! (o/[-MMS))
0.00
-0.
-0.4
-0.15
-0.10-
100
1000
10000
100
1000
10000
1.0
1.2
1.2 C-H distance
Training set size
Training set size
C-H distance in
SchNet
PaINN
SpinClow
eSCN
Equiformer
PIWSL
Baseline
Reference
Figure 2: (a, b) Relative performance gains for MLIPs trained with PIWSL compared to those trained without it and (c, d) potential energy profiles for a C-H bond of the aspirin molecule. Relative performance gains are evaluated for (a) energy (E-) and (b) force (F-) RMSEs as: RMSEPPWsL/ RREEeressLRRSSSeeelliiii These results are presented for the ANI-Ix Ix dataset. Potential energy profiles for C-H bond the aspirin molecule are presented for models trained using (c) 100 and (d) 200 configurations. The red and blue arrows indicate the direction from the original structure (E(S; 0)) the perturbed one (Eor I)), defined by Eq. (2), for the baseline and PIWSL model predictions, respectively.
Equikemes
PaNN
PUNN-GP
PaiNN
Repiformer v2
90
20WAL
Bustie PWWL
Mvll
1.60
100
200
3380
2
10
20
Thsining act size
Training act
Truiting set dize
Time (ps
Time (ps)
(a)
(b)
Figure 3: Stability analysis of the MLIP models during MD simulations. (a) Stability during MD simulations is assessed for the baseline MLIP models and those trained with PIWSL. Left/Middle: Models with the direct force branch. Right: Models with forces computed as negative gradients of the energy. All results are obtained for the aspirin molecule and MD simulations in the microcanonical (NVE) statistical ensemble. We measure stability during MD simulations according to Fu al. (2023). (b) Plots of the total energy conservation observed in MD simulations using MLIP models with a force branch trained on 1,000 samples, with and without the proposed loss.
tion D.3.1. We examine the potential energy varying the length a C-H bond from 0.9 to 1.4 Å. The equilibrium C-H bond length is about 1.09 A. The results figure 2 (c) and (d) demonstrate that the PIWSL method improves the predicted potential energy profile, indicating improved robustness under perturbations of atom coordinates.
rate force branches and do not guarantee the prediction of conservative forces. The proposed method reduces the curl of the predicted forces, as detailed in section D.10, although complete elimination of the curl remains a challenge. In summary, PIWSL minimizes individual energy and force errors, thereby enhancing the overall accuracy of MLIPs.
Although the estimated potential energies do not always match the reference values, the direction between the orig- inal and perturbed configurations, indicated by arrows in figure 2, consistently follows the gradient of the reference potential energy, corresponding the negative force. In fig- perturbation length of | 0.01 Å. This consistency with the energy gradient underscores the effec- tiveness of the PIWSL method, ensuring alignment between predicted total energies and forces, and improving the corre- sponding RMSE values. As discussed in section 4.2, PIWSL also addresses key limitation of MLIPs that employ sepa-
To further assess PIWSL's impact, we evaluate the robust- ness during MD simulations of the MLIP models trained with and without PIWSI. We consider MD simulations of the aspirin molecule, with corresponding results presented in figure 3. We measured stability following the approach proposed in Fu et al. (2023). A detailed experimental setup is provided in section D.3.2. The results demonstrate that PIWSL improves the stability of MD simulations for both the direct and gradient- based force prediction models. The simulation times figure are shorter than those reported by Fu al. (2023). This difference arises from our choice
7

---

Physics-Informed Weakly Supervised Learning for Interatomic Potentials
to perform MD simulations in the microcanonical (NVE) statistical ensemble instead of the canonical (NVT) statis- tical ensemble in Fu et al. (2023) to assess stability more accurately without the influence of thermostat. Results for MD simulations conducted in the canonical statistical ensemble are provided in section D.3.2. Figure 3 shows the time evolution of the total energy during MD simulations, demonstrating that total energy is conserved within a 0.05 % deviation until molecular degradation occurs. These results indicate that PIWSL enhances the stability of MD simula- tions without compromising total energy conservation.
Table 3: Results for models trained on the MDI7(CCSD) dataset without reference atomic forces. All models are trained on aspirin data without force labels. Energy RMSE is given in kcal/mol, while force RMSE is in kcal/mol/A.
Model
Samples
Epoch
Baseline
PIWSL
950
100
E
1.21 + 0.00
0.72 0.01
MACE-OFF
6,90 + 0.01
3.77 0.13
512
100
E
2.03 + 0.03
1.21 = 0.02
MACE-OFF
10.96 +: 0.13
6.55 0.27
950
1000
3.27 + 0.16
2.10 0.30
MACE (scratch)
18.05 + 0.82
10.99 0.18
5.4. Fine-Tuning of Foundation Models
We evaluated the accuracy of predicted atomic forces using corresponding force labels provided in the test dataset (CCSD/ce- pVDZ level accuracy).
In this section, we present case study demonstrating the application of PIWSL to the fine-tuning of foundation mod- els using sparse dataset. The detailed experimental setup is provided in section D.4. First, we consider fine-tuning MACE-OFF (large) (Kovács et al., 2023), a MACE model pre-trained on the SPICE dataset (Eastman et al., 2023), including QMugs and liquid water subsets (Isert et al., 2022; Schran et al., 2021). The foundation model is fine-tuned using the aspirin dataset with energies and forces evaluated at the CCSD level of theory (CCSD/cc-PVD2) (Chmiela et al., 2018). In the following, we consider a more challeng- ing scenario, e.g., where CC energies are extrapolated to the CBS limit, where force labels are typically unavailable due to the high computational cost.
Table 4: Energy (E) and force (F) mean-absolute errors (MAEs) for MACE, MACE-OFF, and MACE-MP mod- els fine-tuned on the MD22 dataset. All models are fine- tuned using 50 training samples of the buckyball-catcher molecule from MD22. Energy MAE is given in kcal/mol, while force MAE is in kcal/mol/A.
Model
Train Epoch
Bascline
PIWSL
800
E
1.046 + 0.095
0.745 + 0.031
MACE (scratch)
F
0.294 + 0.002
0.290
0.002
100
E
2.111 + 0.453
2.033
0.149
MACE-MP
F
0.716 : 0.009
0.700
0.011
We follow the fine-tuning procedure outlined in the official repository," adjusting the learning rate from 10-2 to 10- to enhance the final accuracy of the model. The results, presented in Table 3, show that PIWSL significantly im- proves model accuracy, achieving approximately 40 % lower RMSE values. Notably, the model trained with PIWSL on 512 samples performs similarly to the model trained without it, using nearly doubled training set size, i.c., 950 configu- rations. These results highlight the data efficiency of our approach. Furthermore, both fine-tuned MACE-OFF mod- els outperform models trained from scratch.
MACE-OFF
100
E
1.159
= 0.154
0.992 + 0.051
F
0.346 = 0.003
0.337 + 0.001
underscoring the importance of selecting an appropriate pre- trained model (Kjeldal & Eriksen, 2024). While a more exhaustive hyperparameter search could potentially yield further gains, such tuning is beyond the scope of this work. Our primary aim is to demonstrate the relative efficacy of the PIWSL approach.
5.5. Computational Cost Analysis
We further evaluate the effectiveness of PIWSL on datasets comprising conformations of a single large molecule. Specifically, we consider the buckyball catcher molecule from the MD22 dataset (Chmiela et al., 2023), which con- tains 148 atoms. For this experiment, we fine-tune two foundation models, MACE-MP (large) (Batatia et al., 2023) and MACE-OFF (large) (Kovács et al., 2023), using only 50 samples to explore the sparse data regime, As shown in Table 4, PIWSL consistently improves fine-tuning per- formance. Notably, MACE-OFF outperforms MACE-MP,
Training time. Table 5 presents the measured training times for experiments conducted with and without PIWSL. For all models except MACE-OFF, the reported training time corresponds to a single training epoch, averaged over five epochs. These experiments use 1000 training configura- tions from the ANI-Ix dataset with a mini-batch size of six. For MACE-OFE, the training time is measured per epoch the MD17(CCSD) dataset and averaged over ten epochs, following the setup described in section 5.4.
"As the official MACE repository supports only a gradient- based force model, our analysis is restricted to this case. 'https: htt/:/mmccecoos latest hitees//gide//aee finetuning.nnnn html
Table 5 shows that PIWSL increases training time by a factor of approximately two to three relative to the baseline. This increase is primarily due to PIWSL effectively doubling or tripling the number of training labels, which leads to a
hedocs. io/en/
8

---

Physics-Informed Weakly Supervised Learning for Interatomic Potentials
Table 5: Training time comparison for experiments with and without PIWSL. We measure the time required for a single training epoch and provide the results obtained as an average over five epochs. All training times are provided in seconds.
6. Discussion and Limitations
This work introduces the PIWSL method, encompassing two distinct physics-informed weakly supervised loss func- tions, for learning MLIPs. These losses provide the physics- informed weak labels based on the Taylor expansion (PITC loss) and the spatial consistency (PISC loss) of the potential energy. These physics-informet weak labels enable any MLIP to improve its accuracy and robustness, particularly in scenarios characterized by sparse training data, which are common when investigating a new molecular or material system. The improved accuracy and robustness of MLIPs can allow running sufficiently long MD simulations, result- ing in more effective use of active learning approaches. Our extensive experiments demonstrate notable efficacy and efficiency of our method from various aspects: (i) depen- dence on the training dataset size, (ii) the potential energy prediction variance and robustness in terms of perturba- tion on a C-H bond length as well as robustness during MD simulations, and (iii) application to the fine-tuning of foun- dation models. In particular, it is shown that our PIWSL method enables MLIPs to improve the accuracy of their force predictions even without force labels used explicitly during training. Therefore, PIWSL opens a new possibility for training MLIPs using highly accurate reference methods, such as CCSD(TYCBS.
SchNett
PaiNN
SpinCony
<SCN
Equiformer v2
MACE-OFF
Buseline PIWSL
8.02
100.71
$7.79
18.5
12.84
23.48
86.28
$28.48
177.55
31.7
proportional rise in computational cost."
Importantly, the additional training time introduced by PI- WSL is negligible compared to the substantial cost and time required for data generation using methods such as DFT or CC, especially as the number of atoms in atomic systems increases. Furthermore, even the large MACE-OFF model requires less than double the training time while achieving an approximately 40 % reduction in error. Finally, it is im- portant to emphasize that PIWSL affects only the training time; the inference time remains unchanged.
Total number of training epochs. Table A19 compares extended training iterations, showing that while the baseline model begins to overfit, PIWSL continues to improve perfor- mance. Table A20 extends the results from Table 3, further confirming the robustness of PIWSL even under prolonged training of the baseline model. more detailed discussion is provided in section D.8. This finding is particularly relevant in the sparse data regime, which is the focus of this work.
Limitations. The proposed PIWSL method is tailored to ML models that predict atomic forces and total energies of atomic systems. It cannot be applied to other ML problems unrelated to computational chemistry or materials science. Although this work uses up to the second-order Taylor ex- pansion to obtain weak labels in Eq. (2), employing more sophisticated higher-order ordinary differential equations viable alternative. Furthermore, while PIWSL was evalu- ated the heterogeneous molecular dataset (ANI- 1x), po- tential limitation of our analysis is that we performed it only on homogeneous inorganic datasets (TiO, and LMNTO), which may not fully capture the method's generalizability across diverse inorganic systems.
Training sample efficiency. Throughout this section, we have demonstrated that PIWSL. enhances model perfor- mance. This improvement also implies greater sample ef- ficiency during training. Notably, Table illustrates that PIWSL achieves comparable or superior accuracy while using only half the training data for fine-tuning a foundation model. This property makes PIWSL especially advanta- geous in data-scarce scenarios, which are central to our investigation.
Impact Statement
This paper presents work whose goal is to advance the field of machine learning, in particular, machine learning for science. Due to the generic nature of pure science, there are many potential societal consequences of our work in the far future, none of which we feel must be specifically highlighted here.
Conclusion. In summary, although PIWSL leads to an increase in training time due to the effective expansion of training samples, the corresponding performance gains jus- tify the additional computational cost-ee especially when compared to the high cost of generating new labels via DFT or CC. Moreover, PIWSL enables achieving lower errors with smaller number of training epochs.
Acknowledgements
MN acknowledges support from the Deutsche Forschungs- gemeinschaft (DFG, German Research Foundation) under Germany's Excellence Strategy EXC 2075 390740016
We can reduce this overhead by employing the 2pt-PISC loss introduced in section D.2.2, which removes the need estimate the third conformation.

---

Physics-Informed Weakly Supervised Learning for Interatomic Potentials
and the Stuttgart Center for Simulation Science (Sim'Tech).
Behler, J. and Parrinello, M. Generalized neural-network representation of high-dimensional potential-energy faces. Phys. Rev. Lett., 98: 98:146401, 2007.
References
Bihani, V., Mannan, S., Pratiush, U., Du, T., Chen, Z., Miret, S., Micoulaut, M. Smedskjaer, M. M., Ranu, S., and Kr- ishnan, N. A. Egraffbench: evaluation of equivariant graph neural network force fields for atomistic simula- tions. Digital Discovery, 3(4):759-768, 2024. 2
Akiba, T., Sano, S., Yanase, T., Ohta, T., and Koyama, M. Optuna: A next-generation hyperparameter optimization framework. Int. Conf. Knowl. Discov. Data Min., 2019. 14, 11
Artrith, N. and Urban, A. An implementation of artificial neural-network potentials for atomistic materials sim- ulations: Performance for tio2. Comput. Mater Sci., 114:135-150, 2016. ISSN 0927-0256. doi: https: 1.004. 2, 5, 16
Blank, T. B., Brown, S. D., Calhoun, A. W. and Doren, D. J.
Neural network models of potential energy surfaces. J. Chem. Phys., 103:41 129--139777 1995. 2
Briganti, V. and Lunghi, A. Efficient generation of sta- ble linear machine-lcarning force fields with uncertainty- aware active learning. Mach. Learn.: Sci. Tech- nol., 4:035005, jul 2023. doi: 10.1088/2632-2153/ ace418. URL https:/ //tp//xx//// doi
Artrith, N., Morawietz, T., and Behler, J. High- nigh-dimensiooal neural-network potentials for multicomponent systems: Applications to zinc oxide. Phys. Rev. B, 83:153101, 2.11. 2
2632 2153 ac14... 14
Cai, S., Mao, Z.,, Wang. Z.,, Yin, M., and Karniadakis, G. E. Physics-informet neural networks (pinns) for fluid me- chanics: A review. Acla Mech. Sin., pp. 1-12, 2022.
Bartlett, R. J. and Musiat, M. Coupled-cluster theory in quantum chemistry. Rev. Mod. Phys., 79:291-352, 2007. doi: 10. Mo..1///eoMoPPy..79....
2
Batatia, I., Kovacs, D. P., Simm, G. N. C., Ortner, C.,, and Csanyi, G. MACE: Higher Order Equivariant Message Passing Neural Networks for Fast and Accurate Force Fields. Adv. Neural Inf. Process. Syst., 35:11423-1143 2022. URL https:::...............tt.. wptp//eeeee...../////p net forum? id=YP 20. 2, 5, 14, 18
Carloni, P., Rothlisberger, U., and Parrinello, M. The role and perspective of ab initio molecular dynamics in the study of biological systems. Acc. Chem. Res., 35:455- 464, 2002. doi: 10. 1021/ar010018u
Chanussot, L., Das, A., Goyal, S., Lavril, T., Shuaibi, M.. Riviere, M., Tran, K., Heras Domingo, J., Ho, C., Hu, W., Palizhati, A., Sriram, A., Wood, B., Yoon, J., Parikh, D., Zitnick, C. L., and Ulissi, Z. Open Catalyst 2020 (OC20) Dataset and Community Challenges. ACS Catal., 11:6059-600 6072, 2021. doi: 10. 1a..0///ascatal.004555522 1, 3, 5, 14
Batatia, I., Benner, P., Chiang, Y., Elena, A. M., Kovács,
D. P., Riebesell, J., Advincula, X. R., Asta, M., Baldwin,
W.J., Bernstein, N., Bhowmik, A., Blau, S. M., Cärare, V., Darby, J. P., De, S., Pia, D., Deringer, V. I.., Elijošius,
R. El-Machachi, Z., Fako, E., Ferrari, A. C., Genreith- Schriever, A., George, J., Goodall, R. E. A., Grey, C. P., Han, S., Handley, W. Heenen, H. H., Hermansson, K., Holm, C., Jaafar, J., Hofmann, S., Jakob, K. S., Jung, H., Kapil, V., Kaplan, A. D., Karimitari, N., Kroupa, N., Kullgren, J., Kuner, M. C., Kuryla, D., Liepuoniute, G., Margraf, J. T., Magdāu, I.-B., Michaclides, A., Moore, H., Naik, A. A., Niblett, P., Norwood, S. W. O'eeill, N., Ortner, C., Persson, K. A., Reuter, K., Rosen, A. S., Schaaf, L. L,, Schran, C., Sivonxay, E., Stenczel, T. K., Svahn, V., Sutton, C., van der Oord, C., Varga-Umbrich, E., Vegge. T., Vondrák, M., Wang. Y., Witt, W. C., Zills, F, and Csányi, G. A foundation model for atomistic materials chemistry, 2023. 1, 8, 19
Chmiela, S., Tkatchenko, A., Sauceda, H. E., Poltavsky, I., Schütt, K. T., and Müller, K.-R. Machine learning of accurate coorgy-conserving molecular force fields. Sci. Adv. 3:e1603015, 2017. 3, 5, 16
Chmiela, S., Sauceda, H. E., Müller, K.-R., and Tkatchenko, A. Towards exact molecular dynamics simulations with machine- force fields. Nat. Commun., 9:3887, 2018. 5, 8, 16, 22
Chmiela, S.,
V., Unke, O. T., Kabylda, A., Sauceda, H. E., Tkatchenko, A., and Müller, K.- R. Accurate global machine learning force fields for molecules with hundreds of atoms. Science Advances, 9 (2):eadf0873, 2023. 5, 8, 16, 18
Batzner, S., Musaelian, A., Sun, L.., Geiger, M., Mailoa, J. P., Kornbluth, M., Molinari, N., Smidt, T. E., and Kozinsky, B. E(3i-equiv graph neural networks for data-efficient and accurate interatomic potentials. Nat. Commun,, 13:2453, 2022. 2, 14
Christensen, A. S. and von Lilienfeld, O. A. On the role of gradients for machine learning of molecular energies and forces. Mach. Learn: Sci. Technol., 1:045018, oct 2020. doi: 10. 20000/2632-215//abbba 5, 16
10

---

Physics-Informed Weakly Supervised Learning for Interatomic Potentials
Cooper, A. M., Kästner, J., Urban, A., and Artrith, N. Ef- ficient training of ann potentials by including atomic forces via taylor expansion and application to water and transition- metal oxide. npj Comput, Mater, 6:54, 2020. 2, 3,5, 5, 16, 18, 26
A graph neural network for large-scale quantum calcula- tions. Int. Conf. Learn. Represent, 2021. 3
Iftimie, R., Minary, P., and Tuckerman, M. E. Ab initio molecular dynamics: Concepts, recent developments, and future trends. Proc. Natl. Acad. Sci., 102:6654-6659, 2005. doi: 10. (0073//pmaa00000193000
Drautz, R. Atomic cluster expansion for accurate and trans- ferable interatomic potentials. Phys. Rev. B, 99:014104, 2019. 2
Isert, C., Atz, K., Jiménez-Luna, J., and Schneider, G. Qmugs, quantum mechanical properties of drug-like molecules. Scientific Data, 9(1):273, 2022. 8,22
Duval, A., Mathis, S. V., Joshi, C. K., Schmidt, V., Miret, S., Malliaros, D., Cohen, T., Liò, P., Bengio. Y., and Bronstein, M. A hitchhiker's guide to geometric gnns for 3d atomic systems. arXiv preprint axXiv2311.005111 2023. 2
Kjeldal, F. and Eriksen, J. Transferability of atom-based neural networks. Machine Learning: Science and Tech- nology, 5(4):045059, dec 2024. doi: 10.1088/2632-2113. ad9709. URL https: //xx/xx///////////// doi doi. wwtp//.gggoooooo 1088/ 2 1552-2153/ad9777 8
Eastman, P., Behara, P. K., Dotson, D. L., Galvelis, R., Herr, J. E., Horton, J. T., Mao, Y.. Chodera, J. D., Pritchard, B. P., Wang, Y., et al. Spice, dataset of drug-like molecules and peptides for training machine learning potentials. Scientific Data, 10(1):11, 2023. 8, 22
Kohn, W. and Sham, L. J. Self-Consistent Equations Includ- ing Exchange and Correlation Effects. Phys. Rev., 140: A1133-A1138, 1965. doi: 10. (003/PhysRev.40.A.
Feller, D., Peterson, K. A., and Crawford, T. D. Sources of error in electronic structure calculations on small chem- ical systems. J. Chem. Phys., 124:054107, 2006. doi: 10.1063/1 2137323. 2. 26
Kovács, D. P., Moore, J. H., Browning, N. J., Batatia, I., Horton, J. T., Kapil, V., Witt, W. C., Magdäu, I.-B., Cole, D. J., and Csányi, G. Mace-off23: Transferable machine learning force fields for organic molecules, 2023. 1, 8, 19, 22
Fu, X., Wu, Z., Wang, W., Xie, T., Keten, S., Gomez- Bombarelli, R., and Jaakkola, T. Forces are not enough: Benchmark and critical evaluation for machine learning force fields with molecular simulations. Transact. Mach. Learn. Res., 2023. 2, 7, 8, 14, 15, 22,22,22 25
Lee, K. L. K., Gonzales, C., Nassar, M., Spellings, M., Galkin, M., and Miret, Matsciml: A broad, multi- task benchmark for solid-state materials modeling. arXiv preprint arXiv:23399 05934, 2023. 5
Godwin, J., Schaarschmidt, M., Gaunt, A. L., Sanchez- Gonzalez, A., Rubanova, Y., Veličković, P., Kirkpatrick, J. and Battaglia, P. Simple GNN regularisation for 3d molecular property prediction and beyond. Int. Conf. Learn. Represent., 2022. 3, 5, 15,
Li, Z., Kermode, R., and De Vita, A. Molecular dynamics with on-the-fly fly machine learning of quantum meetttnnnec forces. Phys. Rev. Lett., 14:0096405, 2015. 1, 14
Liao, Y.-L., Wood. B., Das, A., and Smidt, T. EquiformerV2: Improved Equivariant Transformer for Scaling to Higher- Degree Representations. Int. Conf. Learn Represent..
Goodfellow, Shlens, J., and Szegedy, C. Explaining and harnessing adversarial examples. Int. Conf. Learn. Represent. 2014. 4
2023. 2, 3, 5, 16
Liao, Y.-L., Smidt, T., Shuaibi, M., and Das, A. General- izing denoising to non-equilibrium structures improves equivariant force fields. Transactions on Machine Learn- ing Research, 2024. ISSN 2835-8856. URL https: openreview. net Soorumiid=whGGuuIIT. 16
Hobza, P. and Sponer, J. Toward true dna base-stacking energies: Mp2, ccsd(t), and complete basis set calcula- tions. J. Am. Chem. Soc., 124:11802-11808 2002. doi: 10.1021/ja026759n. 26
3.
Hohenberg, and Kohn, W. Inhomogeneous electron gas. Phys. Rev., 136:B864-B871, 1964. doi: 10.1103/PhysRev 136.B864.
Merchant, A., Batzner, S., Schoenholz, S. S., Aykol, M., Cheon, G., and Cubuk, E. D. Scaling deep learning for materials discovery. Nature, 624:80 85, 2023. doi: 10.
Hoover, W. G. Canonical dynamics: Equilibrium phase- space distributions. Physical review A, 316)) 1985. 22
Miyato, T., Maeda, S.-i., Koyama, M., and Ishii, S. Virtual adversarial training: a regularization method for super- vised and semi- sumi-supervieed learning. IEEE Trans. Pattern Anal. Mach. Intell., 41:1979- 1993, 2018. 4,5
Hu, W., Shuaibi, M. Das, A., Goyal, S., Sriram, A., Leskovee, J., Parikh, D., and Zitnick, L. Forcenet:

---

Physics-Informed Weakly Supervised Learning for Interatomic Potentials
Musaelian, A., Batzner, S., Johansson, A., Sun, L.., Owen, C. J., Kornbluth, M., and Kozinsky, B. Learning local equivariant representations for large-scale atomistic dy- namics. Nat. Commun., 14:579, 2023. 2
Shuaibi, M., Kolluru, A., Das, A., Grover, A., Sriram, A., Ulissi, Z., and Zitnick, C. L.. Rotation invariant graph neural networks using spin convolutions, 2021a. 2,
Shuaibi, M., Sivakumar, S., Chen, R. Q. and Ulissi, Z. W. Enabling robust offline active learning for ma- chine learning potentials using simple physics-based pri- Mach. Learn.: Sci. Technol., 2:025007, dec 2021b. URL https: dxx .oi.org//o / 10. 088/ 26222263222222 21 53/ abcc44. 14
Ni, Y., Feng, S., Ma, W.-Y,, Ma, Z.-M., and Lan, Y. Sliced denoising: A physics-informe molecular pre-training method. Int. Conf. Learn. Represent, 2024. 3
doi: 10. 10.11888/2632-21533/bbee.
Nosé, S. A unified formulation of the constant temperature molecular dynamics methods. The Journal of chemical physics, 81(1),511-319, 1984. 22
Shui, Z.,, Karls, D., Wen, M., Tadmor, E., Karypis, G., et al. Injecting domain knowledge from empirical interatomic potentials neural networks for predicting material prop- erties. Advances in Neural Information Processing Sys- tems, 35:1 11114-4844880 2022. 3
Parrinello, M. From silicon to ma: The coming of age of ab initio molecular dynamics. Solid State Commun., 102: 107-120, 1997. ISSN 0038-1098 doi: https://doi.or//10. 1016/50038- 10900000000099
Passaro, S. and Zitnick, C. L. Reducing SO(3) Convolutions to SO(2) for Efficient Equivariant GNNs. Int. Conf. Mach. Learn., 002-2222-227332 2023. 2, 3,5
Smith, S., Isayev, O., and Roitberg, A. E. ANI-1: An extensible neural network potential with DFT accuracy at force field computational cost. Chem. Sci., 8:3192-3203,
2017. doi: 10. 10.1039/C6SC0572AA 1,2
Podryabinkin, E. V. and Shapeev, A. V. Active learning of linearly parametrized interatomic potentials. Comput.
Smith, J. S., Nebgen, B. T., Zubatyuk, R., Lubbers, N., Devereux, C., Barros, K., Tretiak, S., Isayev, O., and Roitberg, A. E. Approaching coupled cluster accuracy with a general-purpost neural network potential through transfer learning. Nat. Commun., 10:2903, 2019. ISSN 2041-1723. doi: 10. 10.10088/41467--1--0003333
Mater: Sci., 14:0::111180 2017. doi: https://doi.or//1.. 101 //..//oommaaccii 2017. 00.00.00..1.... 14
Purvis, G. D. and Bartlett, R. J. A full coupled-cluster singles and doubles model: The inclusion of disconnected triples. Chem. Phys., 76:1910-1918, 1982. doi: 10. 1063/1.443) 1431144.
Smith, J. S., Zubatyuk, R., Nebgen, B., Lubbers, N., Barros, K., Roitberg, A. E., Isayev, O., and Tretiak, S. The ani- Icex and ani- data sets, coupled-cluster and density functional theory properties for molecules. Sci. Data, 134, 2020. 2, 5, 16, 20, 24, 25, 26, 27,
Raissi, M., Perdikaris, P., and Karniadakis, G. F. Physics- informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J. Comput. Phys., 378:686- 707, 2019. doi: 10. 10.1016/j..cp.2001 20118...p...0110.00 10.045.
Thomas, N., Smidt, T., Kearnes, S., Yang, L., Li, L., Kohlhoff, K. and Riley, P. Tensor field networks: Rotation- and canssation-equivariinn neural networks for 3d point clouds, 2018. 2
Schran, C., Thiemann, F. L., Rowe, P., Müller, E. A., Marsalek, O., and Michaelides, A. Machine learning potentials for complex aqucous systems made simple. Proceedings of the National Academy of Sciences, 118 (18))2210077111 2021. 22
Unke, O. T. and Meuwly, M. Physnet: A neural network for predicting energies, forces, dipole moments, and partial charges. J. Chem. Theory Comput., 15:3678-3683 2019.
Schütt, K., Kindermans, P.-J., Sauceda Felix, H. E., Chmiela, S., Tkatchenko, A., and Müller, K.-R. Schnet: A continuous-filter convolutional neural network for model- ing quantum interactions. Adv. Neural Inf. Process. Syst., 30:991-1001, 2017. 5
2
Unke, O. T., Chmiela, S., Sauceda, H. E., Gastegger, M., Poltavsky, I., Schütt, K. T., Tkatchenko, A., and Müller, K.-R. Machine Learning Force Fields. Chem. Rev, 121: 10142-10188 2021. doi:
Schütt, K. T., Unke, O. T., and Gastegger, M. Equivariant message passing for the prediction of tensorial properties and molecular spectra. Int. Conf. Mach. Learn., 139: 9377-9388, 2021. 2, 22
111.
van der Oord, C., Sachs, M., Kováes. D. P., Ortner, C., and Csányi, G. Hyperactive learning for data- driven interatomic potentials. ирj Comput. Mater, 9 (1):168, Sep 2023. ISSN 2057-3960.
Shapeev, A. V. Moment tensor potentials: A class of sys- tematically improvable interatomic potentials. Multi- scale Model. Simul., 11111111117731 2016. doi: 10.1137/ 15M1054182
doi: 10.1038/ 041524-023-01104-6 URL https:ts//// /tte:://dooiieee /ttp://oo..oo....///// 1038/ 0tt5:/44122--02--111100
12

---

Physics-Informed Weakly Supervised Learning for Interatomic Potentials
Vandermause, J., Torrisi, S. B., Batzner, S., Xie, Y., Sun, L., Kolpak, A. M., and Kozinsky, B. On-the-fly active leamn ing of interpretable bayesian force fields for atomistic rare events. npj Comput. Mater, 6:20, 2020. 1, 14
Verlet, L. Computer" experiments" on classical fluids. i. thermodynamical properties of lennard-jones molecules. Physical review, 159(1):98, 1967. 22
Yang, X., Song, Z.., King, I., and Xu, survey on deep semi-supervised learning, IEEE Trans. Knowl, Data Eng., 2022. 3
Zaverkin, V. and Kästner, J. Gaussian Moments as Phys- ically Inspired Molecular Descriptors for Accurate and Scalable Machine Learning Potentials. J. Chem. Theory Comput., 16:5410-5421, 2020. ISSN 1549-9618. doi:
10.1022/aacsjcccc.0000044 2
Zaverkin, and Kästner, Exploration of transferable and uniformly accurate neural network interatomic potentials using optimal experimental design. Mach. Learn.: Sci. Technol., 2(3):035009, 2021. doi: (0.1088/2632-2153/
abe294. URL http: http:///pppccccccccce/// op. org/ article/ /0t.:/000.000//222222/////6
Zaverkin, V., Holzmüller, D., Steinwart, I., and Kästner, Fast and Sample-Efficient Interatomic Neural Network Potentials for Molecules and Materials Based on Gaus- sian Moments. J. Chem. Theory Comput., 17:6658-6670, 2021. ISSN 1549-9618. 8. doi: 10000///as.ccc..c00057.
Zaverkin, V., Holzmüller, D., Steinwart, I,, and Kästner, J. Exploring chemical and conformational spaces by batch mode deep active learning. Digital Discovery, 1:605-620, 2022.
Zaverkin, V., Holzmüllerr, D., Bonfirraro, L., and Kästner, J. Transfer learning for chemically accurate interatomic neural network potentials. Phys. Chem. Chem. Phys., 25: 5383-5396, 2023. doi: 10.10/9/22200
Zaverkin, V., Alesiani, F., Maruyama, T., Errica, F., Chris- tiansen, H., Takamoto, M., Weber, N., and Niepert, M. Higher-rank irreducible cartesian tensors for equivariant message passing, 2024a. 2
Zaverkin, V. Holzmüller, D., Christiansen, H., Errica, F., Alesiani, F., Takamoto, M., Niepert, M., and Kästner, J. Uncertainty-biasse molecular dynamics for learning uniformly accurate interatomic potentials. npj Comput. Mater, 10:83, 2024b. 1,
13

---

Physics-Informed Weakly Supervised Learning for Interatomic Potentials
A. Related Work
Addressing Data Sparsity in MLIPs. Generating training datasets suitable for learning reliable MLIPs is challenging,
especially when considering unexplored molecular and material systems. Numerous computationally expensive calculations with either ab initio or first-principles approaches are required for the latter. To mitigate this challenge, active learning (AL) methods, which utilize prediction uncertainty, can be applied (Li et al., 2015; Podryabinkin & Shapeev, 2017; Vandermause et al., 2020; Shuaibi et al., b021b; Briganti & Lunghi, 2023; Zaverkin et al., 2024b). Furthermore, equivariant MLIPs often reduce required training dataset sizes through improved data efficiency (Batzner et al., 2022; Batatia et al., 2022).
B. Experimental Setup, Baselines, and Datasets
B.1. Experimental Setup
Code for Experiments. The code used to run our experiments builds upon the recent work (Fu et al., 2023) and extends it to integrate the latest Open Catalyst Project code (Chanussot et al., 2021). We adopt hyper-parameters from the Open Catalyst (OC) project, tuned to the corresponding OC dataset. Note that we do not use this dataset in the presented work, whose main focus is training general-purpose MLIPs that can be used to run molecular dynamics (MD) simulations and geometry optimization. However, the OC dataset has been designed to investigate the latter, making it less suitable for the current study, Our modifications include adjusting the learning rate scheduler, details of which can be found in our repository. For potential energy and force prediction, we utilize mean-absolute error (MAE) and Lg-norm (L2MAE) losses with coefficients of 1 and 100, respectively. More details on the model hyperparameters are provided in our repository. For the PITC and PISC loss functions, we use the mean square error (MSE) loss based on an experiment in section D.5. When noise is applied to inorganic material datasets with periodic boundary conditions, atoms outside the unit cell are wrapped back into the cell when noise is applied.
Training Details. For training MLIPs, we followed the setup in the Open Catalyst Project. We kept the mini-batch size consistent across all models, as shown in Table A1. We have chosen the mini-batch size based on the maximum memory needed by the most demanding models, such as eSCN and Equiformer v2. All experiments are performed on a single NVIDIA 1 GPU with 81.92 GB memory. To avoid overfitting, we stopped training when the validation loss stopped improving-the specific number of training iterations is provided in Table A2.
We used perturbation vectors ór drawn from uniform random distribution; see also section 4.4. Particularly, we defined 5r = <g. with each component of g drawn from a uniform random distribution in the interval (-1, The magnitude e is also drawn from a uniform random distribution and < fmax- This definition of or differs from the one in Eq. (6), improving the computational efficiency of PIWSL by avoiding the calculation of square root and division.
The remaining
are the coefficients for the PITC and PISC losses (Crrre, CPISc) and the maximum magnitude 'max of the perturbation vector őr; see Table A3. These hypee-parameters tuned using Optuna (Akiba et al., 2019) for PaiNN and Equiformer v2. We used 1000 configurations drawn randomly from the original ANI-1x dataset for training. Due to multiple local minima, Optuna identified several optimal hyper-parameter sets in each run. We selected the following representative combinations (CPrro, CPSCC Emax) = Case A: (1.2, 0.8, 0.025), Case B: (1.0, O, 0.01), Case C: (0.1, 0.01, 0.01), Case D: (1.2, 0.01, 0.025), Case E: (1.2, 0.01, 0.01), Case F: (1.2, 0.01, 0.015), Case G: (0.01, 0.001, 0.025), and Case H: 10.1, 0.01, 0.025). We selected the hyper parameters listed in Table A3 based on the validation dataset performance.
Splitting Datasets. We split the original datasets into training, validation, and test sets for our experiments. We shuffled the original datasets using random seed and selected the training datasets of predefined sizes. For validation, we selected the same number of configurations as in the training dataset if it exceeded 100 configurations; otherwise, we used 100
Table A1: Employed mini-batch sizes. We provide mini-hatch sized for all datasets and models employed in this work.
ANI-Ix
TiOg"
rMD17
LMNTO
Mini-batch size
6
16
" As explained in Table 2, the mini-batch size of the SchNet model was changed to 32 due to high RMSE values observed with a mini-batch size of four as the training dataset size increased. A more detailed discussion of the results for SchNet is provided in section D.
14

---

Physics-Informed Weakly Supervised Learning for Interatomic Potentials
Table A2: Total number of training iterations. We provide the total number of training iterations for all datasets and training set sizes employed in this work. The number in the parentheses demonstrates the corresponding total number of training epochs.
Nurain
ANI-Ix
Ti02
rMD17
LMNTO
50
7500 (900)
100
10,000 (600)
10,000 (400)
7500 (1200)
10,000 (400)
1000
40,000 (240)
10,000 (100)
10,000 (160)
10,000 (100)
10,000
100,000 (60)
100,000
400,000 (30)
1,000,000
400,000 (3)
5,000,000
420,000 (1)
Table A3: Optuna (Akiba et al., 2019): (Cprre, CPISC, Cmax)= == Case A: (1.2, 0.8, 0.025), Case B: (1.0, O, 0.01), Case C: (0.1, 0.01, 0.01), Case D: (1.2, 0.01, 0.025), Case (1.2, 0.01, 0.01), Case F: (1.2, 0.01, 0.015), Case (0.01, 0.001, 0.025), and
for the PIWSL loss. We selected the following hyper-parameter combinations using
Case H: (0.1, 0.01, 0.025).
Dataset
Size
Equiformer v2
eSCN
PaiNN
SpinConv
SchNet
50
A
C
B
A
A
ANI-Ix
1000
100
DDD
A
D
C
D
A
BB
DD
B
A
10,000
G
C
B
C
C
100,000
C
1,000,000
C
5,000,000
B
100
A
A
A
A
H
TiO2
1000
G
A
C
A
C
100
B
B
B
A
B
LMNTO
1000
B
A
B
B
B
configurations ensure sufficient validation size. For the rMD17 dataset, following (Fu et al., 2023), we used 9000 configurations validation dataset and another 10,000 for testing. We used the same test dataset across different sizes of the training datasets for fair performance comparison. We used 10,000 test configurations for ANI-1x 1x and 1000 for TiO, and LMNTO.
Training with Adversarial Directions. In our experiments, which defined perturbation vectors adversarially (see section E), we determined adversarial directions using Eq. (7). More concretely, we only considered the potential energy, i.e., Ypred and Ylabel, avoid Hessian calculations. addition, we considered the loss function for the potential energy as Laist Eq. (7). The expression of g Ldisis is then
gs V., (E E
(F, (8; F.ol (E E
(AI)
L
Note that we used the relation Vr, Lye Frof to obtain the final expression. Though ge can also be interpreted as a constant regarding atom positions. We have chosen this expression to avoid the case where the adversarial direction g points to F other than the very beginning of the training. Our experiments indicate that the employed expression is slightly better than its alternative. In our experiment, also randomly flip the sign of gs avoid overfitting to adversarial directions.
B.2, Baseline Methods
Data Augmentation with NodsyNode. In our experiments, we used the NoisyNode approach (Godwin et al., 2022) as one the baseline methods. This method aims improve the performance of MI. models by adding perturbation to
15

---

Physics-Informed Weakly Supervised Learning for Interatomic Potentials
node features, i.e., atomic coordinates, and makes ML models recover original labels. This approach enables ML models to be more robust noise in the data. Although the original method recommends adding decoder network to learn the denoising process, we do not utilize it following previous work (Liao et al., 2023) and add the perturbation vector to atomic coordinates similar to PIWSL losses, fixing energy and force labels. We implement the NoisyNode approach in our code. Thus, we can expect slightly different behavior compared to the recent work (Godwin et al., 2022; Liao et al., 2023) 8
Eetoor-Eppssiio---- Baade Weak Labels. Recent work proposed a similar Taylor-expansion-bar based weak label ap- proach (Cooper al.. 2020). Nonetheless, the loss different from the one in Eq. (3) as the authors used reference energy and atomic force labels to estimate weak energy labels E for perturbed atomic configurations Sör
Na.
Eref sJ
pref
Σ
Ör,, Fi.s) + O (11)111²)
(A2)
The trainable parameters of MLIPs are optimized by minimizing the weak label (WL) loss
((:0)=(E((S)))
Nut
LWL (S:0) = (E(Sor:0) 0),
(rr, i,5)
(A3)
Figure (a) illustrates the corresponding approach (Cooper et al., 2020), which computes the energy of a perturbed atomic configuration using a Taylor expansion based on reference energy and atomic force labels. This approach was originally applied to train MLIPs without explicit force labels.
B.3. Description of the Datasets
ANI-1x Dataset. The ANI-Ix dataset a heterogeneous molecular dataset and includes 63,865 organic molecules (with chemical elements H, C, N, and O) whose size ranges from to 64 atoms (Smith et al., 2020). The ML model requires learning total energies and atomic forces for various molecules and their conformations. Total energies and atomic forces are obtained through DFT calculations.
Ti02 Dataset. Titanium dioxide (TiO2) is an industrially relevant and well-studied material. TiO, dataset includes 7815 bulk structures of several TiO2 phases whose reference energies and forces are obtained through DFT calculations (Artrith & Urban, 2016). The number of atoms in a single configuration ranges from 6 to 95.
rMD17 Dataset. The rMD17 dataset includes ten small organic molecules, including 100,000 configurations obtained by running MD simulations for each (Christensen & von Lilienfeld, 2020). The ML model requires learning the total energies and atomic forces for each molecule. In this revised version of the MD17 dataset, the molecules are taken from the original MD17 dataset (Chmiela et al., 2017; 2018). However, the energies and forces are recalculated the PBE/deI2-SVP level of theory using very tight SCF convergence and a very dense DFT integration grid.
LMNTO Sataset. The Li-Mo-Ni-Ti oxide (LMNTO) is of technological significance as potential high-capacity positive electrode material for lithium-ion batteries. It exhibits substitutional disorder, with Li, Mo, Ni, and Ti all sharing the same sub-lattice. This dataset includes LMNTO with the composition LigMo,,ii,Ti.O0 and configurations obtained from an MD simulation, resulting in approximately 2600 structures in total (Cooper et al., 2020).
MD22 Dataset. The MD22 dataset (Chmiela et al., 2023) includes seven larger organic molecules, such as a small peptide and wallle-walled nanotube, whose size ranges from 42 to 370 atoms. The dataset consists of MD trajectories sampled at temperatures between 400 and 500 K. The ML model requires learning the total energies and atomic forces for each molecule. The energies and forces are calculated at the PBE+MBD level of theory.
'Note that NoisyNode assumes the unperturbed state to be the equilibrium structure, which may have contributed to the limited performance improvements observed in our experiments. Recently, efforts have been underway to develop extended version of Node tailored for non-equilibrium structures, such as (Liao et al., 2024).
16

---

Physics-Informed Weakly Supervised Learning for Interatomic Potentials
Table A4: Energy (E) and force (F) root-mean-square errors (RMSEs) for the ANI-Ix dataset. The results are obtained by averaging over three independent runs. Energy RMSE is given in kcal/mol, while force RMSE is in kcal/mol/A.
N train 50 Noisy Nodes
train 10, Noisy Nodes
Baseline
PIWSL
Baseline
PIWSL
E
90.08 + 1.24
76.83
0.75
83.90 + 2.82
24.88 + 0.01
24.86 + 0.00
24.88 + 0.00
Schnet
F
35.49 + 0.36
31.13 = 0.13
35.30 + 0.87
13.36 + 0.01
13.36 + 0.00
13.36 + 0.00
E
212.64
1.14
440.11
11.68
121.36 4.13
19.14 + 0.38
165.25 + 4.87
14.10 + 0.14
PaiNN
F
22.61 + 0.04
22.50 = 0.22
20.83 + 0.28
8.24 = 0.10
9.22 + 0.09
7.89 = 0.02
E
222.75 + 7.12
219.85 6.99
175.38 + 9.77
19,42 = 0.67
46.31 + 10.31
18.81 + 0.60
SpinConv
F
24.88 + 0.88
24.61 = 0.35
25.12 + 0.58
10.31 + 0.33
10.78 + 0.66
9.94 0.12
E
517.17 + 31.98
583.90 = 33.04
454.40 + 11.10
12.65 + 0.63
165.30 + 33.11
10.66 + 0.31
eSCN
22.51 + 0.09
24.04 = 0.15
22.28 + 0.08
5.11 = 0.30
11.51 + 0.23
4.35 0.15
E
498.58 + 17.44
630.32 + 0.32
433.88 + 79.63
8.03 = 0.21
970.95 + 236.90
7.77 = 0.14
Equiformer
22.86 + 0.04
22.92 = 0.00
22.72 + 0.04
2.97 = 0.00
29.28 + 5.63
2.98 0.00
C. Differences in Gradients for Physics-Informen Losses
The following considers the gradients of the proposed two losses. First, considering squared emrrrs, we obtain the following gradients the loss in Eq. (A2) with respect to trainable parameters
dCwL d0
No..
dE (Sori 0)
=
E (S Es
Cr Feef 1,
(A4)
d0
In contrast, for the PITC loss in Eq. (3) we obtain
dCprTc
Nar
F. (
frr F; (S:0))
d0
(A5)
dE (Sox: 0) dE (S: 0) d0
No. dōr,, F; (S;
d0
d0
The above equations indicate that the direction of the derivative of the PITC loss Eq. (A5) is different from that the weak label loss because of the incorporation of the predicted potential energy at the original and the force at the reference point. The gradient of PISC loss in Eq. (4) reads
dCPISC d0
No. or, F, (S; 0))
(8;
(S 0) +
(81 , (Sor; 0))
(A6)
dE (S:0)
No
d(rr; F, (S: 0))
dE (Sar:; 0)
Nat d(ōr" (Sor:; 0))
d0
d0
d0
d0
D. Experiments
D.I. Benchmark Results
The following section provides additional results, complementing those provided in the main text.
Additional Results for ANI-1x. Table A4 provides results for ANI-Ix dataset and a training set sizes of 50 or 10,000; see also figure 2. The table demonstrates a considerable reduction of energy and force RMSEs for models trained using small training dataset sizes of configurations. Furthermore, we find around 5 to 25 error reduction for a larger training set size of 10,000, indicating the effectiveness of the PIWSL method for relatively large training set sizes. Finally, we provide
17

---

Physics-Informed Weakly Supervised Learning for Interatomic Potentials
Table A5: Energy and force erorrs of PaiNN model trained on the ANI-Ix dataset with 100,000, 1,000,000, and 5,000,000 samples. The results are obtained by averaging over three independent runs. Energy errors are given in kcal/mol, while force errors are in kcal/mol/A.
Nirain
Force MAE
Force RMSE
Energy MAE
Energy RMSE
Baseline PIWSL
100,000
0.92 0.00
3.70 E 0.01
4.28 + 0.15
6.14 0.21
0.91 + 0.01
3.72 + 0.04
4.07 + 0.17
5.83 0.20
Baseline PIWSL
1,000,000
0.67 = 0.00
2.74 + 0.01
4.90 + 0.23
6.56
0.26
0.68 + 0.00
2.77 + 0.04
4.48 + 0.05
6.06
0.01
Baselinc PIWSL
5,000,000*
0.53
2.18
3.94
5.24
0.55
2.24
3.25
4.75
Because of the computational cost, we performed only one training in the case of 5,000,000 training samples.
Table A6: Energy (F) and force (F) root-mean-squar errors (RMSEs) for the TiO2 dataset obtained for the SchNet model with a mini-batch size of four. The results are obtained by averaging over three independent runs. Energy RMSE given in kcal/mol, while force RMSE is in kcal/mol/A.
Narain 100 Noisy Nodes
Ntrain 1000 Noisy Nodes
Baseline
PIWSL
Bascline
PIWSL
E
18.85 + 0.00
17.48 + 0.00
17.58
0.00
35.58 + 0.00
58.08 + 18.44
15.28 + 0.12
SchNet
F
2.74 = 0.00
2.51 + 0.00
2.74 = 0.00
6.54 - 0.00
18.40 0.00
3.61 = 0.27
a result of PaiNN model trained on ANI-1x dataset with 100,000, 1,000,000, and 5,000,000 samples in Table A5, which demonstrates that PIWSL still improves the performance around 5% to 10%of the energy RMSE.
Additional Results for SchNet Applied to TiO In Table 2, we set the mini-batch size to 32 for training the SchNet model. This adjustment was made because training SchNet with small mini-batch size of four increases RMSE values with growing training dataset size. Table A6 demonstrates the performance of the SchNet model for mini- size of four. Table A7 provides the results obtained for the SchNet model with mini-batch size of four for the following training set sizes: 100, 200, 500, and 1000. This figure demonstrates that SchNet, with a mini-batch size of four, reaches its best performed with Nevain = 200. These results indicate the difficulty of learning training data statistics from small mini- batihs, probably due to the limited expressive power of SchNet.
Results for LMNTO. Table A8 presents RMSE errors for LMNTO (Cooper et al., 2020). PIWSL shows the error reduction for most cases for this benchmark dataset, especially for small training set sizes (i.e., a training set of 100 configurations).
Molecular Dynamics Trajectories for Large Molecules - MD22. Table A9 evaluates the impact of PIWSL on datasets containing conformations of single large molecule. For this purpose, we selected the buckyball catcher molecule with altoo = 148 atoms. To demonstrate the applicability of PIWSL, we trained a MACE model (Batatia et al., 2022) to prove that PIWSL enhances even the performance a recent state-of-the-ar model. The model structure and training configuration were based on those provided in the official repository" with slight modifications: "max__" was set to one and mini-batch size was adjusted to four. Following the original training setup (Chmiela al., 2023), we randomly sampled 600 configurations for the training dataset and 400 for the validation dataset and retained the remaining 5102 configurations for testing. To further validate PIWSL's effectiveness in sparse data scenarios, prepared smaller training dataset comprising only 50 configurations while keeping the validation dataset size unchanged. The model was trained for 450 and 800 epochs for the 600-sample and 50-sample training datasets, respectively. The results presented in Table A9 demonstrate that our approach remains effective on average, particularly in the sparse data regime. In this study, the coefficient of the PITC and PISC losses are set as 0.01 and 0.001 with émax = 0.01.
"https://mane doceaaacceeos....o readthedocs..... wttp://ateet//eeeeetttttt/tttaet htt.ec/eei..s..eeeeaaeeee///eaer
html
18

---

Physics-Informed Weakly Supervised Learning for Interatomic Potentials
Training dataset size dependence of SchNet with a mini-batch size of four. The results are presented for the T dataset and are obtained by averaging over three independent runs. Energy RMSE given in kcal/mol, while force RMSE is in kcal/mol/A.
Table A7:
N train 100
Nerain 200
Ntrain 500
Nraai 1000
E
18.85 + 0.00
16.28 + 0.00
24.42 + 0.00
35.58 + 0.00
SchNet
F
2.74 + 0.00
2.56 + 0.00
4,43 + 0.00
6.54 + 0.00
Table A8: Energy (E) and force (F) root-mean-square errors (RMSEs) for the LMNTO dataset. The results are obtained by averaging over three independent runs. Energy RMSE is given in kcal/mol, while force RMSE is in kcal//ol//..
Nrain = 100
Ntrain = 1000
Model
Baseline
NoisyNode
PIWSL
Baseline
NoisyNode
PIWSL
E
4,46 + 0.00
6.10 + 0.00
4.45 + 0.00
3.09 + 0.00
3.25 + 0.00
3.09 + 0.00
SchNet
F
9.24 + 0.00
8.31 + 0.00
9.24 + 0.00
5.09 + 0.00
5.21 0.00
5.09
0.00
E
6.91 + 0.02
7.09 + 0.04
5,99 + 0.02
3.26 + 0.01
4.61 + 0.03
2.98 = 0.01
PaiNN
F
4.75 + 0.00
7.20 + 0.01
4.75 + 0.00
2.03 + 0.00
2.55 + 0.00
2.03 0.00
E
7.90 + 0.00
7.83 + 0.04
7.83 + 0.01
4.90 + 0.33
7.20 + 0.06
3.95 = 0.02
SpinConv
F
4.63 + 0.01
5.14 + 0.04
4.71 + 0.02
1.81 + 0.01
2.33 + 0.00
1.74 + 0.00
E
7.92 + 0.00
7.92 0.00
1.92
7.93 + 0.00
7.93 0.00
6.40 = 0.14
eSCN
F
4.67 + 0.01
7.59 0.02
4.64 0.01
0.01
1.54 + 0.00
1.98 + 0.06
1.53 = 0.00
E
7.40 + 0.03
7.92 + 0.00
7.32 + 0.08
3.57 + 0.05
7.04 0.03
3.60 0.02
Equiformer v2
F
4.26 + 0.00
7.60 + 0.02
+
1.34 + 0.00
1.99 + 0.00
1.34 + 0.00
In addition to the experiment of training from scratch, we conducted an additional study using the MACE foundation model to evaluate whether PIWSL. could be effectively applied in the context of foundation models with finetuning. Specifically, we utilized two foundation models: MACE-MP ("large") (Batatia et al., 2023), a universal model, and MACE-OFF "'large_off") (Kovács et al., 2023), a model designed for organic force fields. Training was performed on the buckyball catcher molecule from the MD22 dataset. To simulate finetuning on a smaller dataset, the models were trained using 50 samples, consistent with the previous experiment. The official settings from the respective repositories were used, with adjustments made the banc- size (set to 4) and the number of epochs (set to 100).
The results, presented in Table 4, demonstrate that PIWSL is effective for the fine-tuning of foundation models. Moreover, the result indicates that the MACE-MP model provides worse results than MACE-OFF, emphasizing the importance of selecting an appropriate pre-trained model. For this experiment, we employed the second-order PITC loss and the two-point consistency loss defined Eq. (A14) and Eq. (A11), with the coefficients set to 0.08.
Table A9: Energy (E) and force (F) mean-absolute errors (MAEs) for the MD22 dataset. We have chosen buckyball catcher for our experiments. The results are obtained by averaging over three independent runs. Energy MAE is given in kcal/mol/atom, while force MAE is in kcal/mol/A..
Nuain
50
Ntrain 600
Dataset
Model
Baseline
PIWSL
Baseline
PIWSL
Buckyball catcher
E
1.046 1 0.095
0.745 1 0.031
0.587 1 0.107
0.548 1 0.015
MACE
F
0.294 + 0.002
0.290
0.002
0.082 + 0.001
0.082 + 0.001
19

---

Physics-Informed Weakly Supervised Learning for Interatomic Potentials
D.2. Different Configurations for the Physics-Informed Spatial-Consistenny Loss
D.2.1. TRIANGLE-BASED
Table A10: Results for different configurations of the PISC loss. The presented numerical values are the root mean square enrors (RMSEs) for the ANI-Ix dataset (Smith et al., 2020). Energy (in kcal/mol) and force (in kcal/mol/A) errors obtained by averaging over three independent runs. All models trained using 1000 configurations. The case 1, 2, and 3 correspond to Eq. (4), Eq. (A8) and Eq. (A9), respectively.
Model
Baseline
PISC (Case 1)
PISC (Case
PISC (Case 3)
60.11
45.24
46.32
57.29
PaiNN
13.10
12.33
12.42
13.28
In section 4.2, we consider the following form of the PISC loss
LPISC (5: = E (Sor: E (Ssri 0)
Σ (or, F. (Sor;
(A7)
where ōr, 88', òr" are related as ōr + .r" 8r. In this section, as a variant of Eq. (4), we also consider the following three PISC losses
Na (rr, F, (S;0)), E (Sor);
0)))
LpisC,Cuse2 (8:0) - (S: Σ
Σ (or" F, (Seri; (Sp::)))))
(A8)
Nat
Lass 3 (S:0) =( E (Sór: 0), (Stri 0)
>
(-rr., F; (Sór: 0))
(A9)
where the point at r- or the point where PIRC loss is imposed (see Eq. (3)). The results are provided Table A A10 and indicate that Eq. (4) (Case 1) shows a better performance than the other cases for both the potential energy and the force predictions. In this study, we used the ANI-Ix dataset with 1000 training samples different from the one used train the model used in the main body avoid overfitting the test dataset. For the coefficient of the PITC and PISC losses, used 0.1 and 0.001 with max 0.01.
D.2.2. FURTHER VARIATIONS
Two-Point Spacial Consistency. The flexibility of the PISC loss allows us to explore additional forms of spatial consistency. For example, instead of using a triangular configuration, we can impose spatial consistency between two points at r and r + ôr, leading to the following expression:
Nar Σ
01:).
Er, F. (Sér: (S)r::)))
LPISC.2pt (8:0) = (S: (Str: 0)
(-r,,
(A10)
While not thoroughly investigated, we empirically observed that this loss delivers competitive performance when applied with the same coefficient value as the PITC loss.
Two-Point Spacial Consistency with Label. Since the training sample includes label, can utilize this label information instead of predicting the potential energy of the original conformation to enforce two-point spatial consistency.
Nas
LpiSC,2pewl (8:0) = EEse E (Sor; 0)
Σ
(-r,,
òr, F, (Str; 0))
(A11)
We observed that this loss function performs well when applying the MACE model to datasets without force labels, as discussed in subsection 5.4.
20

---

Physics-Informed Weakly Supervised Learning for Interatomic Potentials
Second-order Term Consideration for PITC Loss. In Eq. (2), we considered only the first-order Taylor expansion. Here, we introduce straightforward approach approximately account for the second-order term. Using the energy and forces + ôr, the second-order derivative of the potential energy can be approximated using the explicit finite difference method as:
5r 0p0,E = 6r 8r F = (őr, F (Sbr: 0) - F (S:
(A12)
Then, Eq. (2) with the above second-order term can be written as:
E (Stri 8) 2e E (S:
Nat
Kör; F, (S:0); + kand (ōr;, F; (Sor; 0) - FF; (S:0)) + O (1]r)11),
Nos
E (8;
Σ
frr, (1- (2nk Fs; (S: k2na (S 0)) O (1 13)
(A13)
where k2nd is numeric parameter that controls the contribution of the second-order term. Note that setting kand 0.5 recovers the exact second-order expression, while knnd 0 reverts to the first-order approximation.
Using Eq. (A13), the expression of the second-orde PITC loss is given as:
Nat
LPIrc (S:0) = ( (Srr:
Σ
frr (1 k2nd F, (S:0) + kendF, (Sori 0))):
(A14)
Note that the second-order term becomes significant under the following conditions: (1) the accuracy of the MLIP model surpasses the contribution of the first-order term, and (2) the original conformation is near the equilibrium state. The latter can be understood as follows. Considering spring model as a two-body interaction, the potential energy can be expressed as:
(ir) khond (r r,)
(A15)
where kisond is the constant characterizing the strength of the two-body interaction, and ro denotes the equilibrium bond length. Introducing a small perturbation or in the bond length, the potential energy becomes:
Aboud ro ōr)² V(r) + 2kbond (r ro; ör) kbond 5r
(rr + ör)
(A16)
2
At the equilibrium state (r = ro), the above equation demonstrates that the second-order term becomes dominant. Conse- quently, the second-order accuracy of the PITC loss function becomes crucial in such scenarios.
PITS for Curl of Forces Another potential direction is enforcing reduction in the curl of the forces. This can be achieved by leveraging Stokes' theorem: Js V x F dS = foys F dl = where Σ represents a specific surface regime, and OΣ denotes its boundary. Similar to the PISC loss, the right-hand side of this equation can be effectively described by considering a triangular configuration, where the midpoints of the three sides correspond to r, r ör, ôr'.
D.3. Detailed Setups for Qualitative Analysis
1.3.1. C-H POTENTIAL. ENERGY PROFILE OF ASPIRIN
This section describes the detailed setup and procedure for section 5.3. First, we trained PaiNN with and without PIWSL losses using the aspirin data from rMD17 with training set sizes of 100 and 200. For PIWSL. we used (CPITC, CPISC, m (1.2, 0.01, 0.015). The other experimental setups are the same as for rMD17 experiments presented in section B.I. We used the PaiNN model with gradient-basse forces obtain the reference model and tuned the model pprrmmparreeter with Optuna (Akiba et al., 2019). The obtained models' performance is provided in Table A11. Then, we prepared the aspirin molecule structures, including the corresponding atomic coordinates and atomic types. For these structures, we perturbed one of the C-H bonds with a bond length from 0.8 to 1.8 Å. We prepared 100 structures and estimated the corresponding potential energy with the pre-trained models. The aspirin data provided in our publicly available source code.
21

---

Physics-Informed Weakly Supervised Learning for Interatomic Potentials
Table All:
Performance of PaiNN employed in figure 2 (c, d). All the models other than the reference model (Nirain 1000) use the OC20's hyper- parammerressrss.. For the reference model, we tuned the hyper-parameter of PaiNN model following the original paper (Schütt et al., 2021).
train
100
Ntrain Baseline
200
N Ntrain 1000 Baseline
Baseline
PIWSL
PIWSL
E
6.55
5.64
5.11
4.48
0.68
PaiNN
F
7.38
7.36
3.95
3.97
1.44
D.3.2. MD SIMULATION STABILITY ANALYSIS
NVE-Ensembles This section describes the detailed setup and the procedure for our analysis of MD simulations in section 5.3. Because our implementation builds upon the source code provided by Fu et al. (2023), we used their scripts for performing MD simulations. However, we added a minor modification to enable MD simulations in the microcanonical (NVE) statistical ensemble, i.e., the particle position and velocity are updated with velocity Verlet algorithm(Verlee, 1967) 10 We set the initial temperature to 300 K and the integration time step to 0.5 fs for all simulations. As defined by Fu et al. (2023), the stability of an MD simulation for a target molecule is defined as the time T during which the bond lengths satisfy the following condition
max (T) - (x,))x,())I) وويو > A, (K,))EB
(A17)
where B denotes the set of all bonds, {i.j} denote the pair of bonded atoms, and b,, denotes the equilibrium bond length. Following Fu al. (2023), we set A = 0.5Å. This definition indicates when the molecule experiences significant structural changes during the MD simulation.
We trained PaiNN and Equiformer v2 models with and without PIWSL losses using the aspirin data from rMD17. We used training set sizes of 100 and 200 for PaiNN-GF and 1000 for PaiNN and Equiformer v2 with direct force. The corresponding stability values are presented in Table A12. The hyperparameters for the PIWSL loss are provided in Table A13. To train direct force model with 1000 samples, we used second-order PITC and PISC losses because of their good accuracy. The results for the stability of the PaiNN (direct and gradient-basee based force) and Equiformer v2 models are shown Table A14, To select the hyperparameters in Table A13, a series of MD simulations with a fixed random number seed for the initial atomic velocities was used. For our final stability results in Table A12, a series of MD simulations with three different random seeds for velocities and the selected hyperparameters
used.
NVT-Ensembles To investigate the effect of thermostats, we also performed MD simulations in the canonical (NVT) statistical ensemble, where temperature is maintained constant. To keep the temperature constant, we used thermostat (Nosé, 1984; Hoover, 1985). The initial and target temperatures were both set to 300 K for all simulations. The integration time step was set to 0.5 fs and the characteristic parameter T for the thermostat is set to 20 fs. The result, shown in figure A1, demonstrate that the thermostat stabilizes the simulations by mitigating the increase in kinetic energy.
D.4. Training setup for MDI7-CCSD(T) Experiments
In this section, we provide the training setup of finetuning of MACE-OFF discussed in section 5.4. The considered model is MACE-OFF (large) (Kovács et al., 2023) which is the MACE model pretrained on SPICE (Eastman et al., 2023), QMugs (Isert et al., 2022), and liquid water (Schran et al., 2021) datasets. The foundation model is finetuned on the aspirin molecule data in CCSD dataset (Chmiela et al., 2018) whose potential energy is obtained at quantum-chemical CCSD level of accuracy. The data includes 1500 samples which are split into 950/50/500 train/validationntest datasets. To emulate general coupling-cluster method dataset, we only use potential energy label for training. We utilize the set up provided in the official repository" !! with modifying learning rate from 10-2 to 10 to improve the performance. For the model trained scratch, we use the same architecture with setting energy weight to 40 and learning rate to 10-3 to improve the performance. For PIWSL, we used the second-order PITC and 2pt-with-label PISC losses Equation A10 with the following parameters:
10 Note that the total energy conservation necessary for the microcanonical statistical ensemble is in general not perfectly satisfied due to the numerical error, in particular, when the force is not calculated as the curl of the force. "1 httpps: :otps://mmacess /maccm doce. readthhedoss. 1 / lttest/ gui de/ felefiineee winefunnnnnn ing. html
22

---

Physics-Informed Weakly Supervised Learning for Interatomic Potentials
Table A12: Stability of the models employed in the MD analysis. The presented numerical values are the stability defined by Eq. (A17) measured in ps. The results are obtained as an average over three different random seeds for the initial velocity of the atoms the target aspirin molecule. "GF" denotes the gradient-based force prediction.
N train 100
N Miaain 200
Ntrain 1000
Baseline
2.68 = 0.13
PaiNN
PIWSL
4.65 = 0.72
Baseline
14.53 + 8.61
Equiformer
PIWSL
24.65 + 8.24
Baseline
3.25 + 3.98
220.5 + 137.7
PaiNN-GF
PIWSL
15.07 + 10.09
267.7 56.0
Table A13: Hyper-parameters for the PIWSL loss used in the MD analysis. We used the following hyper-parameter for MD simulation analysis: (Cprte, CPISC, mmx )= Case (0.01, 0.001, 0.025), Case 8: (1.2, 0.01, 0.01), Case (1.2, 0.01, 0.025). Case &: (1.2, 0.01, 0.015). Case e: (0.1, 0.01, 0.01), and Case Ç: (1.0, O., 0.01). "GF" denotes the gradient- based force prediction.
Dataset
Size
Equiformer v2
PaiNN
PaiNN-GF
rMD17
100
a
(Aspirin)
200
3
1000
?
(CPrte, CrISc, c) 40.55,4.5.0..
4.5, 0.01).
D.5. Metric Dependence of PITC
Table A15 provides the result of the metric dependence of PIWSL. For simplicity, we only consider the PITC loss (the coefficient of the PITC and PISC losses set as 0.1 and 0). For the ReLU metric, consider
Na
LRELU (5:0) = ReLU (S:
fr, F, (S:0)) - (S (Sér: 0) ||ór|12
0)
(A18)
This metric is zero when the difference between the two terms is less than the second-order term in ôr. The results indicate that taking the second-order term into account does not improve the performance (see PITC MAE Loss and PITC ReLU Loss results), and the MSE loss function shows the best performance. In this study, we used the ANI- Ix dataset and the 1000 training samples. These samples differ from the one used train the model the main text to avoid overfitting the test dataset.
D.6. Perturbation Magnitude Dependence
In this section, we provide the result the perturbation magnitude dependence of PIWSL, i.e., ||ór|| = e. First, we consider only the Ist-order PITC loss (the coefficient of the PITC and PISC losses are set as 0.1 and 0.0). The results are provided in Table A16 and demonstrate that the longer perturbation vector length is fruitful for force predictions. However, values that are too large are harmful to predicting potential energy. In this study, we used the ANI-Ix dataset and the 1000 training samples. These samples differ from the one used to train the model in the main text to avoid overfitting the test dataset.
Second, we examine the sensitivity of the second-order PIWSL in the finetuning of MACE-OFF on the MD17 (CCSD) dataset, as discussed in section 5.4. The results, presented in Table A17, indicate that the model's performance is not strongly affected by the choice of maxx even in the second-order setting.
23

---

Physics-Informed Weakly Supervised Learning for Interatomic Potentials
Table A14: Energy and force errors for the models employed in the MD analysis. The presented numerical values are the root-mean-squee errors (RMSEs) energy (E) and force (F). Energy RMSE is given in kcal/mol, while force RMSE is kcal/mol/A, "GF" denotes the gradient-bane based force prediction.
N train 100
itain
200
N Mtrain Baseline
1000
Baseline
PIWSL
Baseline
PIWSL
PIWSL
E
6.55
5.64
5.11
4.48
2.30
0.99
PaiNN-DF
7.38
7.36
3.95
3,97
1.63
1.61
E
4.79
4.64
4.92
4.82
1.39
1.19
Equiformer v2
F
4.86
4.90
2.50
2.42
0.76
0.75
6.05
6.03
6.01
6.02
PaiNN-GF
6.41
6.33
3.50
3.53
Table A15: Metric dependence of PITC. The presented numerical values are the root mean square errors (RMSEs) for the ANI-Ix x dataset (Smith et al., 2020). Energy (in kcal/mol) and force (in kcal/mol/A) errors are obtained by averaging over three independent runs. All models are trained using 1000 configurations. MAE refers to the mean absolute error, and MSE denotes the mean square error.
Model
Baseline
PITC MAE Loss
PITC MSE Loss
PITC ReLU Loss
E
60.11
58.84
47.09
60.47
PaiNN
F
13.10
13.18
12.19
13.06
D.7. Dependence of PITC on the Number of Perturbed Atoms
This section provides the result of the perturbed atom number dependence of PIWSL. For simplicity, we only consider the PITC loss (the coefficient of the PIRC and PISC losses are set as 0.1 and 0). In this study, we randomly selected atoms in a training sample following the ratio of O %, %, %, 50 %, 75 %, 90 %, 100 %. The results are provided in Table A A18, which indicates that around 75% to 100% ratio cases result in the best performance for the force and the potential energy prediction. However, the number dependence is rather complicated. Therefore, in the main text, perturbed all the atoms (100 %) as conservative choice. In this study, we used the ANI-1x x dataset and the 1000 training samples. These samples differ from the one used to train the model in the main text to avoid overfitting the test dataset.
D.8. Dependence on the Number of Training Iterations
To show the effectiveness of our approach even in the case of longer training, we provide the result of the dependence of PIWSL on the number of training iterations. In this study, we performed training twice as long as in the main text, that is, 80,000 iterations for ANI-Ix with 1000 training samples. The results are provided in Table A19 and indicate that our approach performs better in the longer training case. On the other hand, the training without PIWSL shows an overfitting to the validation dataset, reducing its performance compared dataset and the 1000 training samples. These samples differ from the one used to train the model in the main text avoid overfitting the test dataset. The coefficients of the PITC and PISC losses are 1.2 and 0.01, respectively.
shorter training case. In this study, we used the ANI- 1x
In Table A20, we also present the performance dependence on training length (measured in epochs) of MACE-OP model finetuned on MD17 aspirin data with CCSD label. The results demonstrate that PIWSL achieves substantially better performance than the baseline, even when the latter is trained for ten times as many epochs.
D.9. Additional Experiments with Cradient-Based Forces
In this section, we provide the result of the training with the gradient-basee force predictions. The results are provided in Table A21 and demonstrate that our PIWSL loss enables a better force prediction, even in the case of gradient- based force predictions. These results also indicate that our PIWSL method can improve the ML model performance in the case of MI.IPs commonly applied in computational chemistry and materials science. We consider that this partly due the
24

---

Physics-Informed Weakly Supervised Learning for Interatomic Potentials
Equiformer
PaiNN
300
300
Stability [ps]
Stability [ps]
200
200
100
100
0
0
100
200
1000
100
200
1000
Training set size
Training set size
Baseline
PIWSL
Figure A1: Stability analysis of the MLIP models during MD simulations. Stability during MD simulations is assessed for the baseline MLIP models and those trained with PIWSL. All results are obtained for the aspirin molecule and MD simulations in the canonical (N V T) statistical ensemble. We measure stability during MD simulations according to (Fu et al., 2023).
Table A16: Perturbation magnitude dependence of PITC. The presented numerical values the root mean square errors (RMSEs) for the ANI-Ix dataset (Smith et al., 2020). Energy (in kcal/mol) and force (in kcal/mol/A) errors are obtained by averaging over three independent runs. All models are trained using 1000 configurations.
Model
Baseline
Emax
0.0005
Emax
00.005
Emax
0.05
E
60.11
60.43
47.09
109.17
PaiNN
F
13.10
12.75
12.19
11.70
effectiveness of the weak label at r + ör as indicated by the WL results, which show improvement of the performance different from the case with the direct force branch (see also section E). We hypothesize that the further improvement results from the additional gradient calculation as indicated in Eq. (A5) and Eq. (A6). This observation also indicates that our PIWSL method can potentially improve other generic property prediction tasks by calculating their first derivatives in terms of the atomie coordinate and utilizing the proposed loss functions. this study, the coefficient of the PITC and PISC losses set as 0.1 and 0.01 with max 0.01. The weak label loss coefficient is set as 0.1.
D.10. Reducing Curl of Forces for Models with the Force Branch
In this section, we study effect our loss functions on the curl of forces the case of the model with the force branch. The results are provided in Table A22, which shows that our PITC loss reduces the curl of the predicted forces, allowing potentially better energy conservation during MD simulations. In this study, we used the ANI- Ix dataset and the 1000 training samples. These samples differ from the one used to train the model in the main text to avoid overfitting the test dataset. The hyper-parane parameters of the PITC and PISC losses are (Crrre, CrIsc, fmax) (1.2, 0.01, 0.025). It is theoretically possible to define loss function aimed at directly minimizing the absolute value of the curl of forces. However, this approach necessitates calculating the Hessian matrix, which requires a substantial memory cost given the limitations of current computational resources. Developing a method to train with such a loss function while mitigating memory requirements is promising direction for future research.
25

---

Physics-Informed Weakly Supervised Learning for Interatomic Potentials
Table A17: Performance dependence of the second order PIWSL on émax. The numerical values are MAES for the energy in kcal/mol and force kcal/mol/Aof MACE-OFF model finetuned on MD17-CC dataset.
Model
Baseline
Emax
0.005
6max
0.01
6max
0.02
E
1.21
0.73
0.72
0.75
MACE-OFF
6.90
4.20
3.77
4.04
Table A18: Dependence of PITC on the number of perturbed atoms. The presented numerical values are the root mean square errors (RMSEs) for the ANI-Ix dataset (Smith et al., 2020). Energy (in kcal/mol) and force (in kcal/mol/A) errors are obtained by averaging over three independent runs. All models are trained using 1000 configurations.
Model
Baseline
10%
20%
50%
75%
90%
100 %
E
60.11
46.68
52.37
54.51
46.94
45.92
46.32
PaiNN
F
13.10
13.03
12.62
12.16
12.14
12.24
12.42
E. Further Analyses of PIWSL
The following provides further analyses of our approach. We provide the results for Equiformer v2 and PaiNN since these models employ equivariant features and demonstrate high accuracy on the ANI-Ix dataset when trained using 1000 configurations.
E.1. Training MLIPs without Reference Forces
In the following, we explore scenarios where only potential energy labels are available. This situation commonly arises when calculating energy labels with chemically accurate approaches, such as CCSD(T)CBS (Hobza & Sponer, 2002; Feller et al., 2006), for which force calculation is infeasible. To consider practical applications, force by force branch (FB) and (2) predicting force as gradient of the potential energy (GF). The former enables fast force prediction and is popular in the machine learning community, while the latter requires additional gradient calculation but yields curl-free force predictions. It is popular in computational chemistry as it ensures the conservation of the total energy during MD simulations. The results are provided in Table A23; training without reference forces is achieved by setting the relative force contribution to zero Eq. (1). The PIWSL method consistently performs better than the baseline for the FB and GF cases. However, more sigmificant improvement in the force prediction performance is observed in the GF case. We attribute this phenomenon to the inherent nature of PIWSL, which requires consistency between the potential energy and atomic forces, discussed in section 5.3. This result aligns with expectations, confirming the capability of our PIWSL method to enable ML models to reduce the error in the predicted forces. Overall, PIWSL opens a new possibility for training MLIP models using highly accurate reference methods, such as CCSD(T)/CBS.
examine cases: (1) predicting
E.2. Comparing PITC with the
Weak Label Loss.
We compare the PIWSL method with the Taylor-expansion-based weak label (WL) approach (Cooper et al., 2020), whose loss function is presented in Eq. (A3). For simplicity, we only consider the PITC loss in Eq. (3), For fair comparison, we consider the following two cases. First, we train with reference forces and energies (w. RF). Second, we train the methods without reference forces and use only the reference energies. For the training with reference forces, we set the numeric coefficient of the PITC loss to 1.0; for the training without reference forces, the coefficient set to 0. Note that the WL loss without the reference force is calculated using the predicted force labels. The results are provided in Table A24.
Our PITC loss demonstrates the best accuracy in all cases, with and without the reference forces. Interestingly, PaiNN failed to learn the potential energy with the WL loss and reference forces. We hypothesize this to be due to the imbalance of the training between the energies and forces. Specifically, the WL loss trains only the potential energy, resulting in an inconsistency between the energy and force branches, which share the same readout layer that experiences more frequent updates using the potential energy. This hypothesis is supported by the results for the training without reference forces, where the error in energy is reduced compared to the baseline. A further validation in a similar experiment in the case of GF
26

---

Physics-Informed Weakly Supervised Learning for Interatomic Potentials
Table A19: Dependence on the number of training iterations. The presented numerical values are the root mean square errors (RMSEs) for the ANI-Ix dataset (Smith et al., 2020). Energy (in kcal/mol) and force (in kcal/mol/A) errors are obtained by averaging over three independent runs, All models are trained using 1000 configurations.
Model Iteration Number
Baseline
PIWSL
40,000
56.62 2.80
24.53 0.16
F
12.96 + 0.18
11.43 + 0.05
PaiNN
80,000
59.92 + 1.47
23.78 0.16
13.10 0.19
11.50 0.04
Table A20; Dependence on the number of training iterations of MACE-OFF finetuning, The presented numerical
values are the mean absolute emrors (MAEs) for the MD17-CC, in terms of energy (in kcal/mol) and force (in kcal/mol/A).
Model
Epoch Number
Baseline
PIWSL
100
1.21
0.72
6.90
3.77
MACE-OFF
1,000
E
1.19
F
6.24
provided in section D.9. However, the proposed PTTC loss still performs better here. In summary, PITC loss enables MLIPs leam energies and forces consistent with each other and does better than the previously proposed WL method.
E.3. A blating the Impact PITC and PISC Losses
We conduct an ablation experiment to analyze the impact PITC and PISC losses. Results Table A25 indicate that the PITC loss predominantly improves the accuracy of resulting models, especially for PaiNN. Using just the PISC loss does consistently improve accuracy but stabilizes training when combined with PTTC. This combined approach notably benefits Equiformer v2. For Equiformer v2, repeated the experiment five times reduce the effect from outlier the PITC loss.
E.4. Adversarial Directions for Perturbing Atomic Positions.
The following discusses the dependence of the PIWSL's performance selecting the vector dr in Eq. (5) employed to perturb atomic positions. The detailed implementation and setups are provided in section B.1. Table A26 compares the results obtained for a randomly-sample vector ör and for the one determined adversarially. The results demonstrate that both approaches improve the performance compared to the baseline without weak supervision, though the results might depend on the employed model.
E,5. Performance Difference Between First and Second order PIWSL
In this section, we present a comparison between first- and second-order PIWSL methods. We evaluate a foundation model, MACE-OFF, fine-tuned on the MD17 Aspirin dataset with CCSC(T) labels and the MD22 buckyball catcher dataset, as well as PaiNN-GE GF, a gradient-basee based force model trained on 1.000 samples from ANI-1x. The results are provided in Table A27.
Our findings indicate that the second-order term becomes more significant as model performance improves, particularly when force prediction error becomes small. This suggests that the first-order term, which encourages the model to predict the potential energy of neighboring conformations based first-order Taylor expansion, plays crucial role in promoting smoother potential energy surface. This effect arises because, in MACE and PaiNN-GF models, force predictions are derived as the gradient of the potential energy surface.
27

---

Physics-Informed Weakly Supervised Learning for Interatomic Potentials
Table A21: Results of PIWSL with gradient-based force predictions. The presented numerical values are the root mean square errors (RMSEs) for the A dataset (Smith et al., 2020). Energy (in kcal/mol) and force (in kcal/mol/A) errors are obtained by averaging over three independent runs. All models are trained using 1000 configurations.
Model
Baseline (GF)
PIWSL (GF)
WL (GF)
E
23.57 + 0.62
18.62 + 0.09
22.61 + 0.50
PaiNN
F
11.32 + 0.08
10.94 + 0.01
11.72 = 0.06
29.07 = 2.32
19.53 + 0.32
21.07 0.86
Equiformer
11.90 + 0.13
11.99 0.03
11.90 0.20
Table A22: Curl of forces for models with the force branch, The presented numerical values are the absolute values of the total curl of the force evaluated for the ANI-Ix dataset (Smith et al., 2020). Energy (in kcal/mol) and force (in kcal/mol/A) errors are obtained by averaging over three independent runs. All models are trained using 1000 configurations.
Model
Baseline
PITC
PaiNN
45.18 + 4.07
39.06 + 0.58
Equiformer
29.62 + 0.28
23.42 + 0.09
Table A23: Results for models trained on the ANI-1x dataset without reference forces. All models are trained using 1000 training samples. FB refers to the setting where the force branch estimates the force, and GF denotes the setting where the force is estimated by the gradient of the potential energy with respect to the atomic coordinates.
Model
Case
Baseline
PIWSL
FB
42.36 + 0.30
25.42 + 0.72
F
24.25 + 0.00
20.54 + 0.08
PaiNN
GF
E
41.83 + 1.81
29.71 + 0.55
F
83.36 + 2.85
24.02 + 0.95
FB
E
43.14 + 0.86
29.48 + 0.51
F
24.25 + 0.00
21.99 + 0.49
Equiformer
GF
E
42.55
+ 0.99
32.66 + 1.11
F
35.70 + 0.78
21.83 + 0.27
Table A24: Comparison PITC and the Tayllor-xppansoon-beeee weak label loss. WL (+FP) denotes the Taylor- expansion-based method using reference energies and either reference (w. RF) or predicted (w/o. RF) forces; see Eq. (A3). The listed values the RMSE values for energies in kcal/mol and atomic forces in kcal/mol//.. All models are trained on the ANI-1x 1x dataset using 1000 configurations, with (w.) and without (w/o.) reference atomic forces (RF).
Model
Case
Baseline
PITC
WL (+FP)
(w. RF)
56.62 1 2.80
30.94.1 0.56
81.86.1 9.39
12.96 0.06
12.04 0.04
14.54 0.12
PaiNN
(w/o. RF)
42.36 0.30
25.42+ 0.72
41.77 4.82
24.25 0.00
20.54 0.08
24.68+ 0.54
(w. RF)
E
54.524 4.52
23.16.t 0.19
31.02+ 3.99
F
10.10 0.00
10.03 0.05
13.43.1 0.92
Equiformer
(w/o. RF)
43.141 0.86
29.48 + 0.51
88.59+ 11.36
F
24.25 0.00
21.99 0.49
293.411 26.96
28

---

Physics-Informed Weakly Supervised Learning for Interatomic Potentials
Table A25: Results for models with direct-force trained the ANI-1x dataset with ablated weakly supervised losses. All models are trained using 1000 training samples. All results are obtained by averaging over three independent runs. Energy RMSE is given in kcal/mol, while force RMSE is given in kcal/mol/A.
Model
PITC
PISC
E
X
X
56.62 + 2.80
12.96 + 0.06
V
X
24.60 0.18
11.51 + 0.03
PaiNN
X
V
58.30 + 2.10
13.18 + 0.29
V
V
24.53 = 0.48
11.43 + 0.05
X
X
54.52
10.10 + 0.00
X
32.64 + 26.48
9.64 + 0.03
Equiformer
X
V
48.96 + 4.96
10.30 + 0.06
V
20.89 0.50
9.68 + 0.03
Table A26: PIWSL's performance dependence the atomic position perturbation vector. The numerical values RMSEs for the energy in kcal/mol and force in kcal/mol/A. All results provided for the ANI-Ix dataset and models trained using 1000 configurations.
Baseline
Random (Eq. (6)
Adversarial (Eq. (7))
E
56.62 = 2.80
24.53 = 0.48
33.67 + 1.12
PaiNN
F
12.96 = 0.18
11.43 = 0.05
12.74 + 0.14
E
54.52 + 4.52
23.16 = 0.50
20.54 0.21
Equiformer
F
10.10 = 0.00
10.03 = 0.03
9.93 + 0.04
Table A27: Performance difference between first and second order PIWSL. The numerical values are RMSEs (ANI- and MAE (MDI7-CC/MD22-1) for the energy kcal/mol and force in kcal/mol/Ā. MD17-CC denotes MD17 aspirin data with CCSD label, MD22-BB denotes MD22 buckyball catcher data, and ANI-Ix (IK) denotes 1000 samples from ANI-Ix.
Dataset
Model
Baseline
Ist-order PIWSL
2nd-order PIWSL
E
1.21 1: 0.00
0.90 1: 0.03
0.72 1 0.01
MD17-CC
MACE-OFF
F
6.90 + 0.01
4.59 + 0.16
3.77 + 0.13
1.16 + 0.15
1.16 0.28
0.99 + 0.05
MD22-BB
MACE-OFF
F
0.35 1 0.00
0.35 1 0.00
0.34 1 0.00
E
23.57 + 0.62
18.62 + 0.09
18.88 0.23
ANI-Ix (IK)
PaiNN-GF
F
11.32 + 0.08
10.94 + 0.01
11.01 0.00
29


Reviews:
[{"review_id": "DzC0UeG2sJ", "reviewer": "Reviewer_T8Zw", "summary": "In this paper, the authors propose two auxiliary loss functions to improve the generalization of machine learning interatomic potentials (MLIP). Using molecular and crystal datasets, they demonstrate that the proposed method enhances the accuracy of energy and force predictions across multiple MLIPs.\n\n## update after rebuttal\nThe author's responses addressed my questions. In particular, my major concerns about the convergence of the training of the baseline methods were resolved, so I raised my evaluation. However, I'm not convinced that simply using Taylor expansion is called 'physics informed'. More explanation is needed. \n\nI would like those responses to be reflected in the main text.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer,\n\nThank you for your constructive feedback. We have carefully considered your comments and will incorporate the necessary revisions in the camera-ready version. Below, we provide detailed responses to each point.\n\n$\\\\textbf{Q1}$. The iterations in Table A2 apply to both the baseline and PIWSL. Baseline methods were trained until convergence based on standard criteria, ensuring a fair comparison. Table A20 shows that while the baseline overfits with prolonged training, PIWSL continues improving due to physics-informed weak labels, which is especially beneficial in sparse data regimes.\n\nTo further support this, we extended MACE-OFF training:\n\n$$\\\\begin{array} {|r|r|}\\\\hline  & Model & Epoch & E-RMSE & F-RMSE \\\\\\ \\\\hline MACE-OFF & Baseline & 100 & 1.21 & 6.90 \\\\\\ \\\\hline  &  & 1000 & 1.19 & 6.24 \\\\\\ \\\\hline  \\\\end{array}$$\n\nThese results confirm PIWSL’s effectiveness even when the baseline trains significantly longer. We will include this in the final version.\n \n$\\textbf{Q2}$. Below, we present new results on $\\\\epsilon$ dependence for fine-tuning MACE-OFF, supplementing Table 3:\n\n$$\\\\begin{array} {|r|r|}\\\\hline  & Model & \\\\epsilon & E-RMSE & F-RMSE \\\\\\ \\\\hline MACE-OFF & PIWSL & 0.005 & 0.73 & 4.20 \\\\\\ \\\\hline  &  & 0.01 (Table 3) & 0.72 & 3.77 \\\\\\ \\\\hline  &  & 0.02 & 0.75 & 4.04 \\\\\\ \\\\hline  \\\\end{array}$$\n\nThese results show that PIWSL maintains stable performance across reasonable $\\\\epsilon$ variations. The choice of $\\\\epsilon$ = 0.3 of the original bond length was empirically determined to ensure the validity of the Taylor expansion. Table A18 shows that perturbations closer this threshold degrade performance, indicating the breakdown of the approximation. This aligns with constraints observed in prior studies, which we will clarify in the final version.\n\n$\\textbf{Q3}$. No. MACE does not include a direct force module; we used official MACE models. We will explicitly mention this.\n\n$\\textbf{Q4}$. The RMSE ratio is computed as RMSE_PIWSL / RMSE_Baseline to determine whether the PIWSL error is lower than that of the baseline. Thank you for your observation—we will add an explicit definition in the final version.\n\n$\\textbf{Q5}$. To clarify, CCSD/CBS was not used to generate the energy and force labels in this experiment. Instead, CCSD/cc-pVDZ was used [1]. During training, we relied solely on energy labels, while CCSD/cc-pVDZ force labels were used only in the test dataset to evaluate whether PIWSL could improve force predictions without force labels during training. This setup is motivated by real-world applications where force labels are often unavailable due to the high computational cost of advanced coupled cluster methods, such as CC/CBS. By testing in this way, we assess PIWSL’s ability to generalize force predictions from energy supervision alone.\n\n[1] Chmiela. et al. Nat Commun 9, 3887 (2018).\n\n$\\textbf{Comparison with DeNS (Liao+2024)}$\n\nThe key differences between PIWSL and DeNS are as follows: \n\n1) DeNS requires force labels, whereas PIWSL does not. This allows PIWSL to fine-tune MLIP models using datasets containing only potential energies, as demonstrated in Table 3. This scenario is particularly interesting for methods like CC/CBS.  \n\n2) While DeNS introduces a smaller increase in training time, PIWSL offers greater flexibility by eliminating the need for force supervision, making it more broadly applicable in scenarios where force labels are unavailable. \n\n3) DeNS incorporates an equivariant force encoding module within the target MLIP model, which requires integration into the training pipeline. In contrast, PIWSL directly refines energy-based models without such modifications. \n\n4) DeNS primarily discusses energy and force errors but does not explicitly discuss robustness in molecular dynamics (MD) simulations, which is one of the key aspects in our work.\n\nAdditionally, an interesting open question is whether techniques from DeNS could complement PIWSL to further enhance MLIP performance. While we have not explored this combination in this work, investigating such synergies could be valuable for future research.\nWe acknowledge that DeNS was recently accepted on December 27th. Although we have already cited it in Appendix C.2 as a related work, we will provide a more detailed discussion of these differences in the “Related Work” section of the main body in the camera-ready version.\n\n$\\textbf{Periodic boundary condition}$\n\nAs noted by the reviewer, atoms outside the unit cell are wrapped back into the cell when noise is applied. We will add this clarification to the camera-ready version.\n\n$\\textbf{Physics-Informed Claim for PITC}$\n\nConcerning the physical motivation behind PITC: It is valid to locally approximate a potential energy surface using a Taylor series.", "claims_and_evidence": "The authors claim that the proposed \"Physics-Informed\" auxiliary loss functions improve the accuracy of MLIPs (Machine Learning Interatomic Potentials). They experimentally demonstrate improvements in the accuracy of various MLIPs on molecular and crystal datasets.\n\nThe proposed auxiliary loss functions consist of PITC (Physics-Informed Taylor-Expansion-Based Consistency Loss) and PISC (Physics-Informed Spatial-Consistency Loss).\nPITC uses a second-order Taylor approximation to create pseudo-labels for energy, but it does not explain the physical justification. Hence, it does not fully support the \"Physics-Informed\" claim. PITC might be an analogy with the harmonic oscillator approximation that uses a second-order approximation around stable equilibrium points.\nOn the other hand, PISC is based on the physical principle that the energy difference is path-independent, and it successfully supports the claim of being \"Physics-Informed.\"", "methods_and_evaluation_criteria": "Since generating training data for MLIP is time-consuming, there is a need to train or fine-tune with a small amount of training data, and the proposed method addresses this issue. Moreover, evaluating the accuracy of energy and forces on molecular and crystal datasets is a fundamental and appropriate method of evaluation. Evaluation with CCSD(T), which is high-accuracy but high-cost, is an experiment closely aligned with practical applications and likely to attract the readers' interest.", "theoretical_claims": "There is no theoretical claim.", "experimental_designs_or_analyses": "The experimental setup follows existing research. However, there are a few points that are slightly concerning due to a lack of clear explanation:\n\n1. The proposed method takes 2-3 times longer to train compared to the baselines, so I'm concerned whether the advantage over the baseline methods wouldn't be slight if the number of training iterations was increased. There would be no issue if it could be confirmed that the baseline methods have sufficiently converged with the current number of training iterations.\n2. Since the proposed method is expected to be sensitive to the value of $\\epsilon$, I would like to ensure that the value of epsilon was not cherry-picked.", "relation_to_broader_scientific_literature": "Several foundational models for machine learning potentials have been proposed, but these foundational models are often fine-tuned for specific targets. Additionally, there are various methods for first-principle calculations used to generate training data; methods like CCSD(T) that have smaller errors concerning experimental values tend to have high computational costs.\nThe proposed method is effective for tasks with limited training data, and it is expected to be beneficial for fine-tuning foundational models as well as for applications that use training data generated from high-cost first-principles calculations.", "essential_references_not_discussed": "I think many readers want to see the comparison with (Liao et al., 2024) since they also improved EquiformerV2 and eSCN on both crystal and molecule datasets.", "other_strengths_and_weaknesses": "A strength of this paper is that it provides evaluation results with varying training data, and it consistently demonstrates improved accuracy across various MLIP methods.", "other_comments_or_suggestions": "It would be good to describe how the periodic boundary conditions for bulk were handled when adding noise. Probably, noised atoms outside the unit cell will be wrapped back into the cell.\n\nTypo\n* Page 5: \"nergy\" should be \"energy\" in Section 5.", "questions_for_authors": "1. Is the total number of training iterations in Table A2 only for the proposed method? Let me confirm the training iterations are appropriately selected for the baseline methods. Since they are faster to train than the proposed method, they should be able to run longer iterations in the same amount of time.   Did the baseline methods require more iterations to converge compared to the proposed method?\n2. It is expected that PISC, being a second-order approximation, would be sensitive to the value of the maximum perturbation length, $\\epsilon$. Please provide the performance evaluation results when varying epsilon. Also, how was the 30% of the original bond length determined for epsilon?\n3. Did you even use direct force prediction for MACE? As shown in the Stability evaluation results in Figure 3, direct force prediction cannot be used in MD simulations and is known to have a very limited range of applications. Therefore, readers are likely to be more interested in the results of force calculations using gradients.\n4. Could you please provide the formula used to calculate the Relative performance gains shown in Figure 2? Wouldn't it be more intuitive and easier to understand if you showed it as $1 - \\frac{RMSE_{PIWSL}}{ RMSE_{baseline}}$?\n5. Could you provide a more detailed explanation for the annotation regarding \"F\" in Table 3? Does it mean that the forces calculated using CCSD(T)/CBS were not used, and instead, the DFT forces were used as the ground truth?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "l1z7hQtIEW", "reviewer": "Reviewer_Q1cJ", "summary": "The paper proposes a physics-informed weakly supervised learning (PIWSL) framework to improve the accuracy and robustness of machine-learned interatomic potentials (MLIPs). PIWSL incorporates two new loss functions: Physics-Informed Taylor-Expansion-Based Consistency (PITC) loss and Physics-Informed Spatial Consistency (PISC) loss. These losses help refine energy and force predictions, particularly when training data is sparse. The paper demonstrates significant improvements in predictive accuracy and robustness across various molecular and material datasets. Additionally, PIWSL enhances the fine-tuning of foundation models.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer,\n\nThank you for your constructive feedback. We have carefully considered your comments and will incorporate the necessary revisions in the camera-ready version. Below, we provide detailed responses to each point.\n\n$\\\\textbf{W1}$: \nA detailed discussion regarding the computational time required for both the baseline and PIWSL is provided in Tables A10 and A11 in Appendix E2. Additionally, data sample efficiency is partly analyzed in Table 3, demonstrating that PIWSL achieves comparable or better accuracy while requiring only half the training data for finetuning a foundation model.\nSince our primary focus is on the sparse data regime, as explained in the paper, data efficiency is a critical factor for us. This makes PIWSL particularly valuable in our target setting. We will reference these results more prominently in the main paper. \n\n$\\\\textbf{W2}$: \nWe acknowledge the reviewer’s concern that PIWSL’s relative improvement decreases as dataset size increases. This trend is expected because, in data-rich scenarios, supervised learning already provides strong generalization, reducing the additional benefit of weak labels. However, PIWSL remains valuable, particularly in materials discovery, where data is scarce and MLIP models must be trained from scratch or fine-tuned from foundation models.\nEven in data-rich settings, PIWSL still provides meaningful improvements. For instance, with the ANI-1x 5M dataset, PIWSL achieves over a 10% error reduction, demonstrating that weak labels enhance learning even when abundant supervised data is available. While the marginal gains decrease, the improvements remain significant given the high precision requirements in MLIP applications.\n\n$\\textbf{Q1}$: Although training stability depends on the coefficients of PITC and PISC, we do not observe instability in the initial stages of training with PIWSL. This is because, at the beginning of training, the energy and force regression losses are significantly larger than the PIWSL losses. While increasing the PIWSL loss coefficients could lead to instability in the initial phase, it would also degrade overall performance. If instability were to arise, a potential mitigation strategy would be gradually increasing the PIWSL loss weight during training (similar to curriculum learning).\n\n$\\textbf{Q2}$: As stated in the final sentence of Section 5.2 (\"Heterogeneous Molecular Dataset\"), we attribute this issue to NoisyNode's inability to properly capture the response of energy and atomic forces to perturbations in atomic positions.", "claims_and_evidence": "The claims in the paper are generally well-supported with empirical evidence. The authors provide extensive benchmarking on diverse datasets, including ANI-1x, TiO2, and MD17(CCSD), showcasing improvements in accuracy. They also validate robustness through MD simulations. Theoretical justifications, particularly regarding the formulation of loss functions, are sound. However, further analysis of the trade-offs in computational cost associated with PIWSL would strengthen the claims.", "methods_and_evaluation_criteria": "The proposed methods are well-aligned with the problem of training MLIPs. The authors employ relevant benchmark datasets, including molecular and material science datasets, and compare their approach with state-of-the-art methods such as SchNet, PaiNN, and Equiformer. Evaluation criteria include RMSE and MAE metrics for energy and force predictions, along with robustness assessments via MD simulations. The experimental setup is comprehensive and appropriately chosen for the task.", "theoretical_claims": "The paper provides a theoretical derivation of PITC and PISC loss functions. The use of Taylor expansions to approximate perturbed energy values is mathematically sound.", "experimental_designs_or_analyses": "The experimental design is appropriate. The authors apply the proposed training method to multiple datasets and baseline models. The authors also conduct experiments to study the impact of PIWSL on training set sizes, and the robustness during MD simulations.", "relation_to_broader_scientific_literature": "The paper aims to improve the generalizability of MLIPs in tasks where a limited amount of training data is available. In the field of atomistic modeling, data scarcity is a common challenge given the computational complexity of first-principle methods (DFT) or the demanding cost of conducting wet-lab experiments.", "essential_references_not_discussed": "The paper presents a solid literature review. However, existing works that utilize weak supervision signals for training MLIPs [1] may offer a more comprehensive review.\n\n[1] Shui, Zeren, et al. \"Injecting domain knowledge from empirical interatomic potentials to neural networks for predicting material properties.\" Advances in Neural Information Processing Systems 35 (2022): 14839-14851.", "other_strengths_and_weaknesses": "Strengths:\n\n1. The proposed method is innovative and effective in improving the accuracy and robustness of MLIPs in the data sparse scenario.\n\n2. The proposed method outperforms existing methods such as NoisyNodes on a wide range of benchmark datasets and base models.\n\nWeaknesses:\n\n1. Limited discussion on computational costs and efficiency.\n\n2. The method’s improvement diminishes as the training data size increases.", "other_comments_or_suggestions": "Typo\nLine 232, \"nergy\" -> \"energy\".", "questions_for_authors": "1. Does the method lead to unstable training in the initial stages when the neural networks are randomly initialized? The predicted energy and forces used to compute the supervision signal may deviate significantly.\n\n2. In Table 2/3, why does NoisyNodes significantly degrades the performance of the base models?\n\nI will adjust my score once these questions and the first weakness are addressed.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "CWJfv0czA4", "reviewer": "Reviewer_gwDq", "summary": "The paper proposes a new method for training machine learning interatomic potentials (MLIPs) using loss functions based on the Taylor expansion of the potential energy and the notion of conservative forces. The papers starts by describing ab-initio computational chemistry simulation methods and motivates the need for MLIPs as tool to bridge ab-initio methods and classical force fields. Next, the paper describes data generation as one of the major challenges for MLIP training given the high computational cost of computational chemistry methods that serve as the primary source of MLIP data. Furthermore, the authors claim that MLIPs have trouble generalizing and claim that their methods helps mitigate such challenges. The proposed method, physics-informed weakly supervised learning (PIWSL), is based on two physics-informed losses: physics-informed Taylor-expansion-based consistency (PITC) and physics-informed spatial consistency (PISC). The authors claim that MLIPs trained with PIWSL require less training data, produce more accurate energy prediction and increases robustness of MLIPs.\n\nSection 2 covers related work on MLIPs, focusing on architectures and methods, and physics-informed machine learning covering various approaches to infuse physics principles into machine learning methods. Section 3 outlines relevant background on training MLIPs, including the common training loss definition centered on energy and force labels. Section 3 also introduces weakly supervised learning for MLIPs, which relies on training on structures with small perturbations. Section 4 describes the details of weakly supervised learning focusing on the two PIWSL losses. The PITC takes the Taylor expansion of the energy a perturbed structure and re-writes it as the inner product of forces and positions. PITC also introduces a parameter that modulates the contribution of the second order, thereby enabling training with different contributions of the force field terms. PISC, the second loss, leverages the conservative nature of forces to create weak supervision. The method leverages perturbations by different paths to the same perturbed state to enforce a consistency loss. The paper then combines the two losses into a joint loss and discusses considerations for perturbation directions and magnitudes. \n\nSection 5 outlines the experiments starting with energy and force error experiments on ANI-1x and TiO2 while concurrently ablating dataset size. The experiments ablate five different models and compares results to noisy nodes pre-training for different dataset sizes. The results in Table 1 and Table 2 generally show that PIWSL outperformance baseline training and noisy nodes for most of the cases studied. Section 5.3 details a qualitative assessment of the effects of PIWSL for the aspirin molecule which claim greater prediction improvement and stability in MD simulations. Section 5.4 outlines experiments related to fine-training pretrained MLIPs, such as MACE, with a regular training loss compared to PIWSL. The results generally show that MACE trained with PIWSL performs better than baseline losses both for pretrained and randomly initialized MACE models. Section 6 provides a conclusion and brief discussion of limitations.\n\n## update after rebuttal\n\nThe authors answered my questions to my satisfaction and agreed to include relevant details that will make the paper stronger. These include details on limitations, computational cost trade-offs and additional details on comparing with DeNS. As such, I maintain my support of the paper.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer,\n\nThank you for your constructive and encouraging feedback. We have carefully considered your comments and will incorporate the necessary revisions in the camera-ready version. Below, we provide detailed responses to each of your encouraging proposals and questions. Please let us know if any of your concerns remain unaddressed.\n\n$\\\\textbf{Q1}$. We agree that evaluating PIWSL on additional datasets would be beneficial. However, we have evaluated PIWSL across various MLIP models and datasets, including heterogeneous datasets such as ANI-1x. As recommended, we will discuss the limitations of the benchmark dataset in the \"Limitations\" section and plan to explore a broader range of datasets in future work.\n\n$\\\\textbf{W2/Q2}$. While a third-order extension may improve accuracy, the computational cost of calculating second and third-order derivatives of the potential may outweigh the benefits. An alternative approach would be to explore new PISC spatial configurations, which offer greater flexibility than PITC, as discussed in Appendix E.2.\nWe will mention the computational cost trade-off mentioned in Weakness 2 in the main-body in the camera-ready version. The performance improvements achieved by PIWSL justify the additional computational cost. For example, Table A20 compares training with extended iterations, showing that while the baseline method overfits the training data, PIWSL continues to improve. This is particularly important for the sparse data regime, which is our primary focus. Moreover, the increase in training time is negligible compared to the substantial computational cost and time required for data generation, particularly when using DFT or coupled-cluster methods. This becomes even more significant as molecular size increases.\n\n$\\textbf{Q3}$. Thank you for your insightful comment! PITC is applicable whenever the target property depends on atomic coordinates. For instance, DeNS, the recommended work by the reviewer, applied their method to relaxation energy estimation in OC20/22, and we expect that PIWSL could also be effectively applied to this work. We are excited about the potential to explore new application domains in future research.\n\n$\\textbf{Comparison with DeNS (Liao+2024)}$\n\nWe appreciate the recommendation of DeNS. We acknowledge that DeNS was recently accepted on December 27th. Although we have already cited it in Appendix C.2 as a related work, we will provide a more detailed discussion of these differences in the “Related Work” section of the main body in the camera-ready version.\n\nThe key differences between PIWSL and DeNS are as follows:\n\n1) DeNS requires force labels, whereas PIWSL does not. This allows PIWSL to fine-tune MLIP models using datasets containing only potential energies, as demonstrated in Table 3. This scenario is particularly interesting for methods like CC/CBS.\n\n2) While DeNS introduces a smaller increase in training time, PIWSL offers greater flexibility by eliminating the need for force supervision, making it more broadly applicable in scenarios where force labels are unavailable.\n\n3) DeNS incorporates an equivariant force encoding module within the target MLIP model, which requires integration into the training pipeline. In contrast, PIWSL directly refines energy-based models without such modifications.\n\n4) DeNS primarily discusses energy and force errors but does not explicitly discuss robustness in molecular dynamics (MD) simulations, which is one of the key aspects in our work.\n\nAdditionally, an interesting open question is whether techniques from DeNS could complement PIWSL to further enhance MLIP performance. While we have not explored this combination in this work, investigating such synergies could be valuable for future research.", "claims_and_evidence": "The claims made in the paper, that PIWSL based training requires less data and improves MLIP performance, are generally well supported with the experiments shown in Section 5. The claim related to increased robustness in MD simulation has some supporting evidence, but is mainly on a single case study, and could be further strengthened with additional evidence.\n\nFurthermore, the paper mainly shows experiments on molecular structures (outside of the TiO2 dataset). The claims and evidence could be further strengthened by providing studies of solid-state structures (examples datasets can be found here [1] [2]). At the minimum, the limitations should mention that future work is needed to assess PIWSL for a broader set of materials.\n\n[1] Bihani, V., Mannan, S., Pratiush, U., Du, T., Chen, Z., Miret, S., Micoulaut, M., Smedskjaer, M.M., Ranu, S. and Krishnan, N.A., 2024. EGraFFBench: evaluation of equivariant graph neural network force fields for atomistic simulations. Digital Discovery, 3(4), pp.759-768.\n\n[2] Lee, K.L.K., Gonzales, C., Nassar, M., Spellings, M., Galkin, M. and Miret, S., MatSciML: A Broad, Multi-Task Benchmark for Solid-State Materials Modeling. In AI for Accelerated Materials Design-NeurIPS 2023 Workshop.", "methods_and_evaluation_criteria": "The methods and evaluation criteria generally make sense for the claims presented. The varying of dataset size in Section 5.2 is particularly useful for supporting the data efficiency claims and providing ideas for future research. As mentioned in the previous in the previous box, the paper could be improved by discussing the limitations of the benchmark used and how it relates to the broad set of materials and molecules that MLIPs aim to cover.\n\nOne additional baseline that would be useful to include, or discuss at the minimum, is DeNS [1], which also outlines a pre-training method based on perturbed non-equilibrium structures. \n\n[1] Liao, Y.L., Smidt, T., Shuaibi, M. and Das, A., Generalizing Denoising to Non-Equilibrium Structures Improves Equivariant Force Fields. Transactions on Machine Learning Research.", "theoretical_claims": "The theoretical claims in the paper are generally well supported. I would recommend re-writing Equation 2 to include relevant gradient/derivatives that relate energy and force to make it easier for the reader to follow.", "experimental_designs_or_analyses": "I checked the experiments and analyses presented in the main paper and some of the additional details in the supplementary material.", "relation_to_broader_scientific_literature": "The contributions and findings provide a new useful formulation and insights for the training of MLIPs. As mentioned in prior boxes, the paper could be strengthened by discussing MLIP application beyond molecular structures.", "essential_references_not_discussed": "I would recommend the paper discuss the papers mentioned in the previous boxes. [1] is especially important for the MD analysis section as it provides relevant metrics and analysis techniques for that case. [2] discusses datasets relevant for MLIPs for solid-state materials and [3] outlines a different pre-training method.\n\nOn top of that, [4] is a useful review of model architectures and methods related to atomistic modeling that would be useful to provide a broader perspective on MLIPs and geometric deep learning models used for property prediction. A recent paper [5] might also be worth mentioned as it reinforces many of the arguments made in the paper related to costly training data and scaling MLIPs to more difficult challenges.\n\n[1] Bihani, V., Mannan, S., Pratiush, U., Du, T., Chen, Z., Miret, S., Micoulaut, M., Smedskjaer, M.M., Ranu, S. and Krishnan, N.A., 2024. EGraFFBench: evaluation of equivariant graph neural network force fields for atomistic simulations. Digital Discovery, 3(4), pp.759-768.\n\n[2] Lee, K.L.K., Gonzales, C., Nassar, M., Spellings, M., Galkin, M. and Miret, S., MatSciML: A Broad, Multi-Task Benchmark for Solid-State Materials Modeling. In AI for Accelerated Materials Design-NeurIPS 2023 Workshop.\n\n[3] Liao, Y.L., Smidt, T., Shuaibi, M. and Das, A., Generalizing Denoising to Non-Equilibrium Structures Improves Equivariant Force Fields. Transactions on Machine Learning Research.\n\n[4] Duval, A., Mathis, S.V., Joshi, C.K., Schmidt, V., Miret, S., Malliaros, F.D., Cohen, T., Liò, P., Bengio, Y. and Bronstein, M., 2023. A hitchhiker's guide to geometric gnns for 3d atomic systems. arXiv preprint arXiv:2312.07511.\n\n[5] Miret, S., Lee, K.L.K., Gonzales, C., Mannan, S. and Krishnan, N.M., 2025. Energy & Force Regression on DFT Trajectories is Not Enough for Universal Machine Learning Interatomic Potentials. arXiv preprint arXiv:2502.03660.", "other_strengths_and_weaknesses": "Overall the paper provides an interesting methodological innovation that could be broadly applicable to MLIP training and provides solid evidence for its claims.\n\n**Strengths:**\n* Interesting methodological innovation based on physical principles that can help alleviate data sparsity for MLIP training.\n* The proposed method is model agnostic and hence potentially broadly applicable.\n* The experiments provide support for the claims around data efficiency.\n\n**Weaknesses:**\n* The paper could be strengthened by discussing broader applications of MLIPs and ideally conducting more experiments on different systems to show the applicability of PIWSL. At a minimum, limitations should more clearly outline what the experiments show and what is future work.\n* The mentions a training time analysis in Appendix D that states that PIWSL leads to increased training time. It would be worth mentioning this in the main paper as a trade-off.", "other_comments_or_suggestions": "N/A", "questions_for_authors": "1. What is the reason you focused most of your experiments and analysis on molecular systems and single molecules? Do you believe this provides enough evidence for the benefits of PIWSL? Why or why not?\n2. What extension of PIWSL do you think could strengthen MLIP training (e.g., third order Taylor expansion or other weak supervision)? How does that trade off with additional computational cost?\n3. Do you think PIWSL representations could be useful for property prediction tasks?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: Dear authors,

thank you for submitting your paper to this year's ICML conference. After extensive review, rebuttal period and discussion phase, the reviewers agreed that your paper has good novelty to be accepted. Your paper proposed a physics-informed weakly supervised learning (PIWSL) framework that can improve the training of interatomic potentials. Your method incorporates two novel loss functions—PITC and PISC—motivated by physical principles such as energy conservation and Taylor series approximations. 

Despite the fact that several weaknesses were initially identified (computational cost, limited diversity of applications, ...) most of the concerns have been sufficiently addressed during rebuttal period.
The extensive empirical results in your paper covers several benchmark datasets (e.g., ANI-1x, $TiO_2$, MD17) and models, showing improved accuracy, robustness, and data efficiency.

I would like to congratulate you for acceptance of your work and strongly encourage you to implement all the suggested improvements and results of discussions into the final camera ready paper.

Best, AC
Decision: Accept (poster)