Robust and Interpretable Adaptation of Equivariant Materials Foundation Models via Sparsity-promoting Fine-tuning
Authors: Youngwoo Cho
Seunghoon Yi
Wooil Yang
Sungmo Kang
Young-Woo Son
Jaegul Choo
Joonseok Lee
Soo Kyung Kim
Hongkee Yoon
Venue: ICLR 2026 Poster (2026)
Keywords: machine learning interatomic potentials
equivariance
sparsity-promoting
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)
TL;DR: This work proposes a sparsity-promoting fine-tuning method for equivariant MLIPs that integrates equivariant constraints with selective parameter pruning.

Abstract: Pre-trained materials foundation models, or machine learning interatomic potentials, leverage general physicochemical knowledge to effectively approximate potential energy surfaces. However, they often require domain-specific calibration due to physicochemical diversity as well as mismatches between practical computational settings and those used in constructing the pre-training data. To address this, we propose a sparsity-promoting fine-tuning method that selectively updates model parameters by exploiting the structural properties of E(3)-equivariant materials foundation models. On energy and force prediction tasks across molecular and crystalline benchmarks, our method matches or surpasses full fine-tuning and equivariant low-rank adaptation while updating only ~3 \% of parameters, and in some cases as little as \~0.5 \%. Beyond energy and force calibration, we further demonstrate task generalizability by applying our method to magnetic moment prediction and magnetism-aware total energy modeling. Finally, analysis of sparsity patterns reveals physically interpretable signatures, such as enhanced $d$-orbital contributions in transition metal systems. Overall, our results establish sparsity-promoting fine-tuning as a flexible and interpretable method for domain specialization of equivariant materials foundation models.

Published as a conference paper at ICLR 2026
ROBUST AND INTERPRETABLE ADAPTATION OF EQUIVARIANT MATERIALS FOUNDATION MODELS VIA SPARSITY-PROMOTING FINE-TUNING
Youngwoo Cho"
Scunghoon Yi Scoul National Univ.
Wooil Yang KIAS
KAIST cyw314@kaist.acc ac. kr
jaguar6182 ac. kr
yspacefirst@kias. re.kr
Sungmo Kang
Young-woo Son KIAS hands re.kr
Jaegul Choo
KIAS snkang@kias.re.Kk
KAIST
jchoo@kaist.aco kc.
Joonseok Leet
Soo Kyung Kim
Hongkee Yoon Kangwon National Univ.
Seoul National Univ.
Ewha Womans Univ.
Koonseok@snu.c ac.
sookim@ewha.ac. kr
tongkeeyoonkkangwor. ac .c.kr
ABSTRACT
Pre-trained materials foundation models, machine learning interatomic poten- tials, leverage general physicochemical knowledge to effectively approximate po- tential energy surfaces. However, they often require domain-specific calibration due to physicochemical diversity as well as mismatches between practical compu- tational settings and those used in constructing the pre-training data. To address this, we propose sparsity-promoting fine-tuning method that selectively updates model parameters by exploiting the structural properties -quivvarrn ma- terials foundation models. On energy and force prediction tasks across molecular and crystalline benchmarks, our method matches or surpasses full fine-tuning and equivariant low-rank adaptation while updating only ~3 % of parameters, and in some cases as little as ~0.5 %. Beyond energy and force calibration, we fur- ther demonstrate task generalizability by applying our method to magnetic mo- ment prediction and magnetism-awarc total energy modeling. Finally, analysis of sparsity patterns reveals physically interpretable signatures, such enhanced d-orbital contributions in transition metal systems. Overall, our results establish fine-tuning flexible and interpretable method for domain specialization of equivariant materials foundation models.
INTRODUCTION
Machine learning interatomic potentials (MLIPs, Blank et al. (1995)) aim reproduce the potential energy surface (PES) of material structures, serving as efficient surrogates for density functional theory (DFT, Kohn Sham (1965)). Modern MLIPs offer remarkable efficiency and accuracy in regimes that have been computationally challenging for traditional soysics-based simulations with limited scalability. Inspired by foundation models in computer vision and natural language processing (Caron et al., 2021; Li et al., 2022; Radford et al., 2021), recent work has leveraged large-scale materials datasets develop atomistic foundation models (Barroso-Luque et al., 2024; Batatia et al., 2025; Chen & Ong, 2022; Deng et al., 2023; Park et al., 2024), achieving impressive accuracy on multiple benchmarks (Levine al., 2025; Riebesell et al., 2025).
Due to the extreme diversity of material systems, however, even large-scale pre-training dataset cannot fully cover the complicated and subtle landscape of materials, under-representing elements and motifs, well as diverse physicochemical variations (e.g., pressure and temperature). Thus,
Equul contribution. 'Corresponding authors.

---

Published as a conference paper at ICLR 2026
direct application of pre-trained foundation MLIP often fails to generalize to unseen conditions, requiring Nomain-speciffe calibration fine-tuning) of the foundation model. Moreover, downstream applications may rely on different levels of theory or richange-corrllatin functional settings than those used to construct the pre-training data, introducing systematic discrepancies.
Given this necessity, the question shifts to how such calibration should be performed. Full fine-tuning, which updates all model parameters, would be the most straightforward adaptation method. However, domain-specific datasets are typically small relative the vast configurational and chemical space they should represent, making full fine-tuning highly prone to overfitting, aside from its computational and memory costs. Parameter-effichee Adapters (Houlsby et al., 2019) and low-rank adaptation (LoRA, Hu et al. (2022)) provide a natural alternative by introducing small set of trainable parameters. Applying these methods to equivari- ant graph neural networks (GNNs), which are commonly used in sfate-of-the-ar MLIPs, requires preserving equivariance constraints, i.e., translational, rotational, and reflectional symmetries in 31) space. To this end, GeoAda (Zhao et al., 2025) and ELoRA (Wang et al., 2025) redesign Adapters and LoRA, respectively, to preserve these constraints.
fine-tuning methods such as
These methods focus on how the parameters are updated, e.g. via low-rank structures or regularized magnitudes. A complementary yet unexplored perspective is to instead control which parameters updated, encouraging the model to selectively modify only those most relevant the target domain while preserving the rest intact. In other words, rather than constraining how updates are parameterized, one can directly regularize the number of impacted parameters.
counsity-rrmooting adaptation offers a natural realization of this idea, while also addressing a long- standing goal in scientific machine learning (ML): enhancing model interpretability by reducing unnecessary degrees of freedom (Brunton et al., 2016; Hoefler et al., 2021). MLIPs offer unique possibilities in this regard, as their internal layers often encode physically meaningful basis func- tions such as spherical harmonics (Geiger & Smidt, 2022). By selectively updating the weights as- sociated with these bases, one can precisely identify which channels are modified during fine-tuning and which remain unchanged, providing clear insight into the model's internal physical represen- tations. However, pearsity-rromoting approaches remain underexplored in the context of MLIPs, particularly regarding their application to fine-tuning.
To bridge this gap, we propose a sparsity-pramoting fine-tuning method for equivariant MLIPs, which carefully maintains equivariance with selective parameter updating. Applied to MACE po- tentials (Batatia et al., 2022; 2025; Kovács et al., 2025) across molecular, crystalline, and magnetic benchmarks, our method matches or exceeds the accuracy of full fine-tuning while updating only 0.5-3 % of parameters. Beyond recalibrating energy and force predictions, it extends effectively to other material properties such magnetic moments, demonstrating broader applicability of founda- tion MLIPs. Finally, analysis of sparsity patterns reveals which coefficients are selectively updated, yielding physical insights into the learned representations.
Our contributions are threefold- (i) Sparse equivariant fine-tuning: We introduce a fine-tuning method top of equivariant MLIPs that unifies symmetry preservation with sparsity-promotte approach, matching or surpassing full fine-tuning even with extremely sparse updates, (ii) Flexible and accurate adaptation: Across molecular, crystalline, and magnetic benehmarks, we demon- strate the versatility of our method in accurately predicting physical properties across diverse do- mains. (iii) Physically interpretable sparsity: Analysis of sparsity patterns reveals physically interpretable signatures of the physicochemical representations learned by fine-tuned MLIPs.
2 RELATED WORK
This section presents a brief survey of related work. The mathematical and physical background needed to understand our study is provided in Appendix A.
Machine Learning Interatomic Potentials. In the early stages of MLIP development, simple feed- forward networks were employed on limited target systems, using atomic descriptors to predict energy (Behler al., 2008; Blank et al., 1995; Le & Raff, 2008; No et al., 1997; Smith et al., 2017). Later, GNNs enabled sophisticated modeling of chemical interactions through message passing by representing atoms nodes and bonds edges (Gilmer et al., 2017; Schütt al., 2017; Unke &
2

---

Published as a conference paper at ICLR 2026
Meuwly, 2019). However, these models handle only invariant representations, without considering directional information crucial for vector and tensor quantities (e.g., forces or dipoles).
Recognizing this limitation, architectures that preserve geometric symmetries have been developed, with features and outputs transforming under (3) or E(3) group operations. These transforma- tions rely on type-f vectors and spherical harmonies, combined with Clebsch-Gordal tensor prod- ucts, inspired by physical principles (Geiger & Smidt, 2022; Kondor et al., 2018; Satorras et al., 2021; Thomas al., 2018). Building upon these foundations, representative equivariant architec- tures such as MACE (Batatia et al., 2022), NequlP (Batzner et al., 2022), Allegro (Musaelian et al., 2023), and Equiformer (Liao & Smidt, 2023) are capable of capturing many-bode interactions while strictly preserving physical symmetries.
These advances have further enabled the emergence of foundation MLIPs, following trends in vision and language domains (Caron et al., 2021; Li et al., 2022; Radford et al., 2021). Such foundation models are pre-trained on large-scale databases of DFT calculations (Barroso-Luquee et al., 2024; Deng et al., 2023; Jain et al., 2013; Levine et al., 2025) to achieve broader generalization. Notable examples include M3GNet (Chen & Ong, 2022), CHGNet (Deng et al., 2023). MACE-MP-0 (Batatia et al., 2025), SeveNNet (Park et al., 2024), and ORB (Neumann et al., 2024).
While these foundation MI.IPs provide broad generalization, task- sasc-specific adaptation often requires fine-tuning process. Generally, existing studies have employed full fine-tuning, sometimes with partial layer freezing to reduce computational cost (Gardner et al., 2025; Kong et al., 2025; Liu et al., 2026; Radova et al., 2025). Recently, equivariant adapters (Zhao et al., 2025) and ELoRA (Lu et al., 2025; Wang et al., 2025) have been proposed for equivariant architectures.
Sparse Neural Networks and Sparsity-promotie maintain only a fraction of their weights, offer significant benefits such as model compression, inference acceleration, and improved generalization with reduced model complexity (Hoefler al., 2021; Louizos et al., 2017; Molchanov et al., 2017; Roh et al., 2022; Wen et al., 2016). Early work by Han et al. (2015); Li al. (2017); Srinivas & Babu (2015) proposed to prune trained networks and then fine-tune to recover the accuracy. Subsequently, the lottery ticket hypothesis (Frankle & Carbin, 2019) and Liu et al. (2019) shifted focus toward the subnetworks obtained after pruning and their initialization strategies (You et al., 2020; Zhang et al., 2021). Some studies (Azarian et al., 2020; LIU et al., 2020; Louizos et al., 2018; Savarese et al., 2020; XIAO et al., 2019) proposed to simultaneously learn weights and their sparse masks to avoid multiple rounds of model training and pruning. Soft threshold weight reparameterization (STR, Kusupati et al. (2020)), which we employ and describe below, belongs to this line of methods.
Approaches. Sparse neural networks, which
The concept of sparsity is particularly powerful in scientific ML. For example, the family of SINDy frameworks (Brunton al., 2016) leverages sparsity to identify simple governing equations, based on the principle that physical phenomena are often governed by a few core variables (Kaheman et al., 2020; Mangan et al., 2019; 2016; Rudy et al., 2017). This reflects Occam's 's razor, where reducing model complexity enhances both interpretability and generalization in scientific applications. De- approaches remain largely underexplored in the MLIP domain (Sandberg et al., 2024; Torabi et al., 2026), particularly in fine-tuning scenarios.
spite these promising results, sparsity-promotiee
3 METHOD
Our approach builds on equivariant GNNs (EGNNs), which are designed to preserve the inherent geometric symmetries in atomic systems. We first summarize their symmetry-preservine operations and then describe how we incorporate prommttig techniques into the fine-tuning of EGNNs.
3.1 PARAMETER UPDATES UNDER EQUIVARIANCE CONSTRAINTS
As MLIPs are designed to model atom-atom interactions, a key requirement is that their internal representations preserve the geometric symmetries of atomic systems, ensuring predictions remain consistent under changes of the coordinate system. This principle reflects the underlying spherical symmetry of atomic orbitals, which motivates the adoption of equivariant models such as EGNNs.
Equivariant Representation. In EGNNs, node and edge features are expressed as irreducible rep- resentations (irreps) of the rotation group, indexed by their order é: = 0 for scalars, & 1 for
3

---

Published as a conference paper at ICLR 2026
vectors, and & = 2 for rank-2 tensors. When two input irreps combined, the admissible output orders fout must satisfy 1411-1n2 S lout I f +42 These couplings, or cimmetry-allowesd in- teraction paths (Geiger & Smidt, 2022), are realized through Clebsch- Gordan coefficients (CGCs), which are predefined, non-trainable constants that mathematically guarantee equivariance.
The strength of each symmetry-allowe interaction path is modulated by learnable sealar weight, and the collection all such scalars forms a path-specific weight tensor W. Equivariance is strictly enforced by the predefined CGCs, while the capacity learn lies entirely in the weights W. There- fore, when adapting a pre-trained model, the natural intervention point is this path-weight tensor. In other words, updating the path- weight tensor does not affect equivariance constraints. For a detailed explanation of the equivariant representation, we refer the reader to Appendix A.
Weight Decomposition for Fine-tuning. To adapt a foundation MLIP target domain, we de- compose the path weights W to be fine-tuned, into frozen and trainable components:
W' W + AW,
where W denotes the frozen weights from the pre-traine treinrained model and AW contains the adaptation parameters. From this perspective, fine-tuning corresponds to reweighting the relative contributions of interaction paths, thereby adjusting the model to the characteristics of the target dataset (e.g., composition, pressure/temperatuuee regime, or the level of theory used in DFT calculations).
Existing approaches such as ELoRA (Wang et al., 2025) parameterize AW as the product of two low-rank matrices, leading dense updates in which every interaction path perturbed to some degree. We hypothesize that more effective adaptation can be achieved through selective updates, enabling the model to adaptively identify and modify only the most critical paths.
Sparse Adaptation in Practice. To this end, we promote sparsity in AW by encouraging most path coefficients to remain close to zero, thereby suppressing unnecessary degrees of freedom during fine-tuning. Because AW modulates the learnable weights on CGCs in individual tensor interaction paths (see Appendix A), symmetry constraints are maintained, and equivariance is strictly preserved. In practice, inject AW directly into the path-weight tensor, introducing negligible computational overhead (see Section 5.5) while enabling flexible domain adaptation. Here, we note that sparsity is not introduced reduce training-time compute or memory usage, since the pre-trained weights W remain dense; rather, it serves to induce selective and physically meaningful updates in AW.
3.2 CONTROLLING SPARSITY DURING ADAPTATION
To enable selective adaptation, we learn AW in a sparse manner based on STR (Kusupati et al., 2020), sparsity-promoting approach originally introduced in the computer vision domain. In STR, dynamic pruning mechanism is governed by learnable, per-layer scalar parameter т, which con- trols the pruning threshold 8 g(r). where g is the sigmoid function. This threshold removes entries of AW with magnitudes below the threshold during training, making sparse and yielding compact updates. In our case, only subset of path weights corresponding to specific interaction paths is updated during adaptation, highlighting only the physically relevant interactions while suppressing symmetry-trivial paths.
Our training procedure starts by initializing the threshold parameter 5 for each layer. Then, AW is initialized from a narrow Gaussian distribution ('00, T²)). This initialization is necessary because AW were set to zero, the threshold mechanism would cause vanishing gradients and prevent any learning. Before the forward pass at timestep t, parameters with magnitudes below ò in AW, are pruned by applying a sparsify function S as:
AW, + S(AW,, 6,) sign(AW) @ ReLUU[AW.]] oc),
where @ denotes the element-wise Hadamard product.
In our work, we further decouple the updates of and AW. First, AW is updated as:
AW + (1 - >W>aWWW - n) AWW Sioal c Mask (IAW,) > 0},
(3)
where 77, is the learning rate, na is the weight decay for AW and Mask passes gradient flows only through the non non-pruned elements. Simultancously, T is updated independently:
T+ (1 .T,AAr Tr. V Lautal:
(4)
4

---

Published as a conference paper at ICLR 2026
where it is governed by its own learning rate 1/, and separate weight decay parameter X allowing for fine- geeeggraied control over the final sparsity. We empirically found that applying the naive STR causes instability during the fine-tuning process in equivariant MLIPs due to the coupled decay of and AW. Decoupling these updates resolves this issue and provides stable, controllable fine-tuning.
EXPERIMENT
We demonstrate the effectiveness of our adaptation method MACE and NequlP potentials (Bata- tia et al., 2022; 2025; Kovács et al., 2025; Tan et al., 2025) through comparison with two major base- lines: standard full-parameter fine-tuning and ELoRA (Wang et al., 2025). This section presents the experimental setup, including datasets and hyperparameters. In addition, we report experiments that extend the task predicting magnetic properties, beyond the standard force and energy predictions.
4.1 DATASETS
We use four benchmark datasets in experiments. By default, each dataset is randomly split into training, validation, and test sets with an 8:1:1 ratio.
Inorganic erystals. For inorganic crystals, we use nine subsets from the LAM benchmark (Peng et al., 2026), including both single-element and compound systems. Specifically, we employ Cu (Zhang et al., 2020), Sn (Chen, 2023), Ti (Xu et al., 2024). V (Wang, 2022), W (Wang et al., 2022). H2O-PD (Zhang, 2021), SSE-PBE (Huang et al., 2021), AgUAu-PBE (Wang et al., 2021), and AIUMg JCu (Jiang et al., 2021).
Revised MD17. For molecules, we use the revised MD17 (rMD17) dataset (Christensen & von Lilienfeld, 2020), which provides reliable energies and forces derived from tighter quantum- mechanical calculations compared to the original MD17 dataset (Chmiela et al., 2017). The rMD17 dataset contains snapshots from long molecular dynamics trajectories of ten small organic molecules, with 100,000 structures each. Following the evaluation protocol of rMD17, we perform training and testing using 1,000 samples for training and a distinct set for validation.
TM-O-Spin. We construct the TM-O-Spin dataset via DFT calculations using Quantum ESPRESSO (QE, Giannozzi et al. (2009)). The dataset covers transition metal systems, specifi- cally elemental Mn, Ni, and Fe, as well as transition metal oxides MnO and NiO. We generate ~1,000 structures across varying pressures, magnetic orders, and random displacements. Here, we employ the ACBNO functional scheme through patched version of QE (Agapito et al., 2015; Gi- annozzi et al., 2009; Lee & Son, 2020; Yang et al., 2021) to properly account for strong electronic correlations in transition metal oxides. Details of DFT computations provided in Appendix C.
MP-mag. Further, we construct MP-mag dataset, a magnetic subset of the Materials Project trajec- tory (MPTrj) dataset (Deng et al., 2023), which is large-scale collection of relaxation trajectories derived from the Materials Project database (Jain et al., 2013). The MPTrj dataset comprises ap- proximately 1.6 million bulk crystal configurations covering 89 elements and has been widely used for training foundation MLIPs such as MACE-MP-Ob3. From MPTrj, we select systems in which least one atom carries a magnetic moment larger than 0.1 μn, and the total number of atoms per structure is less than 100. This filtering resulted in a focused set of magnetic crystals suitable for benchmarking spin-related tasks.
4.2 BASE MODELS AND TRAINING DETAILS
In our main fine-tuning experiments, we use MACE-MP-0b3 (Batatia et al., 2025) and MACE-OFF23 (Kovács et al., 2025) as foundation models. These architectures are updated and extended variants of the original MACE model (Batatia et al., 2022), pre-trained on the MPTrj and SPICE datasets (Deng et al., 2023; Eastman et al., 2023), respectively, and tailored to crystal and molecular domains. These models are commonly trained to reproduce the PES by minimizing energy, force, and optionally stress prediction losses. We also conduct experiments using NequlP- OAM-L (Tan et al., 2025) to demonstrate that our method generalizes across different equivariant MLIPs. Experimental details and results are provided in Appendix D.
5

---

Published as a conference paper at ICLR 2026
To extend the applicabilityy of MLIPs beyond energy and force predictions, conduct additional experiment
experiments on predicting magnetic properties. Specifically, a set of spin-aware layers, which re- ceives the final node and edge embeddings inputs, is added on top of the foundation model and predicts vector-valued non-collinear magnetic moments on each atom fág and edge-wise energy cor- rection 6ij arising from spin-spin exchange interactions. The total energy is the of the energy predicted from the foundation model and the spin contributions. Consequently, the total loss Croual weighted sum of the four properties: Liaal agLe + aF + av Lv + an where Ey denotes magnetic moment loss, and the terms are the corresponding weight coefficients. Each loss term Lg, Lr, Lv. and C is computed using Huber loss (Huber, 1964). For detailed methods and background on both magnetic-awane energy and magnetic moment estimation, see Appendix B.
For the full fine-tuning and ELoRA baselines, we adopted the optimal hyperparameters from Wang et al. (2025) to ensure fair comparison. For our method, initial learning rates were selected via grid search: x 10-2 for rMD17, x 10-3 for LAM,, and x 10-3 for MP-mag. The sparsity threshold A was set to 0.01 and 0.3 for the low and high sparsity regimes, respectively. To initialize AW, we used an empirical value of a = 0.01 for the initial weight distribution, and set the threshold parameter 8 0.001 in Equation 2.
Across all experiments, we used a batch size of 64 and a weight decay of 1 x 10-8, and employed the schedule-free AdamW optimizer (Defazio al., 2024; Loshchilov & Hutter, 2019). All models were trained on a single GPU, with results averaged over three independent runs using different set the coefficients for energy,
random seeds. For magnetic systems (TM-O-Spin and MP-mag), force, stress, and magnetic moment losses to an equal ratio of 1: 1: 1 : 1; otherwise, we follow the ratios of the MACE foundation models (Batatia al., 2025; Kováes et al., 2025).
RESULTS AND ANALYSIS
In this section, we present experimental results demonstrating that our method is comparable or superior to baselines such as full fine-tuning and ELoRA across all domains. Remarkably, these gains are achieved by modifying fewer than 3 % of the total parameters, with some tasks requiring updates to few 0.5 %. In addition, present preliminary interpretation of the model, which, to the best of our knowledge, represents a novel contribution to fine-tuning MLIPs.
5.1 ADAPTATION ON MOLECULAR AND CRYSTAL SYSTEMS
We first evaluate our method on the fundamental task of adapting foundation models for energy and force predictions. To this end, we adapt the MACE-OFF25 model on the rMD17 dataset (Christensen & von Lilienfeld, 2020) for the molecular domain and the MACE-M, 0b3 model on a subset of the LAM benchmark (Peng et al., 2026) for the inorganic crystal domain. To compare the enhancements achieved by each method, we also measure the zero-shot accuracy of the foundation models and the accuracy of models trained from scratch. We report the mean absolute error (MAE) for energy and force predictions, alongside two sparsity metrics for our method. Total sparsity measures the fraction of all model parameters that remain unchanged during fine-tuning. Layer-wise sparsity quantifies sparsity within the updated parameters AW, defined as the proportion of entries in A AW that remain exactly zero. Our method is presented in two configurations, detailed in Section 5.2: a standard setup (A, = 0.01) denoted as L that yielded the best overall predictive performance, and a high-sparsity setup (A, 0.3) denoted as
Results on Organic Molecules. The left side of Table presents our results on the rMD17 dataset. First, the results of zhoo foundation models and models trained from scratch highlight the neces- sity of fine-tuning ML.Pss Among the fine-tuning approaches, our method demonstrates competitive with the baselines. In the standard setup (I.), it surpasses both full fine-tuning and EL.RA on eight out of the ten molecules. Even more impressively, method in high-sparsity setup (H) also outperforms ELoRA across all molecules, showing the robustness of our sparse adaptation strategy on diverse organic systems.
Results on Inorganic Crystals. The results on inorganic crystal datasets shown on the right side of Table 1 further confirm the strength of method. In the standard setup, our method achieves results comparable to or superior to both full finettning and ELoRA. The strength of our approach is particularly evident for systems with large initial zero-shot errors, indicating significant dis-

---

Published as a conference paper at ICLR 2026
Table 1: Evaluation on the rMD17 (left) and the LAM benchmark (right) datasets. Energy
MAE (E), force MAE (F), and total sparsity values (Sp.) are reported in m/atommm meA, and percent (%), respectively. We evaluate our method in both low-siarsity (L., X = 0.01) and high- sparsity (H, A 0.3) regimes. Additional results including standard deviation and layer- layer-wise sparsity values are provided in Appendix D.
(MD17
7ero shor
Scratch
Method Full
ORA
Ours (1.
Our (H)
LAM
7em shot
Scratch
Method Full
FLaRA
Chrrs (1.)
Ours (H)
E
4.72
0.00
10.1
0.21
0.37
0.20
E
238.42
27.90
1:33
1.63
1.83
2.23
Aspirin
F
354.42
25.55
8.09
8.52
7.56
8.22
Al
43.17
8.23
5.59
6.82
6.10
6.63
96.84
99.62
4,50
0.16
1002
0.02
001
359,05
3.31
0.66
0.79
0.65
0.71
Benzene
233:45
7,03
1.67
0.9
30.94
7.08
3.96
5.29
4,32
83.50
06.85
Sp.
97.57
99.58
7,47
0.98
10.00
0.24
0.21
0.22
31905 00
8.60
32.74
9.33
2.18
2.49
Malonaldehyde
405.57
22.66
7.91
8.61
7.58
8.03
Sa
F
114.22
29.58
25.82
32.46
23,92
25.39
63.54
97.39
97.46
99.42
5.33
0.44
10.12
0.13
0.11
0.13
20.33
0.53
0.29
0.41
0.33
0.32
Parseetamol
232.60
6.74
SSE-PRI
38
8.92
63.29
96.92
Sp.
96.77
99.33
0.19
10:05
0.06
0.05
0.05
126. 38
8.61
2.57
3.50
2.50
3,28
Tobocne
280.54
11.78
3.31
3.01
2.70
2.92
Ti
F
133.56 56
51.38
30.24
43.65
39.01
41.03
83.31
96.87
Sp-
97.23
99.45
4.96
0.22
0.10
0.11
0.08
0.10
E
226.44
12.09
1:45
2.65
1.07
3.16
Azobenzene
353.55
17.05
6.49
6.59
6.26
6.90
V
F
168.02
41.26
23.15
31.25
25.38
29.26
Sp
97.30
Sp.
97.60
99.65
10008
0.09
378.04
7.22
2.9
4.33
3.15
Eihanol
369.59
13.28
4.24
4.58
4.12
4.49
W
F Sp.
449.23
96.13
66.00
89.26
71,71
79.01
Sp.
83.21
96.32
97.44
99.45
E
5.63
0.19
10.06
0.07
0.05
0.06
E
80. 155
14.27
3.64
1.89
1.26
1.40
Naphthalene
F
288.43
13.59
3.75
3,34
3.25
3,26
Agu PDE
F
93.36
17.55
14.03
10.39
8.55
9.24
97.33
Sp.
99.41
0.32
0.09
0.08
381.19
5.10
4.98
3,67
Salicytic
351.31
20.87
5.99
5.87
5.51
5.72
Hg D-PD
346.92
80.64
69.41
62,51
67.71
Sp.
83.05
97.17
Sp.
97.47
99.48
6.91
0.41
10.08
0.08
0.07
0.08
Uracil
365.80 86
17.41
6.03
4.96
4.52
4.92
97.44
tribution shift from the pre-training data (e.g., Sn and H₂O-PD). This was achieved by modifying approximately % of the model's parameters. For the high-sparsity setup, by updating merely 0.5- 0.7 % of the total parameters, our method still demonstrates comparable accuracy to full fine-tuning and better prediction accuracy than ELoRA for energy and forces across nearly all subsets. This highlights method's ability to achieve performance gains through highly targeted and compact modifications. These results suggest that our approach is not confined to a single material class but can generalize across diverse chemical domains, from inorganic crystals to organic molecules. To further verify that our method is not specific to the MACE architecture, we additionally evaluate on the NequiP-OAM-L architecture (Tan et al., 2025). As shown in Table I in Appendix D, our method achieves comparable or superior accuracy to full fine-tuning and ELoRA across all bench- mark datasets, demonstrating its generalizability across different equivariant architectures.
5.2 ABLATION STUDIES
Our framework allows selective adaptation in two directions: by targeting specific model layers and by controlling the pruning strength via the threshold weight decay X.. We conduct ablation studies to explore both aspects and describe our experimental design, then report key findings.
First, we leverage framework's 's ability to perform modulc-specific tuning. We investigate three distinct strategies: (I) updating only the linear layers, (2) only the fully-connected tensor product (FCTP) layers, or (3) both, which we refer to Linear, FCTP, and All settings.
As shown in Figure 1, the predictive performance trends for the Linear and All settings are nearly identical across wide range of X, values. Conversely, the FCTP setting consistently yields the poorest performance, indicating that the majority of the adaptation capability is captured by linear layers. Results on TM-O-Spin (shown in Figure II in Appendix D) demonstrate similar tendency.
7

---

Published as a conference paper at ICLR 2026
Energy (meV/alom)
Force (meviA)
55
Layer sparsity (96)
100
FCTP
75
50
45
3-
Linear
25
All
40
0
10"2
10
10"2 10
10
10
Ar
Ae
Ae
Figure 1: Ablation study on the effect of the threshold weight decay (A,) for Ti dataset. Energy and force MAEs along with layer-wise sparsity are plotted against varying X, values. Three different update strategies are compared: updating only the linear layers, only the FCTP layers, or both layers (denoted as All). The tendency of the linear and All configurations similar and significantly superior to that of FCTP, showing a stable, high-sparsity regime at large A values.
Table 2: Evaluation on TM-O-Spin dataset. Energy (me V/atom,, force (me V/A), and magnetic moment (p) MAEs along with sparsity values (%) are presented for comparison with the baselines.
Method
Metric
Scratch
Full
ELoRA
Ours (L)
Ours (H)
Energy
27.28
11 2.88
12.44
9.50+a.11
10.57
Force
174.70
96.89 0.029
116.31
70.46
74.15
Magnetic moment
0,035
0.038.10014
0.028
0.030
Sparsity (total)
90.2111001
91.83.1000
Sparsity (layer)
43.17:10.0
89.70...29
However, a closer look at the module sparsity trends reveals a key difference between the Linear and All settings. As shown in Figure 1, for the Linear-only strategy, sparsity increases gradually with increasing X,. In contrast, for the All strategy, module sparsity increases sharply at much earlier stage. This suggests that the linear parameters are more critical for maintaining performance and thus pruned more reluctantly by the model, point we revisit in our interpretation section.
Furthermore, the energy plots for both the Linear and All settings commonly exhibit distinet, effective adaptation regimes, First, a low-sparsity or high-performance regime (labeled as Ours (L) in Table 1) exists small 7 with low module sparsity. For our main experiments, selected the optimal point from this regime, defined as the X, value yielding the best performance with the most compact update. This corresponds X, 0.01 for most datasets, except for the TM-O-Spin, where the optimum occurs at X, 0.05.
Second, we identified a high-sparsity, stable regime (labeled as Ours (H)) at larger X from 0.3, which maintains strong performance with extreme sparsity despite being slightly sub-optimal. To represent this regime, we consistently choose X, 0.3 across all datasets. We hypothesize that this high-sparsity regime forces the model to distill the adaptation into the most essential parameter updates. This provides hints for interpretability, as we will explore further Section 5.4.
5.3 EXTENSION TOWARDS MAGNETIC SYSTEMS
Magnetism often emerges in transition- metal systems, where spin degrees of freedom play a central role. The relevant energy scale typically on the order of 10 metomm, making accurate prediction essential. Extending non-magnetic foundation models is therefore highly efficient strategy, as it enables the reuse of pre-trained knowledge while capturing the subtle energy differences associated with magnetic configurations. We add spin-aware layers (+8.6 % parameters) to MACE-MP 0b3, which are trained from scratch, while our original foundation model parameters. Further architectural details are provided in Appendix B.
adaptation is applied exclusively to the
We first evaluate this approach using our custom TM-O-Spin dataset. As shown Table 2, fine- tuning from the pre-trained model provides substantial improvements derscoring the importance domain-specinn mine-tunngg.... While predictions for spin-aware energy showed only slight improvements, the accuracy of total energy and forces improved significantly,
training from scratch, un-
8

---

Published as a conference paper at ICLR 2026
Table 3: Evaluation on the MP-mag dataset. Energy (meV/atom), force (meV/A), and magnetic
moment (p) MAEs along with sparsity values (%) are presented for comparison with the baselines.
Metric
Method
Energy
Force
Magnetic moment
Sparsity (total)
Sparsity (layer)
Full
18.75
37.56_0.16
0.015
El.oR
17.37
40.34
0.01
Ours
Linear, A 0.01
17.56
42.11 +0.35
0.012
90.15
41.27++100
Linear, A = 0.3
18. 48.4
39.60=0.77
0.014
91.83++001
89.79+t.3
All, X 0.05
76.561071
40.75
0
30.7810.12
10.90
All, 0.3
17. 17.20
39.59
0.013:10000
87.531004
94.00
Ours: AgAu
Ours: TM-O-Spin
ELORA: AgAu
ELORA: TM-O-Spin
10-2
s-like 1st
Interactions
p-like
10
d-like 1st
f-lkk 1st
10-5
s-like 2nd
10-6
40
60
80
10
20
30
40
60
80
10
20
30
Atomic Species
Atomic Species
Atomic Species
Atomic Species
Figure 2: Comparison of parameter update patterns our method and ELoRA, fine-tuned on the TM-O-Spin dataset. The y-axis denotes interaction channels corresponding to orbital-like symmetries (s, p, d, f' different layers: /st indicates the first layer and 2nd the second layer. The z-axis shows atomic species indices, and colors represent the magnitude of parameter changes quantified by 1 R² (coefficient of determination), with darker colors indicating stronger updates.
indicating the effectiveness of our approach. Moreover, compared to the baselines, our method with the high-sparsity setting (i.e., Ours (H)) attains accuracy comparable to full fine-tuning while sub- stantially outperforming ELoRA. An ablation study shows trends consistent with those observed on the dataset (see Figure in Appendix D).
Furthermore, to assess scalability, we evaluate our approach on the large-scale MP-mag dataset, shown in Table 3. On this broader and more diverse dataset, differences predictive performance among training methods diminished. Full fine-tuning achieved the best force prediction accuracy, while our All setting with low sparsity (A, 0.05) was optimal for other metrics. This suggests that dense updates become more competitive when trained on sufficiently large and diverse in-domain datasets, revealing the practical trade-offs among different fine-tuning strategies.
5.4 MODEL INTERPRETATION
Our sunsstt-promoting framework provides fundamental interpretation of the fine-tuned model, enabling us to identify which physical interaction pathways are most critical for adapting to new task. We analyze the updated parameters from the high-sparsity All setting, as results in highly selective changes across both linear and FCTP layers. We quantify the magnitude of these changes, especially for FCTP layers, by measuring the R² difference of the path-wise weights before and after fine-tuning.
Primary results from TM-O-Spin dataset, shown in Figure 2, reveal physically intuitive adaptation patterns. When fine-tuned with our method, the model exhibits prominent parameter updates in pathways associated with p- and d-orbital-like features for the transition metals, and s- and p-like features for oxygen. This aligns with fundamental physical principles that the valence electrons of transition metals occupy d-orbitals, while those of oxygen are the s- and p-orbitals, which govern their chemical bonding. In contrast, ELoRA produces parameter updates for elements outside the training set and modifies all layers, which may not be necessary. Similar update patterns are also observed in the AgAu dataset.

---

Published as a conference paper at ICLR 2026
Such update pattern arises from the low-rank construction of AW in ELoRA, where AW AB. In this formulation, any parameter update applied to A or B propagates across entire rows or columns of AW. Since these low-rank factors are shared across all atomic species channels, updates originating from species present in the training data inevitably propagate to parameters for diffuse, non-specific update pattern that
all atomic species in the reconstructed AW, leading hinders clear interpretation. This demonstrates that our sparsity-promotte approach can provide fundamental interpretability, highlighting how a foundation model calibrates its physical knowledge to new domains.
5.5 COMPUTATIONAL COST
We measured the average training iteration time and peak memory consumption for full fine- tuning. ELoRA, and our method, as shown in Table 4. The experiments were con- ducted on the rMD17 Aspirin dataset using the OF model with single NVIDIA H100 GPU. We note that during inference, AW merged with W, resulting in identical execu- tion speed and memory footprint to those of the original model; therefore, only training iteration time and memory consumption are measured. In Appendix D, we further provide comparison of convergence speed across the three methods.
Table 4: Computational cost of each method. We report the average training iteration time (msec) and peak memory consumption (GB) of each fine-tuning method.
ELORA (r 16)
Ours (Lineur)
Ours (FCTP)
Ours (All)
Full
Time (msee)
144.92
165.56
159.49
149.77
161.75
Memory (GBB
5.67
5.18
Training Iteration Time. Our method exhibits a small overhead in training iteration time, typically 3-11 % slower than full fine-tuning, However, it outperforms ELoRA in terms of training speed, as the matrix multiplications required to reconstruct AW in ELoRA create a computational bottleneck.
Memory Usage In the case of our Linear setting, the memory overhead is marginal compared to full fine-tuning. While the FCTP and All settings exhibit increased memory usage relative to full fine-tuning, they remain comparable to that of ELoRA. This behavior stems from the distinctive training mechanism of MLIPs. In conventional neural network training, memory consumption is of- ten dominated by optimizer states that scale with the number of parameters, where ELoRA achieves memory savings by reducing this overhead. In contrast, MLIP training is primarily dominated by intermediate activations and gradients required for autograd-basen force prediction. Consequently, the size of the optimizer state has only minor influence on the overall memory footprint, resulting in comparable memory usage between ELoRA and our method.
CONCLUSION AND DISCUSSION
In this paper, we propose a novel prommttig fine-tuning method tailored for equivariant MLIPs. By updating small subset of parameters, few as 0.5 % in some cases, method achieves comparable or superior accuracy to full parameter fine-tuning and existing equivariant fine- tuning approach. Furthermore, demonstrate its versatility by extending foundation ML.P the challenging task of magnetie-aware energy and magnetic moment prediction. These results present the promise of extending foundation models to broader tasks beyond traditional force and energy approach offers interpretability, clearly revealing
prediction. In addition, our physically meaningful insights into the fine-tuned model. Taken together, our approach suggests promising direction for interpretable adaptation of materials foundation models.
Outlook. While fine-tuning is a common paradigm, its inherent nature often limits computational efficiency gains, one of the key advantages associated with sparse neural networks. In light of this, structured sparse pre-training represents a promising direction toward providing MLIPs with both computational efficiency and enhanced interpretability (Wen et al., 2016). Such an approach would enable genuine hardware acceleration through structured sparsity patterns, allowing MLIPs to perform large-scale simulations faster than current dense implementations. Furthermore, sparse pre-training presents opportunities for designing sparsity- aware interpretable MLIP architectures fundamentally distinct from current MLIPs, building upon the physically meaningful basis selection demonstrated in our work. These directions suggest that the intersection of structured sparsity and interpretable design could define the next generation of equivariant MLIPs.
10

---

Published as a conference paper at ICLR 2026
REPRODUCIBILITY STATEMENT
To enhance reprodueibility of experiments in this work, we described the overall experimental setup in Section 4. For the magnetism-related experiments, we constructed dataset through our own DFT calculations and extended the MACE model architecture with spin- pin-prediction layers predict magnetic properties, with the details provided in Appendix B and C. The implementation code for our proposed method included in the Supplementary Materials.
ETHICS STATEMENT
This work presents pannsttypromoting fine-tuning method for equivariant MLIPs and does not raise specific ethical concerns. Our research does not involve human subjects, sensitive data, or potential dual-use applications.
ACKNOWLEDGMENTS
This research was supported by the National Research Foundation of Korea (NRF) grants (RS-
2021-NR05515, RS- 2S-2002--03366576 RS-2023-0026663, RS-2024-00342044, RS-2025-1 16063688, RS-2025-00555621, RS- (S-20025-02293161, the Institute of Information & Communications Tech-
nology Planning & Evaluation (IITP) grants IRS-2019-1119005,, RS-2022-1220264, RS-2024- 00353131, RS-2025-026531133
the National Supercomputing Center (KSC-2025-CRE-00766 the High-Performance Computing Support Project funded by the Korea government. This work was also supported by Samsung Electronics, the SOFT Foundry Institute at SNU, the Youlchon Foun- dation, the Ewha Global Excellence Program Research Grant, and the Korea Institute for Advanced Study, including computing resources from the Center for Advanced Computation in KIAS.
REFERENCES
Luis A. Agapito, Stefano Curtarolo, and Marco Buongiomo Nardelli. Reformulation of DFT as a pseudohybrid hubbard density functional for accelerated materials discovery. Phys. Rev. X, 5:01 1006, 2015. doi: 10.1103/hysReex.5.001100
Cited on p. ii
Kambiz Azarian, Yash Bhalgat, Jinwon Lee, and Tijmen Blankevoort. Learned threshold pruning.
2007:20003000075. 2020. Cited on p.
Luis Barroso-Luque, Muhammed Shuaibi, Xiang Fu, Brandon M Wood, Misko Dzamba, Meng Gao, Ammar Rizvi, C Lawrence Zitnick, and Zachary W Ulissi. Open materials 2024 (omat24) inorganic materials dataset and models. arXiv:2410. /2771, 2024. - Cited on p. 1,3
Ilyes Batatia, David P Kovacs, Gregor Simm, Christoph Ortner, and Gábor Csányi. MACE: Higher order equivariant message passing neural networks for fast and accurate force fields. In Proc. the Advances in Neural Information Processing Systems (NeurIPS), 2022. Cited on p. 3,
Ilyes Batatia, Philipp Benner, Yuan Chiang, Alin M. Elena, Dávid P. Kovács, Janosh Riebesell, Xavier R. Advincula, Mark Asta, Matthew Avaylon, William J. Baldwin, Fabian Berger, Noam Bernstein, Arghya Bhowmik, Filippo Bigi. Samuel M. Blau, Vlad Cärare, Michele Ceriotti, Sang- gyu Chong, James P. Darby, Sandip De, Flaviano Della Pia, Volker L. Deringer, Rokas Elijošius, Zakariya El-Machachi, Edvin Fako, Fabio Falcioni, Andrea C. Ferrari, John L. A. Gardner, Mikolaj Gawkowski, Annalena Genreith-Scchrieeed Janine George, Rhys E. A. Goodall, Jonas Grandel, Clare P. Grey, Petr Grigorev, Shuang Han, Will Handley, Hendrik H. Heenen, Kersti Hermansson, Cheuk Hin Ho. Stephan Hofmann, Christian Holm, Jad Jaafar, Konstantin S. Jakob, Hyunwook Jung, Venkat Kapil, Aaron D. Kaplan, Nima Karimitari, James R. Kermode, Panagio- tis Kourtis, Namu Kroupa, Jolla Kullgren, Matthew Kuner, Domantas Kuryla, Guoda Liepuo- niute, Chen Lin, Johannes T. Margraf, Ioan- Bogdan Magdău, Angelos Michaelides, J. Harry Moore, Aakash A. Naik, Samuel Niblett, Sam Walton Norwood, Niamh O'Neill, Christoph Ortner, Kristin A. Persson, Karsten Reuter, Andrew S. Rosen, Louise A. M. Rosset, Lars L. Schaaf, Christoph Schran, Benjamin X. Shi, Eric Sivonxay, Tamás K. Stenczel, Christopher Sut- ton, Viktor Svahn, Thomas D. Swinburee, Jules Tilly. Cas der Oord, Santiago Vargas, Eszter Varga-Umbrich, Umbrich, Tejs Vegge, Martin Vondrák, Yangshuai Wang, William C. Witt, Thomas Wolf, Fabian Zills, and Gábor Csányi. A foundation model for atomistic materials chemistry. The

---

Published as a conference paper at ICLR 2026
Journal of Chemical Physies, 163(18):184110, 2025. ISSN 0021-9606. doi: 10.1063/5.029706.
-Cited on 1, 2, 3,5,6
Simon Batzner, Albert Musaelian, Lixin Sun, Mario Geiger, Jonathan P. Mailoa, Mordechai Ko- rbbluth, Nicola Molinari, Tess E. Smidt, and Boris Kozinsky, E())-qqiivariant graph neural net- works for data-effiiieen and accurate interatomic potentials. Nature Communications, 13(1):2453,
2022. - Cited on p. ii
Jörg Behler, Karsten Reuter, and Matthias Scheffler. Nonadiabatic effects in the dissociation of oxygen molecules at the al (111) surface. Physical Review Condensed Matter and Materials Physics, 77(11):115423 2008 Cited on p.2
Thomas B Blank, Steven D Brown, August W Calhoun, and Douglas J Doren. Neural network models of potential energy surfaces. The Journal of chemical physics, 103(10):4129-4133 1995.
Cited on 1.
Steven L. Brunton, Joshua L. Proctor, and J. Nathan Kutz. Discovering governing equations from data by sparse identification of nonlinear dynamical systems. Proc. of the National Academy of Sciences, (3((15):333-3399 2016. - Cited on 2,
Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski. and Armand Joulin. Emerging properties in supersuue vision transformers. In Proc. of the IEEE conference on computer vision and pattern recognition (CVPR), 2021. -Cited on p. 1,3
Chi Chen and Shyue Ping Ong. A universal graph deep learning interatomic potential for the periodic table. Nature Computational Science, 2(11):718-728 2022. Cited on p. 1.3
Tao Chen. Modeling the high-pressure solid and liquid phases of tin from deep potentials with ab initio accuracy. Physical Review Materials, 7(5), 2023. doi: M0a1 053603. - Cited on p.5
Stefan Chmiela, Alexandre Tkatchenko, Huziel F. Sauceda, Igor Poltavsky, Kristof T. Schütt, and Klaus-Robert Müller. Machine learning of accurate energy-conserving molecular force fields. Science Advances, 365)xe1 May 2017. doi: 10.1126/sciadv.11000555
Cited on p.
Anders S Christensen and O Anatole von I Lilienfeld. On the role of gradients for machine learning of molecular energies and forces. Machine Learning: Science and Technology, 1(4):045018, 2020. ISSN 2632-2153. doi: 1..11///632-255//bbbbaee
-Cited on 5,
Aaron Defazio, Xingyu Yang, Harsh Mchta, Konstantin Mishchenko, Ahmed Khaled, and Ashok Cutkosky. The road less scheduled. In Proc. the Advances in Neural Information Processing Systems (NeurIPS), 2024. Cited on 6
Bowen Deng, Peichen Zhong, KyuJung Jun, Janosh Riebesell, Kevin Han, Christopher Bartel, and Gerbrand Ceder. Chgnet pretrained universal neural network potential for charge-informed atomistic modelling. Nature Machine Intelligence, 1031110311 2023. ISSN 2522-5839. doi:
Cited 3,
Peter Eastman, Pavan Kumar Behara, David L. Dotson, Raimondas Galvelis, John Herr, Josh T. Horton, Yuezhi Mao, John D. Chodera, Benjamin P. Pritchard, Yuanqing Wang, Gianni De Fabri- tiss, and Thomas E. Markland. Spice, a dataset drug-like molecules and peptides for train- ing machine learning potentials. Scientific Data, 10(1):11, 2023. ISSN 2052-4463, doi: 10. 10010388/41557--22-00888
-Cited on 5.5
Jonathan Frankle and Michael Carbin. The lottery ticket hypothesis: Finding sparse, trainable neural networks. In Proc. the International Conference on Learning Representations (ICLR), 2019. Cited on p.3
John LA Gardner, Daniel F Toit, Chiheb Ben Mahmoud, Zoé Faure Beaulieu, Veronika Juraskova, Laura-Biam Pasca, Louise AM Rosset, Fernanda Duarte, Fausto Martelli, Chris J Pickard, et al. Distillation of atomistic foundation models across architectures and chemical domains. arXiv:2506. 10956, 2025. -ited on p.3
12

---

Published as a conference paper at ICLR 2026
Mario Geiger and Tess Smidt. e3nn: Euclidean neural networks. arXhv:2207.09453, 2022. Cited on 3,4,
Paolo Giannozzi, Stefano Baroni, Nicola Bonini, Matteo Calandra, Roberto Car, Carlo Cavazzoni, Davide Ceresoli, Guido Chiarotti, Matteo Cococcioni, Ismaila Dabo, Andrea Dal Corso, Ste- fano de Gironcoli. Stefano Fabris, Guido Fratesi, Ralph Gebauer, Uwe Gerstmann, Christos Gougoussis, Anton Kokalj, Michele Lazzeri, Layla Martin-Samos, Nicola Marzari, Francesco Mauri, Riccardo Mazzarello, Stefano Paolini, Alfredo Pasquarello. Lorenzo Paulatto, Carlo Sbraccia, Sandro Scandolo, Gabriele Sclauzero, Ari P Seitsonen, Alexander Smogunov, Paolo Umari, and Renata M Wentzcovitch. Quantum espresso: a modular and open-source software project for quantum simulations of materials. Journal of Physics: Condensed Malter, 21(39): 395502, 2009. doi: 10. 22.088/0053-888422179999955
-Cited on p. ii
Justin Gilmer, Samuel S. Schoenholz, Patrick F. Riley, Oriol Vinyals, and George E. Dahl. Neural Message Passing for Quantum Chemistry. In Proc. the International Conference on Machine Learning (ICML), 2017. Cited on p.2
Song Han, Jeff Pool, John Tran, and William Dally. Learning both weights and connections for effi- cient neural network. In Proc. the Advances in Neural Information Processing Systems (NeurIPS), 2015.
Cited on 3.3
Torsten Hoefler, Dan Alistarh, Tal Ben-Nun, Nikoli Dryden, and Alexandra Peste. Sparsity in deep learning: Pruning and growth for efficient inference and training in neural networks. Journal of Machine Learning Research, 22(241):1-124, 2021. Cited on p.2.
Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, An- drea Gesmundo, Mona Attariyan, and Sylvain Gelly. Parameter-efficient transfer learning for nlp. In Proc. the International Conference Machine Learning (ICML), 2019. Cited on p.2
Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In Proc. the Interna- tional Conference on Learning Representations (ICLR), 2022. - Cited p.2
Jianxing Huang, Linfeng Zhang, Han Wang, Jinbao Zhao, Jun Cheng, and Weinan E. Deep potential generation scheme and simulation protocol for the Li10GeP2SI 12ttype superionic conductors. The Journal of Chemical Physics, 154(9):094703, 2021. ISSN 0021-9606 doi: 10.1063/5.004 0066//500048499
-ited 5.5
Peter J. Huber. Robust Estimation of Location Parameter. The Annals of Mathematical Statistics, 35(1):73-101, March 1964. -Cited on 6
Anubhav Jain, Shyue Ping Ong, Geoffroy Hautier, Wei Chen, William Davidson Richards, Stephen Dacek, Shreyas Cholia, Dan Gunter, David Skinner, Gerbrand Ceder, and Kristin a. Persson. The Materials Project: A materials genome approach to accelerating materials innovation. APL. Materials, 1(1):011002, 2013. ISSN 2166532X. doi: 10.1063/1. .88 48112223. - -Cited on 3.
Wanrun Jiang, Yuzhi Zhang, Linfeng Zhang, and Han Wang. Accurate Deep Potential model for the Al-Cu-Mg alloy in the full concentration space*. Chinese Physics B, 30(5):050706, 2021. ISSN 1674-1056. doi: 10.1088/1611 0188//677-1055660 104410b///bb/44 -Cited p.5
Kadierdan Kaheman, J. Nathan Kutz, and Steven L. Brunton. SINDy-PI: a robust algorithm for parallel implicit sparse identification of nonlinear dynamics. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, 27602242):20200279, 2020. -Ctee on p.3
W. Kohn and L. Sham. Coff-Consistent Equations Including Exchange and Correlation Effects. Physical Review, 140(4A):1113--1111 1965. doi: 10. 10.1 10.1103/PhysRev144000 140. 140.A1133. - -Cted on p.
Risi Kondor, Zhen Lin, and Shubhendu Trivedi. Clebsch- gordan nets: fully fourier space spherical convolutional neural network. In Proc. the Advances in Neural Information Processing Systems (NeurIPS), 2018. Cited 3.3
13

---

Published as a conference paper at ICLR 2026
Lingyu Kong, Nima Shoghi, Guoxiang Hu, Pan Li, and Victor Fung. Mattertune: integrated, user- friendly platform for tine-tuning atomistic foundation models to accelerate materials simulation and discovery. Digital Discovery, (18):2253-2266) 2025. -Ctee p.
Dávid Péter Kováes, J. Harry Moore, Nicholas J. Browning, Ilyes Batatia, Joshua T. Horton, Yixuan Pu, Venkat Kapil, William Witt, Ioan-Bop -ogddan Magdäu, Daniel J. Cole, and Gábor Csányi MACE-OFF: Short-Range Transferable Machine Learning Force Fields for Organic Molecules. Journal of the American Chemical Society, (77221):75588111 17611, 2025. ISSN 0002-7863. doi: 10. (a.102/jacc.4c000000 - Cited on 2, 6,
Aditya Kusupati, Vivek Ramanujan, Raghav Somani, Mitchell Wortsman, Prateek Jain, Sham Kakade, and Ali Farhadi. Soft threshold weight reparameterization for learnable sparsity. In Proc. the International Conference on Machine Learning (ICML), 2020. - -ited on p.
Hung M Le and Lionel M Raff. Cis-> trans, trans-+ cis isomerizations and n-- bond dissociation of nitrous acid (hono) on an ab initio potential surface obtained by novelty sampling and feed- forward neural network fitting. The Journal of chemical physics, 128(19), 2008. Cited on p.
Sang-Hoon Lee and Young- Young-Woo Son. First-principles approach with pseudohybrid density func- tional for extended hubbard interactions. Phys. Rev. Res., 2:043410, 2020. doi: 10.1103/ PhyssRerReserrchh.2.4...1.
Cited p.5,
Daniel S Levine, Muhammed Shuaibi, Evan Walter Clark Spotte-Smith, Michael G Taylor, Muham- mad R Hasyim, Kyle Michel, lyes Batatia, G' G'abor Cs' anyi, Misko Dzamba, Peter Eastman, al. The open molecules 2025 (omol25) dataset, evaluations, and models. 1rXiv:2505.0876. 2025.
Cited on p. 1,
Hao Li, Asim Kadav, Igor Durdanovic, Hanan Samet, and Hans Peter Graf. Pruning filters for efficient convnets. Proc. the International Conference on Learning Representations (ICLR), 2017. - -Cied on p.3
Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language-image pre- training for unified vision-language understanding and generation. Proc. the International Conference on Machine Learning (ICML), 2022. Cited on p. 1,3
Yi-Lun Liao and Tess Smidt. Equiformer: Equivariant graph attention transformer for 3d atomistic graphs. In Proc. the International Conference on Learning Representutions (ICLR), 2023. Cited on p.3
Junjie LIU, Zhe XU, Runbin SHI, Ray C. C. Cheung, and Hayden K.H. So. Dynamic sparse training: Find efficient sparse network from scratch with trainable masked layers. In Proc. the International Conference Learning Representations (ICLR), 2020. Cited on p.
Xiaoqing Liu, Kehan Zeng, Zedong Luo, Yangshuai Wang. Teng Zhao, and Zhenli Xu. Fine-tuning universal machine-learned interatomic potentials: A tutorial on methods and applications. Journal of Applied Physics, 1 139(4)x04110, 2026. ISSN 00021-8979. doi: 3
Cited on
Zhuang Liu, Mingjie Sun, Tinghui Zhou, Gao Huang, and Trevor Darrell. Rethinking the value of network pruning. In Proc. the International Conference on Learning Representations (ICLR), 2019. Cited on 3.3
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In Proc. the International Conference on Learning Representations (ICLR), 2019. -ited on p.
Christos Louizos, Karen Ullrich, and Max Welling. Bayesian compression for deep learning. In Proc. the Advances Neural Information Processing Systems (NeurIPS), 2017. -Cited on p.
Christos Louizos, Max Welling, and Diederik P. Kingma. Learning sparse neural networks through Lo regularization. In Proc. the International Conference on Learning Representations (ICLR), 2018. Cited on p.3
14

---

Published as a conference paper at ICLR 2026
Muyu Lu, Siyu Hu, Chengxi Zhao, Chen Wang, Weile Jia, Guangming Tan, Jun Jiang, and Lin- jiang Chen. Efficient fine-tuning of foundation atomistic models for reversible polymorphic phase transitions in organic molecular crystals. ChemRxiv preprint, 2025, doi: 10.26434/
chemrxiv-2025-445 st.st.. - CCted on p.3
N. M. Mangan, T. Askham, S. L. Brunton, N. Kutz, and J. L. Proctor. Model selection for hybrid dynamical systems via sparse regression. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, 475((223):201 80534, 2019. Cited p.3
Niall M. Mangan, Steven L.. Brunton, Joshua L.. Proctor, and Nathan Kutz. Inferring biologi- cal networks by sparse identification of nonlinear dynamics. IEEE Transactions Molecular, Biological, and Multi-Scale Communications, 2(1):52 63, 2016. Cited on p.3
Pavlo Molchanov, Stephen Tyree, Tero Karras, Timo Aila, and Jan Kautz. Pruning convolutional neural networks for resource efficient inference. In Proc. the International Conference on Learn- ing Representations (ICLR), 2017. -Cited on 3.
Albert Musaelian, Simon Batzner, Anders Johansson, Lixin Sun, Cameron J Owen, Mordechai Ko- rnbluth, and Boris Kozinsky. Learning local equivariant representations for large-scale atomistic dynamics. Nature Communications, 14(1):579, 2023. - Cited on p.3
Mark Neumann, James Gin, Benjamin Rhodes, Steven Bennett, Zhiyi Li, Hitarth Choubisa, Arthur Hussey, and Jonathan Godwin. Orb: A fast, scalable neural network potential. arXiv:2410.2550, 2024. Cited on p.
Kyoung Tai No, Byung Ha Chang. Su Yeon Kim, Mu Shik Jhon, and Harold Scheraga. Descrip- tion of the potential energy surface of the water dimer with an artificial neural network. Chemical physics letters, 271(1-3):152-155 1997. - Cited on p.2
Yutack Park, Jaesun Kim, Seungwoo Hwang, and Scungwu Han. Scalable paraliel algorithm for graph neural network interatomic potentials molecular dynamics simulations. Journal of chem- ical theory and computation, 20(1 (1(11)))) 1:4857-48661 2024. -Cited p. 1,3
Anyang Peng, Chun Cai, Mingyu Guo, Duo Zhang, Chengqian Zhang, Wanrun Jiang, Yinan Wang, Antoine Loew, Chengkun Wu, Weinan E, Linfeng Zhang, and Han Wang. Lambench: a bench- mark for large atomistic models. npj Computational Materials, 12(1):62, 2026. ISSN 2057-3960. doi: 10. 10.1038/s41524-005
0202244-25-000 025-01929-3 -itee on 5,
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In Proc. the International Conference on Machine Learning (ICML), 2021. Cited on p. 1.3
Mariia Radova, Wojciech G. Stark, Connor S. Allen, Reinhard Maurer, and Albert Bartók. Fine-tuning foundation models of materials interatomic potentials with frozen transfer learning. upj Computational Materials, 11(1):237, 2025. ISSN 2057-3960. -Cited on p.
Janosh Riebesell, Rhys E. A. Goodall, Philipp Benner, Yuan Chiang, Bowen Deng, Gerbrand Ceder, Mark Asta, Alpha A. Lee, Anubhav Jain, and Kristin A. Persson. A framework evaluate machine learning crystal stability predictions. Nature Machine Intelligence, 7(6):836 847, 2025. -Cited on p.
ISSN 2522-5839. doi:
00158//42255-025-01000
Byungseok Roh, Jae Woong Shin, Wuhyun Shin, and Saehoon Kim. Sparse DETR: Efficient end-to- end object detection with learnable sparsity. In Proc. the International Conference on Learning Representations (ICLR), 2022. - -Cted on p.3
Samuel H. Rudy, Steven L.. Brunton, Joshua L. Proctor, and Nathan Kutz. Data-driven discovery of partial differential equations. Science Advances, 3(4):e160261 4, 2017. - -Cited on p.
Johannes Sandberg, Thomas Voigtmann, Emilie Devijver, and Noel Jakse. Feature selection for high- dich-dimensional neural network potentials with the adaptive group lasso. Machine Learning: Science and Technology, 5(2):025043, 2024. doi: 10. 108822632-2155/a44555 Cited p.3
15

---

Published as a conference paper at ICLR 2026
Victor Garcia Satorras, Emiel Hoogeboom, and Max Welling. E (n) equivariant graph neural net- works. Proc. the International Conference on Machine Learning (ICML), 2021. - Cited on p.3.i
Pedro Savarese, Hugo Silva, and Michael Maire. Winning the lottery with continuous sparsification. In Proc. the Advances in Neural Information Processing Systems (NeurIPS), 2020. Cited on p.
Kristof Schütt, Pieter-Jan Kindermans, Huziel Enoc Sauceda Felix, Stefan Chmiela, Alexandre Tkatchenko, and Klaus-Robert Müller. SchNet: A continuous-filte convolutional neural network for modeling quantum interactions. In Proc. the Advances in Neural Information Processing Systems (NeurIPS), 2017. Cited on p.2
J. S. Smith, O. Isayev, and A. E. Roitherg. ANI-1: extensible neural network potential with dft accuracy at force field computational cost. Chemical Science, 8:3192-3203, 2017. -Citdd on p.
Suraj Srinivas and R. Venkatesh Babu. Data-free parameter pruning for deep neural networks. In Proc. of The British Machine Vision Conference (BMVC), 2015. doi: 10.5244/C.29.31. - -ited on 3
Chuin Wei Tan, Mare L. Descoteaux, Mit Kotak, Gabriel de Miranda Nascimento, Seán R. Ka- vanagh, Laura Zichi, Menghang Wang, Aadit Saluja, Yizhong R. Hu, Tess Smidt, Anders Jo- hansson, William C. Witt, Boris Kozinsky, and Albert Musaclian. fiigh-perfomance training and inference for deep equivariant interatomic potentials, April 2025. -Cited on p. 7,
Nathaniel Thomas, Tess Smidt, Steven Kearnes, Lusann Yang. Li Li, Kai Kohlhoff, and Patrick Riley. Tensor field networks: Rotation-and translation-equiie ieuillation-qqiviiianll neural networks for 3d point clouds. arXiv: 088.002221111 2018 Cited on p.3
Tina Torabi. Matthias Militzer, Michael P. Friedlander, and Christoph Ortner. Scalable data-driven basis selection for linear machine learning interatomic potentials. Machine Learning: Science and Technology, 2026. Cited on p.3
Oliver T Unke and Markus Meuwly. PhysNet: A neural network for predicting energies, forces, dipole moments, and partial charges. Journal of chemical theory and computation, 15(6))3678- 3693, 2019. - Cited on p.2
M.J. van Setten, M. Giantomassi, Bousquet, M.J. Verstraete, D.R. Hamann, X. Gonze, and G.- M. Rignanese. The pseudodojo: Training and grading 85 element optimized norm-conserving pseudopotential table. Computer Physics Communications, 226:39-5:: 54, 2018. ISSN 0010-4655. doi: http:://chh..gg/1l0...dddp.p.p..
-Cited on p.
Chen Wang, Siyu Hu, Guangming Tan, and Weile Jia. ELoRA: Low-rank adaptation for equivariant GNNs. In Proc. the International Conference Machine Learning (ICMI), 2025. Cited on 2. 5,
Rui Wang. Classical and machine learning interatomic potentials for BCC vanadium. Physical Review Materials, 6(11), 2022. doi: 10.1 103/PhysRevMaterria..k 6.363333... -Cited p.5
Xiaoyang Wang, Yinan Wang, Linfeng Zhang, Fuzhi Dai, and Han Wang. tungsten deep neural- network potential for simulating mechanical property degradation under fusion service environ- ment. Nuclear Fusion, 62(12):126013, 2022. ISSN 0029-5515. doi: 10.1088/1441-44266c388 -Cited 5.5
YiNan Wang, LinFeng Zhang, Ben Xu, Xiao Wang, and Han Wang. A generalizable machine learning potential Ag-Au nanoalloys and its application surface reconstruction, segregation and diffusion. Modelling and Simulation in Materials Science and Engineering, 30(2):025003, 2021. ISSN 0965-0393. doi: 10. 00.1088/1311-63 X/ac4002. -Cited on on
Wei Wen, Chunpeng Wu, Yandan Wang. Yiran Chen, and Hai Li. Learning structured sparsity in deep neural networks. Proc. the Advances in Neural Information Processing Systems (NeurIPS), 2016. - Cited on p.3, 10
16

---

Published as a conference paper at ICLR 2026
XIA XIAO, Zigeng Wang, and Sanguthevar Rajasekaran. AutoPrune: Automatic network pruning by regularizing auxiliary parameters. In Proc. the Advances Neural Information Processing Systems (NeurIPS), 2019. Cited on 3.3
Zhao Xu, Haiyang Yu, Montgomery Bohde, and Shuiwang Ji. Equivariant graph network approxi-
mations of high-degree polynomials for force field prediction. arXiv:2411.04211. 2024. -Ciedd
on p.
Wooil Yang, Seung-Hoon Jhi, Sang- Hoon Lee, and Young- Woo Son. Ab initio study of lattice dynamics of group iv semiconductors using pseudohybrid functionals for extended hubbard inter- actions. Physical Review B, 104:10431, 2021. doi: 10. 1...1000/sssee..100 10...e.3..0..141133 -Cited on 5. ii
Haoran You, Chaojian Li, Pengfei Xu, Yonggan Fu, Yue Wang, Xiaohan Chen, Richard G. Baraniuk, Zhangyang Wang, and Yingyan Lin. Drawing early-bird tickets: Toward more efficient training of deep networks. In Proc. the International Conference on Learning Representations (ICLR), 2020. -Cited on 3.3
Linfeng Zhang. Phase Diagram of Deep Potential Water Model. Physical Review Letters, 126(23), 2021. doi: lo..i 126.236011... - Cited on p.5
Yuzhi Zhang, Haidi Wang. Weijie Chen, Jinzhe Zeng, Linfeng Zhang, Han Wang. and Weinan E. DP-GEN: A concurrent learning platform for the generation of reliable deep learning based potential energy models. Computer Physics Communications, 253:107206, 2020. ISSN 0010-
4655. doi: 10.1016fj.pc.2220.1077766
Cited on p.5
Zhenyu Zhang, Xuxi Chen, Tianlong Chen, and Zhangyang Wang. Efficient lottery ticket finding: Less data is more. In Proc. the International Conference on Machine Learning (ICML), 2021.
-Cited on p.3
Wanjia Zhao, Jiaqi Han, Siyi Gu, Mingjian Jiang, James Zou, and Stefano Ermon. Geoada: Effi- ciently finetune geometric diffusion models with equivariant adapters. In Proc. the Advances in Neural Information Processing Systems (NeurIPS), 2025. - Cited on 2,
17

---

Published as a conference paper at ICLR 2026
A BACKGROUND FOR EQUIVARIANT OPERATIONS IN EGNNS
Tensor Products EGNNS and Adaptation Strategy. The fundamental operations in EGNNS, such message passing and feature updates, performed using tensor product operations in Equa- tion 5. These operations are typically implemented using e e3n library (Geiger Smidt, 2022), and designed to guarantee two crucial symmetries required for atomistic systems: permutation in- variance and -qqiavvuvva Permutation invariance is automatically satisfied by the inherent nature of message passing on graphs, message aggregation is invariant under node permutation (see Satorras et al. (2021) for more details).
To satisfy (33))equivariane (invariance to translation, and equivariance to rotation and reflection), EGNNs for atomic systems typically use relative position vectors between atoms input, which are inherently translationally invariant. Rotational equivariance then handled by representing features as geometric objects called irreducible representations, which are tensors of specific order or type-E. This concept physically analogous to atomic orbitals:
An f 0 feature is scalar (like an s-orbital), remaining invariant under rotation.
An = feature is a vector (like a p-orbital), which transforms linearly under rotations.
An 2 feature is rank-2 tensor (like d-orbital), transforming according higher-order
rotation rules.
Furthermore, each feature indexed with the magnetic quantum number m, with -6 < m S L, and channel index k, which is the same concept of channels in deep learning, allowing the model to learn multiple distinct features for each geometric type.
For general formulation of interactions, tensors u and v with orders & and fg. Specific TPs are defined per interaction path (f1, f2 + (a). which generates output feature of type Ez. This is physically analogous to the orbital interactions, for example, interaction between p-orbital like features (fi = la = 1) can contribute to 8, p. and d-orbital-like features, as dictated by the triangle inequality: 161 Eg S L3 e Lz. The general form of the tensor product for given interaction path is:
start with tensor product (TP) operation between two
u &0 v)
Σ
Σ
Whakeh Clama Cimy Wky fym U
(5)
where & denotes a TP layer in EGNNs, Chmmi are the predefined CGCs, which enforce equiv- ariance of individual interaction paths, derived from group theory. Finally, the learnable weight tensor Wezest, E * (Mxxx(Kx))) xk2) models how strongly each interaction path should contribute the final output, based on the training data. For further information, see Satorras et al. (2021), Batatia et al. (2022), and Geiger & Smidt (2022).
Adaptation of Equivariant MI.IPs. mathematically guaranteed by the CGCs, while the learnable weights W simply modulate the strength of each symmetry-allower interaction path. Based on this principle, ka2kk without breaking the model's model to a domain by modifying path-wise weights 69622 fundamental equivariance.
As established in Equation 5, the equivariance of the network
adapt the
B DETAILS IN SPIN LAYERS
We extend the baseline MACE architecture with minimal spin-aware layers, denoted as the SpinProductio roductBl oock (fa), which processes atomic features from the foundation model, alongside spin information. These new layers take spin orientations as an additional input to predict atomic magnetic moments and their energetic contributions. If spin inputs not provided, these layers deactivated, allowing the model to operate as the standard non- -on-maanetic MLIP.
In this design, the model receives the spin directions as input (encoded as unit vectors) and the model predicts the magnitude of the magnetic moment during readout. The raw spin directional vectors first embedded into equivariant representation through linear map from the physical spin irrep (1o) to higher-dimensioaa. space with hidden irreps of 16 x (Oe + lo + 2e + 30). These embedded spin features are combined with the node and edge hidden features x; and ei from the

---

Published as a conference paper at ICLR 2026
MACE backbone via fully connected tensor product, producing spin-augmented node and edge features x and e' that incorporate both local atomic environments and spin degrees freedom.
From these apin-augmented features, the spin layers predict per-atom spin vectors S; and edge-level energy contributions Etj as
(S;, Exz) = So (x': ely):
(6)
with j.j E (1....,, N) in a system of atoms. The edge-level contributions are subsequently
aggregated across the graph,
F Pispin
Σ
Gg::
(HI))
yielding the spin-induced energy contribution Espin. The total energy can then be expressed as
Emnal Eg + Espin:
where Eg denotes the energy predicted by the foundation model. Finally, the framework predicts the magnetic moment S, associated with each atom.
This formulation can be interpreted as learning effective exchange-like interactions directly from data: the function fo acts as flexible mapping that generalizes beyond fixed analytic forms while remaining sensitive spin-dependent features. Importantly, as spin vectors enter only through their equivariant embeddings, the model automatically preserves time-reversal symmetry (TRS): flipping all spins S; ト SS leaves the energy invariant. In this sense, adding spin layers provides data- driven representation of spin interactions that captures complex spin-lattice couplings while ensuring essential physical symmetries.
C DFT CALCULATIONS FOR TM-O-SPIN DATASET CONSTRUCTION
The TM-O-Spin dataset was constructed via DFT calculations using the QE (Giannozzi et al., 2009) package. Norm-conservn cone-conervinggg pseudopotentials from the PseudoDo jo library (van Setten et al., 2018) within the PBE exchange-correlation functional were employed. A plane-wave cutoff of 100 Ry was used, and Brillouin zone sampling was performed with uniform k-point mesh gener- ated using spacing of 0.05 Á All calculations were carried out in the collinear-spin framework with a self-consistent convergence threshold of 10 Ry.
To properly account for strong electronic correlations Hubbard U in transition-men metal oxides, employed the ACBNO functional scheme, which automatically and self-consistently determines the on-site Hubbard U from the electronic structure (Agapito et al., 2015; Lee & Son, 2020; Yang al., 2021). The Hubbard U parameters were evaluated in the G-type antiferromagnetic ground state of the corresponding transition-metal oxide (MnO and NiO). This was realized through patched version of QE (Giannozzi et al., 2009; Lee & Son, 2020; Yang et al., 2021).
D ADDITIONAL RESULTS
This section provides additional experiments and analyses. We first evaluate our method on the NequlP-OAM-L architecture (Tan et al., 2025) to demonstrate generalizability across different equivariant architectures (Table 1), and then analyze the convergence behavior of our method com- pared to baselines (Figure l). We also include an ablation study on the threshold weight decay (Xz) for the O--O-Spin dataset (Figure II) and detailed evaluation results on the rMD17 (Table II) and LAM (Table III) benchmark datasets.
Results on NequIP Foundation Model. To demonstrate that our method generalizes across dif- ferent equivariant architectures, we additionally evaluated on the NequIP-OAM-L. model (Batzner et al., 2022; Tan et al., 2025), another state-of-the-art foundation MLIP built on a different archi- tecture from MACE. Following the main experiments on MACE models, we evaluated the same rMD17 and LAM benchmark datasets. Here, we adjusted the learning rates for the NequIP-OAM-L architecture, using 5 10 for full fine-tuning and x 10-2 for our method. The results presented in Table indicate that our method achieves performance comparable to or surpassing that of the full fine-tuning baseline.
ii

---

Published as a conference paper at ICLR 2026
10
Full
3 10
LAM LAM-AgAu Agāu
x
rMD17 Aspirin
ELORA
10
10
Ours (Linear)
6x10
4
10a
10*
10a
10a
10a
Training steps
Training steps
Figure I: Convergence speed analysis comparing full fine-tuning, ELoRA, and our method. We plot the total loss on the test data against the training steps for the AgUAu-PBE and Aspirin dataset. The lines represent averages three independent experiments, and the shaded regions indicate the corresponding standard deviations.
Energy (meViatom)
14
Spin E. (meV/atom)
0.040
Layer sparsity (96
100
(meliAA)
0.035
75
FCTP
70
0.030
60
10
Linear
0.025
25
All
0.020
0
10
10
10-2
10
10
10
10"2
10
Ar
Ar
Av
Figure II:
Ablation study on the effeet of the threshold weight decay (A₂) for TM-O-Spin dataset. Energy MAE, force MAE, and layer-wise sparsity are plotted against varying A values.
Analysis of Convergence Speed. We analyzed the convergence behavior of our method (Linear setting) in comparison full fine-tuning and ELoRA. For this evaluation, we selected the rMD17 Aspirin dataset and the AgUAu-PBE dataset as representative examples of organic and inorganic systems. The baseline configurations are consistent with those used the main experiment, and all models were trained for 500 epochs with their respective optimal hyperparameters. As shown Figure I, our method achieves significantly faster convergence on the AgUAu-PBE: dataset com- pared to both full fine-tuning and ELoRA. On the Aspirin dataset, our method exhibits modest but consistent advantage in convergence speed over the baselines.
E
USE OF LARGE LANGUAGE MODEL
Large language models were used solely for minor English grammar corrections. They were not involved in the research ideation, experimental design, analysis, or substantive writing.

---

Published as a conference paper at ICLR 2026
Table I: Evaluation on the rMD17 and LAM benchmark datasets, with NequIP-OAM-L as the foundation model. Energy MAE (E) and force MAE (F) values with standard deviation reported in meV/A and me V/atomm, respectively. Sparsity (Sp.) values are reported as total sparsity with layer- wise sparsity in parentheses, in percent (%). We evaluate our method with the sparsity of (), 0.03) for rMD17 dataset, and (), 0.003) for LAM datasets.
Method ELoRA
Method ELoRA
rMD17
LAM
Full
Ours
Full
Ours
E
0.37
2.38
0.33
1.33
1.17
1.10
Aspirin
F
16.34
20.50
+o.5
AlUMgUCO
5.194027
5.22 +0.34
4.39
Sp.
98.22 (71.7)
Sp.
93.90 (3.73)
E
0. 162006
0.05 0.00
00
0.67
0.35 0000
0.49
Benzene
F
4.59 9.43
4.26.10
Cu
2.81
2.76
3.11
Sp.
98.80 (81.84)
Sp.
94.23 (8.87)
E
0.35 0.42
0.35
1.55
1.60
1.56+004
Malonaldehyde
F
13.521003 14.64
13.6010.00
Sn
20.8 20.8310.2
20.9410.4
20.08
Sp.
98.40 (74.75)
Sp.
93.86 (3.14)
E
0.29
0.40
0.25
0.2 zaol
0.22
0.24
Paracetamol
F
15. 13
21.59 =0.87
15.82+t91
SSE-PBE
6.40+a21 Sp.
6.91 10.26
7.58
Sp.
98.26 (72.47)
94.41 (1181)
E
0.
08
0.16
0.14
1.77 1.62
1.58 1.005
Toluene
F
11.85 2.
8.92 oz
Ti
F
13. 33.02
30.82+1.2
Sp-
98.52 (76.59)
Sp.
93.89 (3,7)
E
0.24
0.27 +0.01
0.17
1.16
0.88 00007
0.78
Azobenzene
F
4-33
17.04+0.04
11.69
19.00+139
19.66
18.37++109
Sp.
98.54 (76.94)
Sp.
93.96 (4.61)
E
0.324004 0.34
0.01
10400
2.27 1.006
1,93 18
Ethanol
F
11.80 11.93
10.88
W
S1. +0.
54.54
52.86+11.54
Sp.
98.40 (74.81)
Sp.
93.83 (2.67)
E F
0.13
0.11 1000
0.11:0000
1.29
0,91 10005
0.78
Naphthalene
9.87
8.47
8.70
AgUAu-PBI PBE
5.5 1025
0.
5.88
Sp.
98.54 (7.01)
Sp.
94.39 (11.49)
E
0.20
0.32
0.2 1000
10.1
2.67 0.02
Salicylic
F
14.1640.17
14.43 o
13.07
H3O-PD
76.05
1..
42.39
Sp.
98.30 (73.1))
Sp.
93.76 (1.48)
E
0.
180m
0.37
0.30 to
Uracil
F
11.
16. 16.4.91 18
12. 20.500
Sp.
98.44 (75.3))
iv

---

Published as a conference paper at ICLR 2026
Table II: Evaluation on the rMD17 datasets. Energy MAE and foree MAE values with standard deviation are reported in me V/atom and me V/A, respectively. Sparsity values are reported as total sparsity with layer-wise sparsity in parentheses, in percent (%).
Method
Subset
Metric
Zero shot
Scratch
Full
ELoRA
Ours (L)
Ours (H)
Energy Force
4.72
0.60
0.19
0.21
0.17
0.20
Aspirin
354.42
25.55
8.09
8.52
7.56=0.0
8.22++0.M
Sparsity
83.19 (27.0)
96.84 (8628)
Energy
4,50
0.16=0007
0.02
0.02
0.00
0.01
Benzene
Force
233.45
7.63-0.1
1.67
0.94
0.90=0.04
0.9611005
Sparsity
83.50 (28.3))
96.85 (86.40)
Energy
7.47
0.98=n.12
0.20+aoo
0.24
0.21=004
0.22
Malonaldehyde
Force
405.37
22.661038
1.00
8.61 10.06
7.58
8.03
Sparsity
83.54 (29.00)
97.39
Energy
5.33
0.44=004
0.12
0. 13.001
0.11
0.13
Paracetamol
Force
232.60
21.39
6.74
6.65
5.92=aa5
6.53
Sparsity
83.29 (2700
96.92 (86.72)
Energy
5,54
0.19
4.00
0.06
0.05=
0.05
Toluene
Force
280.54
11.78
3.11
20.03
2.70=0.02
2.92
Sparsity
83.31 (28.0)
96.87 (86.1))
Energy
4,96
0.28
+00
0.1 +0.01
0.08 =0.01
0.10#a.01
Azobenzene
Force
353.55
+7.05
4.09
6.59
6.26=0.59
6.90
Sparsity
83.20 (27.56)
97.30 (68
Energy Force
6,93
0.33=000
0.08
0. 200000011
0.09=0.01
0.10.00.1
Ethanol
369.69
13.28
4.24
4..0
4.12-0.11
4.49
Sparsity
83.21 (27.58)
96.32 (84.15)
Energy Force
5,63
0.19
0.06
0.07
0.05
0.06.10.01
Naphthalene
288.43
13.59
3.75
3.04
3.25
3.26.10.04
Sparsity
83.29 (27.0)
97.33 (88.50)
Energy
6,13
0.32=am
0.09
0.08
0.08
0.08
Salicylic
Force
351.31
20.87
1.09
5.87
5.51 10.18
6.72
Sparsity
83.05 (26.88)
97.17 (87.38)
Energy
6.91
0.41
0.08
0.0 2000
0.07=2.00
0.08
Uracil
Force
365.86
17.41
6.03
4.9620077
4.52-0.21
4.92.10.7
Sparsity
83.69 (29.3))
97.44 (88955

---

Published as a conference paper at ICLR 2026
Table III: Evaluation on the LAM benchmark datasets. Energy MAE and force MAE values with standard deviation are reported me V/atom and meV/Å, respectively. Sparsity values are reported as total sparsity with layer-wise sparsity in parentheses, in percent (%)
Method
Subset
Metric
Zero shot
Scratch
Full
ELoRA
Ours (L)
Ours (H)
Energy Force Sparsity
238.42 43.17
27.9014.22
1.13
1.33
1.83
2.23
AlUMgUCu
8.23
5.59
6.82
6.101007
6.63
97.61 (36.59)
99.62
Energy
359.05 50.94
2..0
0.66
0.79
0.65
0.71 2004
Cu
Force
7.08
3.96
5.29
4.3240o2
4.80
Sparsity
97.57 (35.38)
99.58 (88.82)
Energy
31905.00
8.60
32.74
9.33
2.18:001
2.49
Sn
Force
114.22
29.5810m
25.8 23.53
32.46:004
23.92:10.0
25.39 20
Sparsity
97.46 (32.59)
99.42 (84.7))
Energy
20.33
0.53
0.29 +ou2
0.41 +:.m
0.33+o08
0,32
SSE-PBE
Force
82.38
12.74+t004
1.
10.85
8.92+
9.78
Sparsity
96.77 (14.16)
99.33 (82.12)
Energy
126.38
8.61 +0:45
2.57
3.50
2.
3.28
Ti
Force
133.56
$1.38
39.24=1.96
43.65
39.01+1.68
41.03:1.90
Sparsity
97.23 (26.55)
99.45 (85.47)
Energy
226.44
12.09
1..511.111
2.651:0.16
1.67
3.16
Force
168.02
41.2611294
23.15 1.69
25.38
29.26
Sparsity
97.60 (36.28)
99.65 (90.74)
Energy
378.04
7.22
2.94
4.33++0.52
3.18
4.03
W
Force
449.23
93.13
66.00+2.28
89.26
71.714211
79.01 4:246
Sparsity
97.44 (32.04)
99.45 (85.33)
Energy
80.15
14.271091
3.64
1.89
1.26100
1,40
AgUAu-PBE
Force
93.36
17.55
14.03
10.39
8.554054
9.24
Sparsity
97.38 (30.54)
99.41 (84.22)
Energy
311.19
5. 5.10
11.67
4. 4.98
3.67:1023
4, 4.84
H₂O-PD
Force
346.92
80.6418.12
84.25
69.41
62.31
67.71 1.161
Sparsity
97.47 (32.82)
99.48 (86.30)
vi


Reviews:
[{"review_id": "RLh6z7kb1E", "reviewer": "Reviewer_1j46", "summary": "The paper proposes sparsity-promoting fine-tuning for E(3)-equivariant architectures. The sparsity is achieved by introducing learnable parameters tau that control sparsity. The method is applied to the MACE architecture and benchmarked on the Inorganic crystals, Revised MD17, TM-O-Spin, and MP-mag datasets. For these datasets, the accuracies of the non-fine-tuned model, the model fitted from scratch, the fully fine-tuned model, the model fine-tuned by ELoRA, and the model fine-tuned by the proposed methods with two sparsity levels are presented. Additionally, the applicability of the presented approach for model's interpretation was studied.", "questions": "Is it correct that the memory requirements match those of the full finetuning because of the need to store dense tensors of the tau coefficients? \n\nCould you provide practical finetuning times and memory requirements for full finetuning, ELoRA, and your method for the reported experiments?\n\nIs it correct that the method is also equally applicable for invariant and unconstrained architectures?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The method produces the most accurate models compared to other approaches most of the time. Furthermore, the method is competitive with very high reported sparsity. \n\nThe benchmarking setups are diverse, including small molecules, crystals, and extension to other targets over energies and forces, along with magnetic degrees of freedom. \n\nThe method helps with the interpretability of fine-tuned models.", "weaknesses": "It seems that the method doesn't reduce computational cost and memory requirements during finetuning as the achieved sparsity would suggest. During training, it maintains the coefficients tau, which, to the best of my understanding, mirror each of the dense parameters. So, their total number equals the number of parameters in all layers that undergo finetuning. \n\nThe equivariance is preserved not by incorporating the equivariance constraints into the finetuning method itself, but instead by applying it only to scalar path weights of the model. In other words, any finetuning technique applied to these layers would preserve equivariants. Therefore, I don't think that the positioning of the method for equivariant models, e.g., in the title of the paper, is fully justified.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "EyRzqCjH91", "reviewer": "Reviewer_KPDY", "summary": "This paper proposes a sparsity-promoting fine-tuning method for equivariant materials foundation models, using Soft Threshold Weight Reparameterization (STR) to reduce the number of updated parameters. On molecular and crystalline benchmarks, it updates only a small fraction of parameters while matching or surpassing full fine-tuning.", "questions": "1. STR is applied to \\Delta W rather than W (Sec. 3.2). Does this double the number of parameters during training? Without structured sparsity, are these computations still dense?\n2. In Eq. (3), the mask uses \\Delta W > 0, which blocks gradients for negative weights. Is this a typo or an intentional design choice?\n3. The definition of sparsity is confusing. Lines 316-317 state “Total sparsity measures the fraction of updated parameters,” implying that higher sparsity means more updates. Yet Table 1 shows 80-100\\% “sparsity,” while the abstract claims only 0.5-3\\% updated. Please unify the definition.\n4. Baselines are misaligned with prior work. Taking aspirin as an example, MACE [1] reports 2.2 meV (energy) and 6.6 meV/Å (force) from scratch, while this paper reports 0.60 meV/atom (12.6 meV) and 25.55 meV/Å. Does this indicate suboptimal tuning for baselines? Please align baselines or explain the discrepancies.\n5. TM-O-Spin: Appendix Fig. I shows force MAE ≥ 70 meV/Å in ablations, but Table 2 reports 48.75 meV/Å for Ours (L). Why the mismatch?\n6. Interpretability: Why does ELoRA show updates for elements outside the training set? With one-hot element embeddings in MACE, absent elements should not activate or receive gradients. Official MACE implementations typically remove such parameters before fine-tuning.\n[1] Batatia, I., Kovacs, D. P., Simm, G., Ortner, C., & Csányi, G. (2022). MACE: Higher order equivariant message passing neural networks for fast and accurate force fields. Advances in neural information processing systems, 35, 11423-11436.", "limitations": "", "rating": 4, "confidence": 2, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "+Strong fine-tuning accuracy across molecules, inorganic crystals, and magnetic tasks.\n+Extends force-field foundation models to magnetic systems for magnetic moment prediction.\n+Provides a novel analysis linking valence electronic structure with sparse update patterns.", "weaknesses": "-Limited novelty: mainly applying STR to equivariant models.\n-Experimental results are not aligned with prior work and inconsistent across sections.\n-The interpretability analysis is not fully convincing.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "VawWaKYfso", "reviewer": "Reviewer_bcJ7", "summary": "The paper proposes a sparsity-promoting finetuning method for E(3)-equivariant MLIPs (e.g., MACE). The authors enforce the finetuned weights ΔW as a sparse format by using a Soft Threshold Reparameterization (STR) method, which only updates a small subset of interaction paths. This method can preserve equivariance during adaptation. Experiments on rMD17 (molecules), LAM subsets (crystals), and two magnetic datasets (TM-O-Spin, MP-mag) show that this sparse fine-tuning method can match or outperform full updating and ELoRA while only 0.5% to 3% of parameters are updated.", "questions": "On the reproducibility of rMD17 dataset: In MACE: Higher Order Equivariant Message Passing Neural Networks for Fast and Accurate Force Fields (Table 1), the aspirin setting reports Etot=2.2 meV (original metric), whereas Table 1 in the author’s paper reports 0.6 meV/atom. Please provide a MACE-aligned reproduction (mean±std over seeds) and, ideally, attach the exact training config/script in the rebuttal for community verification.\n\nThe baseline results appear inconsistent with those reported in prior work. For instance, in the case of aspirin, MACE [1] reports errors of 2.2 meV for energy and 6.6 meV/Å for forces when trained from scratch. In contrast, this paper reports 0.60 meV/atom (equivalent to ~12.6 meV per molecule) and 25.55 meV/Å. Such a significant discrepancy raises the question of whether the baseline models in this study were suboptimally configured or tuned. It is recommended that the authors either realign their baselines with established reference values or provide a clear explanation for these differences.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "Employing STR with a learnable threshold per layer and decoupled optimization for ΔW vs. τ is simple, and the implementation appears lightweight.\nExperimental Results: This paper covers molecules, multiple crystal subsets, and magnetic datasets. The results show that both low and high sparsity scenarios (meaning very few updated parameters), sparse finetuning gets lower error than full finetuning and ELoRA.", "weaknesses": "Evaluation: Sparse training is emphasized, but the training/inference time speed/footprint gains (if any) aren’t measured (e.g., wall-clock per step, peak memory usage). In Eq. (2), ΔW is initialized from a normal distribution and then sparsified via STR. I suspect this operation will not reduce training time and memory. Please provide memory profiles vs full finetuning and ELoRA.\nLimited Benchmarks: The experiments mainly focus on MACE. It would be better to include Nequip-OAM-L to demonstrate the effectiveness of the sparse finetuning method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "fvzpIjtKrc", "reviewer": "Reviewer_cUD3", "summary": "The authors present a method that enables pre-trained equivariant models for materials and molecules to be fine-tuned for new downstream tasks (e.g. different level of theory, different elemental compositions, etc) with very minimal parameters updates (< 5%) while maintaining equivariance. They use soft threshold weight reparameterization (STR) to learn the adaptation weights in a manner that promotes sparseness, so the original weights are only updated when necessary. Results are presented on LAM, MD17, TM-O-Spin, and MP-mag, in nearly all cases the method is reported to be equivalent or better than full fine-tuning.", "questions": "1. How much hyperparameter tuning was done for your method vs that done for full fine-tuning or ELoRA?\n2. Was the schedule-free AdamW optimizer used for all experiments?\n3. The percentage of parameters that get updated is impressively small, is there precedent for this (or examples of this) in other fields? \n4. Are the adaptation weights just added to the original weights? Or is something else happening?\n5. I was confused by line 168, does the method only adapt path weights on scalar features (ignoring higher order features) or is it saying that path weights are scalar values? Also, what weights are being adapted in the linear layers?\n6. What are the limitations of the method? \n7. Quick and easy fine-tuning is great to have, but as the foundational models get bigger distillation will also be important as well. Does any of the work here carry over to distillation?\n8. The S in the table 1. makes me think of stress not sparsity.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The work addresses an important and timely topic.\n- A notable strength is the exceptional parameter efficiency: the model adapts to new tasks with surprisingly few parameter updates while maintaining good performance.\n- The experimental design is rigorous and well-motivated, with appropriate baselines and comprehensive ablation studies.", "weaknesses": "- While reporting the fraction of parameters updated is great, it doesn’t help understand what the practical benefits of the methods are i.e. how much less memory is required?, how much faster is training than full fine-tuning?, can you store different adopters for different tasks?, etc. It would make the paper much stronger with this information included and emphasized. \n- The paper does not discuss in-depth why this fine-tuning method is consistently more accurate than full fine-tuning. This seems like a counter initiative result and is worth digging into more. \n- In particular, the MD17 benchmark is quite low signal i.e. small tweaks in hyperparameters can change the outcome.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)