MoMa: A Simple Modular Learning Framework for Material Property Prediction
Authors: Botian Wang
Yawen Ouyang
Yaohui Li
Mianzhi Pan
yuanhang tang
Haorui Cui
Yiqun Wang
Jianbing Zhang
Xiaonan Wang
Wei-Ying Ma
Hao Zhou
Venue: ICLR 2026 Poster (2026)
Keywords: material property prediction
modular deep learning
AI4Materials
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)

Abstract: Deep learning methods for material property prediction have been widely explored to advance materials discovery. However, the prevailing pre-train paradigm often fails to address the inherent diversity and disparity of material tasks. To overcome these challenges, we introduce MoMa, a simple Modular framework for Materials that first trains specialized modules across a wide range of tasks and then adaptively composes synergistic modules tailored to each downstream scenario. Evaluation across 17 datasets demonstrates the superiority of MoMa, with a substantial 14% average improvement over the strongest baseline. Few-shot and module scaling experiments further highlight MoMa's potential for real-world applications. Pioneering a new paradigm of modular material learning, MoMa will be open-sourced to foster broader community collaboration.

Published as a conference paper at ICLR 2026
MOMA: A SIMPLE MODULAR LEARNING FRAME- WORK FOR MATERIAL PROPERTY PREDICTION
Botian Wang 120
Yawen Ouyang! Yaohui Li
Mianzhi Pan Yuanhang Tang' Yiqun Wang' Haorui Cui
Jianbing Zhang Xiaonan Wang' Wei-Ying Ma
Hao Zhou'
I Institute for AI Industry Research (AIR), Tsinghua University
2 Department of Computer Science and Technology. Tsinghua University
3 School of Artificial Intelligence, Nanjing University
4 Department of Chemical Engineering, Tsinghua University
5 Hong Kong Institute of AI for Science, City University of Hong Kong
ABSTRACT
Deep learning methods for material property prediction have been widely explored to advance materials discovery. However, the prevailing pre-train paradigm often fails to address the inherent diversity and disparity of material tasks. To over- come these challenges, we introduce MoMa, a simple Modular framework for Materials that first trains specialized modules across wide range of tasks and then adaptively composes synergistic modules tailored to each downstream sce- nario. Evaluation across 17 datasets demonstrates the superiority of MoMa, with a substantial 14% average improvement over the strongest baseline. Few-shot and module scaling experiments further highlight MoMa's potential for real-world ap- plications. Pioneering new paradigm of modular material learning, MoMa is open-sourced to foster broader community collaboration.
INTRODUCTION
Accurate and efficient material property prediction is critical for accelerating materials discovery. Key properties such as formation energy and band gap are fundamental in identifying stable and functional materials (Masood et al., 2023; Riebesell et al., 2025). While traditional approaches such as density functional theory offer high precision (Jain et al., 2016), their prohibitive computational cost limits their practicality for large-scale screening (Fiedler et al., 2022; Lan et al., 2023).
Recently, deep learning methods have been developed to expedite traditional approaches (Xie & Grossman, 2018; Griesemer et al., 2023). Pre-trained force field models, in particular, have shown remarkable success in generalizing to a wide spectrum of material property prediction tasks (Shoghi et al., 2024; Rhodes et al., 2025; Wood et al., 2025), outperforming specialized models trained from scratch. These models are typically pre-trained the potential energy surface (PES) data of materials (Barroso-Luque et al., 2024) and then fine-tuned for the target downstream task.
Despite these advances, we identify two key challenges that undermine the effectiveness of current deep learning models for material property prediction: diversity and disparity.
First, material tasks exhibit significant diversity (Fig. which challenges the generalizability of existing models. For instance, prevailing force-field models are only trained on PES-derived prop- erties (e.g., force, energy, and stress) mostly focusing on crystalline materials (Yang al., 2024b; Barroso-Lqque et al., 2024). However, material tasks span a much wider variety of systems (c.g., crystals, organic molecules) and properties (e.g., thermal stability, electronic behavior), making difficult for methods trained on a limited set of data to generalize across the full spectrum of tasks.
Second, the disparate nature of material tasks presents huge obstacles for jointly training broad span of tasks in one model. Material systems vary significantly in atomic composition, bonding
Equual Contribution. Correspondence to Hao Zhou (zhouhao@ ciu.u.)ghuaa..uuucc and Jianhing Zhang (rjh@nju.edu..n). MoMa is open-sourced at https:://///. //gi thub. com com/Gens, T-TUA R/MoMa.

---

Published as a conference paper at ICLR 2026
New Dala
D2
D2
Dn
Formation Energy
Band Gap
Phonons Prediction
Compose
M
seo oofona bebgo $000 000g00
ooo
Crystalline Materials
Organic Molecules
Figure Illustration the diversity of material prop- erties (top) and systems (down). Material tasks also disparate, with different laws governing diverse proper- ties and systems. These characteristics pose challenges for material property prediction models.
Figure 2: The modular learning scheme in MoMa trains and stores a broad spec- trum of material tasks as modules, and adaptively composes them given new material property prediction task.
and structural periodicity, while their properties are governed by distinct physical laws. For exam- ple, mechanical strength in metals is primarily influenced by atomic bonding and crystal structure, whereas electronic properties like conductivity are determined by the material's electronic structure. Consequently, training a single model across wide range of tasks (Shoghi al., 2024) may lead to knowledge conflicts, hindering the model's ' ability to effectively adapt to downstream scenarios.
Drawing inspiration from modular deep learning (Pfeiffer et al., 2023), we propose MoMa, Modular framework for Material property prediction. Respecting the diversity challenge, MoMa trains multiple high- high-resource property prediction datasets into transferrable modules to support wide-span of downstream tasks. In parallel, to address the disparity challenge, MoMa encapsulates each task within a specialized module during training to avoid interference. Furthermore, in adapt- ing to each downstream task, MoMa adaptively integrates synergistic combination of modules to mitigate knowledge conflicts. high-level abstraction MoMa is provided Fig. 2.
Specifically, MoMa comprises two major stages: (I) Module Training & Centralization. MoMa trains dedicated modules for a diverse range of material tasks, offering two versions: full modules and mnemory-effician adapter modules. These modules centralized in MoMa Hub, which en- ables knowledge reuse while keeping proprietary data private. (2) Adaptive Module Composition training-free module composition algorithm. Given a target task, AMC first estimates the performance of each module via kNN label propagation representation space. then infers weighted module composition by solving con- vex optimization problem over a justified proxy error. The composed module is further fine-tuned for improved downstream adaptation. Together, MoMa offers a flexible solution achieve scalable modular learning for material property prediction.
(AMC) Fine-tuning. We devise AMC, a eepresentation-driiie
Empirical results across 17 downstream tasks showcase the superiority of MoMa, outperforming all baselines in 16/17 tasks, with an average improvement of 14% compared to the best non-modular baseline. In few-shot settings, which are common in materials science, MoMa achieves even larger performance gains to the conventional pre-train then fine-tune paradigm. Additionally, MoMa shows improved average improvements as we scale the number of modules in the MoMa Hub, and the AMC-optimized weights provide valuable insights into relationships between material properties. The code and trained modules of MoMa are open-sourced, and we envision MoMa becoming a pivotal platform for the modularization and distribution of materials knowledge, fostering deeper community engagement to accelerate materials discovery.
2 RELATED WORK
2.1 MATERIAL PROPERTY PREDICTION WITH DEEP LEARNING
Deep learning methods have been widely adopted for predicting material properties (De Breuck al., 2021). The seminal CGCNN model (Xie & Grossman, 2018) represents crystalline materials
2

---

Published as a conference paper at ICLR 2026
with multi-edge graphs and applies graph neural networks for representation learning. Subsequent work (Choudhary& & DeCost, 2021; Das et al., 2023; Yan et al., 2024; Taniai et al., 2024) has focused on improving neural network architectures to better model the inductive biases of crystals.
Another line of work develops pre-training strategies for materials (Jha et al., 2019; Magar et al., 2022; Wang et al., 2025). Recently, series of large force field models (Merchant et al., 2023; Batatia et al., 2023; Neumann et al., 2024) are trained on massive Potential Energy Surface data (Barroso- Luque al., 2024) and achieve remarkable accuracy in material tasks (e.g. thermal stability predic- tion (Riebesell et al., 2025)). Notably, the JMP model (Shoghi et al., 2024), trained across multiple domains (small molecules, catalysts, etc.), performs impressively when fine-tuned on both molecular and crystalline tasks.
Extending beyond these methods, MoMa offers modular strategy centralize diverse material knowledge into modules and adaptively compose them, yielding superior downstream performance.
2.2 MODULAR DEEP LEARNING
Modular deep learning (Pfeiffer et al., 2023; Xiao et al., 2024) represents a promising paradigm where parameterized modules are composed, selected, and aggregated for function specialization and reuse. Notable examples of modular networks include mixture-of-expere
(Jacobs et al., 1991; Shazeer et al., 2016), adapters (Houlsby et al., 2019) and LoRA (Hu et al., 2021). Recently, we have seen increasing applications of modular methods across domains such as NI.P (Pfeiffer al., 2020; Huang et al., 2024; Tan et al., 2024) and CV (Puigcerver et al., 2020; Pham et al., 2024), where its strengths in flexibility and minimizing negative interference have been demonstrated.
An important aspect of modular learning is how modules are weighted prior composition. Pre- vious adaptive module composition approaches can be broadly grouped into (1) search-based meth- ods that iteratively optimize weights based on downstream predictive performance after composi- tion (Huang et al.. 2024; Akiba et al., 2025), and (2) router-based methods that learn composition weights via additional routing network (Muqeeth et al., 2023; Lu et al., 2024). Crucially, both paradigms rely on the downstream prediction error of the composed model to guide weight alloca- tion. However, this dependence is problematic in material settings: high task disparity makes the error signals (from arbitrary module mixtures) noisy and unstable for search-based methods, while data-scarcity provides insufficient supervision for router learning. Additionally, loading all material modules during router training becomes prohibitively costly as the number of module scales.
In the context of material property prediction, modular learning remains largely under-explored. The most related work is the router- based mixture-of ofre--of---eg of-experts method MoE-(18) (Chang et al., 2022), which loads all available modules and learns routing network for embedding aggregation.
3 PROPOSED FRAMEWORK: MOMA
MoMa is simple modular framework targeting the diversity and disparity of material property prediction tasks. MoMa involves two major stages. In the first stage (Section 3.1), we train and centralize modules for diverse range of material systems and properties into MoMa Hub. In the second stage (Section 3.2), we devise a representation-driven training-free algorithm to adaptively select and compose MoMa hub modules for target task, and then fine-tune the composed model. A visual overview of MoMa is shown in Figure 3.
3.1 MODULE TRAINING & CENTRALIZATION
To better exploit the transferrable knowledge of open-soures source material property prediction datasets, we first train distinctive modules for each high-resource material task, and subsequently centralize these modules to constitute MoMa Hub.
Module Training Leveraging the power of state-of-the-ar material property prediction models, choose to employ a pre-trained backbone encoder f as the initialization for training each MoMa module. Note that MoMa independent of the backbone model choice, which enables smooth integration with other pre-trained backbones.
3

---

Published as a conference paper at ICLR 2026
(a) Module Training Contralization
Adaptive Module Composition & Fine-tuning
W
4. Fine-tuning
MoMa Hub
Prediction Fstimation
Module Composition
Full Adapter Training
2. Weight Optimization
MoMa Hub
Figure 3: The MoMa framework. (a) During the Module Training & Centralization stage (Sec- tion 3.1), MoMa trains full and adapter modules for a wide spectrum material tasks, constituting the MoMa Hub; (b) The Adaptive Module Composition (AMC) & Fine-tuning stage (Section 3.2) leverages the modules in MoMa Hub to compose tailored module for each downstream task. The AMC algorithm comprises three steps: 1. Prediction Estimation; 2. Weight Optimization; 3. Mod- ule Composition. The composed module is further fine-tuned on the task for better adaptation.
We provide two parametrizations for the MoMa modules: the full module and the adapter mod- ule. For the full module, directly treat each fully fine-tuned model backbone standalone module. The adapter module, in contrast, serves as a parameter-efficien alternative where adapter layers (Houlsby al., 2019) inserted between each layer of the backhone. The adapters updated and the rest of the backbone is frozen. All adapters trained for given task are collectively treated as one module. This implementation trade-offs the downstream performance for much lower GPU memory cost during training, making it especially suitable for compute-constrant set- tings. When training converges, all module parameters are stored into centralized repository H termed MoMa Hub, formally:
(full module) (adapter module)
H = (911 02. (.1.,9.......0 IN
where of and Aj denote the full and adapter module parameters for the 2th task and encoder f.
Module Centralization To support a wide array of downstream tasks, MoMa Hub needs in- clude modules trained on diverse material systems and properties. Currently, MoMa Hub encom- passes 18 material property prediction tasks selected from the Matminer datasets (Ward et al., 2018) with over 10000 data points. These tasks span across a large range of material properties, including thermal properties (e.g. formation energy), electronic properties (e.g. band gap), mechanical prop- erties (e.g. shear modulus), etc. For more details, please refer to Section C.I. Note that MoMa is designed to be task-agnostic and may readily support larger spectrum of tasks in the future.
An important benefit of the modular design of MoMa Hub is that it preserves proprietary data, which prevalent in the field of materials, enabling privacy-aware contribution of new modules. Therefore, MoMa could serve as an open platform for the modularization of materials knowledge.
3.2 ADAPTIVE MODULE COMPOSITION & FINE-TUNING
y. the goal of the second stage is to customize task-specific model by composing modules from MoMa Hub. Due the diversity and disparity material tasks, blindly combining modules often leads to suboptimal performance. The composition must be adaptive, composing only the most synergistic modules for each task. Furthermore, given the vast and expanding scale of the Hub, the method must be data-driven and efficient, avoiding reliance human expertise prohibitively expensive exhaustive search.
Given labeled material property prediction dataset D ((x,, yt) M
However, satisfying these requirements is non con-trivial for existing adaptive weighting paradigms. As discussed in Section 2.2, both search-based and router-based methods rely on downstream predic-
4

---

Published as a conference paper at ICLR 2026
Composed Module
Target Task
Exfoliation Energy (r-
Refractive Index (I 0.63)
Prodiction Estimation
Phonons Mode
y X
0.6
Module
Module 2
Module
MAME
0.4
Module Composition
0.2
X
Proxy Erroe
Convex
0.0
Optimization
Predictions
Weights (NX l)
Ensemble (MX I)
Label
(M (MXN
(MX 1)
0.2
0.6
0.8
Proxy MAE
Figure 4: An analytical decomposition of AMC.
Figure Scatter plot showing the relationship between ANN proxy MAE and post-finettuning MAE of standalone MoMa Hub modules on three representative tasks. Colored lines are lin- ear fits. We observe a clear positive correlation with Pearson's 0.6.
Blue arrows: per-module ANN prediction esti- mation in representation space on target task. Black arrows: convex optimization of ensem- ble proxy error obtain composition weights. Orange arrows: weight-space module composi- tion to construct the final composed module.
tion error derived from composed module as the supervision signal. In our setting, this signal less reliable: the high disparity of modules in inputs (e.g. crystals vs. molecules) and targets (e.g. energies vs. band gaps) induces highly heterogeneous representation spaces. Hence module mix- tures yield unstable representations and uninformative error signals, resulting in noisy optimization landscape that hampers search-based methods. Moreover, the scarcity of downstream data makes router training difficult and prone overfitting.
To address these limitations, we devise the Adaptive Module Composition (AMC) algorithm. In- stead of relying on prediction free strategy. Specifically, it first estimates per-module performance via kNN in the representation space, and then solves for optimal composition weights by minimizing ensemble proxy error via convex optimization. This allows AMC to efficiently identify synergistic compositions without iter- ative search extra trainable parameters. We now introduce AMC in detail. An analytical figure of AMC is provided in Fig. 4, with its formal formulation in Algorithm 1.
supervision, AMC adopts representation-ddiiee and training-
Representation--riien Prediction Estimation AMC begins by estimating the affinity of each module the downstream task. To bypass the unstable optimization landscape of arbitrary module mixtures, we first evaluate the intrinsic representation quality of each module individually. We posit that a task-aligned module should map materials with similar properties to adjacent points in the embedding space.
Formally, for each module 9j E H. we encode the training data into representations X (x(...., )- We then perform leave-one-out ANN label propagation (Iscen et al., 2019) to obtain prediction o for each instance:
Sa (x x
0.'
yku
Z
5x
(I)
2
KENG
kEA;
where N; denotes the indices of the K nearest neighbors of x within 2 and is the exponential cosine similarity function.
We choose &NN as the estimator because it directly probes the local geometry of the representation space without introducing learnable parameters. This strictly aligns with our training- free design principle and ensures robustness against overfitting data-scarce tasks.
5

---

Published as a conference paper at ICLR 2026
Training-free Module Weight Optimization With the module- performance estimates {-") from the representation space, our goal is to identify an optimal weight vector w E R (where wy denotes the weight of module j) to compose these modules. While the ideal objective to minimize the validation error of the fine-tuned model, searching this space directly is compu- tationally infeasible due to combinatorial explosion. Instead, inspired by ensemble learning (Zhou et al., 2002; Zhou, 2016), we propose to use the prediction error of the weighted ensemble (prior to fine-tuning) as a proxy error to guide weight optimization.
Specifically, we formulate the composition prediction as a weighted sum of the individual mod- ule estimations. The proxy error ED defined the mean squared error between the ensemble prediction and the ground truth labels on the training set:
N
1 M
En(w)=
Σ
We can further cast Eq. (2) to constrained optimization problem:
N s.t. Σ = wy 20
argmin ED(W),
(3)
Since the objective is convex and the constraints define a convex feasible set, the problem admits a global optimum that be reliably obtained by standard solvers. Moreover, this weight selection is training-free since it introduces no additional learnable parameters and requires no gradient-based updates hyperparameter tuning beyond the optimizer settings.
Justification for Using the Proxy Error A central premise of AMC is that the kNN-based proxy error (Eq. is reliable indicator of the final model performance. Theoretically, we provide formal risk analysis in Section B to show that, under reasonable assumptions, minimizing this proxy error bounds the risk of the subsequently fine-tuned model. Empirically, when measured in MAE to align with downstream metrics, we observe a strong Pearson correlation (> 0.6) between the per- module proxy errors and their post-fine-tuning performance (see Fig. and Section D. for detailed discussion). This indicates that the proxy is reliable predictor of final performance and supports its use for weight optimization.
Weight-space Module Composition Once the optimal weight vector w is obtained, we com- pose a single customized module gD for the target task. Inspired by recent advances in model merging (Wortsman et al., 2022; Ilharco et al., 2022; Yu et al., 2024; Yang et al., 2024a), merge the modules in their weight space: gD Σラ= W599.
The validity of this averaging is supported by linear mode connectivity (Frankle al., 2020; Zhou et al., 2023; 2024). Since all modules originate from a common pre-trained initialization, their parameters remain structurally compatible despite task-specific divergence. This ensures that the composed module serves as stable and well-conditioned initialization for downstream fine-tuning.
Downstream Fine-tuning Finally, to better adapt the downstream task D, the composed mod- ule gD is appended with a task-specific head and then fine-tuned on D to convergence.
4 EXPERIMENTS
In this section, we conduct comprehensive experiments to demonstrate the empirical effectiveness of MoMa. The experimental setup is outlined in Section 4.1. The main results, discussed in Sec- tion 4.2, show that MoMa substantially outperforms baseline methods. Additionally, we extend MoMa to more architectures in Section 4.3 and conduct an in depth examination of AMC in Section 4.4. Confronted with the data scarcity challenge common real-world materials discov- ery settings, we evaluate MoMa's few-shot learning ability in Section 4.5, where it achieves even larger performance gains compared to baselines. To further highlight the flexibility and scalabil- ity of MoMa, extend MoMa Hub include molecular datasets and present scaling analysis of MoMa Hub in Section 4.6. Finally, we visualize the module weights optimized by AMC in Section 4.7, highlighting MoMa's potential for providing valuable insights into material properties.

---

Published as a conference paper at ICLR 2026
Table 1: Main results for 17 material property prediction tasks. The best MAE for each task is highlighted in bold and the second best result is underlined. The result for each task are the average of five data splits, reported to three significant digits. For each method, the standard deviation of the test MAE across five random seeds is shown in parentheses, Additionally, the average rank and its standard deviation across the 17 datasets are provided to reflect the consisteney of each method.
Datasets
CGCNN
MoE-(18)
JMP-MT
JMP-FT
MoMa (Adapter)
(Pall)
Experimental Band Gap (eV)
0.471 j3:00
0.374 eues:
0.355 (2237)
0.377 ouxe:
0.358 (2.00
0.359 ques)
0.305 (Uoe)
Formation Enthalpy (eVatoom)
0.193 (2555)
0.0949 (20000)
0.192 (2:00
0.134 mom
0.168 (2.97)
0,158 quas)
0,0839 (2.0033)
2D Dielectric Constami
p.12
2.29 auu
2.34
2.25
2.35
2.31
1,89
Fontation Energy (eVlatom)
moxe:
0.120 moms
0.125
0.112 mooe)
0,0496 (0
Exfoliation Energy (me Vitto)
(9.7
52.5 m33
44.4.1..) (11.5)
42.3 raay
35.4 (20)
35.4 (0:0)
36.3 (02)
2D Band Cup (eV)
0.686 13:244
0.532 nuxe:
0.191 12090)
0,546 worar
0.582 (2209))
0,552 (0m14
0,375 (U00))
3D Poly Electronic
32,5 (1.11
27.7
32.7
23.9
23.3
23,3
23,0 (i.))
Refractive (eV)
0.492
0,268
0.249 (3:08)
0:245
0,200
0,0866 (0901))
0,07855 (3.0000
0,0582 (009-))
0.0636 (30000
0.05 8077)
(0000
Elastic Anisotropy
3.65 (0.11
3.01 11:3
3.79 (2.40)
2.53 01251
2.42 (1:90)
2.57 21511
2,86 1n 291
Flectronic Dielectric Constant
0.168 (9:00)
0.157
0.116 3:500
0.137 mints
0,108
105 m000)
(LUKKS (1:00
Dielectric Constunt
0,258
0.236
0,183
0.224
0.171
0.168
0.15%
Phonons Mode Peak (cm-t)
0.127 :3:04
0.0996 (3.0est)
0.081] moxe.)
0,0859 (9:0005)
0,0559 (o0te))
0L0556 12 1009
0.U144 (0.00
Poisson Ratio
0.0326 moroo))
0,0292 (3.000)
0.0225 monit
0.0297 (2
0.0221 (V000))
0.0220 ya p00t
f.112014 c007s
Poly Electronic
2.97 o.m
2.61 (117)
2.33 (ao
2.42 (16
2.11 (000)
2.13 (0.00
2,09 (1:55)
Poly Toral
6.54
5:51
5.61
5.52
4,89
4.89
4,86
Plaznelectric Meduluus
0.232 j3.0m
0,208 mon:
0.208 (980t)
moue
0.174 (3.00)
0.173 moun)y
0.174 (100))
Average Rank
6.88 (0:3))
4.71 (1 mn
(14
4.53 (1.79)
(112
2.59 (1:0
1.35 91
4.1
SETUP
Datasets To better align with real-world material property prediction settings where labels are usually scarce, we conduct experiments on 17 low-data material property prediction tasks from Matminer (Ward et al., 2018) adhering to Chang et al. (2022). This benchmark offers a compre- hensive evaluation of model capability on wide span of properties critical for material discovery. Refer to Scction C.I for more dataset details.
Implementation Details For the pre-trained backbone of MoMa, we employ the open-source JMP model (Shoghi et al., 2024) for representing material systems given its superior performance in property prediction tasks across both crystals and molecules. For a rigorous comparison, we present the MAE averaged across the five splits adopted from Chang et al. (2022). Each experiment is repeated with five random seeds, and the reported standard deviation is computed across the seed- level averages. Additional implementation details, including the details of module architecture, the hyper-parameters for MoMa, and the computational cost, are provided in Section C.2.
Baseline Methods We compare the performance of MoMa with five baseline methods: CGCNN (Xie & Grossman, 2018), MoE-(18) (Chang et al., 2022), UMA (Wood et al., 2025), JMP-FT, and JMP-MT (Shoghi et al., 2024). CGCNN represents a classical method without pre-training. MoE-(18) trains separate CGCNN models for the upstream tasks of MoMa, then ensembles them as one model in a mixture-of of- offeeeertss approach for downstream fine-tuning. UMA is a general- purpose atomic foundation model which achieves state-of-the-att performance in canonical bench- marks (Riebesell et al., 2023). We fine-tune the UMA-Medium model on each downstream task. JMP-FT directly fine-tunes the JMP pre-trained checkpoint on the downstream tasks. JMP-MT trains all tasks in MoMa Hub with a multi-task pretraining scheme and then adapts to each down- stream dataset with further fine-tuning. More discussions basclines are included in Section C.3.
4.2 MAIN RESULTS
Performance of MoMa As shown in Table 1, MoMa (Full) achieves the best performance with the lowest average rank of 1.35 and 14/17 best results. MoMa (Adapter) follows, with an average rank of 2.59. Together, the two variants hold 16/17 best results. They also exhibit the smallest rank deviations, indicating that MoMa consistently delivers reliable performance across tasks. No- tably, MoMa (Full) outperforms JMP-FT in 14 tasks, with an impressive average improvement of 14.0%, highlighting the effectiveness of MoMa Hub modules in fostering material property predic- tion. Moreover, MoMa (Full) surpasses JMP-MT in 16 of 17 tasks with substantial average margin of 24.8%, underscoring the advantage of MoMa's 's modular design in mitigating task interference. Further analyses in this section are done with MoMa (Full) due to its superior performance.
7

---

Published as a conference paper at ICLR 2026
MARS
2051
2.01
300
414
(b)
(c)
Figure 6: (a) Results with Orb-v2; (b) Ablation study of AMC; (c) Analysis experiments of AMC. The axis represents test set MAE and smaller area is better.
Performance of Baselines Among the baseline methods, JMP-FT performs the best with an aver- age rank of 3. 12, followed by JMP-MT and UMA with an average rank of 4.53. Though additionally trained on upstream tasks of MoMa Hub, JMP-MT still lags behind JMP-FT. We hypothesize the in- herent knowledge conflicts between disparate material tasks pose a tremendous risk to the multi-task learning approach. For UMA, it is primarily pre-trained on force-field datasets as a DFT surrogate, so its inductive bias may transfer less well to PES downstream tasks as compared to JMP.
4.3 RESULTS WITH MORE ARCHITECTURES
To verify whether MoMa offers consistent benefits in other model backbones beyond JMP, we con- duet additional experiments on the GNS architecture (Sanchez-Gonnzllz et al., 2020) used by the Orb-v2 model (Neumann et al., 2024), which is not equivariant and much less complex than the GemNet-based architecture (Gasteiger et al., 2022) in JMP. Specifically, we first train and construct an Orb-based MoMa Hub. Then we run AMC and downstream fine-tuning identically as in Sec- tion 3.2. The results (Orb-MoMa) are compared with directly fine-tuning the pre-trained Orb model (Orb-FT). The average test MAE are reported on 5 splits and random seeds.
As shown in Fig. 6a, MoMa outperforms in 13/17 tasks and achieves 6.1% % average boost to direct fine-tuning. This indicates that the effectiveness of MoMa consistent across GemNet-basee and GNS-based architectures.
4.4 ABLATION & ANALYSIS OF ADAPTIVE MODULE COMPOSITION
Ablation Study We conduct a fine-grained ablation study of AMC with three variants: (1) Select Average, which retains the AMC-selected modules (nonzero weights) but averages them uniformly; (2) All Average, simply averages all modules in MoMa Hub, which is equivalent to applying the classical Model Soup strategy (Wortsman al., 2022); (3) Random Selection, which picks random set of modules in MoMa Hub with the same module number as AMC. A visualization of the ablation results is presented in Fig. 6b. The three variants are inferior to AMC in 13, 15 and 15 out of 17 tasks, with an average test MAE increase of 1 11.0%, 18.0% and 20.2%, respectively. This highlight the effectiveness of both module selection and weighted composition in AMC. The average test MAE of 5 splits are reported on one random seed (hereafter). Furthermore, we show in Section D.2 that AMC is robust to kNN configurations and solver tolerances, yielding highly stable weights and consistent post-fine-tuning MAE.
Analysis Experiments To empirically validate the benefit of AMC's representation--diien and training-free pipeline, we replace AMC with three alternatives: (1) LoRAHlub Learning (Huang et al., 2024), a black-box optimization approach for module composition; (2) JMP-(18), where train a routing network over the 18 JMP MoMa modules; and (3) Softmax Weighting, non- optimized heuristic based on ANN proxy. As shown in Fig. 6c, AMC consistently outperforms all baselines, surpassing the three variants 15, 17, and 12 tasks with average MAE reductions of
8

---

Published as a conference paper at ICLR 2026
21.8%, 15.5%, and 13.7%, respectively. This shows the benefit of the AMC over search-based, router-based and performance-basse alternatiess. See more details and discussion in Section C.5.
Efficiency Analysis We highlight that AMC highly efficient: it requires only a single round of forward embedding generation, followed by lightweight kNN prediction and convex optimization. For the largest dataset, AMC converges in under 30 seconds. This efficiency enables MoMa to scale to larger number of modules in future applications. See Section D.3 for a detailed analysis.
4.5 PERFORMANCE IN FEW-SHOT SETTINGS
Motivation & Setup To better assess the per- formance of MoMa in real-world scenarios, where labeled material candidates are costly and often scarce (Abed et al., 2024), we construct few-shot learning setting and compare MoMa with JMP-FT. For each downstream task, we down-sample the training data and apply AMC to compose modules from MoMa Hub, followed by fine-tuning on the sampled subset. The valida- tion and test sets remain consistent with those the standard setting for robust evaluation. Experiments are conducted under 10-shot and 100-shot conditions, representing few-shot and extremely few-shot scenarios.
Table 2: Few-shot evaluation. The average normalized test MAEs of MoMa and JMP-FT under varying data settings, MoMa consistently outperforms JMP-FT in all settings.
10-shot
100-shot
Full data
JMP-FT
0.7003
0.4076
0.2217 0.1871
MoMa
0.5503
0.2990
The average normalized test MAES' for the 17 downstream tasks of MoMa compared to JMP-FT across the full-data, 100-data, and 10-data settings are presented in Table 10. As expected, test loss increases as the data size decreases, while MoMa consistently outperforms JMP-FT in all settings. Notably, the performance advantage of MoMa is more pronounced in the few-shot settings, with the normalized loss margin widening from 0.03 in the full-data setting to 0.11 and 0.15 in the 100-data and 10-data setting. This suggests that MoMa may offer even greater performance gains in real- wolld scenarios, where property labels are often limited, thereby hindering the effective fine-tuning of large pre-trained models. Complete results shown in Section D.4.
Results
4.6 SCALING ANALYSIS OF MOMA HUB MODULES
Motivation & Setup In this section, we study the scaling behavior of MoMa understand whether it benefits from larger MoMa Hub. We first do a hub-scale ablation to progressively expand MoMa hub from 5 to 10 and 18 modules. Then further expand MoMa Hub to include 12 QM9 modules (Ramakrishnan et al., 2014), which are trained on 12 quantum chemical properties for 134,000 stable small organic molecules. We consistently perform AMC and evaluation across all Hub variants. The full setup is described in Section C.6.
Results presented in Table 3, as the MoMa hub scales, the average normalized test MAE across 17 tasks decreases monotonically (from 0.2040 with 5 modules to 0. 0.5 with 30 modules), showing no sign of saturation in this regime. The complete results are provided in Table (Section D.5).
To further analyze the effect of adding the 12 QM9 modules, we plot the test-MAE reduction rate against the AMC proxy-error decrease in Fig. for datasets where QM9 modules are se- lected. We observe that: (1) The integration of QM9 modules leads to an average of 1.7% de- crease in test MAE; (2) larger reduction in the AMC-optimized proxy error correlates with greater performance improvements post-fine-tuning (Pearson correlation 0.69). We highlight the task of MP Phonons prediction, which marks 11.8% decrease in test set MAE after the inclusion of QM9 modules. Overall, these results sup- port our vision of MoMa as a flexible community-dris downstream performance improves and AMC remains effective at larger scale.
Table 3: Scaling with hub size. Average nor- malized test MAE decreases as the number of modules in MoMa Hub increases.
Modules
5
10
30
Norm. MAE 0.2040
0.1910
0.1353
0.1759
platform: as more modules are added,
Computed by dividing the MAF of each task by its standard deviation.

---

Published as a conference paper at ICLR 2026
MP Phonons
10%
Average Test MAE Decrease
Text MAE Decrease (%)
Fit Line (r-0.69)
8%
6%
4%
2%
polscc, mete
phonora
0%
poly sotall
plogeolectri Ierrier
-2%
antue See's
Wid
0%
2%
5%
8%
10%
13%
15%
18%
Proxy Error Decrease (96
Figure 7: Scatter plot showing the relationship between the test MAE decrease and the proxy error (Eq. (3)) decrease after adding QM9 mod- ules. The solid line represents linear regres- sion fit, yielding a Pearson correlation of 0.69.
Figure 8: Heat map of AMC weights on one data split. The x-axis represents the task names of the MoMa Hub modules, while the y-axis shows the 17 material tasks in Table 1. Darker color indicates larger weight.
4.7 MATERIALS INSIGHTS MINING
Motivation We argue that the AMC weights derived in Eq. (3) can provide valuable insights into relationships of material properties. To explore this, we interpret the weights as indicators for the relationships between MoMa Hub modules and downstream tasks. Following Chang et al. (2022), we present a normaizza visualization these weights in Fig. 8.
Results We highlight several noteworthy observations: (I) The weights assigned by AMC ef- fectively capture physically intuitive relationships between material properties. For instance, in predicting electronic dielectric constants, MoMa assigns high weights to the band gap modules, which is reasonable given the inverse relationship between the dielectric constant and the square of the band gap; (2) Less intuitive relationships also emerge. For the task of experimental band gap prediction (row 1), the DFT-computed formation energy module (column 1) is assigned the second- highest weight. In the prediction of dielectric constant (row 9), modules related to thermoelectric and thermal properties (columns 5 and 6) are heavily weighted. However, the first-principles lationship between these tasks is indirect. We hypothesize that addition to task relevance, other factors such as data distribution and size may also influence the weight assignments for AMC.
CONCLUSION
In this paper, we present MoMa, simple modular learning framework for material property predic- tion. Motivated by the challenges of diversity and disparity in materials, MoMa first trains special- ized modules across a wide spectrum material tasks, constituting MoMa Hub. We then introduce the Adaptive Module Composition algorithm, which facilitates tailored adaptation from MoMa Hub each downstream task by adaptively composing synergistic modules. Experimental results across 17 datasets demonstrate the superiority of MoMa, with few-shot and hub-scaling experiments further highlighting data efficiency and scalability.
Limitations and Future Work. The current scope of our study is limited to crystalline and organic materials. Future work includes expanding MoMa Hub with modules for wider range of material data and prediction tasks, and examining how MoMa scales with hundreds or thousands of modules, which may yield deeper insights into the modularity of materials knowledge.
Broader Impact. As an open-source platform for modularizing and distributing materials knowl- edge, MoMa enables secure sharing modules without exposing proprietary data, efficient cus- tomization for downstream tasks, and improved prediction accuracy in low-data scenarios. We envision MoMa fostering a new paradigm of modular material learning and driving broader com- munity collaboration toward accelerated materials discovery.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
The authors would thank Junwei Yang. Ziyao Cao, Fanyou Meng, and Yuxuan Song for their valu- able feedback on the paper. We also thank the anonymous reviewers for reviewing the draft. This work is supported by the National Science and Technology Major Project (2022ZD01 17502), the Natural Science Foundation of China (Grant No. 62376133, 62406170), and Wuxi Research Insti- tute of Applied Technologies, Tsinghua University under Grant 20242001120.
REFERENCES
Jehad Abed, Jiheon Kim, Muhammed Shuaibi, Brook Wander, Boris Duijf, Suhas Mahesh, Hyeon- seok Lee, Vahe Gharakhanyan, Sjoerd Hoogland, Erdem Irtem, al. Open catalyst ex- periments 2024 (ocx24): Bridging experiments and computational models. arXiv preprint arXiv:2411.11783, 2024.
Takuya Akiba, Makoto Shing, Yujin Tang, Qi Sun, and David Ha. Evolutionary optimization of model merging recipes. Nature Machine Intelligence, 7(2):195-2044 2025.
Luis Barroso-Luque, Muhammed Shuaibi, Xiang Fu, Brandon M Wood, Misko Dzamba, Meng Gao, Ammar Rizvi, C Lawrence Zitnick, and Zachary W Ulissi. Open materials 2024 (omat24) inorganic materials dataset and models. arXiv preprint arXiv:2410. /2771, 2024.
Ilyes Batatia, Philipp Benner, Yuan Chiang, Alin M Elena, Dávid P Kovács, Janosh Riebesell, Xavier R Advineula, Mark Asta, Matthew Avaylon, William J Baldwin, et al. A foundation model for atomistic materials chemistry. arXiv preprint arXiv:2401.000 0000000 2023.
Ivano E Castelli, David D Landis, Kristian S Thygesen, Søren Dahl, Ib Chorkendorff, Thomas F Jaramillo, and Karsten Jacobsen. New cubic perovskites for one-and two-photon water splitting using the computational materials repository. Energy & Environmental Science, 5(10):9034- 9043, 2012.
Rees Chang, Yu-Xiong Wang, and Elif Ertekin. Towards overcoming data scarcity in materials science: unifying models and datasets with mixture of experts framework. npj Computational Materials, 8(1):242, 2022.
Kamal Choudhary and Brian DeCost. Atomistic line graph neural network for improved materials property predictions. npj Computational Materials, 7(1):185, 2021.
Kamal Choudhary, Irina Kalish, Ryan Beams, and Francesca Tavazza. High-throughput identifica- tion and characterization of two-dimensional materials using density functional theory. Scientific reports, 7(1):5179, 2017.
Kamal Choudhary, Gowoon Cheon, Evan Reed, and Francesca Tavazza. Elastic properties bulk and low-dimensional materials using van der waals density functional. Physical Review B, 98(1) )): 014107, 2018.
Kamal Choudhary, Kevin F Garrity, Andrew CE Reid, Brian DeCost, Adam Biacchi, Angela R Hight Walker, Zachary Trautt, Jason Pattrick-Simpen Gilad Kusne, Andrea Centrone, et al. The joint automated repository for various integrated simulations (jarvis) for data-driven materials design. npj computational materials, 6(1):173, 2020.
Kishalay Das, Pawan Goyal, Seung-Cheol Lee, Satadeep Bhattacharjee, and Niloy Ganguly. Crys- mmnet: multimodal representation for crystal property prediction. In Uncertainty Artificial Intelligence, pp. 507-517. PMLR, 2023.
Pierre-Paut De Breuck, Geoffroy Hautier, and Gian-Marco Rignanese. Materials property prediction for limited datasets enabled by feature selection and joint learning with modnet. upj computa- tional materials, 7(1):8,, 2021.
Maarten De Jong, Wei Chen, Thomas Angsten, Anubhav Jain, Randy Notestine, Anthony Gamst, Marcel Sluiter, Chaitanya Krishna Ande, Sybrand Van Der Zwaag, Jose J Plata, et al. Charting the complete elastic properties of inorganic crystalline compounds. Scientific data, 2(1):1-13, 2015a.

---

Published as a conference paper at ICLR 2026
Maarten De Jong, Wei Chen, Henry Geerlings, Mark Asta, and Kristin Aslaug Persson. A database to enable discovery and design piezoelectric materials. Scientific data, 2(1):1-13, 2015b.
Jacob Devlin. Bert:
arXiv preprint Pre-training of deep bidirectional transformers for language understanding. 2018.
Luc Devroye, László Györfi, Adam Krzyžak, and Gábor Lugosi. On the strong universal consistency of nearest neighbor regression function estimates. The annals of Statistics, pp. 1371-1385, 1994.
Steven Diamond, Eric Chu, and Stephen Boyd. CVXPY: A Python-embedded modeling language for convex optimization, version 0.2. http: / cvxpy org/, May 2014.
Alexander Dunn, Qi Wang, Alex Ganose, Daniel Dopp, and Anubhav Jain. Benchmarking materials property prediction methods: the matbench test set and automatminer reference algorithm. npj Computational Materials, 6(1):138, 2020.
Rahim Entezari, Hanie Sedghi, Olga Saukh, and Behnam Neyshabur. The role of permutation in- variance in linear mode connectivity of neural networks. In International Conference Learning
Representations.
Lenz Fiedler, Karan Shah, Michael Bussmann, and Attila Cangi. Deep dive into machine learning density functional theory for materials science and chemistry. Physical Review Materials, 6(4): 040301, 2022.
Jonathan Frankle, Gintare Karolina Dziugaite, Daniel Roy, and Michael Carbin. Linear mode nectivity and the lottery ticket hypothesis. In International Conference on Machine Learning, pp- 3259-3269, PMLR, 2020.
Johannes Gasteiger, Muhammed Shuaībi, Anuroop Sriram, Stephan Günneeann, Zachary Ward Ulissi, C Lawrence Zitnick, and Abhishek Das. Gemnet-oc: Developing graph neural networks for large and diverse molccular simulation datasets. Transactions on Machine Learning Research, 2022.
Scan D Griesemer, Yi Xia, and Chris Wolverton. Accelerating the prediction of stable materials with machine learning. Nature Computational Science, 3(11):934-94) 2023.
Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, An- drea Gesmundo, Mona Attariyan, and Sylvain Gelly. Parameter-efficient transfer learning for nlp. In International conference machine learning, pp. 2790-2799. PMLR, 2019.
Edward Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021.
Chengsong Huang, Qian Liu, Bill Yuchen Lin, Tianyu Pang, Chao Du, and Min Lin. Lorahub: Ef- ficient cross-task generalization via dynamic lora composition. In First Conference on Language Modeling, 2024.
Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Suchin Gururangan, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi. Editing models with task arithmetic. arXiv preprint arXiv:2212.00 04089, 2022.
Ahmet Iscen, Giorgos Tolias, Yannis Avrithis, and Ondrej Chum. Label propagation for deep semi- supervised learning. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pp. 5070-5079, 2019.
Robert A Jacobs, Michael Jordan, Steven Nowlan, and Geoffrey Hinton. Adaptive mixtures of local experts. Neural computation, 3(1):79-8) 87, 1991.
Anubhav Jain, Shyue Ping Ong, Geoffroy Hautier, Wei Chen, William Davidson Richards, Stephen Dacek, Shreyas Cholia, Dan Gunter, David Skinner, Gerbrand Ceder, et al. Commentary: The ma- terials project: A materials genome approach to accelerating materials innovation. APL materials, 1()), 2013.
12

---

Published as a conference paper at ICLR 2026
Anubhav Jain, Yongwoo Shin, and Kristin A Persson. Computational predictions of energy materials using density functional theory. Nature Reviews Materials, 1(1):1-13, 2016.
Dipendra Jha, Kamal Choudhary, Francesca Tavazza, Wei-keng Liao, Alok Choudhary, Carelyn Campbell, and Ankit Agrawal. Enhancing materials property prediction by leveraging computa- tional and experimental data using deep transfer learning. Nature communications, 10(1):5316, 2019.
George Kim, SV Meschel, Philip Nash, and Wei Chen. Experimental formation enthalpies for intermetallic phases and other inorganic compounds. Scientific data, 4(1):1-11, 2017.
Janice Lan, Aini Palizhati, Muhammed Shuaibi, Brandon M Wood, Brook Wander, Abhishek Das, Matt Uyttendaele, C Lawrence Zitnick, and Zachary W Ulissi. Adsorbml: a leap efficiency for adsorption energy calculations using generalizable machine learning potentials. npj Compu- tational Materials, 9(1):172, 2023.
Zhenyi Lu, Chenghao Fan, Wei Wei, Xiaoye Qu, Dangyang Chen, and Yu Cheng. Twin-merging: Dynamic integration of modular expertise in model merging. Advances in Neural Information Processing Systems, 37:78905-78935, 2024.
Rishikesh Magar, Yuyang Wang, and Amir Barati Farimani. Crystal twins: self-supervised learning for crystalline material property prediction. npj Computational Materials, 8(1):231, 2022.
Hassan Masood, Tharmakulasingam Sirojan, Cui Ying Toe, Priyank V Kumar, Yousof Haghshenas, Patrick HI. Sit, Rose Amal, Vidhyasaharan Sethu, and Wey Yang Teoh. Enhancing prediction accuracy of physical band gaps in semiconductor materials. Cell Reports Physical Science, 4(9), 2023.
Daniel McNamara and Maria-Florina Balean. Risk bounds for transferring representations. In International Conference on Machine Learning, 2017.
Akshay Mehra et al. Understanding the transferability of representations via hypothesis transfer. NeurlPS, 2024.
Amil Merchant, Simon Batzner, Samuel Schoenholz, Muratahan Aykol, Gowoon Cheon, and Ekin Dogus Cubuk. Scaling deep learning for materials discovery. Nature, 624(7990):80-85, 2023.
Mohammed Muqeeth, Haokun Liu, and Colin Raffel. Soft merging experts with adaptive routing. arXiv preprini (rXXi::300603745, 2023.
Mark Neumann, James Gin, Benjamin Rhodes, Steven Bennett, Zhiyi Li, Hitarth Choubisa, Arthur Hussey, and Jonathan Godwin. Orb: A fast, scalable neural network potential. arXiv preprint arXiv:2410.22570, 2024.
Joannis Petousis, David Mrdjenovich. Eric Ballouz, Miao Liu, Donald Winston, Wei Chen, Tanja Graf, Thomas D Schladt, Kristin A Persson, and Fritz B Prinz. High-throughput screening of inorganic compounds for the discovery of novel dielectric and optical materials. Scientific data, 4(1):1-12, 2017.
Guido Petretto, Shyam Dwaraknath, Henrique PC Miranda, Donald Winston, Matteo Giantomassi, Michiel J Van Setten, Xavier Gonze, Kristin A Persson, Geoffroy Hautier, and Gian-Marco Rig- nanese. High-throughput density-functional perturbation theory phonons for inorganic materials. Scientific data, 5(1):1-12, 2018.
Jonas Pfeiffer, Andreas Rücklé, Clifton Poth, Aishwarya Kamath, Ivan Vulić, Sebastian Ruder, Kyunghyun Cho, and Tryna Gurevych. Adapterhub: framework for adapting transformers. arXiv preprint arXi::200..07779, 2020.
Jonas Pfeiffer, Sebastian Ruder, Ivan Vulić, and Edoardo Maria Ponti. Modular deep learning. arXiv preprint arXiv:2302.115522 2023.
13

---

Published as a conference paper at ICLR 2026
Chau Pham, Piotr Teterwak, Soren Nelson, and Bryan A Plummer. Mixturegrowth: Growing neural networks by recombining learned parameters. In Proceedings of the IEEE/CVF Winter Confer- ence on Applications of Computer Vision, pp. 2800-28099 2024.
Joan Puigcerver, Carlos Riquelme, Basil Mustafa, Cedric Renggli, André Susano Pinto, Sylvain Gelly, Daniel Keysers, and Neil Houlsby. Scalable transfer learning with expert models. arXiv preprint arXiv:2009. 13239, 2020.
Raghunathan Ramakrishnan, Pavlo O Dral, Matthias Rupp, and Anatole Von Lilienfeld. Quantum chemistry structures and properties of 134 kilo molecules. Scientific data, 1(1):-77, 2014.
Benjamin Rhodes, Sander Vandenhaute, Vaidotas Simkus, James Gin, Jonathan Godwin, Tim Duignan, and Mark Neumann. Orb-v3: atomistic simulation scale. arXiv preprint arXiv:2504.06231, 2025.
Francesco Ricci, Wei Chen, Umut Aydemir, G Jeffrey Snyder, Gian-Marco Rignanese, Anubhav Jain, and Geoffroy Hautier. An ab initio electronic transport database for inorganic materials. Scientific data, 4(1):1-13, 2017.
Janosh Riebesell, Rhys EA Goodall, Anubhav Jain, Philipp Benner, Kristin A Persson, and Al- pha A Lee. Matbench discovery-an evaluation framework for machine learning crystal stability prediction. arXiv preprint arXiv:2308. /4920, 2023.
Janosh Riebesell, Rhys EA Goodall, Philipp Benner, Yuan Chiang, Bowen Deng, Gerbrand Ceder, Mark Asta, Alpha Lee, Anubhav Jain, and Kristin A Persson. framework evaluate machine learning crystal stability predictions. Nature Machine Intelligence, 7(6):836-847, 2025.
Lars Ruddigkeit, Ruud Van Deursen, Lorenz C Blum, and Jean-Louis Reymond. Enumeration of 166 billion organic small molecules in the chemical universe database gdb-17. Journal of chemical information and modeling, 52(1 2):2 2178666-755,- 2012.
Alvaro Sanchez-Gonzalez, Jonathan Godwin, Tobias Pfaff, Rex Ying, Jure Leskovee, and Peter Battaglia. Learning to simulate complex physics with graph networks. In International conference on machine learning. pp. 8459-8465 PMLR, 2020.
Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoe Le, Geoffrey Hinton, and Jeff Dean. Outrageously large neural networks: The sparsely-gated mixture-of-experts layer. In International Conference on Learning Representations, 2016.
Nima Shoghi, Adeesh Kolluru, John R Kitchin, Zachary W Ulissi, C Lawrence Zitnick, and Bran- don M Wood. From molecules to materials: Pre-training large generalizable models for atomic property prediction. In The Twelfth International Conference on Learning Representations, 2024.
Charles J Stone. Consistent nonparametric regression. The annals of statistics, pp. 595-620, 1977.
Shaomu Tan, Di Wu, and Christof Monz. Neuron specialization: Leveraging intrinsic task modular- ity for multilingual machine translation. arXiv preprint arXiv:2404. /201, 2024.
Tatsunori Taniai, Ryo Igarashi, Yuta Suzuki, Naoya Chiba, Kotaro Saito, Yoshitaka Ushiku, and Kanta Ono. Crystalformer: infinitely connected attention for periodic structure encoding. arXiv preprint arXiv:2403.1166 2024.
Amanda Wang, Ryan Kingsbury, Matthew McDermott, Matthew Horton, Anubhav Jain, Shyue Ping Ong. Shyam Dwaraknath, and Kristin A Persson. A framework for quantifying uncertainty in dit energy corrections. Scientific reports, 1(II::1499, 2021.
Han Wang, Duo Zhang, Chun Cai, Wentao Li, Yuanchang Zhou, Jinzhe Zeng, Mingyu Guo, Chengqian Zhang, Bowen Li, Hong Jiang, al. graph neural network for the era of large atomistic models. 2025.
Logan Ward, Alexander Dunn, Alireza Faghaninia, Nils ER Zimmermann, Saurabh Bajaj. Qi Wang, Joseph Montoya, Jiming Chen, Kyle Bystrom, Maxwell Dylla, et al. Matminer: An open source toolkit for materials data mining. Computational Materials Science, 1 152:60-69, 2018.
14

---

Published as a conference paper at ICLR 2026
Brandon M Wood, Misko Dzamba, Xiang Fu, Meng Gao, Muhammed Shuaibi, Luis Barroso-Luque, Kareem Abdelmaqsoud, Vahe Gharakhanyan, John Kitchin, Daniel Levine, et al. Uma: A family of universal models for atoms. arXiv preprint arXiv:2506.33971, 2025.
Mitchell Wortsman, Gabriel Ilharco, Samir Ya Gadre, Rebecca Roelofs, Raphael GontijoLLppes, Ari S Morcos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Kornblith, et al. Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time. In International conference on machine learning, pp. 23965-239999 PMLR, 2022.
Chaojun Xiao, Zhengyan Zhang, Chenyang Song, Dazhi Jiang, Feng Yao, Xu Han, Xiaozhi Wang, Shuo Wang. Yufei Huang, Guanyu Lin, et al. Configurable foundation models: Building Ilms from modular perspective. arXiv preprint arXiv:2409.02877 2024.
Tian Xie and Jeffrey C Grossman. Crystal graph convolutional neural networks for an accurate and interpretable prediction of material properties. Physical review letters, 120(14): 145301, 2018.
Keqiang Yan, Cong Fu, Xiaofeng Qian, Xiaoning Qian, and Shuiwang Ji. Complete and efficient graph transformers for crystal material property prediction. arXiv preprint arXiv:2403. 11857, 2024.
Enneng Yang, Li Shen, Guibing Guo, Xingwei Wang, Xiaochun Cao, Jie Zhang, and Dacheng Tao. Model merging in lims, mllms, and beyond: Methods, theories, applications and opportunities. arXiv preprint arXiv:2408.0766 2024a.
Han Yang, Chenxi Hu, Yichi Zhou, Xixian Liu, Yu Shi, Jielan Li, Guanzhi Li, Zekun Chen, Shuizhou Chen, Claudio Zeni, et al. Mattersim: A deep learning atomistic model across ele- ments, temperatures and pressures. arXiv preprint arXiv:2405.0440 2024b.
Le Yu, Bowen Yu, Haiyang Yu, Fei Huang, and Yongbin Li. Language models are super mario: Ab- sorbing abilities from homologous models as free lunch. In Forty-first International Conference on Machine Learning, 2024.
Zhanpeng Zhou, Yongyi Yang, Xiaojiang Yang, Junchi Yan, and Wei Hu. Going beyond linear mode connectivity: The layerwise linear feature connectivity. Advances in neural information processing systems, 36:60853 60877, 2023.
Zhanpeng Zhou, Zijun Chen, Yilan Chen, Bo Zhang, and Junchi Yan. On the emergence of cross- task linearity in pretraining finetunnnng paradigm. International Conference on Machine Learn- ing, pp. 61854-61884. PMLR, 2024.
Zhi-Hua Zhou. Learnware: on the future of machine learning. Frontiers Comput. Sci., 10(4):589- 590, 2016.
Zhi-Hua Zhou, Jianxin Wu, and Wei Tang. Ensembling neural networks: many could be better than all. Artificial intelligence, 137(1-2):239-263 2002.
A ALGORITHM FOR ADAPTIVE MODULE COMPOSITION
The formal description of the Adaptive Module Composition algorithm is included in Algorithm 1.
15

---

Published as a conference paper at ICLR 2026
Algorithm 1 Adaptive Module Composition (AMC)
Input: MoMa Hub H { Downstream training set ((Xi, v:)
2: Output: Composed module gD-
3: {1. Module Prediction Estimation]
4: for w=rj=11 .N do
5: Generate embeddings X'' - (X, (X,, 9i) E D).
6: Compute prediction vector Y = ......., Pm via leave-one-out k-Nearest Neighbors.
7: end for
8: (2. Module Weight Optimization)
9: Let Y= = (y1,..... yms be the vector true labels from D.
10: Find optimal weights w" (wi,.... wi) by solving the convex optimization problem:
11: w + arg min un
Σe= wy Y
subject to: ΣØ= W and wy 2 for j NI.
13: (3. MMdule Composition]
14: 9D - -EA= w3g)
15: Return ODD
B THEORETICAL JUSTIFICATION AND ERROR ANALYSIS FOR AMC
In this section, we provide formal analysis of the kNN-based proxy error used in AMC. Specifi- cally, we show that the ANN proxy risk Ryrozy(w) serves an upper bound for the fine-tuning risk (F (subject to approximation errors). Consequently, minimizing the empirical approximation of Rprosy (w) tightens this bound, thereby providing theoretical justification for using the proxy risk to control the risk of the subsequently fine-tuned model.
B. DEFINITIONS
Let 0g denote the parameters of the i-th module and define its representation of input x as (xxx) g(O;; æ). Given weights (W1,... - WN) E AN-1, define the merged module in parameter space and its representation by
0 we O,,
(x) g(0w::
Let the Bayes regressors associated with each representation mx(x) == E (X) (x)), mw == E[Y (x) = (x)
We define the following risk terms:
Representation Bayes Risk: R2 (w)) == E (m (X) 2
Fine-tuning Risk: RFT(w) == E [(FFT (X;(F((X;;)) y)
Proxy Risk using kNN: Rproxy (w) = E [ūproxy (X: w) y)
Bayes Ensemble Risk: (en)(w) = E (X: - Y)al, where mens (2; w)
>N W; (x)).
Remark (distribution vs empirical proxy objective). Practically AMC optimizes the empirical proxy error
m
Ew(w)
wi DE.
yu
while analysis here is stated for the distribution-less proxy risk Reroxy (w.) Under standard gener- alization results for squared-loss regression and mild capacity control on the family (pproxy (:: w) w AN-1). the empirical proxy error (w) concentrates around R (w)) hence we treat (w)(w in practice finite-sample approximation of Rproxy(1)
16

---

Published as a conference paper at ICLR 2026
B.2 PRELIMINARIES AND ASSUMPTIONS
We recall a standard non-parametric regression result on the consistency of KNN estimators.
Lemma (Universal KNN L2 Jn-Consistency (Stone, 1977; Devroye et al., 1994)). For each upstream task consider training dataset of size n, and let ghe be the k-nearest neighbor regressor in the feature space 9;(2), where the number of neighbours k = kn satisfies kn - 0o, kn/n - O. Define the Le-estimation error
(n) ( (X) mx))))
Then the kNN regression estimator satisfies the exact risk decomposition
E[ (X) r) [mm, - + (NN
and moreover SINN (n) 0 as
Remark on Lemma 1. Our implementation of AMC uses a finite- K kNN ensemble with cosine similarity in the learned representation space. This differs from the asymptotic setting of Lemma (which requires k, approximation. Lemma is used as a conceptual tool to justify the kNN-based proxy objective; we do not claim a full universal consistency result for our exact finite-sample variant.
and kn/n n + 0), but it serves as computationally efficient finite-sample
Assumption (Fine-tuning stability). There exists Copt 2 0 such that for all w E AN-1
RFT(w) S R (w) + Eopt-
That is, after fine-tuning on top of Iw. the resulting predictor is within Copt of the optimal predictor defined on the same representation (McNamara Balcan, 2017: Mehra al., 2024).
Assumption 2 (Representation closeness). There exists ō and high-probability subset No S X' such that for all modules and all x E Ao,
(1).(x) [w))ll S 8.
This models the empirical observation that independently fine-tuned modules from the same pre- trained model can often be aligned into shared, approximately convex basin parameter space, leading to similar internal representations. Recent work on mode connectivity (Frankle et al., 2020; Entezari et al.) and cross-task linearity (Zhou al., 2024) supports this assumption, which we adopt here as a structural modeling assumption rather than general theorem.
Assumption (stability of latent regressor). Let and g' denote any two modules among {g. i [N]] and the merged module gw- Let their associated Bayes regressors be
m(x) := Y g()) g(x)].
m' (x:) == Y g((x) = g')x))
There exists L 0 and a high-probability subset Xo S such that if
-2)))==(xx)) s
Vr E Xo.
then the regressors are close in expectation:
E m(X m' m((x) 1[X Xo}l s Ld.
We also assume YY| < B almost surely.
Assumption formalizes a semantic smoothness condition in the leamned representation: inputs that are mapped to nearby latent points (i.e., with close g(x) and g((x)) are required to have similar predictive behavior for Y on average. This can be viewed as a Lipschitz-type regularity assumption on the Bayes regressors in representation space, and is sistent with the common inductive bias in deep learning that well-trained encoders should support target functions which do not change abruptly under small perturbations of the latent features.
Remark on Assumption
17

---

Published as a conference paper at ICLR 2026
B.3 RISK TRANSFER ANALYSIS
Step I: KNN Ensemble Approximates the Bayes Ensemble. Bayes ensemble risk:
We compare the proxy risk to the
Rproxy(1)) Rens (w))(w)]
EE| (Uproxy r)² (Mens Y)2l
[E| (Üproxy Meus/ (Uproxy + Mens 2Y)
4B IE] [pproxy Mens
(by bounded label in Assumption 3)
Next,
E Uproxy Menss]
= E
w (1)
wam:
We EE (1)*)) mil
We / [B[[[[00 (1)00 - m
N
Wi V (NN
(6)
(by Assumption I)
Σ
W/ ENNN (n)
(by Jensen's inequality)
Define the weighted kNN error
(NNN(n; = Σ Wi (n),
(n; = 4B V EkNN(n;
Then we obtain
Reroxy Rena (en((w)
I 61 (n;,))
Step 2: Bayes Ensemble Approximates the Merged Bayes Predictor. We compare Mens(;; w) and m. (x) using the triangle inequality,
N Σ
B Mens (X;w) m. (X) 1fx E No11 = E
wxm;(X) - mm (X) 11x E 2o)
1=1
EEl [x,(X) m (x) X6Hl
L8
(by Assumption2 and 3)
Then
Rens (w) R² (w)l
E (mens (X: w) - Y)2 - (m (X) - y)
E (mens (3 - m. (X)) (xatimmensxxx;1) (X; + (X) - 2Y)
4B E[ |ens (X; w) m. (X)
< 4B L8 + (small error term x.)
Absorbing the small-prohability contribution from 24 into the constant, we obtain
Rons (w)) - R2 (w) < Co.
where C := 4BT.
18

---

Published as a conference paper at ICLR 2026
B.4 MAIN TRANSFER BOUND AND GUARANTEE
Proposition 1. Under Assumptions 1-3 and Lemma 1, for any w E AN-I,
RET(W) - Rproxy(1)) (w)) + C' + Copt + (1(;w)), where C' = 4BL and e1 (n; w) 4BV ExNN(n;
Proof.
From Step we have
Reroxy(W) 2 Rwna(W) e1 =1
and from Step 2
Reus(w) 2 R (w) C.ó.
Hence
(")w I Rproxy(w) + C8 (1(
Combining with Assumption yields
(FT(w) I Rº (w) + Eopt S Rproxy(w) + C6 + Eopt + (n;
Combining this result with the optimality of á for the proxy objective yields the following theorem. Theorem 1. Let d E arg min Rproxy( (w) be minimizer of the proxy risk. Under Assump- tions 1-3 and Lemma 1, the fine-tuning risk of the merged encoder with weights ie satisfies
RFT(w) I (Bproxy(1) + C'8 + Sopt + E1 (,;
where C = 4B L and (n; u) = 4BV EN ii (N)
C EXPERIMENTAL DETAILS
Here we provide more experimental details regarding the datasets, baselines, and implementation.
C.I DATASET DETAILS
We primarily adopt the dataset setup proposed by Chang et al. (2022). Specifically, we select 35 datasets from Matminer (Ward al., 2018) for our study, categorizing them into 18 high- sigh-resource material datasets, with sample sizes ranging from 10,000 to 132,000 (an average of 35,000 samples), and 17 low-data datasets, with sample sizes ranging from 522 to 8,043 (an average of 2,111 samples).
The high-resource datasets are utilized for training the MoMa Hub modules, as their larger data vol- umes are likely to encompass a wealth of transferrable material knowledge. detailed introduction of these MoMa Hub datasets is included in Table 4.
The low-data datasets serve as downstream tasks to evaluate the effectiveness of MoMa and its baselines. A detailed introduction is included in Table 5. This setup mimics real-world materials discovery scenarios, where downstream data are often scarce. Compared to the benchmark in Chang et al. (2022), exclude two low-data datasets with exceptionally small data sizes (fewer than 20 test samples) from our experiments, as their limited data could lead to unreliable conclusions.
Following Chang al. (2022), all datasets split into training, validation, and test sets with ratio of 1:15:1.55. For the downstream low-data datasets, we follow the exact splitting provided by Chang al. (2022) to ensure a fair comparison.
C.2 IMPLEMENTATION DETAILS OF MOMA
Module Architecture Details We now introduce the architectural details of MoMa modules. Across all our experiments the main text, the JMP (Shoghi et al., 2024) backbone is adopted due to its comprehensive strength across wide range of molecular and crystal tasks. JMP is pre- trained on ~ 120 million oFT-generated force-fiell data across large-scale datasets on catalyst and small molecules. It is 6-layer GNN model with around 160M parameters which is based the
19

---

Published as a conference paper at ICLR 2026
Table 4: Datasets for training MoMa Hub modules. Num stands for the number of samples in each dataset.
Datasets
Num
Description
MP F
132752
The energy change during the formution a compound from its elements. Data from Jain et al. (2013).
MP E
106113
The PBE band gaps, calculated using the Perdew- Prrkee-Enzzerof (PBE) func- tional, represent the energy difference between the valence and conduction bands in a material. Data from Jain ct al. (2013).
MP GvRu
10987
VRH-averraee shear modulus, an approximate value ohtained by averaging the shear modulus of polycrystalline materials, Data from Jain et al. (2013).
MP KvRn
10987
VRH average bulk modulus, calculated by averaging the Voigt (upper bound) Reuss (lower bound) bulk moduli. Data from Jain et (2013).
n-type oe
37390
n-type a, measures the material's s conductivity performance when electrons are the primary charge carriers. Data from Ricci al. (2017).
p-lype de
37390
Similar n-type o. with holes carriers, Data from Ricci al. (2017).
n-yype Ne
37390 n-type Ke evaluates the efficiency of n-ype materials that can conduct both electricity and heat, which is crucial for understanding its performance in ther- moelectric applications. Data from Ricci et al. (2017).
p-lype Ke
37390
Similar to n-lype Ke. with holes as carriers, Data from Ricci et al. (2017).
n-type 8
37390
n-type denotes the average conductivity eigenvalue, which measures thermo- electric conversion efficiency in the hole-conducting state when electrons act as the primary charge carriers. Data from Ricci et al. (2017).
p-type
37390
Similar to a-type S, with holes as carriers. Data from Ricci et al. (2017).
n-type m.
21037 n n-type my denotes the average eigenvalue of conductivity effective mass, which measures the impact the electron's 's effective mass on the electrical conductivity. Data from Ricci et al. (2017).
p-lype m3
20270
Similar n-type mc, with holes carriers. Data from Ricci al. (2017).
Perovskite E,
18928 Perovskite E, refers the heat of formation of perovskite, the amount of heat released or absorbed when the perovskite structure is formed from its con- stituent elements. Duta from Castelli et ul. (2012).
JARVIS Ey
25923
Formation energy from the JARVIS dataset (Choudhary et al., 2020).
JARVIS dielectric constant (Opt)
19027
Dielectric constant measures the material's 's ability to pularize in response to an electric field in twn twn-dimensional systems. Data from Choudhary et al. (2020).
JARVIS Eg
23455
PBE band gaps from the JARVIS dataset (Choudhary et al., 2020).
JARVIS GyRH
10855
VRH-average shear modulus from the JARVIS dataset (Choudhary et al..
2020).
JARVIS KуRи
11028
VRH-average bulk modulus from the JARVIS dataset (Choudhary et al., 2020).
GemNet-OC architecture (Gasteiger et al., 2022). Note that MoMa backbone-agnostic and we include results with the Orb model (Neumann et al., 2024) in Section 4.3.
For the full module parametrization, we exclude the output layer and treat the entire GNN backbone as a single module. For the adapter components, we follow the standard implementation of adapter layers (Houlsby et al., 2019). Specifically, an adapter layer is inserted between every two layers of the JMP backbone. Each adapter consists of a downward projection to a bottleneck dimension, followed by an upward projection back to the original dimension. We adopt BERT-style initializa- tion (Devlin, 2018), with the bottleneck dimension set to half of the input embedding dimension. Note that the merging process for adapters is performed in a layer-wise manner. For each back- bone layer containing adapters, we compute weighted average of the parameters from all selected adapter modules. A single scalar weight for each module, determined by AMC, is applied uniformly across all adapter layers belonging to that module.
Hyper-pan paramterrssssess For the training of JMP backbone, we mainly follow the hyper-parameter con- figurations in Shoghi et al. (2024), with slight modifications to the learning rate and batch size. Dur- ing the module training stage of MoMa, we batch size of 64 and a learning rate of 5e-4 for 80 epochs. During downstream fine-tuning, we adopt a batch size of 32 and a learning rate of 8e-5. We set the training epoch as 60, with an early stopping patience of 10 epochs to prevent over-fitting. We
20

---

Published as a conference paper at ICLR 2026
Table 5: Downstream evaluation datasets.
Datasets
Num
Description
Experimental Band Gap (eV)
2481
The hand gap of material as measured through physical experiments. Data from Ward et al. (2018).
Formation Enthalpy (eV/atom)
1709
The energy change for forming compound from its clements, crucial for defining Gibhs energy of formation. Data from Wang et al. (2021); Kim et al. (2017).
2D Dielectric Constant
522
The diclectric constant of 2D materials from Choudhary et al. (2017).
2D Formation Energy (eV/atom)
633
The energy change associated with the formation of 2D materials from their constituent elements. Data from Choudhary et al. (2017).
Exfoliation Energy V/atom)
636
The energy required to separate a single few layers from bulk materials. Data from Choudhary el al. (2017).
2D Band Gap (eV)
522
The hand gap of 2D materials from Choudhary et al. (2017).
3D Poly Electronic
8043
Poly electronic of 3D materials from Choudhary al. (2018).
3D Band Gup (eV)
7348
The band gap 3D materials from Choudhary al. (2018).
Refractive Index
4764
The quantitative change of speed of light as passes through different media. Data from Dunn et al. (2020); Petousis al. (2017).
Elastic Anisotropy
1181
The directional dependence De Jong et al. (2015a).
material's clastic properties. Data from
Electronic Dielectric Constant
1296
Electronic diclectric constant refers dielectric response caused by clec- tronic polarization under an applied electric field. Data from Petretto et al. (2018).
Dielectric Constant
1296
Dielectric constant of materials from Petretto al. (2018).
Phonons Mode Peak
1265
Phonon mode peak refers the peak in the phonon spectrum caused by specific phonon modes. Data from Petretto al. (2018).
Poisson Ratio
1181 Poisson Ratio quantifies the ratio of transverse strain to axial strain in material under uniaxial stress, reflecting its elastic deformation hehavior. Data from De Jong el al. (2015a).
Poly Electronic
1056 The Average eigenvalue of the dielectric tensor's 's electronic component, where the dielectric tensor links material's 's internal and external fields. Data from Petousis el al. (2017).
Poly Total
1056
The Average dielectric tensor eigenvalue. Data from Petnusis et al. (2017).
Piezoclectric Modulus
941
Piczoclectric modulus measures material's ability to convert mechanical stress into electric charge or vice versa. Data from De Jong et al. (2015b).
adopt mean pooling of embedding for all properties since it performs significantly better than sum pooling certain tasks (e.g. band gap prediction), which echos the findings in Shoghi et (2024).
For the Adaptive Module Composition (AMC) algorithm, set the number of nearest neighbors (K in Eq. (1)) to For the optimization problem formulated in Eq. (3), we utilize the CPLEX optimizer from the evxpy package (Diamond et al., 2014). AMC is applied separately for each random split of the downstream tasks to avoid data leakage.
Computational Cost Experiments are conducted on NVIDIA A100 80 GB GPUs. During the module training stage, training time ranges from 30 to 300 GPU hours, depending on the dataset size. While this training process is computationally expensive, it is one-time investment, as the trained models are stored in MoMa Hub as reusable material knowledge modules. Downstream fine-tuning requires significantly less compute, ranging from 2 to 8 GPU hours based the dataset scale. The full module and adapter module require similar training time; however, the adapter module greatly reduces memory consumption during training. The time cost of AMC is discussed in Section D.3.
C.3 BASELINE DETAILS
The CGCNN baseline refers to fine-tuning the CGCNN model (Xie & Grossman, 2018) separately on 17 downstream tasks. Conversely, MoE-(18) involves training individual CGCNN models for
21

---

Published as a conference paper at ICLR 2026
each dataset in MoMa Hub and subsequently integrating these models using mixture-of-exper (Ja- cobs et al., 1991; Shazeer et al., 2016). For the baseline results of CGCNN and MoE-(18), we reproduce the results with the open-source codebase provided by Chang et al. (2022) and follow the exactly same hyper-parameters as reported in their papers.
For UMA, we fine-tune the UMA-Medium checkpoint (the largest open-sourced UMA model) on each dataset. To determine the batch size, we follow the max-atom batching strategy from the original UMA paper and set the maximum atoms per batch to 200, which ensures consistent memory usage across systems of varying sizes. All remaining hyperparameters (e.g., learning rate, number of epochs) follow the configurations used in JMP baselines.
For JMP-FT, we use the JMP (large) checkpoint from the codebase open-sourced by Shoghi et al. (2024) and fine-tune it directly on the downstream tasks with batch size of 64. JMP-MT adopts multi-task pre-training strategy, training on all 18 MoMa Hub source tasks without addressing the conflicts between disparate material tasks. Starting from the same pre-trained checkpoint as JMP-FT, JMP-MT employs proportional task sampling and trains for 5 epochs across all tasks with a batch size of 16. The convergence multi-task pre-training is indicated by a lack of further decrease in validation error most tasks after 5 epochs. For downstream fine-tuning, both JMP-FT and JMP-MT adopt the same training scheme as the fine-tuning stage in MoMa.
C.4 IMPLEMENTATION DETAILS OF LORAHUB LEARNING & SOFTMAX WEIGHTING
In our analysis experiments (Section 3.3), we compare AMC against two alternative module com- position strategies: Loraflub Learning, a black-box optimization approach, and Softmax Weighting. non-optimized performance-bastt based heuristic.
For the implementation of LoraHub Learning, we strictly follow the ppper-parameters and black- box optimization scheme in its official repository except that we training-five free kNN predictor to obtain the metric in each round of optimization, which is aligned with AMC. This because current capabilities pre-trained models cannot enable zero-shot prediction of material tasks in LLMs.
For the implementation of Softmax Weighting, we convert the predicted MAE from the same initial KNN evaluation into weight for each module. The goal is to directly assign higher weights to modules with better predicted individual performance (i.e., lower MAE). Formally, the weight wy for module j is calculated as:
exp( MTE exp(-MAE,
Wi
(4)
Σ N
where the temperature 7 is set to 1.
C.5 DISCUSSION FOR AMC ANALYSIS EXPERIMENTS
For the router- based JMP-(18) approach, full fine-tuning all parameters induces formidable memory cost, and is impractical considering MoMa Hub may further scale in the future. Hence, resem- bling Chang et al. (2022), we only unfreeze the final MLP layer well as the router network in downstream fine-tuning. We believe it underperforms MoMa because training a router over 18 het- expertss
erogeneous experts with limited supervision per task intrinsically difficult, leading to unstable and suboptimal training of module selection. By contrast, AMC avoids router training and uses a training-fred free convex weighting scheme guided by kNN-based proxy error, which is much better suited to the data-scarce, highly disparate material setting.
We conjecture that AMC outperforms LoraHub Learning for two main reasons. First, LoraHub optimizes weights based on the composed module, where arbitrary mixtures of heterogeneous rep- resentations yield noisy error signals and rugged, non-convex landscape. Second, AMC decouples weight selection from feature mixing by optimizing ensemble predictions. This formulation trans- forms the task into a convex problem, enabling AMC to reliably converge to a global optimum without navigating the instability inherent search-based methods.
The advantage of AMC over the Softmax Weighting highlights the importance of optimizing for synergy. Softmax Weighting determines each module's contribution based solely its isolated performance, overlooking potential synergistic interactions. In contrast, AMC explicitly optimizes for the weight configuration that maximizes collective performance and captures such interactions.
22

---

Published as a conference paper at ICLR 2026
C.6 DETAILS ON MOMA HUB SCALING ANALYSIS
The QM9 dataset (Ramakrishnan et al., 2014) comprises 12 quantum chemical properties (including geometric, electronic, energetic, and thermodynamic properties) for 134,000 stable small organic molecules composed of CHONF atoms, drawn from the GDB-17 database (Ruddigkeit al., 2012). is widely served as comprehensive benchmarking dataset for prediction methods of the structure- property relationships in small organic molecules.
In the continual learning experiments, we expand the MoMa hub by including modules trained on the QM9 dataset. For module training, we adopt the same training scheme as the original MoMa modules, with the exception of using sum pooling instead of mean pooling, as it has been empirically shown to perform better (Shoghi et al., 2024).
D MORE EXPERIMENTAL RESULTS
D.1 CORRELATION ANALYSIS BETWEEN KNN- PROXY AND POST-FINE-TUNING PERFORMANCE
We empirically examine whether the ANN-based proxy used in AMC is reliable indicator of post-fine-tuning performance. We consider three representative Mode Peak, and Exfoliation ties. For each task, we use all 18 modules in the MoMa hub and record (i) the per-module proxy MAE computed during the kNN step of AMC, and (ii) the test MAE obtained by fine-tuning each module individually on the same target. We then compute the Spearman and Pearson correlations between the proxy MAE and the post-fine-tuning MAE over the 18 modules. Across all three tar- gets, we observe consistently strong positive correlations (Spearman > 0.8, Pearson > 0.6, with p-values < 0.01 for Pearson and < 0.0001 for Spearman). Concretely, the Pearson correlations are 0.603, 0.628, and 0.699 for Phonons Mode Peak, Refractive Index, and Exfoliation Energy, re- spectively; the corresponding Spearman correlations are 0.851, 0.825, and 0.816. As visualized in Fig. 5, modules that achieve lower proxy error systematically attain lower post-fine-tune tunien MAE, providing direct empirical support that the kNN-based proxy is a reliable signal for guiding weight optimization in AMC.
Index, Phonons covering optical, vibrational, and energetic material proper-
D.2 SENSITIVITY ANALYSIS OF KNN PROXY AND OPTIMIZER
We perform an additional sensitivity analysis of the kNN components in AMC. Specifically, we vary the number of neighbors k, switch the distance metric from cosine similarity MAE, and modify the normalization of similarity scores from a weighted average to a uniform average over the kNN set. We study robustness by (i) computing the average pairwise correlation of the resulting module weight vectors across variants, and (ii) comparing the final post-finettuning MAE across variants.
Table 6 shows that the learned composition weights are highly robust under different kNN configu- rations. Over the 17 datasets, the average pairwise Pearson and Spearman correlations of the weight vectors are typically above 0.7. This indicates that changing k. the distance metric, or the normal- ization scheme perturbs the proxy predictor but AMC consistently recovers a very similar relative weighting over modules, i.e., the inferred relationships between modules remain stable
Table reports the resulting fen-ffinetttingg MAEs for each kNN variant. This analysis is con- ducted on single rriinvalidation split and a single random seed per dataset. Even under this stringent setting, MAE remains reasonably stable for most tasks across the five kNN configurations. The main exception is the Elastic Anisotropy dataset, where we observe larger variation between variants; in main experiments this target. Elastic anisotropy is a derived mechanical metric that depends on the full clastic response of the material, and we find in practice that the corresponding mapping from structure to target is more challenging and sensitive to initialization, which can amplify small differences in the proxy into larger differences in final MAE.
already noted substantial fluctuations across random seeds for
Finally, we also varied the CPLEX optimizer tolerances (optimality and MIP gap) from the default 10-6 to 10 and 10-9. These changes had effect the optimized weights, which is consistent
23

---

Published as a conference paper at ICLR 2026
Dataset
Avg. Pearson Corr. of Weights
Avg. Spearman Corr. of Weights
Experimental Band Gap
0.9776
0.7553
Formation Enthalpy
0,9972
0.8300
2D Dielectric Constant
0,9752
0.7545
2D Formation Energy
0,9935
0.7495
Exfoliation Energy
0.6139
0,8193
2D Band Gap
0,9594
0,5308
3D Poly Electronic
0,7660
0.771
3D Band Gap
0,9865
0.9267
Refractive Index
0.8591
0.8249
Elastic Anisotropy
0,978
0.8910
Electronic Dielectric Constant
0,9761
0.7768
Dielectric Constant
0,9731
0.7486
Phonons Mode Peak
0,9349
0.8936
Poisson Ratio
0,8570
0,8511
Poly Electronic
0.8876
0.8222
Poly Total Piezoelectric Modulus
0,9309
0.7178
0.7319
0.7972
Table 6: Stability of AMC weights under different kNN variants. Average pairwise Pearson and Spearman correlations between module weight vectors obtained from varying k, the distance metric, and the similarity normalization. Across most datasets, both correlations are typically above 0.7, indicating that AMC recovers highly consistent relative weight patterns over modules despite changes in the kNN setup.
Dataset
weighted
weighted weighted K10
weighted KS
uniform K5
Experimental Band Gap
0.2139
0.2284
0.2439
0,2356
0.2314
Formation Eathalpy
0.0142
2D Dielectric Constant
0,3152
0,3096
0.3137
0,3251
0.3120
Formation Fnergy
0,0292
Exfoliation Energy
0,6078
0,6395
0.7421
0,7084
0.6753
Rand Gap
0.1468
3D Poly Electronic
0,5243
0,5093
0.5138
0,5207
0.5038
Gap
0.0295
Refractive Index
0,2503
0,2560
0,2624
0.2626
0.2576
sniropp
1816
0.5717
Electronic Dielectric Constant
0,3564
0,3532
0.3585
0.3393
0.3560
Dielectric Constant
0.5257
Phonoos Mode Peak
0,1062
0,1310
0.1869
0,1446
0.1414
Prisson Ratio
0.3345
Poly Electronic
0.7307
0,7684
0 8036
0.7265
0.7691
Poly
0.5606
Piozocloctric Modulus
0,5843
0,5940
0.6093
0.5971
0.6002
Table 7: Sensitivity of post-finettuinng MAE to kNN design choices. Test MAE for different kNN configurations in AMC: varying (3/5/10), distance metric (cosine vs. MAE), and normaliza- tion (weighted vs. uniform) across 17 datasets. For most tasks, MAE differences between variants are modest, showing that downstream performance is relatively stable with respect kNN design choices.
with our formulation: our optimization problem is small and strongly convex MIQP over continu- ous variables, and the solver consistently reaches the global optimum under all tested settings.
D.3 EFFICIENCY ANALYSIS OF AMC
Time Cost For the prediction estimation stage, we further divide it into the embedding generation and kNN prediction step. While these steps should be conducted separately for each module and each downstream dataset, the process can be parallelized and the runtime mainly depends the size of the downstream dataset. As shown in Table 8, the maximum total time is below 30 seconds. For the weight optimization stage, report the minimum and maximum time required for convergence of each downstream task (Eq. (3)). As shown in Table 9, the time cost negligible and remains roughly constant as the number of modules scales.
24

---

Published as a conference paper at ICLR 2026
Table 8: Module prediction estimation time
Table 9;
Weight optimization time
Min time (s)
Max time (s)
Module #
Min time (s)
Max time (s)
Embedding generation
7.29
24.06
0.07
0.08
KNN prediction
0.05
4.02
0.12
0.15
Total time
7.34
28.08
18
0.14
0.25
Table 10: Test set MAE and average test loss of JMP-FT and MoMa under the full-data, 100-data, and 10-data settings. Results are averaged over five random data splits on one random seed. Results are preserved to the third significant digit.
Datasels
JMP-FT
MoMa JMP-FT (100)
MoMa (100)
JMP-FT (10)
MoMa (10)
Experimental Band Gap
0,380
0.305
0.660
0.469
1.12
1.245
Formation Enthalpy
0.156
0.0821
0.273
0.101
0,514
0.143
2D Diclectric Constant
2.45
1.90
3.19
2.35
7.74
3.31
2D Formation Energy
0.135
0.0470
0.366
0.113
0.842
0.214
2D Exfoliation Energy
38.9
36.1
54.4
56.1
87.3
2D Band Gap
0.611
0.366
0.890
0.517
1.23
1.05
3D Poly Electronic
23.7
23.0
33.6
24.8
$4.0
48.9
3D Band Gup
0.249
0.201
1.71
0.686
2.10
1.47
Dielectric Constant
0.0552
0.0535
0.134
0.102
0.289
Elastic Anisotropy
2.11
2.85
4.85
3.79
5.26
Electronic Dielectric Constant
0,108
0.0903
0.260
0.178
0,568
0,500
Total Dielectric Constant
0.172
0.155
0.361
0,287
0.543
0.527
Phonons Mode Peak
0.0710
0.0521
0.221
0.199
0,493
0.485
Poisson Ratio
0.0221
0.0203
0.0345
0.0317
0.0466
0.057
Poly Electronic
2.10
2.13
3.24
2.88
6.08
5.10
Total Poly
4.83
4.76
6,54
6.32
11.2
10.1
Piczoclectrie Modulus
0.169
0.175
0.248
0,258
0,303
0.290
Average Normalized Test MAE
0.222
0.187
0.408
0.299
0.700
0.550
During embedding generation, only one module loaded into GPU at a time, requiring approximately 1.8 GB of memory. The generated embeddings are stored on CPU, with the largest set requiring about 5.5 MB. Overall, AMC lightweight memory usage and scales well with the number of modules.
Memory Cost
D.4 COMPLETE FEW-SHIOT LEARNING RESULTS
We present the complete results of the few-shot learning experiments in Table 10. MoMa con- sistently shows performance improvements across all settings, with the margin of normalized test loss increasing as dataset size shrinks. These results highlight MoMa's strong potential to retain a performance advantage in few-shot scenarios, which are prevalent material property prediction tasks.
D.5 COMPLETE RESULTS FOR SCALING ANALYSIS OF MOMA
We present the complete results for the scaling analysis of MoMa in Table 11. report test set MAE for hub sizes of 5, 10, 18 (full MoMa hub), and 30 modules (by adding QM9 modules). The last row reports the normalized average over all tasks.
E POTENTIAL SOCIETAL IMPACT
MoMa is visioned be an open-source platform for the sharing of materials knowledge modules. Potential positive societal impacts include the acceleration of the discovery of new materials with desirable properties, which benefit industries such as energy, electronics, and manufacturing. How- ever, there are risks associated with the mal-intended use of material knowledge to develop harmful or unsafe materials. To mitigate these risks, is crucial to ensure that the application of this work adheres ethical guidelines. Although do not foresee significant negative consequences in the near future, we recognize the importance of responsible usage and oversight in the application of these technologies.
25

---

Published as a conference paper at ICLR 2026
Table 11: Test set MAE of MoMa under different hub sizes: modules, 10 modules, full MoMa hub (18 modules), and 30 modules (with QM9). Results preserved to the fourth decimal digit.
Number of MoMa Modules
5
10
18 (MoMa)
30 (+QM9)
Experimental Band Gap
0.3478
0.3324
0,2975
0.2960
Formation Enthalpy
0.0799
0,0814
0.0789
0.0819
2D Dielectric Constant
2.2075
1.9482
1.9406
1,8879
2D Formation Energy
0.0513
0.0510
0.0438
0.0470
2D Exfoliation Energy
38.6231
36.6587
34.5769
35.1542
2D Band Gap
0.4624
0.4256
0.3649
0.3605
3D Poly Electronic
23.3909
23.0813
22.7205
23.3679
3D Band Gap
0,3035
0.2555
0,2270
0,2053
Dielectric Constant
0.0549
0.0529
0.0511
0.0529
Elastic Anisotropy
1.9967
2.4103
2.5340
2.6408
Electronic Dielectric Constant
0.1046
0,0878
0.0909
0.0892
Total Dielectric Constant
0.1762
0.1554
0.1571
0.1561
Phonons Mode Peak
0.0528
0.0505
0.0512
0.0460
Poisson Ratio
0.0240
0,0207
0.0206
0.0206
Poly Electronic
2.0588
2.0215
2.0445
1,9837
Total Poly
4.9129
4.9148
4,8804
4.7358
Piezoelectric Modulus
0.1805
0.1713
0.1721
0.1743
Average Normalized Test MAE
0.2040
0.1910
0.
0.1759
26


Reviews:
[{"review_id": "Y7v2R8k5vV", "reviewer": "Reviewer_tuSk", "summary": "This paper proposes MoMa, a modular learning framework for material property prediction designed to tackle the diversity (heterogeneity of tasks and systems) and disparity (incompatible physical laws) in materials datasets.\nIt first trains a collection of task-specific modules (either full fine-tuned models or lightweight adapters) on high-resource material datasets, and later composes these modules adaptively for new downstream tasks via the proposed AMC algorithm, which estimates module relevance using kNN-based proxy prediction, optimizes non-negative weights through convex programming, merges module parameters via weighted averaging, and fine-tunes the composed module.", "questions": "See weaknesses.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- Introducing a hub-and-composition paradigm is new to this domain and nicely parallels modular LLM work (AdapterHub, LoRAHub).\n- AMC is simple, requiring only kNN embedding estimation and a small convex optimization step.\n- Evaluation is pretty comprehensive.\n- The visualization of module weights (Fig. 6) provides interesting physical correlations and hints at cross-property transferability.\n- Open-sourcing MoMa Hub could foster standardized module reuse and privacy-aware data sharing in materials science.", "weaknesses": "- While AMC is well-engineered, its core ideas (kNN prediction + convex weighting + parameter averaging) largely repackage known components from model-merging and ensemble literature. The conceptual advance over prior modular or LoRAHub methods is modest.\n- The paper lacks formal justification or analysis of why the convex proxy optimization correlates with post-fine-tuning accuracy, beyond empirical correlation (r = 0.69). A clearer theoretical connection between proxy error and final generalization would strengthen the contribution.\n- Baselines such as JMP-MT and MoE-(18) are implemented in-house; comparisons to other modern material foundation models (e.g., MatterSim [1], Orb-v3 [2], UMA [3]) are missing, limiting claims of SOTA performance.\n- Also, ablations explore only AMC variants, not architectural or hub-scale sensitivities.\n- Although the framework is claimed backbone-agnostic, all results use JMP. It remains uncertain whether MoMa generalizes to other encoders or representations (e.g., E3NN [4], GemNet-OC [5]).\n- While clear overall, the paper is somewhat long and repetitive; core algorithmic details could be condensed, and figures (e.g., Fig. 3) are descriptive but not deeply analytical. The continual-learning section feels exploratory and under-developed.\n- The work focuses on crystalline and small-molecule systems; polymeric, amorphous, and multi-phase materials are not addressed, limiting generality.\n\n[1] Yang, H., et al. \"A Deep Learning Atomistic Model Across Elements, Temperatures and Pressures.\" arXiv preprint arXiv:2405.04967 (2024).\n\n[2] Rhodes, Benjamin, et al. \"Orb-v3: atomistic simulation at scale.\" arXiv preprint arXiv:2504.06231 (2025).\n\n[3] Wood, Brandon M., et al. \"UMA: A Family of Universal Models for Atoms.\" arXiv preprint arXiv:2506.23971 (2025).\n\n[4] Geiger, Mario, and Tess Smidt. \"e3nn: Euclidean neural networks.\" arXiv preprint arXiv:2207.09453 (2022).\n\n[5] Gasteiger, Johannes, et al. \"GemNet-OC: developing graph neural networks for large and diverse molecular simulation datasets.\" arXiv preprint arXiv:2204.02782 (2022).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "JhK1ZZMLlA", "reviewer": "Reviewer_nxqB", "summary": "This paper presents a straightforward machine learning framework for materials science. The framework is composed of multiple independently trained modules, each tailored for a specific task. For a given downstream application, the framework dynamically selects and integrates a combination of modules based on their estimated predictive performance, followed by joint fine-tuning. Extensive experiments across 17 material property prediction benchmarks demonstrate the framework’s superior accuracy and adaptability.", "questions": "1. Does using the open source cvxpy package guarantee to find the optimal weight? \n2. How much does the accuracy of individual modules affect the weight calculation and model performance? \n3. Is there any correlation between individual modules? Does it affect molecule weight calculation or model performance?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The MoMa framework have individual modules trained on a wide range of properties and tasks. This architecture aligns with the diversity and disparity of material properties. It also enables the framework to extend to other modules trained on specialized material properties easily. By having a full module and an adapter module, the framework is parameter-efficient. It also allows easy fine-tuning on downstream tasks. The author also shows that the module selected by the algorithm correlates with the physical relationships between material properties, showing the effectiveness and trustworthiness of the algorithm.", "weaknesses": "The MoMa is a flexible framework for material property prediction. However, there might be other material properties that could not be captured by the trained modules, resulting in inaccurate property prediction.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7OHKJ6vRxN", "reviewer": "Reviewer_3v4v", "summary": "This paper introduces MoMa, a simple modular learning framework to overcome the diversity and disparity challenges in material property prediction. It first trains specialized modules on a wide range of tasks and stores them in a MoMa Hub. Then, Adaptive Module Composition (AMC) composes the most synergistic modules from this Hub into a single, tailored model for new tasks, achieving substantial performance gains over existing baselines.", "questions": "Please refer to the Weaknesses section above.", "limitations": "", "rating": 4, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This paper effectively tackles the challenge of material learning by applying a modular deep learning framework.\n\n2. The paper is clearly written, presenting an intuitive and well-motivated method.\n\n3. MoMa achieves superior performance, showing substantial average improvements over existing baselines.", "weaknesses": "1. AMC's design relies heavily on heuristics, such as using a kNN performance proxy, and the paper would be stronger if it provided sufficient justification for this design choice.\n\n2. As the authors note for future work, it is questionable whether the kNN-based proxy will remain sufficiently efficient and effective as MoMa scales to hundreds or thousands of modules.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "LWuHYYSt77", "reviewer": "Reviewer_1mck", "summary": "This paper introduces an approach for material property prediction that trains specialised modules for different tasks and then adaptively combines them for new prediction scenarios. An algorithm is proposed to efficiently estimate the performance of each module on a target task and then optimise a weighted combination of these modules to create a tailored model for the specific task.  Experiments across 17 datasets show the effectiveness of the approach, especially in few-shot settings, enabling continual learning and scalable integration.", "questions": "1. Would inclusion of a “Naive Soup” baseline, averaging all eighteen fine-tuned modules with uniform weights, help isolate AMC’s benefits by demonstrating whether ensembling everything is inferior to selection and weighting?\n2. Could validation be provided showing correlation between the kNN-based proxy error and post-fine-tuning performance, e.g., scatter plots comparing modules’ proxy errors versus their results after standalone fine-tuning across representative tasks?\n3. Could sensitivity analyses for K in kNN, distance metric choices, normalization schemes, and optimizer tolerance be included, reporting how MAE and selected weights vary across these settings for robustness measures?\n4. Could a second backbone be added to main results, even on a subset, to validate backbone-agnostic gains, and discuss layer alignment issues impacting weight-space merging beyond primarily JMP-based experiments reported?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The evaluation spans 17 diverse downstream tasks. The authors compare  against strong and relevant baselines, including not just older methods (CGCNN) but state-of-the-art pre-trained models fine-tuned in different ways (JMP-FT, JMP-MT).\n2. The authors have diligently validated their design choices. The ablation study presented in Figure 4 is particularly effective, clearly demonstrating that both the selection of modules (vs. averaging all) and the learned weighting (vs. uniform averaging) are critical to AMC's success.", "weaknesses": "1. The method of weighted-averaging fine-tuned model parameters seems to be nearly identical to the 'Model Soups' approach [1]\n2. The algorithm's effectiveness hinges on the assumption that the kNN-based proxy error (Eq. 2) is a reliable indicator of final, post-fine-tuning performance. This is a strong heuristic, but its validity is not fully demonstrated. The authors should conduct an experiment correlating the proxy error of individual modules with their actual performance after being fine-tuned alone on the downstream task.\n3. Section 3.5 presents a 'Continual Learning' experiment by adding 12 QM9 molecular modules to the Hub. While this demonstrates the framework's ability to incorporate new knowledge, it doesn't align with the standard definition of continual learning, which typically involves sequential training and mitigating 'catastrophic forgetting.'\n\n\n[1] Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time, In ICML'22.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)