Learning from the Electronic Structure of Molecules across the Periodic Table
Authors: Manasa Kaniselvan
Benjamin Kurt Miller
Meng Gao
Juno Nam
Daniel S. Levine
Venue: ICLR 2026 Poster (2026)
Keywords: Interatomic potentials
electronic structure
materials science
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)
TL;DR: Pretraining on electronic structure helps to learn generalizable features for atomistic ML models from limited data.

Abstract: Machine-Learned Interatomic Potentials (MLIPs) require vast amounts of atomic structure data to learn forces and energies, and their performance continues to improve with training set size. Meanwhile, the even greater quantities of accompanying data in the Hamiltonian matrix $\mathbf{H}$ behind these datasets has so far gone unused for this purpose. Here, we provide a recipe for integrating the orbital interaction data within $\mathbf{H}$ towards training pipelines for atomic-level properties. We first introduce HELM ('Hamiltonian-trained Electronic-structure Learning for Molecules'), a state-of-the-art Hamiltonian prediction model which bridges the gap between Hamiltonian prediction and universal MLIPs by scaling to $\mathbf{H}$ of structures with 100+ atoms, high elemental diversity, and large basis sets including diffuse functions. To accompany HELM, we release a curated Hamiltonian matrix dataset, 'OMol\_CSH\_58k', with unprecedented elemental diversity (58 elements), molecular size (up to 150 atoms), and basis set (def2-TZVPD). Finally, we introduce 'Hamiltonian pretraining' as a method to extract meaningful descriptors of atomic environments even from a limited number atomic structures, and repurpose this shared embedding space to improve performance on energy-prediction in low-data regimes. Our results highlight the use of electronic interactions as a rich and transferable data source for representing chemical space.

Published as a conference paper at ICLR 2026
LEARNING FROM THE ELECTRONIC STRUCTURE OF MOLECULES ACROSS THE PERIODIC TABLE
Manasa Kaniselvan 1.2.t, Benjamin Kurt Miller 1, Meng Gao 1, Juno Nam & Danicl S. Levine! FAIR at Meta SMIT ETH Zurich
Work done during internship at FAIR
Seanseella@@eeee ch, levineds@meta.. com
ABSTRACT
Machine-Learned Interatomic Potentials (MLIPs) require amounts of atomic structure data to learn forces and energies, and their performance continues to improve with training set size. Mcanwhile, the even greater quantities of accom- panying data in the Hamiltonian matrix H behind these datasets has so far gone unused for this purpose. Here, we provide a recipe for integrating the orbital interaction data within H towards training pipelines for atomic-level properties. We first introduce HELM ("Hamiltonia--trall Molecules"), state-of-the-arr Hamiltonian prediction model which bridges the gap between Hamiltonian prediction and universal MLIPs by scaling to H of structures with 100+ atoms, high elemental diversity, and large basis sets including diffuse functions. To accompany HELM, we release a curated Hamiltonian matrix dataset, 'OMol. COM
Electronic-structue Learning for
with unprecedented elemental diversity (58 elements), molecular size (up to 150 atoms), and basis set (def2-TZVPD), Finally, we intro- a method extract meaningful descriptors of
duce 'Hamiltonian pretraining' atomic environments even from a limited number atomic structures, and repur- pose this shared embedding space improve performance on mergy-predictte in low daw- regimes. Our results highlight the use of electronic interactions as a rich and transferable data source for representing chemical space.
INTRODUCTION
The availability of large volumes of high quality training data has been key to advancing the state of structure-property prediction models, particularly for applications as Machine-Leamne, Interatomic Potentials (MLIPs). While MLIPs trained on sizable datasets now learn the mappings between atomic positions and inter-atomic forces and energies in a variety of structures, challenges remain when (1) generalizing to highly out-of-distributie
structures (Antoniuk et al., 2025; Sheriff et al., 2025; Zhai et al., 2023) and complex materials (Liu et al., 2024; Miret et al., 2025), (2) satisfying the physics required compute accurate vibrational properties (Fu et al., 2025) and realistic Molecular Dynamics (MD) trajectories (Fu et al., 2022; Bigi et al., 2024), and (3) working with the limited quantity of training data available at more chemically accurate levels of theory. Despite the advent of very large public datasets of DFT calculations, the performance of state-of-the-arl MLIPs is still highly data-limited (Wood al., 2025), and expected to improve as more data becomes available. However, as the current largest models are already trained on over 10 billion core hours worth of DFT data (Levine et al., 2025), further increases in dataset size may not be practically feasible. Alternative sources of data are thus required to further advance MLIP development.
Generating this (typically Density Functional Theory (DFT)-level) training data requires computing the electronic Hamiltonian matrix for each given system of atoms. For every system of N atoms, there is energy label, O(N) force labels, and ( labels in the Hamiltonian matrix. The latter has far not been leveraged towards training large-scale atomic property prediction models, although they are available from DFT calculations with no additional compute.
At the same time, the Hamiltonian matrix contains far more information than the atomic-level prop- erties (c.g., forces, energies) which are computed from it. includes information about excited states, ionization energies, electron density, multipole moments, and more. Incorporating this elec-

---

Published as a conference paper at ICLR 2026
tronic structure data in different ways has served to improve data efficiency and generalizability across atomic-level prediction tasks and challenging interaction regimes, by exploiting patterns in the underlying electronic representations of atomic interactions (del Rio et al., 2023; Foster et al., 2025; Kang et al., 2025; Suman et al., 2025; Sunshine et al., 2023; Tang et al., 2025; Shao et al., 2023). These initial attempts inform MLIPs with electronic interactions have, however, been lim- ited to demonstrations on relatively restricted datasets in terms of both molecular size and elemental composition. Existing models struggle scale the structure sizes (Xia al., 2025), basis sets (Li et al., 2025), and number of atomic elements necessary to advance the state of the art of MLIPs. Here we directly address these challenges enable scalable electromic-structure learning, and pro- vide a recipe for learning energies from shared embedding space via 'Hamiltonian pretraining'. We make the following contributions:
Develop HELM "Hamilonnian-trained Electronic-structee Learning for Molecules", a scal- able Hamiltonian matrix prediction network which enables transfer-learing to energy predic- tion tasks from a unified atomic orbital basis. HELM is state-of-the-art on the most challenging Hamiltonian matrix benchmarks openly available, and is the first 'universal' electronic structure prediction model demonstrated on structure sizes and chemical diversity comparable to those treated by current MLIPs.
2. Introduce curated dataset of Hamiltonian matrices, CSM55CS based on the Open Molecules (OMol25) (Levine et al., 2025) data. Compared to the previous largest publicly- available Hamiltonian matrix benchmark (Khrabrov et al., 2024), OMol.CSH efficiently samples electronic interactions across an unprecedented range of atomic elements, inter-atomic distances, and orbital types, matching the scale and diversity of current atomic force and energy datasets. It presents a realistic and challenging target for Hamiltonian pretraining approaches.
Finally, we explore the use of HELM towards improving energy prediction via 'Hamiltonian pretraining', using CMHLCSSH_58K as well as the second-largest publicly available Hamiltonian database, D2DFT (Khrabrov et al., 2024). We find that embeddings after Hamiltonian matrix train- ing contain fine-grained representations of atomic environments, while this structure does not appear when training directly on energy labels for the same number of molecules. Hamiltonian pretraining thus leverages these representations to achieve up to a 2x improvement energy-prediction aceu- racy in low-data controlled experiments. Achieving such improvements by naively scaling atomic force and energy data alone would require more than an order of magnitude more calculations.
BACKGROUND AND PREVIOUS WORK
Many of the properties being investigated in computational chemistry research are computed using first-principles electronic structure methods such as DFT, which iteratively finds consistent repre- sentation of electron density and potential for a system of atoms, defined by atomic elements and positions (Kohn & Sham, 1965). The computed electronic structure information can then be pro- cessed to directly obtain various quantities at the electronic and atomic levels, such as molecular energies (see Section A.2).
Learning interatomic potentials: MI.PPs approximate the electronic Born-Oppe potential energy surface far lower cost, enabling large-scale MD and high-throughput virtual screening of molecules and materials. Early work moved from empirical force fields with hand-crafted terms data-driven models that learn energies and forces from DFT, using local descriptors with neural networks or Gaussian processes. To improve transferability and data efficiency, descriptors were designed to encode permutation invariance and rotational equivariance (Behler Parrinello, 2007; Bartók et al., 2010; Drautz, 2019).
Motivated by orbital symmetries in DFT, modern MLIPs adopt graph neural networks that maintain rotational equivariance by constructing features in the form of irreducible representations ('irreps") ) and mixing them through message passing iterations performed with tensor products (Anderson et al., 2019; Batzner et al., 2022; Batatia et al., 2022). In these networks, 'descriptors' are effectively learned through data. On this foundation, universal MLIPs pretrain on large, heterogeneous datasets to yield single models usable across diverse elements over the periodic table (Chen & Ong, 2022; Deng et al., 2023: Batatia et al., 2023; Park et al., 2024; Yang et al., 2024: Rhodes et al., 2025). Recently, Meta's UMA model (Wood et al., 2025) demonstrated multi-task training across molecules
2

---

Published as a conference paper at ICLR 2026
and materials to span broad chemical space and achieve strong accuracy and efficiency. However, despite being trained on over 459M energy labels, UMA's performance is still limited by data.
To close the remaining gap to first- fiist-rriciples fidelity, incorporating electronic structure signals may be an essential next step, both in terms of the additional volume of data available at electronic reso- lution, and the rich information contained within it. Long-range electrostatics, dispersion, non-local charge transfer, and field response remain challenging for current models, and initial explorations that learn auxiliary electronic observables responses provide encouraging signals (Anstine al., 2025; Falletta et al., 2025; Cheng, 2025).
Learning orbital interactions: Symmerry--constrained GNN architectures have recently been adapted to instead directly predict the underlying electronic structure of atomic systems. Here, the most common objective is learn the elements of the ground state Hamiltonian matrix (also re- ferred to as the "Fock" matrix in some contexts). It has conventionally been used to study the band structure (Fischetti
Laux, 1996) and non-equilibrium transport properties of materials (Brand- byge et al., 2002), or serve as input to quantum many-body solvers (Kolorenč et al., 2010). In such electronic structure prediction models, each sub-matrix of the Hamiltonian, which represents inter- actions between atomic orbitals of angular degree I and lg. is decomposed and transformed into direct sum of irreps of degree 1/1 - 12] to 1 + l2: (41 61 = 141 - l2] @ (4 L2] + +1) -.... + (4 + 12)) using the Clebsch-Gortd Gordan (CG) coefficients. As many equivariant GNNs already operate in basis of spherical harmonic coefficients (typically mapping the output / = 0 or = 1 coefficients of output embeddings to atomic energies and forces (Fu et al., 2025)) they can be adapted to instead map the full set of ( = O.. coefficients to every submatrix of H.
To learn the H of small molecules, initial models used full tensor products to mix spherical har- monic coefficients while maintaining rotational equivariance (Unke et al., 2021; Gong et al., 2023). Following work focused on eliminating unnecessary tensor product components and taking advan- tage of CG coefficient sparsity to improve the efficiency of these operations (Yu et al., 2023b; Gong al., 2023; Luo al., 2025). More recent models have replaced full tensor products by SO(2) con- volution approaches (Passaro & Zitnick, 2023) to decrease the computational complexity of treating high mmaz from ( to (1 (Li et al., 2025; Yu et al., 2025) - this is critical for treating inter- actions within basis sets containing d and f orbitals. Several works are now integrating electronic structure prediction models with custom datasets to enable new research directions in electronic materials which were previously computationally unfeasible (Xia al., 2025; Haldar et al., 2025).
METHODS
We introduce HELM: "Hamiltonian-traine Flectronic--srucctue Learning for Molecules." HELM is composed of feature-extraction backbone with shared weights and two separate output heads
Backbone
Hamittorian Output
convolation
Gute
20l Jeا - (00
Edge convolation
Gotto
YD
H
Melecide
Figure 1: Architecture of HELM. Each rectangle in the backbone represents an embedding for an atom inter-atomic interaction, has a shape of (4max + 11² x C, and contains set of learnable spherical harmonic coefficients for each (l, up to specified Imax- TC Tensor construction layer to reconstruct the Hamiltonian (see Section B.1).
3

---

Published as a conference paper at ICLR 2026
(Fig. 1).
One head is designed to predict the Hamiltonian matrix H, while the other predicts total energy values from a given atomic structure. In a typical training regime, HELM is first trained to estimate H, then the backbone features are used directly and/or finetuned to estimate energy.
Backbone: The backbone of HELM is an equivariant message passing GNN based on that of Fu et al. (2025). The input to HELM is a molecular graph, represented by displacement vectors rij I23 and a one-hot array of atom types Z; E {0. 1) with i, = ......N .N where N is the number of atoms, and f is the maximum atomic number. Unlike MLIPs, where predictions are made nodewise, learning Hamiltonian matrices requires the model to make predictions for both nodes (intra-atomic orbital interactions) and directed edges (inter-atomic orbital interactions). We initialize node and edge embeddings Zij as follows:
Sgiined (7.) if =
(1) Zij -
,00.00 (e))
[= (k) (A)
(_()) (2s
fgaus (ris) if + J. ||ayl
< Tout
Rij =
i-j=1
(1)
with k = L..... K where [z, "ij [ denotes an N NN sparse array of multi-chamned spherical harmonic coefficient embeddings, and indexes over atoms within rout of i. Each of these em- beddings has shape (Emax + 1)² C, where C is a channel hyperparameter. feihted and fl are atomic element and gaussian radial embeddings, and maax is the spherical harmonic degree required to represent all coupled interactions in the orbital basis. These initial embeddings are transformed (k)
E, Fnndo (Ie!*) .2) 1.,)). T..).
through K layers: With y
(n S (1. (8)
(k+1) 2gg
ot JLayer
(K) 11r,s) (b) , (I=4
ifi=j if & j. (1rslll < reat)"
(2)
where f and C'ge are embedding update operations, internally consisting of SO(2) convolutions and gating mechanisms (Passaro & Zitnick, 2023). Oe, 0g: 02 Oz...., 0% 0 are learnable weights. The key idea is that the node embeddings at layer k + are a function of node embeddings at layer k; however, edge embeddings at layer & are function of node embeddings in the same layer k specifically the nodes they connect. This one-way dependence of edge embeddings on node embed- dings serves two purposes. First, during training, it incorporates information about the 1/r decay of Coulombic electron integrals present in the interatomic interactions of H (see Section B.2). Second, it allows us to omit the edge updates when re-pu re-purposing the trained backbone for energy prediction. We denote the full backbone, and also just the diagonal, with (o is function composition):
fluye k-? flayer finits faiagonal (x) (x) flz
fdiagonal f f(
(3)
Hamiltonian output head (H = =f ([<(()) Zi ()): Each block of H can be decomposed into direct sum of irreps (see Section B.1), which serve as the labels for every node and edge of the atomic graph. The Hamiltonian output head then maps each of the learned embeddings 2; and
(K) to the corresponding label extracted from H. We make two additional architectural choices designed to improve HELM'ss s ability to handle larger Hamiltonian datasets:
2ig
(1) To better distinguish different orbital shells when using large basis sets (e.g. multiple d-type functions for a given principal quantum number), we add additional gated nonlinearity (Weiler et al., 2018). Each irrep of nonzero degree is multiplied by learned scalar, Is passed through sigmoid b o (gg) >/5,1ya)
activation a: { o (2 .)) @
(K)
(2) The irreps in the diagonal blocks of H, which represent interactions between orbitals on the +eg
same atom have additional parity constraints: &f @ E
+
[ 14 (-1)s (g)
-
Here, /4, l represent the degree of the interacting orbitals within atom A, 8 the Kronecker delta, and the fgs represent their tensor product in the coupled basis. To take advantage of constraints, we compute only unique and non-zero node values.
4

---

Published as a conference paper at ICLR 2026
Energy output head (E= [2 While the total energy E of molecule be directly com- puted from H', this process involves re-assembling the predicted H from the internal representation used in HELM. This makes the use gradient-based force prediction or direct energy finetuning computationally expensive for large structures. Additionally, some of the additional DFT quantities (namely the evaluation of the exchange-correlation energy) require numerical evaluation of the den- sity obtained from H grid and is thus considerably slower than forward-pass of the network.
We therefore design an additional output head to learn a mapping = ([x directly from
the final node embeddings. In practice, [[= ())) is implemented with single embedding update block, followed by linear transformation w of the = 0 (scalar) output components, and sum
over the nodes in each molecule to obtain total energy prediction E ΣE w 2,,1=0
(K+1)
3.1 OMOL BOOSED-SHELL 58K
GAJORIM
SARMIN
intenction)
(6)
Figure 2: Breakdown of atomic and electronic interactions within the OMoL.SHH558 dataset. (a)
Four sample molecules of increasing size and (b) magnitude plots of their Hamiltonian matrices. (c) The full dataset spans 58 atomic elements, as shown in the element distribution histogram; the inset details the corresponding orbital count within the def2-TZVPD basis across the dataset. (d) Element-eeemmntt interaction heatmap indicating the diversity of interactions across the dataset. (e)
Histogram of the distribution of atoms per structure (ranging from 10 150).
Although Hamiltonian datasets for molecules have become increasingly diverse (Schütt et al., 2019 Yu et al., 2023a; Khrabrov al., 2024), they remain heavily limited in chemical scope compared to recent datasets of atomic forces and energies (Levine et al., 2025). Achieving 'universal' elec- tronic structure learning and potential transferability other tasks via pretraining requires data with sufficient sampling of both electronic structures and atomic energies for different atomic elements, structure sizes, configurations, and interaction distances.
To bridge this gap in chemical diversity and complexity between electronic and atomic property datasets for molecules, we introduce 'OMoL.CSH558K entries in the Open Molecules 2025 (OMol25) dataset which contains both electronic structure (KS Hamiltonian) data as well as atomic forces and energies for over 110M structures. This subset con- tains 56,657 entries. OMoI_CSH_588 is designed to efficiently sample three degrees of freedom which generate maximally informative training structures while avoiding redundancy in orbital in- teractions: (1) atomic elements, (2) orbital interactions, and (3) inter-atomic distance. The dataset includes 58 elements, which comprise all the first 83 atomic elements other than the first-row tran- sition metals and the lanthanides." It has roughly 1000 occurrences of all heavy elements. Each
curated subset of the more than 110M
Given the availability "These two sets of elements were excluded because they contain very high angular momentum orbitals (g-functions) in the OMol basis set (def2-TZVPD) which, while posing no technical challenge to the model framework, do dramatically increase memory consumption during training.
few other mnm-itrrativelycompptte DFT quantities (sec Section A.2)
5

---

Published as a conference paper at ICLR 2026
structure contains between 10 and 150 atoms, and interaction distances within the largest struc- tures range up to 15 A. With > 50k structures, 58 atomic elements, and long-range interactions, OMol CSH 58k is the largest Hamiltonian matrix dataset to date in structure size, matrix size, and number of elements.
The presence of heavy atoms and large structures make this dataset d especially challenging for Hamil- tonian matrix prediction, by requiring the treatment of higher moz (6 to consider interactions be- tween f-orbitals), high orbital multiplicity, and large target sizes account for the multiple compact and diffuse orbitals in the basis. We introduce two test splits to assess the performance of HELM and future models: @MMolall_5ke and (OMol.common.Il are drawn from the OMol25 test data, and therefore have unique compositions from all training data. These data span the same elemental composition as the OMol CSH 58k dataset. The second split consists of 1. 1,006 structures also drawn from the OMol25 test data, but only containing the elements present in DFT (H, N, O, F, S, CI, Br). This allows for easier comparison of performance ex- isting models and datasets. An overview of the molecules and corresponding Hamiltonian matrices contained within the train and test splits is summarized in Table 1.
The first consists of 4,937 structures that
Elements
Molecules
N atoms
Nedges
# Fu
# Hu
CSH..CS OMal common 1k OMol.all.k
all 58 H, C, N, O, F, S, CI, Br all 58
56.657
59
4,258
3,362,755
241,227,574
1,006
98
9,086
98,399
9,140,166
4,937
75
7,396
370,924
36,514,364
Table 1: Summary of the OMol train and test datasets, including the number of molecules, average number of atoms (N atoms) and edges (Nodge) per molecule, and the total number of intra- atomic (H₃)) and inter-atomic (Hig) orbital interaction matrices over all molecules in the datasets. The last column indicates the presence of total energy values per molecule.
Loss and scalar referencing: Previous Hamiltonian prediction networks treating small molecules have typically computed the loss with combinations of mean absolute errors (MAE) or mean squared errors (MSF) over the matrix elements (Yu et al., 2023b; Luo et al., 2025). However, straightforward element-wise losses are not comparable across different structure sizes and atomic elements, mak- ing it difficult to develop a meaningful training signal for large datasets with both size and element diversity. To address this, Li et al. (2025) introduced the 'wavefunction alignment loss', which ef- fectively re-weights matrix elements according their relative importance the eigenvalue spectra. However, implementing the wavefunction alignment loss in practice involves re-assembling H after every forward pass during training. When treating large structures with 100+ atoms, building these intermediate matrices becomes a computational bottleneck.
Here, we posit additional two reasons why naive element-wise losses may fail. First, as the number of heavy elements increases, the distribution of magnitudes in the matrix elements of H increases disproportionatel. The parallel problem in energy prediction has typically been mitigated by computing reference values per atomic element, and using them to rescale the training data. Sec- ond, MAE losses are performed component-wise, which may bias the loss computed over otherwise rotationally-ide -ddentiial edges (Wang et al., 2024). To mitigate these effects, we first pre-process the node labels to scale and center the = 0 components, which stem roughly from dement-speeife lo- calized core electron states (see Section B.3). This follows a similar process applied in other recent work on Hamiltonian prediction for materials (Haldar et al., 2025; Qian al., 2025), and serves to effectively homogenize the variance of the data across different elements. Finally, train with MOSt-MSE+MME loss over the labels, which is independent of irrep orientation.
RESULTS
In Section 4.1, evaluate the ability of HELM learn Hamiltonian matrix elements for two existing benchmarks, and show that it is state-of-the-art for this task. In Section 4.2, we use the backbone of HELM, pretrained on the Hamiltonian matrices, to perform low-data controlled training experiments on energy labels in the DFT and OMol MMMM_CSSH... CSH_58K datasets, demonstrating an increase in per-molecule data efficiency stemming from pretraining.
Inn practice, we clip the matrices to maximum inter-atomic interaction distances of 12 A, which is suffi- cient to capture matrix elements greater than 1x 10 (see Section B.2)

---

Published as a conference paper at ICLR 2026
4.1 HAMILTONIAN MATRIX BENCHMARKS
Table 2 summarizes results for benchmarks two previous datasets, MD17/QM7 and DFT. Note that matrix element losses are not comparable across different datasets due to the difference in molecule size and elemental diversity. Prediction of the overlap matrix is omitted, these rapidly obtained with PySCF. Model/training parameters are in Section C.1/ Section C.2.
MD17
DFT Train/Tet-5k (41/38)
Water (3)
Ethanol (9)
Malondialdehydd (9)
Uracil (12)
Train/ Test:/ (39/37)
Train/Test-10 (42/39)
Her (x10
Ex.)
H yerr (x10-=BX) Ex)
SchNOrb
165.4
187.4
191.1
227.8
21500.0
20700.0
20700.0
PhiSNet
17.59
12.15
12.32
10.73
180
330.0
350.0
QHNet
10.79
20.91
21.52
20.12
840.0
730.0
520.0
SPHNet
23.15
21.02
20.67
19.36
HELM
9.33
5.79
4.86
3.61
60.33
57.41
59.21
gery (meV)
gerr (meV)
HELM
23.67
28.93
20:15
22.69
440.82
399.840
415.55
Table 2: Comparison of H matrix element prediction error on the MD17 and DFT molecule datasets, for SchNOrb (Schütt et al., 2019b), PhiSNet (Unke et al., 2021), QHNet (Yu et al., 2023b), and SPHNet (Luo et al., 2025). In parentheses under each dataset are the average number of atoms per molecule. Element-wise matrix errors are computed as He N E., [H H The best performing model is indicated in bold, and the second best is underlined. Benchmark numbers for DFDFT are taken from Khrabrov et al. (2024). In the last row, we summarize the MAE energy error (Eerr) computed from H. and Higf using PySCF, with the functional specific to each dataset.
MD17/QM7 is a commonly considered small-molecule benchmark for Hamiltonian prediction. It tests generalization to conformers of single molecules with the same set of atomic elements (Schütt et al., 2019b). The dataset consists of four small molecules (water, uracil, ethanol, malondialde- hyde) evolving along a Molecular Dynamics trajectory, with Hamiltonian matrices in a def2-SVP basis provided for each snapshot. When training on the Hamiltonian matrices of these molecules, HELM is competitive with all previous 'direct' (single evaluation) models achieving state-of the art accuracies in the MAE over matrix elements (Herr). When using the predicted H to recompute the total energy (see Section A.2), we can reproduce the total energy computed from the reference matrices within 30 V per molecule.
DF is currently the largest openly available molecule benchmark with electronic structure in- formation (Khrabrov al., 2024). It contains drug-like molecules with C. N. S, O. CI, Br, and H atom types, up to 62 atoms per structure, computed at the -B977-D/def2-SVV DFT level of the- ory. Multiple low-energy conformations with atomic forces, energies, and Hamiltonian matrices are provided for each molecule. In Table 2, we compare performance on the 2k, 5k, and 10k train/test splits with those of previous models. HELM predicts matrix elements to ~ 60pEE, which is ~3-5x better than those of the previous best-performing model. Further analysis of the connection between matrix element errors, energy eigenvalues, and total energy errors are provided in Section D.1.
4.2 HAMILTONIAN PRETRAINING
We demonstrate the benefit of Hamiltonian pretraining with controlled experiments on a limited volume of energy data (c.g., the energy labels available for V2DFT's 2k, 5k, and 10k splits, and OMol CSH 58k). D2DFT's differently-sized conformations train and test splits allow for an in- vestigation of scaling with data, while OMol CSH 58k presents a significantly more challenging learning task, as it contains highly limited quantity of energy labels for considerably larger struc- tures and increased elemental diversity.
'A recent work adapting the QHNet architecture with flow-mater wwwtmat training process and eigenvalue- based loss function has extended this accuracy to below 5 pbb (Kim et al., 2025). This flow-matching approach can be applied on top of any Hamiltonian prediction method to improve its accuracy, albeit with increased cost. As such, we facus the underlying architectures.
7

---

Published as a conference paper at ICLR 2026
64
Direct
Pretrained-froze
Finetuned
Backbone
Hamiltonian head
Hg
Backbone
Hamiltorian head
Hij
Bacibonn
Energy head
E
Backbone
E
Backbone
Energy head
E
PDFT Train
[c]
Lam
Thakiing
Thiiiin Lom
Thei
10"
Protraieed Srove
Fietored
2500 5997
Mice 10337
2500
5000
7907 1000
10"1
200
400
200
400
200
400
600
Spoch
Epech
Eaeth
Epoch
Epoch
Spech
Figure 3: (a) The three training schemes we use to investigate the effect of Hamiltonian pretrain- ing on the backbone of HELM for downstream energy prediction. (b) and (c) contain the corre- sponding loss over epochs for each of these schemes, when applied to DFFFT's train-2k split and OMol CSH _8k. For the latter, the fill-between indicates distribution of loss over each batch.
We compare three cases: first, we train the model (backbone and output head) on energies this is the "direct" case. Second, we train HELM on Hamiltonian matrices, detach the backbone, freeze its weights, and connect it to trainable energy output head which we subsequently train on scalar energy labels. We refer this as the ppretrained-frozenn model. Third, we start from the same Hamiltonian-pretrane pretrained backbone model, connect to an energy output head, and finetune on energy data. The total number of parameters in the backbone and output head are equal in all three cases, and the number of learnable parameters are equal for the direct and finetuned models. Model sizes and training parameters are specified Table and Table 8. For fair comparison, we used fixed number of epochs and initial learning rate with a cosine scheduler for each dataset. Illustrations of the three training schemes, along with training loss plots, are shown in Fig. 3. Train/test dataset evaluation results are summarized in Table 3 and Table 4. We additionally report matrix element errors (Herr) achieved after the pretraining phase on CMol_CSH_588 for the two test splits (example matrix predictions presented in Section D.2). The increased Herr for OMol all_5k primarily highlights the variation in matrix element magnitudes resulting from the presence of heavy elements.
Split
Molecules
gorr (meV)
Direct
Pretrained-frozen
Finetuned
2k
Train
8,097
97.16 4.12
217.58 +:2 5.17
76.30 +: 1.56
Test
2,774
791.02 40.68
324.64 27.41
266.23 + 24.34
5k
Train 18,908
240.631 6.20
264.91 + 4.79
116.52 1.35
Test
592.54
15.69
303.92
10k
Train
33,150
285.39 + 4.23
234.91 # 2.77
176.75 1.37
Test
12.06
Table Energy performance metrics (MAE) for the direct, pretrained-frozen, and finetuned HELM models on the different conformers splits of D'DFT, along with 95% confidence intervals.
HELM composition
Metric
Training method
OMol,
OMol_common.jt
OMol
#err (n.Ex)
961 651.64
2122.19
== 1 1
g (meV)
Direct
353.15 = 15.17
3631.68 + 227.36
5976.40 = 209.33
(meV)
Pretrained-frozen
361.98 6.10
2119.76
178.64
3255.69 :: 154.70
Eerr
(meV)
Finetuned
254.44 + 9.68
2533.22 + 172.14
3926. 12 155.21
Table 4: Performance metrics for Energy for HELM trained on OMol_CSH.S Energy errors are reported with 95% confidence intervals.
In all cases, the "direct" model rapidly over-fits to limited number of energy labels in the training data. For DFT, the "pretrained-froot -frozer"" and "finetuned" models show both reduced overfitting and improved test accuracy, with the relative improvement in test accuracy from Hamiltonian pretraining
8

---

Published as a conference paper at ICLR 2026
being highest for the smallest conformer split (Train/Test-2k). Note that despite using limited quantity of energy data, the energy errors are already lower than those computed directly from the Hamiltonian matrices in Table 2 computing the energy directly from the Hamiltonian is highly sensitive to small perturbations in the matrix elements (see Fig. 10), and finetuning on high-precision energy labels helps bridge much of the remaining accuracy. The numerical improvements from pretraining are lower for the OMol test splits, as the pretrained-froot exhibit some degree of overfitting to the training data (Fig. 3). This likely stems from the increased diversity of atomic elements, and thus reduced sampling of energy labels over heavier elements. In this case, the pretrained- --rozen model shows Finetuning increases the extent of overfitting, and shows slightly lower improvement of ~ 1.5x.
and finetuned models also
1.8x improvement over the direct model.
To visualize the effects of Hamiltonian pretraining. we extract and analyze the backbone embeddings from all three models when processing the first 250 molecules in OMoI_CS__58k, In Fig. 4 we plot the UMAP analysis over the norm of each individual irrep in this set of embeddings (Mclnnes et al., 2018), colored by atomic element. The (I) dramatic increase in the number of distinct clusters within the "pretrained-frozent embeddings as opposed to those of the "direct" model, as well as the (2) clearer distinction of heavier atomic elements which are less well-represented in the datasets (as opposed to Hydrogen o and Carbon o) suggest that more detailed descriptors of atomic en- vironments are being learned from the Hamiltonian matrix data. Finetuning OMol CSH 58k visibly diminishes this fine-grained structure, while finetuning on DRDFT energy data preserves it (an extended analysis is in Section D.4), consistent with their relative trends in test accuracy.
Direct
Pretrainad-frozar
Finatunad
15-
UPPP Cropeett
10
-10
-10
-15
-10
10
LBN Componert
bein 200
Argon (1 141
Bronin (2- To
Maybdwen 2
undian 12 AMd
Ia (2.54)
Parours 27.
Uthm 2--11
2-131
Press ar 12-190
Kyysse 12-36
Techretio
Ti 77:22-50
Hathoum (2-12)
Roree 12 20
Naminknn 331
Gel (2 313
Seontiin 2 30
atackin 2 451
Selurin (2 531
Tangewe (2 24g
Thals
Caben 12-6)
ton 12-20
Commanumm 22-32)
Yantum 12-30/
Nellctiom
Jodine (2-32)
Rharkus (2-12)
Orygen 40
44h (2-
Figure 4: UMAP visualization of the output node embeddings (2) (k) of the first 250 molecules of OMol_CSH BFa__SH_5880 25kk, extracted from the direct, pretrained-frooee, finetuned models, and colored by element. There are 100,233 datapoints in total, from 14,319 atoms x 7 l-components (/moz 6)
While we use low-energy-das regimes to demonstrate improvements from Hamiltonian pretraining and investigate their origins, finetuning the pretrained HELM backbone models larger fraction of the energy data available in OMol25 is the next step towards achieving practically usable energy accuracies comparable with current MLIPs. Beyond that, scaling the pretraining phase larger number of structures may be helpful to include increased sampling of different elements. Doing so is currently constrained by memory requirements, as the number of required edge-labels and basis set size (e.g, required angular momenta) grow. Computational innovations are thus required to tackle the memory and data processing requirements of larger training labels, as well to incorporate eigenvalue- based (Li et al., 2025) and symmetry-basee (Qian et al., 2025) loss functions.
CONCLUSION
Combining the spherical harmonic representations used in equivariant GNNs with the immense vol- umes of information-rice training data available in the Hamiltonian matrix allows for more general- izable features to be learned from fewer atomic structure samples. Here, we introduced HELM, an efficient architecture for Hamiltonian matrix prediction, combined with OMol CSH 58k, curated

---

Published as a conference paper at ICLR 2026
Hamiltonian matrix dataset covering umprecedented range of atomic elements and orbital inter- action regimes, and demonstrated its use in learning orbital interactions and energies from a shared embedding space. HELM is state-of-the-ae aft in Hamiltonian prediction, and using its learned features for energy prediction improves achievable test accuracy in low-data controlled experiments by the same amount as increasing training data by order of magnitude. Overall, we establish recipe towards integrating electronic structure data into MLIP training to learn generalizable descriptors for atomic structure-property prediction. We believe such techniques are necessary continue to advance MLIPs.
REPRODUCIBILITY STATEMENT
Code and the OMol CSH 58k dataset (and corresponding test splits) will be made available after publication.
REFERENCES
Brandon Anderson, Truong-Son Hy, and Risi Kondor. Cormorant: Covariant molecular neural networks, 2019. URL https://arxiv www.aiiii....aaaaaeeeeeooeeeeeee wwwp/br////////ooo 04015. (cited on page
Dylan M Anstine, Roman Zubatyuk, and Olexandr Isayev. Aimnet2: neural network potential to meet your neutral, charged, organic, and elemental-organte needs. Chemical Science, 16(23): 10228-10244, 2025. (cited on page 3)
Evan R. Antoniuk, Shehtab Zaman, Tal Ben-Nun, Peggy Li, James Diffenderfer, Busra Demirci, Obadiah Smolenski. Tim Hsu, Anna M. Hiszpanski, Kenneth Chiu, Bhavya Kailkhura, and Brian Van Essen. Boom: Benchmarking out-of-distribution molecular property predictions of machine http://aa.....o/ab///////////
learning models, 2025. URL https: / /ww.wvawxvvv/aaa///a///eeeeaaavv wtt.ooo//////..////ooeo /505. 01 912.. (cited on page 1)
Albert P. Bartók, Mike C. Payne, Risi Kondor, and Gábor Csányi. Gaussian approximation po- tentials: The accuracy of quantum mechanics, without the electrons. Physical Review Let- ters, 104(13), April 2010. ISSN 1079-7114. doi: 10.1 10.1103pphyrevleett httph//seeeeet..4.13//../. 104.136403. URL http://dee doi. org/10. 1111//1/y//eeeeeeeee 104. 136403. (cited on page 2)
Ilyes Batatia, Dávid Péter Kováes, Gregor N. C. Simm, Christoph Ortner, and Gábor Csányi. Mace: Higher order equivariant message passing neural networks for fast and accurate force fields, 2022.
URL //atps://arxiv....// www.ogg///aoo. 2226 07697. (cited on page 2)
Ilyes Batatia, Philipp Benner, Yuan Chiang, Alin M Elena, Dávid Kovács, Janosh Riebesell, Xavier R Advincula, Mark Asta, Matthew Avaylon, William J Baldwin, et al. A foundation model for atomistic materials chemistry. arXiv preprint arXiv:2401.00090 2023. (cited on page 2)
Simon Batzner, Albert Musaelian, Lixin Sun, Mario Geiger, Jonathan P. Mailoa, Mordechai Korn- bluth, Nicola Molinari, Tess E. Smidt, and Boris Kozinsky. E(3)-equivariant graph neural net- works for data-efficient and accurate interatomic potentials. Nature Communications, 13(1), May 2022. ISSN 2041-1723. doi: 10. 1038/44 467-022-29939-5 29939-5. (cited on page 2)
URL http: www. http://dxx.dii.or///..../
Jörg Behler and Michele Parrinello. Generalized neural-network representation of high-dimensional potential-energy surfaces. Physical review letters, 98( (1))::144440,, 2007. (cited on page 2)
Filippo Bigi, Marcel Langer, and Michele Ceriotti. The dark side of the forces: assessing non- conservative force models for atomistic machine learning, 2024. URL https://arxiv.orr abs/2412. 11569, (cited on page 1)
Mads Brandbyge, José-Luis Mozos, Pablo Ordejón, Jeremy Taylor, and Kurt Stokbro. Density- functional method for nonequilibrium electron transport. Physical Review 65(16), March 2002. ISSN 1095-3795.
doi: 10. 103/physrevb.65. 165401. URL http://dx. doi..doi.oo. org/10.
1103/PhysRev8. 65. 165401. (cited on page 3)
Chi Chen and Shyue Ping Ong. A universal graph deep learning interatomic potential for the periodic table. Nature Computational Science, 2(11):718-72) 2022. (cited on page 2)
10

---

Published as a conference paper at ICLR 2026
Bingqing Cheng. Latent ewald summation for machine learning of long-range interactions. upj Computational Materials, 11(1):80, 2025. (cited on page 3)
Beatriz G. del Rio, Brandon Phan, and Rampi Ramprasad. A deep learning framework to emulate
density functional theory. npj Computational Materials, 9(1,, August 2023. ISSN 2057-3960. doi: 10. 1,11038844152--023001011 15-3.
URL http://dk: doi. org/
0212244-222-111555 01115-3.
(cited page 2)
Bowen Deng, Peichen Zhong, KyuJung Jun, Janosh Riebesell, Kevin Han, Christopher J Bartel, and Gerbrand Ceder. Chgnet as pretrained universal neural network potential for charge-informe informed atomistic modelling. Nature Machine Intelligence, 5(9):1031 111000111110000 1144,, 2023. (cited on page 2)
Ralf Drautz. Atomic cluster expansion for accurate and transferable interatomic potentials. Physical doi: 10. (00///hhsteeb.9900411444 URL
Review B, 99(1), January 2019. ISSN 2469-9969,
http://dx. doi. org/oo.///. 1103/PhysRevB. 99. 0141 04, (cited on page 2)
Stefano Falletta, Andrea Cepellotti, Anders Johansson, Chuin Wei Tan, Marc Descoteaux, Albert Musaelian, Cameron J Owen, and Boris Kozinsky. Unified differentiable learning of electric response. Nature communications, 16(1))4031, 2025. (cited on page 3)
M. V. Fischetti and S. E. Laux. Band structure, deformation potentials, and carrier mobility in strained ge, and sige alloys. Journal of Applied Physics, 80(4):2234-2552, August 1996. ISSN 1089-7550. doi: 10. 10.1063/1.363052. URL http://dx www.:ooi..ooo.o.. 1063/.. 363855.
(cited on page 3)
Adam Foster, Zeno Schätzle, Bernát Szabó, Lixue Cheng, Jonas Köhler, Gino Cassella, Nicholas Gao, Jiawei Li, Frank Noé, and Jan Hermann. An ab initio foundation model of wavefunctions that accurately describes chemical bond breaking, 2025. URL htttp::/aaaxiiorr.ee wttplv..orrabbb///55// abs//bbs/20000 19960. (cited page 2)
Xiang Fu, Zhenghao Wu, Wujie Wang. Tian Xie, Sinan Keten, Rafael Tommi Jaakkola. Forces are not enough: Benchmark and critical evaluation for machine learning force fields with molecular simulations. 2022. doi: 1044550/ARXXIV2220.77233 URL https: //arxiv.
and
www.iv..rg.bb///.eee 221 07237. (cited on page 1)
Xiang Fu, Brandon M. Wood, Luis Barroso-Luque, Daniel S. Levine, Meng Gao, Misko Dzamba, and C. Lawrence Zitnick. Learning smooth and expressive interatomic poten- tials for physical property prediction, 2025. URL
(cited on pages 1, 3, and 4)
Xiaoxun Gong, He Li, Nianlong Zou, Runzhang Xu, Wenhui Duan, and Yong Xu. General frame- work for e())-qquivariant neural network representation of density functional theory hamiltonian. Nature Communications, 14(1), May 2023. ISSN 2041-1723. doi: 10. 4178//4447--03-884466
URL http://dx..t.o////// doi. wrg.r/////..0 000 1038/ 00788//4447-023--11
8. (cited on page
Anubhab Haldar, Ali K. Hamze, Nikhil Sivadas, and Yongwoo Shin. Gears h: Accurate machine- learned hamiltonians for gext-generation device-scale modeling, 2025. URL https://arxis: org/abs/ 10298. (cited pages and 6)
Beom Seok Kang, Vignesh C. Bhethanabotla, Amin Tavakoli, Maurice D. Hanisch, William A. God- dard, and Anima Anandkumar, Orbitall: A unified quantum mechanical representation deep learn- ing framework for all molecular systems, 2025. URL https: httpss/////riiooorraaa//////// 038533. (cited on page 2)
Kuzma Khrabrov, Anton Ber, Artem Tsypin, Konstantin Ushenin, Egor Rumiantsev, Alexander Telepov, Dmitry Protasov, Ilya Shenbin, Anton Alekseev, Mikhail Shirokikh, Sergey Nikolenko, Elena Tutubalina, and Artur Kadurin. dft: A universal quantum chemistry dataset of drug-like molecules and benchmark for neural network potentials, 2024. URL https://arxxt org/
abs/2406. 14347. (cited on pages 2, 5, 7, and 20)
Seongsu Kim, Nayoung Kim, Dongwoo Kim, and Sungsoo Ahn. High-order equivariant flow match- ing for density functional theory hamiltonian prediction, 2025, URL https: //arxi/....ii/////a org/abs/ 2505. 18817. (cited page 7)
11

---

Published as a conference paper at ICLR 2026
W. Kohn and L. Sham. Self-consistent equations including exchange and correlation effects. Phys- ical Review, (0004A))A113-A11333 November 1965. ISSN 19999. doi: 10. 10.ph00//yys 140.al133, 133. URL http::/ppppptt////// www.rrr//.... 1103/PhysRev. 140 AA1 133. (cited on page 2)
Jindfich Kolorenč, Shuming Hu, and Lubos Mitas. Wave functions for quantum monte carlo calcu- lations in solids: Orbitals from density functional theory with hybrid exchange-corrllation func- tionals. Physical Review B, 82(11), September 2010. ISSN 1550-235X. doi: 10. 103/physrevb.
82.115108. URL http://dx. orgioorg//00 1ttp////1.10///heeeeeeee 82. 115108. (cited on page 3)
Kin Long Kelvin Lee, Mikhail Galkin, and Santiago Miret.
Deconstructing equivariant 08131.
representations in molecular systems, 2024. URL
(cited on page 24)
Daniel S. Levine, Muhammed Shuaibi, Evan Walter Clark Spotte-Smith, Michael G. Taylor, Muhammad R. Hasyim, Kyle Michel, Ilyes Batatia, Gábor Csányi, Misko Dzamba, Peter Eastman, Nathan C. Frey, Xiang Fu, Vahe Gharakhanyan, Aditi S. Krishnapriyan, Joshua A. Rackers, Sanjeev Raja, Ammar Rizvi, Andrew Rosen, Zachary Ulissi, Santiago Vargas, C. Lawrence Zitnick, Samuel M. Blau, and Brandon M. Wood. The open molecules 2025 (OMol25) dataset, evaluations, and models, 2025. URL pattps://arxiit http://ww.rgg/b////aoee 08762. (cited pages 1, 2, and 5)
Yunyang Li, Zaishuo Xia, Lin Huang, Xinran Wei, Han Yang, Sam Harshe, Zun Wang. Chang Liu, Jia Zhang, Bin Shao, and Mark B. Gerstein. Enhancing the scalability and applicability of kohn- sham hamiltonians for molecular systems, 2025. URL https://arxiv. .wwriiioorr..ioo org/ abs/ 19227.
(cited on pages 2, 3, 6, and! 9)
Yuanbin Liu, Ata Madanchi, Andy S. Anker, Lena Simine, and Volker L. Deringer. The amorphous state as a frontier in computational materials design. Nature Reviews Materials, 10(3):228 241,, December 2024. ISSN 2058-8437, 8437. doi: 10. 10.1088//41578 0188//41578-024-007554 URL http: //tp:///xddooooooo doi.
org/10. 1038/ s41 10011178-000 024-004--000 00754-2. (cited on page
Erpai Luo, Xinran Wei, Lin Huang, Yunyang Li, Han Yang. Zaishuo Xia, Zun Wang, Chang Liu, Bin Shao, and Jia Zhang. Efficient and scalable density functional theory hamilto- nian prediction through adaptive sparsity, 2025. URL
01171.
(cited pages 3, 6, and 7)
Leland Mclnnes, John Healy, and James Melville. Umap: Uniform manifold approximation and projection for dimension reduction, 2018. URL https://aardv..org/abb//// 03426.
(cited page 9)
Peter R. Miedaner, Kin Long Kelvin Lee, Shiang Fang, Tess Smidt, and Keith Nelson. DIRECT PREDICTION OF TENSORIAL PROPERTIES WITH EQUIVARIANT MESSAGE--PASSING: APPLICATIONS TO NONLINEAR OPTICS. In AI for Accelerated Materials Design ICLR 2025, 2025. URL
net/forumm:d=44EoTq0o (cited on page 24)
Santiago Miret, Kin Long Kelvin Lee, Carmelo Gonzales, Sajid Mannan, and N. M. Anoop Krish- nan. Energy & force regression on dft trajectories is not enough for universal machine learning interatomic potentials, 2025. URL https://arxixs org/ aaa/ab///// 2abs/250..am 03660. (cited on page 1)
Yutack Park, Jaesun Kim, Seungwoo Hwang, and Seungwu Han. Scalable parallel algorithm for graph neural network interatomic potentials molecular dynamics simulations. Journal of chem- ical theory and computation, 20(11):457-44888 2024. (cited on page 2)
Saro Passaro and C. Lawrence Zitnick. Reducing s0) convolutions to so(2) for efficient equivariant gnns, 2023. URL
http://aaiiv..orr/////////////// 03655. (cited on pages and 4)
Chen Qian, Valdas Vitartas, James Kermode, and Reinhard J. Maurer. Equivariant electronic hamil- tonian prediction with many-body message passing, 2025. URL https://arxiv. org.:/iiii.ooreaeeee. /bs/ 2508.15108. 15108. (cited on pages 6 and 9)
Benjamin Rhodes, Sander Vandenhaute, Vaidotas Šimkus, James Gin, Jonathan Godwin, Tim Duignan, and Mark Neumann. Orb-v3: atomistic simulation at scale. arXiv preprint arXiv:2504.06331 2025. (cited on page
12

---

Published as a conference paper at ICLR 2026
K. T. Schütt, M. Gastegger, A. Tkatchenko, K.-R. Müller, and R. J. Maurer. Unifying machine learning and quantum chemistry with a deep neural network for molecular wavefunctions. Nature Communicationss 10(1), November 2019a. ISSN 2041-1723. doi: 10. URL htttp://dx... doi. org/10. 0038//4146777 019 12875-2 2. (cited on page 5)
12875-2..
K. T. Schütt, M. Gastegger, A. Tkatchenko, K.-R. Müller, and R. J. Maurer. Unifying machine learning and quantum chemistry with a deep neural network for molecular wavefunctions. Nature Communicationss 10(1), November 20196 9b. ISSN 2041-1723. doi: 10. 1038/s4 447-019-112-111112777222
URL http://dx: doi. org/10..g1.... 1038/ $4144444444 467-4177-9991 12875-2. (cited on page 7)
Xuecheng Shao, Lukas Pactow, Mark E. Tuckerman, and Michele Pavanello. Machine learning electronic structure methods based on the one-electron reduced density matrix. Nature Commu- nications, 14(1), October 2023. ISSN 2041-1723. doi: 10.1038/s41460
URL
http://dx. doi wrgooog//// 0. 0038/0400400 /1////44445523333 -21---33--444-- (cited on page 2)
Killian Sheriff, Daniel Xiao, Yifan Cao, Lewis R. Owen, and Rodrigo Freitas. Machine learning potentials for modeling alloys across compositions, 2025. URL https::/arxiv..or//aaaaa a000//// 12592. (cited on page 1)
Divya Suman, Jigyasa Nigam, Sandra Saade, Paolo Pegolo, Hanna Türk, Xing Zhang, Garnet Kin-Lic Chan, and Michele Ceriotti. Exploring the design space of machine learning mod- els for quantum chemistry with a fully differentiable framework. Journal of Chemical The- ory and Computation, June 2025. ISSN 1549-9626. doi: 10. 1a.102/acs.cctc.5c0000
URL
http://dks doi. www.o.gg//.. 1021/accs jctc. wtt. 5c00522. (cited on page 2)
Ethan M. Sunshine, Muhammed Shuaibi, Zachary W. Ulissi, and John R. Kitchin. Chemical prop- erties from graph neural network-predicted electron densities. The Journal of Physical Chemistry doi: 10.1021 1a..02/acs..pcc.336611
C, 127(48) 23466, November 2023. ISSN 1932-7455.
URL http: /tx. doi. org/10. aeo0/aac...aa jpc. 3c06157. (cited on page 2)
Hao Tang, Brian Xiao, Wenhao He, Pero Subasic, Avetik R Harutyunyan, Yao Wang. Fang Liu, Haowei Xu, and Ju Li. Approaching coupled-cluster accuracy for molecular electronic structures with multi-task learning. Nature Computational Science, 5(2):144-1555 2025. (cited on page 2)
Oliver Thorsten Unke, Mihail Bogojeski, Michael Gastegger, Mario Geiger, Tess Smidt, and Klaus Robert Muller. -quia-equivariant prediction of molecular wavefunctions and electronic densities. In A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan (eds.), Advances in Neural Information Processing Systems, 2021. URL https: wwwpr/eerreiiee.eeee/oorereeee net/formmmid auGY2UQfhSu. (cited on pages and 7)
Yuanqing Wang, Kenichiro Takaba, Michael S. Chen, Marcus Wieder, Yuzhi Xu, Tong Zhu, John Z. H. Zhang, Arnav Nagle, Kuang Yu, Xinyan Wang, Daniel J. Cole, Joshua A. Rackers, Kyunghyun Cho, Joe G. Greener, Peter Eastman, Stefano Martiniani, and Mark E. Tuckerman. On the design space between molecular mechanies and machine learning force fields, 2024. URL https: / aavi.iivvvoioo abgg 01931. (cited on page 6)
Maurice Weiler, Mario Geiger, Max Welling, Wouter Boomsma, and Taco Cohen. 3d steerable cnns: Learning rotationally equivariant features in volumetric data, 2018. URL https://arxix: abs/1807.022 02547. (cited on page 4)
Brandon M. Wood, Misko Dzamba, Xiang Fu, Meng Gao, Muhammed Shuaibi, Luis Barroso- Luque, Kareem Abdelmaqsoud, Vahe Gharakhanyan, John R. Kitchin, Daniel S. Levine, Kyle Michel, Anuroop Sriram, Taco Cohen, Abhishek Das, Ammar Rizvi, Sushree Jagriti Sahoo, Zachary W. Ulissi, and C. Lawrence Zitnick. UMA: A family of universal models for atoms, 2025. URL httpps//arrdee wttp///wwww...//////5...... 23971. (cited pages 1 and 2)
Chen Hao Xia, Manasa Kaniselvan, Alexandros Nikolaos Ziogas, Marko Mladenovié, Rayen Mahjoub, Alexander Maeder, and Mathicu Luisier. Learning the electronic hamiltonian of large atomic structures, 2025. URL httph::tttt/ttttee org. abs/2501 19110. (cited on pages and 3)
Han Yang, Chenxi Hu, Yichi Zhou, Xixian Liu. Yu Shi, Jielan Li, Guanzhi Li, Zekun Chen, Shuizhou Chen, Claudio Zeni, et al. Mattersim: A deep learning atomistic model across ele- ments, temperatures and pressures. arXiv preprint arXiv:2405.0490 2024. (cited page 2)
13

---

Published as a conference paper at ICLR 2026
Haiyang Yu, Meng Liu, Youzhi Luo, Alex Strasser, Xiaofeng Qian, Xiaoning Qian, and Shuiwang Ji. Qh9: quantum hamiltonian prediction benchmark for qm9 molecules, 2023a. URL https: //arxiv. org.org///bb. /tt.//bb.........//// 09549. (cited page 5)
Haiyang Yu, Zhao Xu, Xiaofeng Qian, Xiaoning Qian, and Shuiwang Ji. Efficient and equivariant graph networks for predicting quantum hamiltonian, 2023b. URL https://arxiv.org org/abs/ 04922. (cited on pages 3, 6, 7, 17, 19, and 20)
Haiyang Yu, Yuchao Lin, Xuan Zhang, Xiaofeng Qian, and Shuiwang Ji. Efficient prediction of so(3) -o(3)-equvvariant hamiltonian matrices via so(2) local frames, 2025. URL wtttht:///aaaiii org/ abs/2506, 09398. (cited on page 3)
Yaoguang Zhai, Alessandro Caruso, Sigbjørn Løland Bore, Zhishang Luo, and Francesco Paesani. A "short blanket" dilemma for state-of-the-ard neural network potential for water: Reproducing experimental properties or the physics of the underlying many-body interactions? The Journal of Chemical Physics, 158(8), February 2023. ISSN 1089-7690. doi: 10.1063/5.014288 URL http:://dx doi org/oogg//// /00 1063/5 0142843, (cited on page !)
14

---

Published as a conference paper at ICLR 2026
APPENDIX
A Background
15
A.1
The electronic structure problem in DFT
15
A.2
Extracting atomic-level observables from the Hamiltonian matrix
15
B Hamiltonian Data Processing
16
B.1
Processing the Hamiltonian matrix into learning targets
16
B.2 Decay of orbital interactions with inter-atomic distance
17
B.3 Normalizing scalar node block irreps
17
Model & Training
19
C.I Model parameters
19
C.2 Training parameters
................................
19
C.3 Ablations on node-pane node-parity expansion and = 0 scaling
21
Additional results
21
D.I Hamiltonian matrix error analysis DPDFT
21
D.2 OMoL_CSH_588 Hamiltonian predictions
23
D.3 Energy prediction on different conformers splits DFF
................................
23
D.4 Extended UMAP embedding analysis
24
Use of LLMs
25
A BACKGROUND
A. THE ELECTRONIC STRUCTURE PROBLEM IN DFT
In Kohn-Sham density functional theory (DFT), the electronic structure problem recast as a set of single-particle equations for non-interacting electrons in effective potential. When expressed in finite atomic orbital (AO) basis {Xa}, the problem. The molecular orbitals 2h are expanded as
equations become a generalized eigenvalue
(r) Cuixx(r).
(4)
where Cpi are the molecular orbital coefficients. The Kohn-Sham equations in matrix form are:
Fnr = SprC
(5)
where Fav is the Hamiltonian matrix, S = (xulxx) is the overlap matrix, Cui are the molecular orbital coefficients, and & are the orbital energies. Solving Eq. (5) yields the molecular orbital co- efficients C' and the orbital energies e, which are used to construct the electron density and compute the total energy.
A.2 EXTRACTING ATOMIC-LEVEI. OBSERVABLES FROM THE HAMILTONIAN MATRIX
Given a Hamiltonian matrix F of system of atoms, it is relatively straightforward extract the total energy without further computation of quantum mechanical two electron (2e)-integrals or SCF procedure. The remaining cost of evaluating the energy is thus relatively computationally inexpensive. Any number of QM packages supply the required additional terms, such as the open-source PySCF package.
First, the density matrix is obtained from the molecular orbitals which themselves obtained by solving the generalized eigenvalue problem FC SCe where F is the Hamiltonian matrix and S is the atomic orbital overlap matrix (the latter coming from PySCF) and selecting the first
15

---

Published as a conference paper at ICLR 2026
Netectron/ / (assuming restricted Kohn Sham formalism) eigenvectors. These the occupied
molecular orbitals Coce- Then the density matrix is:
P = 2Cocc C occ
The DFT energy is:
1
E Tr + F)] Exc - Tr[PVxc] ENN 2
where Heore is the matrix of one electron (Ie) integrals, F' is the Hamiltonian matrix, Exc is the axchange-correlation energy, Vxc the exchange-correhe correlatoo potential, and ENN is the nuclear- nuclear repulsion.
atoms
ENN Σ LiZg/roj
where Zi is the atomic number of atom and raj the distance between atoms and j. Heores Exc- and Vxc can be obtained from QM codes (e.g., through PySCF). The first consists of small number of Gaussian integrals. The latter two are obtained by evaluating the exchange-correlann of this density P over an XC grid to obtain Exc and Vxe (note that the exact exchange portion of this range-separated hybrid already part of and does not need be separately computed here).
functional
B HAMILTONIAN DATA PROCESSING
B. PROCESSING THE HAMILTONIAN MATRIX INTO LEARNING TARGETS
Each sub-matrix block of the Hamiltonian, which represents interactions between atomic orbitals of degrees l and (2, can be decomposed and transformed into direct sum of irreducible representa- tions (irreps) of degree 11 1zl to + 122
4
4 484- =
L
1=111-4
where indexes the irreps in the direct sum. This decomposition is performed using the Clebsch- Gordan coefficients, which couple the uncoupled basis of spherical harmonics Y,m, and Yiamy into the coupled basis ILM):
LM) Σ
Jama [mmm))lmme))
mi.72
In practice, the transformation from the uncoupled to the coupled basis can be expressed using Wigner 3j-symbols as:
L.M Ohmanama = (-1)/ 12 /2L +1
4
l2
mi
m2
-M
where CLM famp are the Clebsch-Gordan coefficients in tensor of dimensions (24, + 1) x (2l2 + 1) x (2L + 1), as implemented in c3mn.o3. w3..winer..3. 3j.
The Hamiltonian block (h (2) in the uncoupled basis, with matrix elements H (g) is thus trans- formed into the coupled basis as:
L H
Σ
CLM ( (amyJamm mi (1mm Jams)
(CL
MAP
m........ m//
where (L) is the block corresponding to the irrep L.
16

---

Published as a conference paper at ICLR 2026
0
10
-5
-10
20
-15
bode bode Todee bocle bett a: 1130
(18xC0 (odec 3xderix2o
1230 Sedex13o
Figure 5: Dimensions the (padded) target created for an example water molecule in a DZVP basis (taken from the MD17 dataset), and its decomposition into direct sum of irreducible representa- tions. The colored circles indicate the Ls required to represent the interactions in each submatrix.
We have irreps for every orbital interaction in H. Next, need to create learning targets of consistent size for every node and edge of the GNN, which involves populating the orbitally-resolved direct sum of irreps into targets of fixed size. To do this, adopt the layout used previously by Yu et al. (2023b) and first identify a minimal 'basis' of irreps large enough to accommodate every set of orbital interactions. Figure 5 illustrates this process for water molecule in DZVV basis. To account for all orbital interactions within the 9 sub-matrices of the Hamiltonian (3 node blocks, edge blocks), we need a basis of [0, representations which takes the shape [14 Oe + 21 x le + 15 x 2e + 5 x 3e + 1 x 4e]. The orbital interaction blocks present within every node and edge can then be populated into this skeleton and processed into the coupled space with fixed basis transformation.
1, 2], which corresponds to direct sum irreducible
Note that accommodate all atomic interactions in the OMol CSH 58k dataset, each set of orbital interactions (corresponding to the nodes and edges of the atomic graph) is processed into learning target with dimension 114 x + 229 1e + 239 x + 161 3e + 73 4e + 24 5e + 6c. This translates to 4,096 matrix elements per node/edge.
B.2 DECAY OF ORBITAL INTERACTIONS WITH
DISTANCE
In Fig. 6 we plot the elements in the Hamiltonian matrices of two representative structures from the OMoLCSH_550 CSH_58k dataset (one containing long- cong-distance interactions, and another containing heavy elements), as a function of inter-atonn atter-atomic distance. The matrix elements are available to six decimal places, and truncation error may limit the accuracy of the last decimal place. Therefore, when processing the raw matrices to build the dataset, we trim inter-atomic interactions to a cutoff of 12A, beyond which all Hi are below X 10 Note that this also sets a limit on the precision to which the total energy be recomputed from this data.
B.3 NORMALIZING SCALAR NODE BLOCK IRREPS
The Hamiltonian matrices of structures containing heavy elements often have large-magnitudee trace components corresponding to the energies of the lowest-lying core electronic states. These sub- matrix traces contribute to the scalar components (irreps with angular momentum (l 0)) in the targets corresponding to every node, which can be arbitrarily scaled and shifted without breaking the rotational equivariance of the data. Figure 7 shows the set of average scalar values found in the node blocks of every atomic element for MMoL.CSH_50 dataset. Note that the use of effective core potentials (ECPs) for the heavy elements causes a shift in the energies of the lowest / = 0 irreps for Rb-Xe, [Cs, Ba, La], and Hf-Rn (each these have different set of ECPs). For many of the heavy elements, the values range across three orders of magnitude. The corresponding scalar node values for DFT are shown on the same plot, covering much smaller range of atomic elements and few heavy atoms (with the exception of Br).
averaged over the structures in the
17

---

Published as a conference paper at ICLR 2026
102
10a
Nater cluster
Niobius complex
101
10
O-H
100
10
H,O-Ho0
10
10-1
Hg (E,)
Hy (En)
10
10-2
10-3
10-3
10-4
10
10-5
10-5
10-6
10-6
10
15
5
10 (A)
15
(A)
Figure 6: Magnitude of the matrix elements of Hyy plotted as function of interatomic distance for two representative structures from OMo._CSH_58k. The scatter points at 0 Å come from the diagonal blocks His
Let h (n) E Rdd be the vector of scalar components (corresponding to I irreps) for node n in structure where d is the number of scalar irreps per node. For each atomic number Z, we collect all scalar component vectors h (n(") for nodes corresponding element Z across the dataset. We then compute the mean and standard deviation vectors:
1
h(m
(m)
1
Σ
uz
NZ im:Z,==2
oz
N% inn.n,=Z
pz
where Nz is the total number of scalar vectors for element in the dataset. Any zero standard deviations, corresponding interactions which not sampled within specific element's node interactions, are set to I:
OZ max(oz,j e), = 10-4
Before training. the scalar components each node (*)) are normalized by subtracting the mean and dividing by the standard deviation corresponding to the node's element Z:
h (n)
h
PZJ
j=1...., d
where (1) is the scaled scalar vector used as the target for the equivariant graph neural network. To reconstruct the predicted Hamiltonian matrix during inference, we recover the original scale of the scalar components from the network's 's output (n) with the inverse transformation:
R
R OZ, + pz.j
In Section C.3, we include ablation studies to demonstrate the impact of this scaling procedure on the performance of HELM the DFFFT Hamiltonian matrix prediction benchmark.
18

---

Published as a conference paper at ICLR 2026
orteeeed (1- 0) (Bluc-Red)
600
(Ew)
200
-400
Ae (1p0o1 60p 10 ( "ROAA Seggeeeeeeeeeeeeeeeeeeee Sepp
Atamic Element
Figure 7: Scatter plot of average = O values for every atomic element in the OMoLCSH.558. and DFT datasets, which are used in the scalar element referencing scheme. Each point corresponds to different = irrep within the target for a specific atomic element, averaged over the dataset. For OMoL_CSH_5E the points colored by their order of appearance in the target.
C MODEL & TRAINING
C.I MODEL PARAMETERS
Table 5 shows the model hyperparameters used when training each of the datasets. Note that the maa parameter determined by the maximum degree of the basis required to represent all of the atomic elements in the corresponding dataset (def2-DZVP or def2-TZVPD). Scalar energy referenc- ing is omitted for MD17 due to the lower variation in element magnitudes, and the overall simplicity of the learning task. The edge cutoff distance Teut determined either by the maximum interaction distance or (in the case of OMol CSH 58k) by the inter-atomic distance at which matrix elements are greater than
10-5 5 En. Table shows the number of learnable parameters available in each of HELM's components. Note that the Hamiltonian output head is designed to be relatively learnable features within
lightweight, in order to maintain most of the wwtpllooniinmmaiii---sea the backbone embeddings.
DFTFT
OMoI.CSH.58K
4
Irrep reference subtraction
None
I U
1= 0
Number Message Passing layers
Number of embedding channels C
128
128
128
Edge cutoff distance News (A)
16.0
20.0
12.0
Gaussian cutoff (A)
16.0
20.0
12.0
Table 5: Model parameters used for MD17, DFDFT, and OMol.CSH.58. datasets.
Model Component
Parameters (!mas 4)
Parameters (Ima 6)
Backbone Node Edge updates U%
33.958,528
84,549,245
Backbone Node updates (f(,.))
17,073,280
42,305,280
Hamillonian Output Head (ff))
133,329
279.093
Energy Output Head (f2)
5,430,017
13,757,185
Table 6: Number of learnable parameters in each model component, using the model hyperparame- ters specified in Table for V2DFT.
C.2 TRAINING PARAMETERS
Table contains the details behind the number train/validation/test structures used for the MD17/QM7 and DF Hamiltonian matrix benchmarks in Table 2. For MD17/QM7, we fol- low the train/validation/tsst splits QHNet Yu et al. (2023b). We limit the total training time to 24 hours GPUs (192 total node hours). The total compute cost for training thus ranges from 24 GPU hours (MD17 Water benchmark) to 192 GPU hours (MD17 Uracil, Ethanol, Malondialde- hyde benchmarks). Loss epoch plots for all four molecules shown in Figure 8 note that not
19

---

Published as a conference paper at ICLR 2026
all of the training converges by this point. Due to the similarity of orbital interactions between the training and test data, there is little to no overfitting observed during the training process.
For DFT, we use the train/test splits provided in Khrabrov et al. (2024), although subtract the last 64 structures in each training split to use for validation, and use message- saaesagg--asiiggggg layers rather than 5. Similarly to the MD17 training, the DFF benchmarks also do not converge we stop training after 1,536 GPU hours.
To train on OMol CSH 58k, do not fixed batch size, as the large distribution of molecule sizes would introduce significant load imbalance during training (see Fig. 2). Instead, we batch the data by 'binning' molecules roughly 22k graph edges per bin, which typically translates to between 1 and ~83 molecules. Doing this allows for better load balancing during distributed training. since the time to process molecule determined almost entirely by the number of edges to which its graph representation is connected (which determines the number of messages processed within each layer of the model). When training distributed on 64 processes, we then have roughly 1, 1,408k graph edges per batch.
DF
OMol.CSH.50K
Train
Test
Uracil
Ethanol
Malonddaldehydee
2k
# Train
500
25k
25k
25k
12,145
28,
362
49.725
56k
# Val.
500
500
500
500
64
64
64
64
Test
3800
1478
4.5k
4,5k
2,774
5,634
9.532
5k/1k
Batch size
64
~1 4.08k edges
Scheduler
Reducel, HonPlateuu
ReducalFRonPattou
CosineAmeelingL
Initial
10-4
10
10-3
Opömizer
Adam
Adam
AdamWW single
Precision
double
double
Table 7: Hamiltonian matrix training parameters for MD17, 'FFFT, and OMol CSH 58k datasets. MD17 is shown per molecule, and D'DFT shows parameters for multiple splits. MD17 splits are taken from Yu et al. (2023b).
10-1
10-1
10-1
10-2
Ns training Validation
No Training
Mi training
Pt training Validation
Va Validation H/ Training
Validation M, Training
10-2
Ho Training
10-2
10-2
10-2
Hy Training
H, Validation
Va Validation
Validation
Validation
10-3
Loos
10-3
10-3
10
10-1-
10-1
10-1
10
Water
10-3
Uracil
10-3
Ethanol
10-1
Malondialdehyde
1000
2000
0
100
50
100
150
100
200
300
Epoch
Epoch
Epoch
Epoch
Figure 8: Loss vs. Epochs for the four molecules in the MD17 database.
Dala
PFT
OMoL.CSH.58K
Train
Test
5k
10k
2k
5k
10k
B Train
8,097
18,908
33,150
56k
Validation
640
640
640
Test
2.774
5,634
9.532
Batch size
640
640
640
1, 1,408k edges 700 CosineAmeellnggLR
Number Epochs
10,000
5,000
3,000
Scheduler
CosineAmealingLR
Initial learning rate
1 10
x 10
Minimum learning rate
e AdamW single
Oplimizer
Adam double
Precision
Table 8: Energy training parameters for DFF and MMoL.CSH.50 datasets.
20

---

Published as a conference paper at ICLR 2026
Table 8 contains the training parameters used for the energy training results presented in Table and Table For DFTT we train on 2/3 of the molecules in the dataset, and use the remaining 1/3 for validation. We use the same root- coot-MSE+MSE loss as for Hamiltonian prediction for consistency.
C.3 ABLATIONS ON PARIIY EXPANSION AND 0 SCALING
Here we preform ablation studies to understand the impact of two specific features of HELM: (1) the node parity expansion blocks described in Section 3 ("Hamiltonian output head"), and (2) the / = component scaling described in Section B.3. In Table 9 report the results of these ablations using the DFDFT 2k train/test splits. All of the training here was done with a fixed number of epochs (500) and cosine scheduler. We use fixed number of epochs specifically to compare between these different settings this is generally not long enough to converge the training (which was done using& ReduceLRonPlateer scheduler for the results Table 2), also report the error on the train split.
Scaling Node-Parlty Expansion
H" train
train
1"" test
E test
201.21
169,026.00 08
324.35
329,923.53
95.73
1,163,54
101.73
1,530.83
115.35
837.07
120.18
973.47
92.00
Table 9: Effect of node-parity reduction and 1 O referencing scheme on matrix element emror evaluated using the VDDFT train/test splits. Herr is in units of x10-6F, E,,, while Ferr is in units of meV. The best case performance in each column is in bold.
The node parity expansion block only impacts the error within diagonal blocks of the Hamiltonian. These diagonal blocks contain the largest-magnitude elements (resulting from nearsightedness of orbital interactions), and contribute disproportionately towards the eigenvalue error and total energy prediction. Therefore, improving the loss over diagonal orbital blocks has significant impact the total energy computed from the matrices, bringing the error down from 329,92 02923 meV to 1 1,530 meV. The 1 = component scaling procedure serves to reduce the numerical range of the data from the order of 100 to the order of ~ (see Fig. 7 in Section B.3), thus making it easier to learn with -oorali-dorm model. This brings the energy error down further 706 meV. Together, these two architectural choices contribute significantly towards the improved performance of HELM over other models in Table 2.
D ADDITIONAL RESULTS
D.1 HAMILTONIAN MATRIX ERROR ANALYSIS DF
In Fig. 9 we show the correlation between matrix element errors and eigenvalue errors, collected over all three of the considered test splits in DFT. In general, a weak correlation exists between matrix element errors and resulting eigenvalue errors. We also note that larger molecules tend to have lower matrix element and total energy errors, likely stemming from increased sparsity in H.
(b)
10-1
10-1
10
Awua (63)
20
10
10
10
18.00
10a
10"0
30-4
10-a
10'4
1H (E.)
1Z H87 (EU)
-X (5))
Figure 9: Correlation between matrix element emrors (separated into (a) edge block errors, (b) node block eπrrr and (c) total matrix elements) and resulting MAE errors over the occupied energy cigenvalues (AMAr) of the predicted and reference matrices, shown for all the data over the 2k, 5k, and 10k test splits of DFF..
21

---

Published as a conference paper at ICLR 2026
In Fig. 10 we additionally examine the relationship between matrix element errors and the error in the total energy computed from the predicted H. Most errors are on the order of x 1002 (272. 14 meV), however several outliers can be seen with total energy emrors of > 1. When inspect- ing these cases manually, we found that they occur from cases where hearly-degeneran eigenvalues (to within the prediction error) cause inconsistent electron occupation patterns between the refer- ence and predicted matrices, leading to different density matrices computed from them, and thus to different total energies. To address this, we filtered out linearly dependent atomic orbitals by first removing eigenvalues of the computed overlap (S) matrix to within 1 10-3, followed by manually filtering out inconsistent occupations. We note that the removal of linear dependencies is performed internally within many DFT codes, and typically makes negligible difference on the value of the computed total energy.
Belore Filtering
After Filtering
10
50'
10
30
2
10-1
12-1
10-11
10
10-1
1511
Figure 10: Correlation between matrix element errors, occupied energy eigenvalue errors (AMAE) and total energy computed from the predicted and reference matrices, shown for all the data over the 2k, 5k, and 10k test splits of DFT. The plots are shown before/after filtering the eigenvalues of S to under 1 x 10-3 The black dashed boxes indicate outliers stemming from different occupation, which are reduced after the filtering process.
To understand how these matrix errors are distributed as function of inter-atomic distance, in Fig. 11 we plot the predicted matrix elements and labels for three representative test conformers in the 10k set. The data is plotted in the coupled basis before the awvess-transformatiie reconstruct the Hamiltonian in order to additionally differentiate between irreps of different degree I. An overall matrix element error of ~60 nE, (Table 2) results in the decay in magnitude of orbital interactions with inter-atomic distance being captured up 15 Å. The highest remaining relative errors are found in the longer-range orbital interactions, where the magnitude of the matrix elements drops below 100 J. These long-range interactions are not found in the small molecules of MD17, where the largest interatomic distances are typically less than A, which partially contributes to the large difference in matrix element errors between datasets. We also note that the contributions of different degree / to the matrix elements are learned simultaneously, so the model does not prioritize lower or higher angular degrees.
required to
C17H15CI2225 (58 atomsi
C16H20N203 (41 atoms)
C14H1 C14H1 MAHH1NN50455.0 (4) atoms)
10.
31
10*
10-1
10-1
10
10"
10-1
10"1
10">
10.
10
03
25
5.0
7.5
10.0
12.5
15.0
10.5
20.6
03
25
50
75
160
12.5 15.0 175 20.0
60
25
58
7.5
100
12.5
15.0
17.5
26.0
Intordtones (1500nue
Inte wlanea Miltoncc (4)
Figure 11: Decomposition of the matrix elements of Href and Hpred by interatomic distance and angular degree 1, shown for three molecules in DFT. Points at an inter-atomic distance of 0.0 Á correspond to elements within the diagonal Hamiltonian submatrices/node outputs. The inset shows the connectivity of graph representation of each molecule with an Feut of 20 A.
22

---

Published as a conference paper at ICLR 2026
D.2 OMOL.CSH.58k HAMILTONIAN PREDICTIONS
In Fig. 12, we show computed Hamiltonian matrices with HELM for the first few structures in the OMolall Mioo_allS5 5k test split. We also show heatmaps of reference Hamiltonian matrices, as well as the energy eigenvalues computed between them. For numerical stability, the eigenvalues of the overlap matrix used to solve the generalized eigenvalue problem with both reference and predicted Hamiltonian matrices were filtered to 1 10 Fl In general, reproduce the matrix elements ~2x10 FA (see Table 4), which in practice yields visually indistinguishable matrices and eigen- values for the H in this test split. Deviations from the reference energy eigenvalues are typically concentrated around band edges.
To further analyze the utility of the predicted matrices, we them re-compute the electron density with PySCF (see Fig. 13), Visually, the electron density isosurfaces computed for all three datasets are nearly indistinguishable from those computed directly from the reference Hamiltonian matrices.
0.6
22
H
Hpnd
H8 Hpre
0.0
500
1000
Egenvalue
-000
Iprd
Figenuelue
k
03
-200
-600
0.1
50C
50c
Elgenvolue
6
Homa
0.3
-000
011
Eigenwelue
Figure 12: Predicted Hamiltonian matrices from the first four atomic structures in the OMol all 5k test split, showing heatmaps of the label and predicted H (bounded to color range of [-0.1, +0.1] to better distinguish patterns inter-orbital interactions). the element-wise error between them, as well as a comparison of the energy eigenvalues computed from them.
D.3 ENERGY PREDICTION ON DIFFERENT CONFORMERS SPLITS DFFFT
In Table 3, we evaluated energy errors on matching 2k, 5k, and 10k conformer train/test splits in DDDFT, assessing generalization to different conformers of molecules present in the training set. Hamiltonian pretraining significantly reduced energy prediction errors. In Table 10, we extend this analysis to test generalization to unseen molecules by evaluating each model, trained on 2k, 5k, or 10k splits, across all test splits. Hamiltonian pretraining consistently outperforms direct training on unseen molecules, even when finetuned model trained the 2k split is tested the additional 8k molecules the 10k split (8.62 me V/atom) compared to an energy- model trained on the 10k split and tested on 2k (11.)..
23

---

Published as a conference paper at ICLR 2026
(b]
1d
Computed from H.
Computed from HELM outpor Ht
Figure 13: Isosurfaces of clectron density visualized for a random molecule taken from the (a) QM7 (isovalues: 0.1, 0.2, 0.5 /ag), (b) DFFFT Test-2k (isovalues: 0.05, 0.2 e /a/a) and (c) OMol_CSH_58k (isovalues: 0.03 e-
ee/ee-/a// /a)) test splits. each case, the upper row shows the electron density plotted with the label Hamiltonian input, and the bottom row shows the one predicted from HELM's output.
Test-2k (2,774)
Test-Sk (5,634)
Test-10k (9,532)
Train
Direct
Train 2k (8,097)
2.68
21.7
20.3
23.4
Train-Sk 1889088
6.08
11.9
15.6
14.2
Train 10k (33,150)
7.00
11.0
10.2
13.0
Pretrained-frozen
Train-2k (8.097)
5.66
9.00
10.1
11.0
Train-5k (18,908)
7.61
8.02
8.18
Train- 10k (33,150)
5.75
6.84
6.42
6.93
Finetuned
Train-20 2k (8,097)
3.74
7.32
7.62
8.62
Train-5k (18.908)
3.71
4.29
5.22
4.87
Train- 10k 150)
4.20
4.80
4.49
5.05
Table 10: MAE for the direct, pretrained-frozen and finetuned HELM models trained/evaluated the different conformers splits of DFF. In brackets are the number of molecules used for training/testing.. All units are m//tom.m.
D.4 EXTENDED UMAP EMBEDDING ANALYSIS
In Fig. 14 and Fig. 15, show extended version of Fig. 4 for both the DDFFT and OMoLCSH_558 datasets. We extract embeddings from 100 molecules of the train-2k split for the former, and 250 molecules for the latter. Clusters from the "direct" model indicate that the ability to distinguish hydrogen and carbons atoms has been learned, consistent with the ubiquity of these ele- ments in the training data. After Hamiltonian pretraining, a greater variety features for different atomic elements can be distinguished. The remaining similarities occur between pairs such C-H and O-F, which may have more structurally similar atomic environments within the training data.
In the second row of each figure, we show the UMAP visualization. now colored by angular degree I. The embeddings after Hamiltonian pretraining on OMol CSH 58k exhibit greater separation the space of irreps of different degree l. Our results are thus consistent with the observations of Lee et al. (2024) and Miedaner et al. (2025), where training on rank-2 targets (molecular typerpolarizabiiities)) indicated better utilization of higher-degree irreps than training scalar (rank-0) targets (energies). Extending this idea, the Hamiltonian matrix can be thought of as containing the highest-rank tensors required to encode the properties of system of atoms in given atomic orbital basis. Training to learn these matrix elements results the model maximally leveraging different components of the spherical harmonic embeddings extract structural descriptors from the training data.
24

---

Published as a conference paper at ICLR 2026
Nyrrogen P-11
Carosn (2-4)
Mivgge (2-3)
Deyget 2..
Flevine (2.40
Okarie Oh (2.3
Inceb 2-33
Direct
Finatuned
Pretrained fredin
-10
-15
-28
-10
Direct
Pretrained-frqear
Finetuned
-10
Figure 14: UMAP node embeddings for the first 100 molecules of the DFTT train-2k 2k split, colored by (top) atomic element and (bottom) angular degree I. There are 19,040 data points extracted in total, from 3,808 atoms I-components (lmax 4).
Zalk
Eltaro ne 2hatt
Camaun Dodds
1tars 2--00
argaiam 20-4
Hage moreeaant LC-40
(0-41)
Siow (hvld
an 20-90
gi-da
Direct
Pratrained frezainnod-frgaen
Finetuned
-10
UMbe Comporet
(9977 Congorent
Protrained
UM4
Figure 15: UMAP node embeddings for OMol TMol.CSH.55 58k. The lower of visualizations show the same data as Fig. 4, but now colored by angular degree I.
E USE OF LLMS
We made use LLMs to refine the LaTeX and math formatting within this document. No new ideas text were generated by LLMs.
25


Reviews:
[{"review_id": "1m2pqLc0uf", "reviewer": "Reviewer_atd4", "summary": "This paper presents HELM, a novel approach for predicting molecular potential energies with the help of large amounts of electronic Hamiltonian matrix data, and a curated Hamiltonian matrix dataset OMol_CSH_58k. HELM adopts a novel backbone model architecture, first pre-trains model to predict Hamiltonian matrix then trains model to predict energy. Experiments show HELM achieves better accuracy than baselines in Hamiltonian prediction and good energy prediction accuracy.", "questions": "No additional questions.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "- Propose a novel approach for molecule energy prediction. The idea of first pre-training on Hamiltonian prediction task then training on energy prediction task is quite useful and generalizable to multiple different models.\n- Experiment results demonstrate good performance of the proposed HELM approach.\n- The writing of this paper is clear and well-organized.", "weaknesses": "- Need clarification on results comparison in Table 2: are HELM and all baselines in Table 2 pre-trained on the OMol_CSH_58k Hamiltonians to ensure fair comparison? If so, authors are encouraged to highlight what is the advantages of the architecture design in HELM over baselines that leads to better Hamiltonian prediction accuracy.\n- In Table 2, there is no baseline comparison in energy prediction so there is no idea how good the performance of HELM is. Authors are suggested to add baseline performance. If baseline methods are using the predicted Hamiltonian matrices to compute energy, compare with the energy prediction accuracy of HELM under the same setting.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "QcTLRbhezm", "reviewer": "Reviewer_7bnb", "summary": "This paper introduces HELM, a scalable equivariant graph neural network that learns from Hamiltonian matrices and proposes “Hamiltonian pretraining” as a route to enhance molecular property prediction. The idea is well-motivated, technically sound, and experimentally verified on large and diverse datasets. The proposed OMol CSH 58k dataset is likely to have significant value for the community.", "questions": "The idea of pretraining on Hamiltonian matrices is certainly reasonable and physically motivated. However, this approach requires maintaining detailed edge-level information in the molecular graph, which could substantially increase memory usage. In particular, traditional MLIPs often rely on relatively low-order spherical harmonics for edge features, whereas predicting Hamiltonian elements likely demands much higher angular degrees to capture orbital interactions. This would significantly raise both computational and memory costs.\nCould the authors comment on this trade-off? Specifically, how does the computational and memory footprint of HELM compare to that of conventional MLIPs trained only on energies and forces? Moreover, under the same computational budget, does HELM still outperform existing energy-prediction models?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Strong and well-motivated idea — leveraging electronic-structure information that is usually discarded in MLIP training is both physically meaningful and practically impactful.\n2. Technical soundness — the HELM architecture is carefully designed with rotational equivariance and SO(2) convolutions for scalability to large basis sets and diverse elements.\n3. Dataset contribution — the new OMol CSH 58k dataset fills a gap between small-molecule Hamiltonian data and modern large-scale MLIP datasets.", "weaknesses": "1. Conceptual novelty is moderate: the idea of predicting or using Hamiltonian matrices has been explored in literature; this paper mainly provides a unified and scalable realization.\n2. Performance improvements are modest: while pretraining improves energy prediction in low-data regimes, the gain is typically 1.5–2×, and the benefit diminishes for larger datasets.\n3. No demonstration on forces or MD trajectories: the method is not yet shown to improve dynamical stability or energy–force consistency, which are critical for MLIP applications.\n4. Dataset construction: The details about the settings of how the data points were computed are not clear to me. The authors claimed that the dataset expands over many elements in the periodic table. However, Gaussian basis sets calculations are unknown to fail for many heavy elements. So, how did you handle these issues, to be specific, how to choose basis or effective core potentials for those molecules?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1vCc1Ay7cG", "reviewer": "Reviewer_RqiD", "summary": "This paper introduces HELM (Hamiltonian Electronic Learning Model), an SO(3)-equivariant graph neural network that jointly learns to predict Kohn-Sham Hamiltonians and molecular energies. The key innovation is leveraging the rich electronic structure information encoded in Hamiltonian matrices—routinely computed but typically discarded during DFT calculations—as a pretraining task to improve machine learning interatomic potentials (MLIPs). The authors construct OMol-CSH-58k, a new dataset of ~56.7k closed-shell molecules covering 58 elements with up to 150 atoms, and demonstrate that Hamiltonian pretraining yields substantial improvements in energy prediction accuracy, particularly in low-data regimes.", "questions": "1. Can a model pretrained on def2-SVP (∇²DFT) transfer to def2-TZVPD (OMol) with light head retraining, or vice versa?\n2. Can you also add a force head to your model to see pretraining on H helps with force prediction?\n3. What modifications would HELM require for open-shell/charged systems?\n4. Please report detailed computational performance metrics, including:  (i) Wall-clock time per epoch for ∇²DFT-2k on V100 and A100 GPUs  (ii) Peak GPU memory usage as a function of $l_{\\\\text{max}}$ (iii) Inference time per molecule across different molecular size ranges", "limitations": "", "rating": 6, "confidence": 5, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "*  The paper makes a good case for exploiting Hamiltonian matrices as supervisory signals. The connection between electronic structure and atomistic properties is conceptually novel, and the two-head design bridges DFT and MLIP domains.\n*  OMol-CSH-58k addresses a  gap in existing Hamiltonian datasets by covering 58 elements (vs. 8-10 in prior work), molecules up to 150 atoms, and using the larger def2-TZVPD basis. The dataset composition analysis (Fig. 2, Table 1) is thorough.\n*  HELM achieves ~60 µEh error on ∇²DFT compared to 520-20700 µEh for prior methods (Table 2). The margin is meaningful  The analysis connecting matrix errors to eigenvalue and energy errors (App. D.1-D.2) strengthens the claims.\n*  The pretraining protocol shows energy MAE improvements, especially on ∇²DFT with limited training data (up to ~2× reduction, Table 3). The frozen vs. finetuned head comparisons are informative, and UMAP visualizations (Fig. 4) support the claim of learning better representations.\n*  Distance-decay analysis justifying the 12Å cutoff (App. B.2, Fig. 6), element-referenced scalar normalization (App. B.3), and batching strategies for handling size heterogeneity (Table 7) demonstrate careful implementation.", "weaknesses": "* There is a concerning mismatch between the exchange–correlation (XC) functionals used for generating the training data and those used for reconstructing energies from predicted Hamiltonians. Appendix A.2 reports ωB97M-V for energy reconstruction, whereas the ∇²DFT data rely on ωB97X-D/def2-SVP (Table 2). Such inconsistency may induce systematic biases and plausibly account for outliers observed in Fig. 10. The paper should clearly specify: (i) the XC functional used for energy reconstruction in each dataset, (ii) whether this matches the functional that produced the reference data, and (iii) the quantitative impact of any mismatch.\n\n* The model integrates several design elements—SO(2) convolutions, gated nonlinearities per irrep, parity-aware constraints, and element-referenced normalization—but their individual contributions are not analyzed. It remains unclear whether the observed gains stem primarily from architectural innovations or auxiliary components such as normalization or loss design. A systematic ablation table is necessary to disentangle these effects and to provide guidance for future model development.\n* Table 4 and Fig. 3c show cases where fine-tuning on OMol leads to degraded performance relative to the frozen pretrained head, suggesting overfitting. While the authors attribute this to high elemental diversity, the explanation remains speculative. A more rigorous analysis should explore regularization strategies (e.g., layer freezing, dropout, weight-decay sweeps, early stopping) and report per-element error breakdowns to identify whether rare elements disproportionately drive the issue.\n* Energy errors are inconsistently reported—sometimes per-molecule (Table 3) and sometimes per-atom (Table 9)—without clear indication. The notation alternates between H and F (Fock) across sections. Units and symbols should be standardized throughout (e.g., meV / atom) and summarized in a concise notation table for clarity.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "55j5rTwlcr", "reviewer": "Reviewer_QD9Z", "summary": "This paper introduces HELM, a model that predicts Hamiltonian matrices for molecules, enabling the integration of orbital interaction into interatomic potential training. Accompanying HELM, the authors release OMolCS-H58k—a large-scale Hamiltonian dataset featuring 58 elements and molecules up to 150 atoms. They also propose “Hamiltonian pretraining,” a method that leverages Hamiltonian-derived embeddings to enhance energy prediction accuracy.", "questions": "* In the Introduction, the paper states: “For every system of N atoms, there is 1 energy label, O($N$) force labels, and O($N^2$) labels in the Hamiltonian matrix.” However, the number of labels in the Hamiltonian matrix should be the total number of orbitals, not the number of atoms.\n* In \"Learning orbital interactions\" of Background and Previous Work, both the initial models and following work are cited as [1]. Is this expected?\n* In Fig. 2(d), the “Element–element interaction” is shown. How are these numerical values computed?\n* In Fig. 3(c), are the results shown the same as those reported in Table 4? In Fig. 3(c), the pretrained-frozen model appears to achieve better validation results, whereas Table 4 shows that the finetuned model performs better. These seem inconsistent. I guess that this be due to the loss functions being different (MSE in the figure vs. MAE in the table)? In the section Loss and scalar referencing, the paper describes the loss function used for Hamiltonian prediction, but since the model is also trained to predict energy, I could not find a clear explanation of the loss used for energy prediction. Could the authors clarify this?\n* The work applies a cutoff to the interactions. Appendix B.2 mentions: “Note that this also sets a limit on the precision to which the total energy can be recomputed from this data.” Could a quantitative error analysis be provided here? How much error does this cutoff introduce in downstream tasks?\n* In Appendix D.1, Fig. 9(c) shows a trend that appears inconsistent with Fig. 6 in [2] and Fig. 1 in [3], both of which display a divergent trend. Could the authors explain the source of this discrepancy?\n\n[1] Gong X, Li H, Zou N, et al. General framework for E (3)-equivariant neural network representation of density functional theory Hamiltonian[J]. Nature Communications, 2023, 14(1): 2848.\n \n[2] Wang Z, Liu C, Zou N, et al. Infusing self-consistency into density functional theory hamiltonian prediction via deep equilibrium models[J]. Advances in Neural Information Processing Systems, 2024, 37: 89652-89681.\n\n[3] Li Y, Xia Z, Huang L, et al. Enhancing the scalability and applicability of kohn-sham hamiltonians for molecular systems[J]. arXiv preprint arXiv:2502.19227, 2025.", "limitations": "", "rating": 4, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "* The paper is well-structured and clearly written, providing a concise and accessible overview of the problem and the proposed solution.\n* The authors conduct a thorough analysis, enhancing understanding through techniques such as UMAP visualization.\n* Moreover, they introduce a new dataset of Hamiltonians, which is a valuable contribution to the open-source community in this field.", "weaknesses": "I do not find any obvious weaknesses in this paper. I believe that many researchers working in the areas of Hamiltonian prediction and machine learning force fields may have had similar ideas, so the concept itself is reasonable, and the experimental results appear solid. However, I do have some questions about the paper—please see Questions. If the authors address my concerns, I will consider increasing my score.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)