Advancing Universal Deep Learning for Electronic-Structure Hamiltonian Prediction of Materials
Authors: Shi Yin
Zujian Dai
Xinyang Pan
Lixin He
Venue: ICLR 2026 Poster (2026)
Keywords: AI for Materials
Deep Learning
Electronic-Structure Hamiltonian Prediction
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)
TL;DR: An advanced deep learning method and a comprehensive benchmark for universal Hamiltonian prediction across materials

Abstract: Deep learning methods for electronic-structure Hamiltonian prediction have offered significant computational efficiency advantages over traditional density functional theory (DFT), yet the diversity of atomic types, structural patterns, and the high-dimensional complexity of Hamiltonians pose substantial challenges to the generalization performance. In this work, we contribute on both the  methodology and dataset sides to advance  universal  deep learning paradigm for Hamiltonian prediction.  On the method side, we propose NextHAM, a neural E(3)-symmetry and expressive correction method for efficient and  generalizable materials electronic-structure Hamiltonian prediction. First, we introduce the zeroth-step Hamiltonians, which can be efficiently constructed by the initial charge density of DFT, as informative input descriptors that enable the model to effectively capture prior knowledge of electronic structures. Second, we present a neural Transformer architecture with strict E(3)-symmetry and high non-linear expressiveness for Hamiltonian prediction. Third, we propose a novel training objective to ensure the accuracy performance of Hamiltonians in both real space and reciprocal space, preventing error amplification and the occurrence of ``ghost states'' caused by the large condition number of the overlap matrix. On the dataset side, we curate a broad-coverage large benchmark, namely Materials-HAM-SOC,  comprising $17,000$ material structures spanning more than $60$ elements from six rows of the periodic table and explicitly incorporating spin–orbit coupling (SOC) effects, providing  high-quality data resources for training and evaluation. Comprehensive experimental results demonstrate that NextHAM achieves excellent accuracy in predicting Hamiltonians and band structures, with spin-off-diagonal blocks  reaching the accuracy of sub-$\mu$eV scale. These results establish NextHAM as a universal and highly accurate deep learning model for electronic-structure prediction, delivering DFT-level precision with dramatically improved computational efficiency.

Published as a conference paper at ICLR 2026
ADVANCING UNIVERSAL DEEP LEARNING FOR FLECTRONIC-STRUCTURE HAMILTONIAN PREDIC-
TION OF MATERIALS
Shi Yin!* Zujian Dai'; Xinyang Pan"; Lixin He2..3
Institute of Artificial Intelligence. Hefei Comprehensive National Science Center
2 Laboratory of Quantum Information, University of Science and Technology of China
3 Hefei National Laboratory, University of Science and Technology of China
ABSTRACT
Deep learning methods for electronic-structure Hamiltonian prediction have of- fered significant computational efficiency advantages over traditional density functional theory (DFT), yet the diversity atomic types, structural patterns, and the high-dimensional complexity of Hamiltonians pose substantial challenges to the generalization performance. In this work, we contribute on both the method- ology and dataset sides to advance universal deep learning paradigm for Hamil- tonian prediction. On the method side, we propose AM neural E(3)- symmetry and expressive correction method for efficient and generalizable mate- rials electronic-structue Hamiltonian prediction. First, we introduce the zeroth- step Hamiltonians, which can be efficiently constructed by the initial charge den- sity of DFT, as informative input descriptors that enable the model to effectively capture prior knowledge of electronic structures. Second, Transformer architecture with strict E(3)-symmetry and high non- non-linear expres- siveness for Hamiltonian prediction. Third, propose novel training objective to ensure the accuracy performance of Hamiltonians both real space and recip- rocal space, preventing error amplification and the occurrence of "ghost states" caused by the large condition number of the overlap matrix. On the dataset side, we curate a broad-coverage large benchmark, namely Materials-HAM-SSO comprising 17, 000 material structures spanning more than 60 elements from six rows of the periodic table and explicitly incorporating spin-orbi coupling (SOC) effects, providing high-quality data resources for training and evaluation. Com- prehensive experimental results demonstrate that NextHAM achieves excellent accuracy in predicting Hamiltonians and band structures, with blocks reaching the accuracy of sub-ueV scale. These results establish NextHAM universal and highly accurate deep learning model for electronie-structure prediction, delivering DFT-level precision with dramatically improved computa- tional efficiency.
present neural
INTRODUCTION
Understanding the electronic structure is fundamental to unraveling how electrons govern the prop- erties of condensed matter systems. This knowledge essential for predicting a wide range of material characteristics, such as electrical conductivity, magnetism, optical behavior, and chemi- cal activity, which are vital for technologies spanning from electronics to sustainable energy and advanced catalysis. At the heart of these calculations is the challenge of determining the system's Hamiltonian matrix, whose eigenvalues and eigenstates yield important quantities like energy levels, band structures, and electronic wavefunctions. Traditionally, Density Functional Theory (DFT) (Ho- henberg & Kohn 1964 Kohn & Sham 1965) has been the standard approach for these problems.
"quual contributions.
Corresponning authors. Emails: shiyingial .ustc. ustc..du.crr he1x@ustc.edu.co

---

Published as a conference paper at ICLR 2026
Elcttronic Struture Calculation Methods
Materbik Structure
(AA's o (00) A
Op
of 2al
Materal Properties
TC- (065%) wills
Lighhnnign complesit
stream Physical Quantitees
Elestrone
(00% loge
Band Structure
Magnetic
doportttrr luck pf cull prezs undimeyalll
desapact
thermaal
spantiy mealing it ellicut for networks
be Gurnerday hare sathadd
Berry
Optical
Vhey
consatationll complesis (n
Cremizal leactiviy
Transport
(olATs (ot4) lace
Eqpesieecngd input
PM
powdle- and
leyning
performonce M
Adwrcoa
124
Figure 1: Comparison of paradigms for electronic-structure Hamiltonian calculation, highlighting the fundamental differences between our method and both classical DFT methods and existing deep learning approaches.
However, as shown in Fig. (a), DFT relies heavily on the self-consistent (SC) procedure, which de- mands repeated (denoted as T turns), computationally intensive diagonalizations of large matrices, each scaling as ((N3) with system size N, making simulations of large or complex materials ex- tremely resource-consuminng Recently, deep learning has emerged as powerful tool in the physical sciences (Zhang et al. (2025). As shown in Fig. (b), modern deep neural network methods (Gong cct al. 2023 Yu et al. 2023b; Zhang et al. 2024 Wang et al. 2024c Li et al. 2025 Yin et al. 2025) can predict Hamiltonians, L.e., the core physical quantities in electronic structure calculations, di- rectly from atomic configurations in an efficient way, circumventing the computationally expensive SC loop and dramatically accelerating computations. This paradigm shift lowers the computational barriers associated with electronic structure calculations, unlocking the simulation and design of unprecedentedly large-scale materials systems, driving new innovation in materials discovery and engineering. Please refer to Appendix A for background introduction.
However, deep learning methods still face substantial challenges achieving accurate and general- izable Hamiltonian prediction, primarily due to the extremely complex and fundamentally difficult nature of the input-output mapping that the neural network must learn, making difficullt to general- ize across diverse material systems. Consequently, it has become common practice to constrain the scope, such as limiting the range of supported elements, neglecting spin-orbit coupling (SOC) ef- fects, reducing the number of orbitals considered, as thoroughly discussed in Appendix C While such strategies help alleviate modeling burdens, they also restrict the applicability of these meth- ods to the full diversity and complexity of real-world materials. What's 's more, large open-source materials datasets for the training and evaluation of general Hamiltonian learning models are also rare.
To solve these challenges, in this work, we make contributions on both methodology and benchmark toward advancing universal deep learning for cncetronic-structur Hamiltonian prediction of mate- rials. On the method side, as shown in Fig. (c), we propose NextHAM, a neural E(3)-symmetry and expressive correction framework for efficient and accurate Hamiltonian prediction:
2

---

Published as a conference paper at ICLR 2026
First, we dive deeply into the traditional DFT computational process outlined in Appendix A and introduce a physical quantity that helps mitigate the complexity of the input- mapping en- countered by deep learning models for Hamiltonian prediction. This quantity is the zeroth-step Hamiltonian ("), which is efficiently constructed from the initial electron density (()) (r). given by the sum of the charge densities of isolated atoms, without the requirement of matrix diagonalization. As H()) efficiently encodes essential information about the system's electronic structure, we inno- vatively incorporate it as one of the input features to the neural network. Unlike existing methods that rely on randomly initialized atom and edge embeddings, which lack physical prior knowledge and may suffer from issues of sparsity as analyzed in Appendix M H provides richer physical context by embedding the intrinsic characteristics of diverse elements into unified representation space, thereby enabling robust generalization across chemically complex material systems. More- over, inspired by the delta-learning paradigm (Bowman et al. 2022), we predict the correction term AH = HT)) - ()) rather than directly predicting the entire HV, reducing both the dimensional- ity and numerical range of the regression target, and enabling the model to focus on capturing only the essential differences rather than reconstructing the entire Hamiltonian from scratch.
Second, we present a network architecture that strictly adheres to E(3)-symmetry while maintaining high non-linear expressiveness for Hamiltonian prediction by extending the TraceGrad (Yin et al.) 2025) method to Transformer framework, thereby providing ample capacity for flexible and accurate modeling of atomic systems for Hamiltonian prediction across wide range of elements in the periodic table. Furthermore, we introduce model ensemble techniques to enhance the capacity of the framework for handling complex scenarios in Hamiltonian prediction.
Third, we propose a joint optimization framework that simultaneously refines both real-space (R- space) and reciprocal-space (k-space) Hamiltonians. Most existing methods regress only the real- space Hamiltonian, but the large condition number of the overlap matrix can amplify errors in pre- dicted eigenvalues and eigenfunctions, leading suboptimal physical fidelity. Although recent work (Li et al. (2025) has explored strategies to mitigate this error amplification, their attempts are tailored to finite molecular systems, and also overlook the inherent gauge freedom in Hamiltonian represen- tations (Wang et al., 2024c). In contrast, our method explicitly targets the decoupling of energy subspaces k-space for infinite periodic systems to eliminate spurious "ghost states" and strictly preserve the band topology. Furthermore, we resolve the gauge ambiguity by analytically determin- ing the optimal gauge parameter within our joint optimization framework, thereby stabilizing the optimization landscape and ensuring unique, physically consistent predictions.
On the dataset side, we curate a diverse-collection large benchmark dataset, Materials-HAM-SOO containing 17, 000 material structures generated using DFT softwares. The dataset spans more than 60 elements from the first six rows of the periodic table and explicitly incorporates opintorbit cou- pling (SOC) effects. To ensure the accuracy of the DFT calculations, we employ high-quality pseu- dopotentials that include as many valence electrons as possible, enabling our model to handle phys- ically complex and highly challenging systems. We adopt high-quality atomic orbital basis sets, up to 4s2p2dif orbitals for each element, to ensure line-grained description of electronic structures. This dataset establishes a challenging yet comprehensive benchmark for evaluating generalization across chemically and structurally diverse systems.
Extensive experiments on the Materials-HAMa prediction error of 1.417 VeV across full Hamiltonian matrices in R-space, with spin-off-diagom (SOC) blocks suppressed to the sub-pev scale. Moreover, the band structures derived from k-space Hamiltonian exhibit excellent agreement with first-principles DFT. Furthermore, our method offers a substantial computational advantage over traditional DFT. These results establish a new paradigm for electronic-structus calculations, combining high accuracy, broad generalization capability, and significant computational efficiency. Besides general tasks, our method also achieves state-of-the- art performance in more specialized scenarios from the databases of DeepH Series Li et al. (2022); Gong et al.] (2023). These breakthroughs open new avenues for practical applications, including rapid screening of candidate materials, modeling of nano-structures, and simulation of large-scale quantum devices.
dataset demonstrate that NextHAM achieves a
3

---

Published as a conference paper at ICLR 2026
Reciprocal Space
(High Energy- Sub-Space
Sub-Space
Energy- Sub Sub-Space
Real Space
Joint
Supervised Training
T= TAH (AH)')
Neural Networks
Wigner-Eckart Converter
E'uiiiquivariant Decoder
O(3)-Invriiant Decoder
(edget
E(3)-Symmetry TraceGrad
Symm-tyy Feed-Forward Network
x
Symmerry Layer Normalization
Sy3mmeyyyr Multi-Head Graph Attention
Symmetry Layer Normalization
(odee)
pledge)
Ensemble of models
(lr 1-4, Z0r
Y
Descriptors
H
Terothbstep Homilttttens
Diplarement Vectors
Divide-and-conguer models for partitioned distance ranges
Input Material Structure a crost-sectional view
Figure 2: Illustration of the proposed NextHAM framework.
2 METHOD
As shown in Fig. 2 to effectively handle wider chemical and structural variability of materials, develop a unified Hamiltonian prediction framework along three aspects:
2.1 INPUT DESCRIPTORS
As shown at the lower part of Fig. we use the displacement vector-based descriptors between atoms that lie within the cutoff distance, together with the zeroth-step Hamiltonian (calculated as detailed in Appendix A), as the input features for the neural network. Introducing the zeroth-step Hamiltonian the input features for the neural network is one of the core innovations of our frame- work. The zeroth-step Hamiltonian H derived from the initial charge density (0) (r), obtained
4

---

Published as a conference paper at ICLR 2026
as the sum of neutral atomic charges, reflects the information of different elements in the system, including the strength of the electron-ion interactions (pseudopotential) and a preliminary estimate interactions. These components directly influence the system's 's electronic of the clectron-electrr structure. By embedding (9)) the inputs, our method encodes the characteristics of different ele- ments into a unified representation space, bringing a powerful capability to generalize across diverse material systems.
The zeroth-step Hamiltonian ())) can be decomposed into on-site sub-matrices, which represent the Hamiltonian blocks corresponding to each atom and its own orbitals, and off-site sub-matrices, which capture the interactions between different atoms. These two types of sub-matrices can natu- rally serve as the initial descriptors for nodes and edges, respectively, in the graph neural network, as detailed in Appendix E
What's 's more, as detailed in Appendix A computing the zeroth-step Hamiltonian requires no matrix diagonalization, its cost scales with the number of non-zero matrix elements: it is approximately O(N)) for small systems with N atoms and asymptotically approaches ( for sufficiently large systems as the neighbor count saturates for each atom. This matches the scaling behavior of the message passing mechanism of graph neural networks, ensuring that incorporating the zeroth-step Hamiltonian new input descriptor does not worsen the O-asymptotics.
2.2 NEURAL NETWORK ARCHITECTURE
Accurate Hamiltonian prediction requires the neural network to strictly adhere to the symmetries of the E(3) group. While translation symmetry can be easily implemented using relative coordinates, maintaining -()-equuiveeiance while also achieving significant expressive power presents chal- lenging and fundamental problem. To provide the necessary background on the directly relevant basic concepts, we refer the reader to Appendix B
We present a Transformer architecture that not only maintains striet B(3)-symmetry but also achieves strong non-linear expressiveness. as shown in Fig, 2 Our E(3)-ymmettr graph attention mechanism
is developed from Equiformer (Liao & Smidt (023). J. While Equiformer was designed for regression tasks where the target quantity is essentially node-level atomic property (e.g.. force fields), our Hamiltonian target is fundamentally an edge-level property defined on atomic pairs. This distinction necessitates stronger modeling of edge features and motivates the development of our attention mechanism. First, we explicitly maintain and update edge features across multiple layers, rather than generating them only temporarily from node features on demand (Liao & Smidi (023). this way, the computation of attention weights incorporates both the node features and the persistently maintained edge features. Second, motivated by the decay behavior of Hamiltonian matrix elements with respect interatomic distance, we explicitly incorporate interatomic distances by introducing distance embeddings as additional signals in the computation of attention weights, enabling the model to better exploit distance information for inference. Third, the attention weights between nodes are directly applied to update edge features via multiplicative operations, and are subsequently refined through equivariant transformations. Together, these developments substantially enhance the capacity of the model to represent edge features, from which the Hamiltonian is regressed.
As analyzed in Section C the TraceGrad mechanism (Yin al., 2025) can maintain strong non- linear expressiveness while preserving strict E(3) E(3)-symmetry We extend TraceGrad into the Trans- former framework for electronic-structure Hamiltonian prediction. As shown in the middle of Fig- 2. for an atomic pair (a, b), the updated -qqiivvrant edge feature ('ledge) is further fed into the ((elge) which subsequently passed to the -ivaainvai decoder and trained under the supervision of the -(3)-invariant trace
TraceGrad module to produce the non-linear O(3)-invrriant feature zab
quantity T = tr(AH- AH'). The learned non-linear expressiveness in znb is subsequently
(edge)
delivered into the equivariant feature by (edge) f (edge)
ar ar where fedge) represents the non-
(wbp)
linearity-enhanced -etiivauuvar edge feature, which, together with the node feature, are fed into the subsequent encoding modules of the Transformer followed by the 0(3) -qqivvaraat decoder and a Wckner-Eckart converter (Gong et al. 2023) to regress the correction term AH = HT)) O)
A lthough the original paper of TraceGradd emphasizes its 50(3)--quivariancc is straightforward to prove that it also preserves F(3)-symmetry, including translation-invariancee and -()eeqquivaeiance in this context.
5

---

Published as a conference paper at ICLR 2026
This residual formulation defines the learning task as a delta-learinng problem, reducing both the dimensionality and numerical range of the regression target.
To enhance model capacity and better capture the complex dependence of Hamiltonian matrix el- ements on diverse inter-atomic distances, we employ an ensemble learning strategy. Specifically, sub-models are trained to predict Hamiltonian sub-matrices corresponding different distance in- tervals between atoms. Although each sub-model specializes in a specific range in the output stage, the input to each sub-model is the entire system, including the zeroth-step Hamiltonian and the dis- placement vectors for all atomic pairs, thereby effectively extracting global information. The final prediction is obtained by aggregating the outputs from all these sub-models.
2.3 TRAINING LOss FUNCTIONS
The objective of training the neural network is to make the predicted Hamiltonian, denoted as H ( + AH, approximate the ground truth AHot as closely as possible. As illustrated in accurately derive down-stream physical quan-
Fig. to ensure that the predicted Hamiltonian tities (such as band structures), we design a joint optimization strategy in both real space (R-space) and reciprocal space (k-space) for the neural network. Crucially, as detailed in Appendix G our entire loss formulation is designed to rigorously resolve the gauge ambiguity Wang et al. (2024c), ensuring the uniqueness and physical consistency of the regression targets.
In R-space, the Hamiltonian and the corresponding trace quantity are jointly supervised. As out- lined in Section 2.2 the trace quantity is used to supervise the non-linear O(3)-invariant features, which contribute to constructing the non-lineat O(3) -())-equivariant features required for predicting the Hamiltonian. The R-space training loss function is defined as:
(uss(()) = ER. AR ((1 Ac) loss (R) + 7(ossH, loser, Ac) losst (R))
(Ross((R))
(1)
where ER[]] denotes the empirical expectation, Ac An are tice vector connecting the reference unit cell and a neighboring unit cell; (oss and losst (R)
R. denotes the lat-
denote the prediction losses of the Hamiltonian and the trace quantity in R-space, respectively; and (loss Josst, Ac) is scaling factor designed to balance their relative contributions for stable training. The detailed forms of these terms are provided in Appendix G
As analyzed in Appendix F due the error amplification mechanism associated with the ill- conditioned overlap matrix, even small numerical errors in R-space can be magnified in k-space, leading deviations in downstream physical quantities. To mitigate this, we introduce k-space loss functions. Specifically, the spectrum is partitioned into a low-energy subspace P. which gov- erns most physical properties, and high-energy complement Q. While downstream phenomena are predominantly determined by P, an inaccurately predicted Hamiltonian may introduce spurious couplings between P and Q. This can result in unphysical abrupt changes in band structures, which are referred to as "ghost states" (see Fig. [0 in Appendix L) Therefore, it is essential not only to emphasize accuracy in P but also to maintain reasonable fidelity Q so that the erroneous PQ cou- plings can be identified and suppressed. To this end, we incorporate differentiated weights for P and Q in the loss design, together with an explicit PQ penalty that eliminates unphysical cross-subspace couplings and suppresses ghost states.
The loss function in reciprocal space defined as:
loss( k) Ex|Ap lossp(k) + λo los) + XpQ los)pQ(k))
(2)
where Ap. Aq, and XpQ are tunable hyper-paramene parameters that adjust the relative importance of the three loss terms, which respectively measure the errors in the P subspace, the Q subspace, and the combined PQ joint subspace. The detailed formulations of these terms are provided in Appendix G
The overall loss function combines the losses from both R-space and k-space:
loss
loss (R) + loss(k)
This consistent treatment of real-space and reciprocal space Hamiltonians provides a robust founda- tion for high-fidelity band structure predictions and, in particular, effectively eliminates ghost states.

---

Published as a conference paper at ICLR 2026
3 DATASET
As broad-coverage open-source Hamiltonian datasets that use fine-grained orbital descriptions and include spin-orbit coupling (SOC) effects across wide range of crystals are still rare, we construct one ourselves and contribute it to the community. Specifically, our dataset, called Materials-HAN SOC, contains 17,000 material structures sampled from the Materials Project (Jain al. 2013). with ground trutt Hamiltonians and band structures generated using the DFT software ABACUE US al.] 2016 Lin et al. (023) and PYATB (Jin al. (023). It spans more than 60 distinct elements from the first six rows of the periodic table and explicitly incorporates SOC effects. For these structures, a high-quality atomic orbital basis set (L.in et al.) 202T), up to 4s2p2d1f orbitals for each element, is employed, providing fine-grained representation of their electronic structure. The dataset contains all quantities required by our method, including atomic structures, zeroth-step Hamiltonians, self-consistent Hamiltonians, and overlap matrices. The dataset is partitioned into 12,000 structures for training, 2,000 for validation, and 3,000 for testing. For details of the dataset construction and comprehensive statistical summaries, please refer to Section
EMPIRICAL STUDY
4.1 STATISTICAL RESULTS
We perform empirical studies on the Materials-HAM-SOC dataset. The implementation details of the network architecture and training configurations are provided in Appendix
First, evaluate the role of H))) discrepancy from the ground truth Hamiltonian H H This quantifies how much (
initial approximation the output stage, measure its
reduces the effective size and complexity of the regression target space for subsequent corrections.
Second, we examine the final prediction accuracy by comparing ()) + ÁH with H thereby measuring the contribution of the learned correction AH in closing the residual gap between (
and (T). These two comparisons together disentangle the effectiveness of the prior H))) and the neural correction on achieving high-fidelity Hamiltonian predictions.
While mean absolute error (MAE) is a straightforward error metric. Hamiltonian prediction presents a unique gauge freedom: adding a global shift JuS, where μ is an arbitrary scalar and S is the overlap matrix, leaves all down-stream physical quantities unchanged (Wang et al. (222c)). This necessitates gauge-invariant error metrie for fair evaluation. To remove this gauge freedom, we adopt the Gauge
MAE (Wang et al.. 2024c) to our context:
Gange MAE(H (0) ()) = min MAE (0) (
+ +JS),
(4)
Gauge M ÄH. H (T) min MA ÄH. ( +x))
where μ is determined by solving μr arg min, Gauge_MAE.
The experimental results for the above metrics are reported in Table [ In addition, we also report Gauge_Man MAE(O, (T)) for comparison. Comparing between Gauge (AE(H(O) HT) quantifies the actual reduction in the effective output space achieved by introducing H (0) at the output stage.
H()) and
As shown in Table the zeroth-step Hamiltonian ( closely matches the self-consistent Hamil- tonian (T) in the spin-flip submatrices (1 and 4f). Similarly, the imaginary parts of the spin- conserving submatrices (1f and 41) also exhibit excellent agreement. In these components, the deviation between ()) and HT)) is negligible, with errors reaching sub-ueV level. Achieving SOC predictions with accuracy of μeV level holds significant value, as small differences in SOC energy can greatly impact the electronic structure of materials (Jing et al. 2025).
Furthermore, the Gauge MAE(H ..))) for the real part of the 1f block reduced by 96% compared to MAE(O. H))), yielding a much narrower numerical range for regression. This substantial reduction eases optimization by allowing the network to concentrate physically mean- ingful residual corrections rather than reconstructing the entire Hamiltonian, thereby improving pre- diction accuracy across diverse atomic configurations. In systems with time-reversal symmetry and
7

---

Published as a conference paper at ICLR 2026
Table I: Comparison of Gauge MAE values computed real space (R-space) on the testing set of Values are reported for four spin-resolved regions (11, 1 41, 44) with sep- arate real and imaginary components, and for the entire matrix (Overall), where real and imaginary components are combined into a single metric. Metrics are averaged over non-zero elements only; entries set to zero due to the truncation distance are masked out. All values are meV.
Gauge_MAEQQ H))
H
Gauge + AH, H())
Region
Real
Imag
Real
Imag
Real
Imag
11
149.145
0.293
5.213
< 0.001
2.834
< 0.001
14
0.301
0.299
< 0.001
< 0.001
< 0.001
< 0.001
1t 0.301
0.299
< 0.001
< 0.001
< 0.001
0.001
44
149.145
0.293
5.213
< 0.001
2.834
0.001
Overall
74.914
2.606
1.417
real-valued atomic orbitals, which constitute the majority of practical cases, the real parts of the tt and H blocks are identical. This symmetry implies that the correction network only needs to predict the real part of the tf block in AH, substantially reducing the number of matrix elements be learned. Finally, with the neural network correction applied, the errors for the M and Li blocks are substantially reduced, achieving superior prediction accuracy: the overall Gauge MAE 1.417 meV, closely matching the ground-truth labels obtained from DFT calculations.
C
N
F
He
Mg
Q
135
3.5
1:53
120
1-B
135
1:34
1.0
Hb
Me
Ru
Kh
AW
Od
Aa
1.50
1.75
3.50
235
Figure 3: Element- wise analysis of prediction errors. For each chemical element, we collect all of the testing structures containing that element and compute the Gauge MAE values for each subset.
In Fig. we report fine-grained evaluation of prediction accuracy by partitioning the test set into subsets defined by chemical elements. For each element, we gather all crystal structures that contain it and compute the mean етrоr within this subset. The resulting per-element statistics are visualized on the periodic table, providing a clear view of how the model generalizes across chemically diverse systems. The analysis shows that for most of the elements, the prediction errors are below 1.5 meV, confirming the robustness of our approach across a broad spectrum of the periodic table.
For more detailed analysis of the contributions of different components in our framework, we duct fine-grained ablation studies, which are detailed in Appendix L These studies show that the Ahysics-informan input descriptor ( the correction-bart Grad mechanism, the ensemble strategy, and the joint R- and k-space training objective each provide significant reductions errors. The combination of all these components leads to the best overall performance, with notable improvements in both band structure prediction and the suppression of
regression target design, the Trace-
8

---

Published as a conference paper at ICLR 2026
unphysical artifacts such ghost states. We also compare method with DeepH-E3 (Gong al. 2023 and the original work of TraceGrad (Yin et al. 2025). demonstrating the sigmificant superiority of our method. For details, please refer to Appendix
4.2 CASE STUDY ON DUT-OO-DDISRRIUUTOO GENERALIZATION
As shown in Figure our model is trained a dataset that does not include structures containing the element Neon (Ne). However, the testing set, a Ne-containing structure included. Despite Ne being unseen during training, the model is able generalize well to this new element, with test- ing error remaining very small, i... 0.1 meV for the R-space MAE, as reported in Figure 3 This demonstrates the model's ability extrapolate knowledge learned from other elements to predict properties for unseen elements. This out-of-distribution generalization capability is a direct result of our approach using the zeroth-step Hamiltonian as descriptor. Unlike traditional methods that rely on randomly initialized embeddings for each element, which cannot generalize to unseen elements included in the training set, our model embeds physical information about the system's 's electronic structure into unified representation space using the zeroth-step Hamiltonian. This helps the model capture the relationships between different elements' electronic structures, enabling it to generalize more effectively to out-of-distribution elements. This case study demonstrates the theoretical po- tential of our method's generalizability to unseen elements. In future work, we will conduct further element-level ofdisstribuutinn evaluations to more rigorously quantify this ability.
a
10.0
Dround Tuttt arriinng poook
b
10.0
Greand Truth Full Nethod
5.0
2.5
2.5
(eV)
-2.
-1.0
-5.0
-1.5
-10.0
166
1c6
3.4
Ground Truth Ab Looo
0:75
Ground Trath
AbvorionaLoss
Ful Methed
Full Mathod
0.50
0.25
Im (5m)
0.8
8.4
-0.50
Rh
-0.5
Sa
Row [ev]
haa (eV)
Figure 4: Panels a and show the band-structure comparison among Ablation Loss-k, the Full Method, and the Ground Truth DFT results. The setting Ablation @ Loss-k drops the k-space loss from the Full Method. Panels e and d display the comparison of the real and imaginary parts of the optical conductivity along the direction computed from the three Hamiltonians.

---

Published as a conference paper at ICLR 2026
4.3 CASE STUDY ON K-SPACE LOss FUNCTION
We present a representative case study regarding the effectiveness of the k-space loss function in the main text, with more cases available in [O Appendix [ As shown in Fig. in panel (a), the model is trained using only the H (R))baaeed loss defined in Eq. ( ablating the k-space loss. In this case, the predicted band structure closely matches the DFT reference near the Fermi level for most k-points. However, several isolated points exhibit abrupt discontinuities and deviate significantly from the ground-truth bands, which is a typical manifestation of ghost states. Notably, the MAE of the H (R) matrix elements is only 0.53 meV. These sharp spectral anomalies show that even small MAE errors in real space (R)) can trigger ghost states in the resulting band structure.
This issue arises from the intrinsic numerical instability of the generalized eigenvalue problem, where the non-orthogonality of the orbital basis leads to pronounced error amplification. As we analyze in Appendix F the sensitivity of both cigenvalues and cigenfunctions is enlarged by the factor a(S(k)) implying that even small perturbations in the predicted H (R) can cause significant deviations the computed band energies, manifesting as ghost states in the spectrum.
[[(kk]ll
In Fig. {b), we present the result from our Full Method, which includes the k-space loss. The pre- dicted bands now align closely with the ground-truth results, and ghost states are nearly eliminated. The MAE of the real-space Hamiltonian H (R) remains at 0.49 meV, similar to the case where only (R))(R)) was used, while the k-space loss loss(k) is reduced by more than 50,, substantially enhancing the overall quality of the predicted band structure.
Beyond band energies, the fidelity of the Hamiltonian is also reflected in wave-function-relate physical observables. To further assess the influence of different loss functions on the wave-function accuracy, we compute the optical conductivity corresponding to the predictions in Figs. 4 and (b). The real and imaginary parts of the conductivity along the x direction, compared with the ground-truth results, are presented in Figs. 4c) and (d). As demonstrated in these comparisons, the Hamiltonian trained with our Full Method yields substantially improved agreement with the reference conductivity. This result indicates that incorporating loss(k) not only suppresses ghost states under comparable H (R) MAE, but also enhances the prediction of physical observables that depend sensitively on the wave-function quality.
5 CONCLUSION
We advance universal Hamiltonian deep learning through both a new method and new dataset. We propose NextHAM, a unified deep learning framework designed for accurate and generalizable prediction of electronic-structure Hamiltonians across the periodic table. First, we leverage zeroth- step Hamiltonians constructed from initial charge densities informative input features, facilitat- ing the model to capture the intrinsic characteristics of electronic structures. Second, we present a Transformer- hansserrer-based neural architecture that enforces strict (3)--evvvvrr while maintaining high expressive capacity, enabling accurate modeling of spatial symmetries in material systems. Third, design novel training objective that jointly optimizes the Hamiltonian prediction in both real space and reciprocal space, ensuring consistency with downstream physical quantities such as band a
benchmark of 17,000 DFT- calculated material structures spanning six rows of the periodic table, with explicit spin-orbit cou- pling and high-resolution orbital representations, providing high-quality resources for training and evaluation. Empirically, NextHAM attains DFT-level accuracy for Hamiltonians and band structures while bringing substantial speedups over conventional DFT workflows, providing powerful tools to efficient simulation and design of new materials.
structures. We also release Mattrrilss-AAM-SOO
ACKNOWLEDGEMENTS
This work was supported by the Advanced Materials-National Science and Technology Major Project (Grant No. 20252D0618401) the National Natural Science Foundation of China (Grant No. 12134012, (5506112), the Strategic Priority Research Program of the Chinese Academy of Sciences (Grant No. XDB0500201), and the Innovation Program for Quantum Science and Tech- nology (Grant No. 2021ZD0301200, The numerical calculations were performed on the USTC High-Performance Computing facilities and Hefei advanced computing center.
10

---

Published as a conference paper at ICLR 2026
REPRODUCIBILITY STATEMENT
The codes of NextHAM are available here: httpe///gtttub..cmm//vvidddd.d
NextHAM The pre-trained weights are available here: httpps///zettl...pppcccca com:
the extraction code QoYA.
The dataset Materials-OO--HAM is available here: https://dzeril:: hww.eee.pcubbe... com:
the extraction code is DGEo. Information regarding the dataset construc- tion along with comprehensive statistical summaries can be found in Appendix H The implementa- tion details of the network architecture and the training setup are presented in Appendix
1:011eefiees/wwi http:///wwb/ddhhhhihtc
Details on loss functions presented in Appendix G
ETHICS STATEMENT
This paper presents work whose goal is to advance the field of deep learning driven by materials research. Although our field has not yet exhibited direct negative social or ethical consequences, we recognize the importance of anticipating broader concerns. A key issue is the limited interpretability of deep learning systems, which obscures the reasoning behind their predictions and constrains their utility for gaining physical understanding. We stress the need for improving model interpretability. particularly regarding how physical knowledge is incorporated and represented, to guarantee both the accuracy of predictions and their applicability a wide range of scientific problems.
REFERENCES
Ilyes Batatia, David P Kovacs, Gregor Simm, Christoph Ortner, and Gábor Csányi. Mace: Higher order equivariant message passing neural networks for fast and accurate force fields. Advances in neural information processing systems, 35:1 15:11122-1136, 2022.
Simon Batzner, Albert Musaelian, Lixin Sun, Mario Geiger, Jonathan P Mailoa, Mordechai Ko- rbblthh, Nicola Molinari, Tess Smidt, and Boris Kozinsky. E (3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials. Nature Communications, 13(1):
2453, 2022.
Joel M Bowman, Chen Qu, Riccardo Conte, Apurba Nandi, Paul L Houston, and Qi Yu. A-machine learned potential energy surfaces and force fields. Journal of Chemical Theory and Computation, 19(1):1-17, 2022.
Mildred S Dresselhaus, Gene Dresselhaus, and Ado Jorio. Group theory: application to the physics of condensed matler. Springer Science Business Media, 2007.
Johannes Gasteiger, Florian Becker, and Stephan Günnemann. Gemnet: Universal directional graph neural networks for molecules. Advances in Neural Information Processing Systems, 34:6790- 6802, 2021.
Mario Geiger and Tess E. Smidt. e3mn: Euclidean Neural Networks. CoRR, abs/2207.09453, 2022.
Gene H. Golub and Charles F. Van Loan. Matrix Computations. Johns Hopkins University Press, Baltimore, MD, USA, 4th edition, 2013. ISBN 978-14284417 1421407944.
Xiaoxun Gong, He Li, Nianlong Zou, Runzhang Xu, Wenhui Duan, and Yong Xu. General frame- work for F(3)-equivariant neural network representation of density functional theory Hamiltonian. Nature Communications, 14(1):2848, 2023.
D. R. Hamann. Optimized norm-conserving Vanderbilt pseudopotentials. Physical Review B, 88(8): 17, 2013.
Pierre Hohenberg and Walter Kohn. Inhomogeneous electron gas. Physical Review, 136(3B):B864, 1964.
Anubhav Jain, Shyue Ping Ong, Geoffroy Hautier, Wei Chen, William Davidson Richards, Stephen Dacek, Shreyas Cholia, Dan Gunter, David Skinner, Gerbrand Ceder, et al. Commentary: The Materials Project: A materials genome approach to accelerating materials innovation. APL mate- rials, 1(1), 2013.
11

---

Published as a conference paper at ICLR 2026
Gan Jin, Hongsheng Pang. Yuyang Ji, Zujian Dai, and Lixin He. Pyatb: An efficient python pack- age for electronic structure calculations using ab initio tight-binding model. Computer Physics Communicationss 291:108844, 2023.
Fang-Ming Jing, Zhen-Xiong Shen, Guo-Quan Qin, Wei-Kang Zhang, Ting Lin, Ranran Cai, Zhuo- Zhi Zhang, Gang Cao, Lixin He, Slang-Xiang Song, et al. senttii-feeld-dddeeeeeeee spin-orbit- coupling gap in -oneesaaasullted bilayer graphene. Physical Review Applied, 23(4):044053, 2025.
Robert Jones. Density functional theory: Its origins, rise to prominence, and future. Reviews of modern physics, 87(3):897, 2015.
Walter Kohn and Lu Jeu Sham. Self-consistent equations including exchange and correlation effects. Physical Review, 140(4A):A1133, 1965.
R. B. Lehoueq, D. C. Sorensen, and C. Yang. ARPACK Users' Guide: Solution of Large-Scale Eigenvalue Problems with Implicitly Restarted Arnoldi Methods. 1998.
He Li, Zun Wang, Nianlong Zou, Meng Ye, Runzhang Xu, Xiaoxun Gong, Wenhui Duan, and Yong Xu. Deep-learning density functional theory Hamiltonian for efficient ab initio electronic- structure calculation. Nature Computational Science, 2(6):367-377, 2022.
Pengfei Li, Xiaohui Liu, Mohan Chen, Peize Lin, Xinguo Ren, Lin Lin, Chao Yang. and Lixin He. Large-scale ab initio simulations based on systematically improvable atomic basis. Computational Materials Science, 11:5033557, 2016.
Yunyang Li, Zaishuo Xia, Lin Huang, Xinran Wei, Samuel Harshe, Han Yang, Erpai Luo, Zun Wang. Jia Zhang, Chang Liu, Bin Shao, and Mark Gerstein. Enhancing the Scalability and Applicability of Kohn-Sham Hamiltonians for Molecular Systems. In ICLR, 2025.
Yi-Lun Liao and Tess E. Smidt. Equiformer: Equivariant Graph Attention Transformer for 3D
Atomistic Graphs. In ICLR, 2023.
Peize Lin, Xinguo Ren, and Lixin He. Strategy for constructing compact numerical atomic orbital basis sets by incorporating the gradients of reference wavefunctions, Physical Review B, 103(23): 235131, 2021.
Peize Lin, Xinguo Ren, Xiaohui Liu, and Lixin He. Ab initio electronic structure calculations based on numerical atomic orbitals: Basic fomalisms and recent progresses. WIREs Computational Molecular Science, 14:e1687, 2023.
Erpai Luo, Xinran Wei, Lin Huang, Yunyang Li, Han Yang, Zun Wang, Chang Liu, Zaishuo Xia, Jia Zhang, and Bin Shao. Efficient and Scalable Density Functional Theory Hamiltonian Prediction through Adaptive Sparsity. In ICML, 2025.
Albert Musaelian, Simon Batzner, Anders Johansson, Lixin Sun, Cameron J Owen, Mordechai Ko- rbblthh, and Boris Kozinsky. Learning local equivariant representations for large-scale atomistic dynamics. Nature Communications, 14(1):579, 2023.
Ágnes Nagy. Density functional. theory and application to atoms and molecules. Physics Reports, 198(1):1-79, 1998.
Saro Passaro and C. Lawrence Zitnick. Reducing SO(3) convolutions SO(2) for Efficient Equiv- ariant GNNs. In ICMI,, pp. 27420-27438, 2023.
John P. Perdew, Kieron Burke, and Matthias Ernzerhof. Generalized Gradient Approximation Made Simple. Physical Review Letters, -7818)8885-,888, 1996.
Kristof T Schütt, Huziel E Sauceda, P-J Kindermans, Alexandre Tkatchenko, and K-R Müller. Schnet-a deep learning architecture for molecules and materials. The Journal of chemical physics, 148(24), 2018
Kristof T Schütt, Michael Gastegger, Alexandre Tkatchenko, K-R Müller, and Reinhard J Maurer. Unifying machine learning and quantum chemistry with a deep neural network for molecular wavefunctions. Nature Communccations, 10(1):5024, 2019.
12

---

Published as a conference paper at ICLR 2026
Nathaniel Thomas, Tess Smidt, Steven Kearnes, Lusann Yang, Li Li, Kai Kohlhoff, and Patrick Riley. Tensor field networks: Rotation-and -equiaaiinnnuu neural networks for 3d point clouds. arXiv preprint arXiv:/802.082/19 2018.
Oliver Unke, Mihail Bogojeski, Michael Gastegger, Mario Geiger, Tess Smidt, and Klaus-Robert Müller. Se (3)-equivariant prediction of molecular wavefunctions and electronic densities. In Advances in Neural Information Processing Systems, pp.
2021.
M. J. van Setten, M. Giantomassi, Bousquet, M. J. Verstraete, D. R. Hamann, X. Gonze, and G.- M. Rignanese. The pseudoDojo: Training and grading an 85 element optimized norm-conserving pseudopotential table. Computer Physics Communications, 226:39-54, 2018.
Yusong Wang, Tong Wang, Shaoning Li, Xinheng He, Mingyu Li, Zun Wang, Nanning Zheng, Bin Shao, and Tie Yan Liu. Enhancing geometric representations for molecules with equivariant vector-scalar interactive message passing. Nature Communications, 15(1):313, 2024a.
Yuxiang Wang, He Li, Zechen Tang, Honggeng Tao, Yanzhen Wang, Zilong Yuan, Zezhou Chen, Wenhui Duan, and Yong Xu. Deeph-2: Enhancing deep-learning electronic structure via an equiv- ariant local-coordinate transformer. arXiv preprint arXiv:2401. 17015, 2024b.
Yuxiang Wang, Yang Li. Zechen Tang, He Li, Zilong Yuan, Honggeng Tao, Nianlong Zou, Ting Bao, Xinghao Liang, Zezhou Chen, et al. Universal materials model of deep-learning density functional theory Hamiltonian, Science Bulletin, 69(16):2514 (116))2511-2222) 2521, 2024c.
Maurice Weiler, Mario Geiger, Max Welling, Wouter Boomsma, and Taco S Cohen. 3d steerable cnns: Learning rotationally equivariant features in volumetric data. In NeurlPS, 2018.
Chen Hao Xia, Manasa Kaniselvan, Alexandros Nikolaos Ziogas, Marko Mladenović, Rayen Mahjoub, Alexander Maeder, and Mathieu Luisier. Learning the Electronic Hamiltonian of Large Atomic Structures. In /CMI,, 2025.
Shi Yin, Xinyang Pan, Fengyan Wang, and Lixin He. Tracegrad: a Framework Learning Expressive SQuuiaaquvvariant Non-linear Representations for Socctronic-Structii Hamiltonian Prediction. In ICMI., 2025.
Haiyang Yu, Meng Liu, Youzhi Luo, Alex Strasser, Xiaofeng Qian, Xiaoning Qian, and Shuiwang Ji. QH9: A Quantum Hamiltonian Prediction Benchmark for QM9 Molecules. In NeurlPS, 2023a.
Haiyang Yu, Zhao Xu, Xiaofeng Qian, Xiaoning Qian, and Shuiwang Ji. Efficient and Equivariant Graph Networks for Predicting Quantum Hamiltonian. ICML, pp. 40412-40424, 2023b.
He Zhang, Chang Liu, Zun Wang. Xinran Wei, Siyuan Liu, Namning Zheng, Bin Shao, and Tie- Ran Liu. Self-consistency Training for Hamiltonian Prediction. In ICML, 2024.
Xuan Zhang, Limei Wang, Jacob Helwig, Youzhi Luo, Cong Fu, Yaochen Xie, Meng Liu, Yuchao Lin, Zhao Xu, Keqiang Yan, et al. Artificial intelligence for science in quantum, atomistic, and continuum systems. Foundations and Trends@ in Machine Learning, 18(4):385-912 2025.
Yang Zhong, Hongyu Yu, Jihui Yang, Xingyu Guo, Hongjun Xiang, and Xingao Gong. Universal machine learning kohn-sham hamiltonian for materials. Chinese Physics Letters, 41(7):077103, 2024.
Larry Zitnick, Abhishek Das, Adeesh Kolluru, Janice Lan, Muhammed Shuaibi, Anuroop Sriram, Zachary W. Ulissi, and Brandon M. Wood. Spherical Channels for Modeling Atomic Interactions. In NewrlPS, 2022.
13

---

Published as a conference paper at ICLR 2026
A ELECTRONIC STRUCTURE CALCULATIONS: FROM DENSITY FUNCTIONAL THEORY TO DEEP LEARNING METHODS
Density Functional Theory (DFT) (Hohenberg & Kohn 1964 Kohn & Sham 1965) has established itself as foundational tool in modern electronic structure theory, with wide-ranging applications in condensed matter physics, quantum chemistry, and materials science. First developed in the 1960s by Hohenberg, Kohn, and Sham, DFT reformulates the many-electron problem by replacing the complex many-body wavefunction with the electron density p(r) as the central variable. This shift dramatically simplifies the computational treatment of quantum systems while retaining the essen- tial physics, making it feasible to study realistic systems under accepted computational cost. Over the years, DFT has become indispensable for tasks such computing band structures and orbital energies, performing structural optimizations, and predicting a variety of electronic, magnetic, and optical properties. Its broad applicability and computational efficiency have cemented its role key methodology across multiple scientific domains.
At the heart of density functional theory (DFT) lies the Kohn-Sham (KS) equation (Kohn & Sham 1965), which reformulates the many-body electronic problem into a tractable set of single-particle equations:
h
( (r), with A
V + (ra(T) Vixccpl(r),
(5)
2m
where A is the effective Lingle-partcct Hamiltonian. The potential includes the external potential Vox (r), and the Hartree- cantree-exchange- correlation (HXC) potential VHxc[p](r) VHlpl(() + Vxclp)(
which is a functional of the electron density p(r). The density itself is obtained from the KS orbitals via:
M
p(r) = Σ (mm(r)
(6)
m=1
where M is the number of occupied single-particle states.
To numerically solve Eq. (5) basis set is introduced. Atomic orbitals (Lin al. 2023) are a widely adopted choice due to their localized nature and computational efficiency- typically
require fewer basis functions to reach a given level of accuracy compared to plane-wave or other delocalized bases. The atomic basis functions are products of radial function and a spherical harmonic, that is,
@xcim (r) = fxc((r) (T).
where & denotes the element type, Im denotes the angular momentum and the magnetic quantum number. Usually, real spherical harmonic functions are used. The radial functions are typically tabulated numerically the radial functions (r) are expanded in terms of spherical Bessel functions and truncated beyond cutoff distance Te
fine radial mesh, and hence these basis functions referred to NAOs.
Cactaillar), r <<R To:
fne((T)
(8)
0.
2 Te:
The KS eigenfunctions are expanded in terms of these atomic orbitals:
(an(F) VN R
ΣΣ
Cna, ik th - TT- -R),
(9)
( - T; R) are the uth atomic orbitals centered on the ith atom in the unit cell R. and {u, i} is the composite index for the NAOs. Cna.k the coefficients of orbitals of band at point, and N₂ is the number of unit cells in the Born-von-áámen boundary conditions, equivalent to the number of k points in the first Brillouin zone (BZ).
where
supercell under the periodic
Given the expansion of the KS states in terms of atomic orbitals in Eq. 9.. the KS Eq. 5 becomes ageneralized eigenvalue problem,
H(k) S(k)
(10)
14

---

Published as a conference paper at ICLR 2026
where (k), S(k), and Ck are the Hamiltonian matrix, overlap matrix and eigenvectors at given point, respectively. Ek is a diagonal matrix whose entries are the KS eigenenergies, Enk denotes the energy eigenvalue of the n-th KS eigenstate. To obtain the Hamiltonian matrix H (k), first calculate the Hamiltonian real space as
(000
h2
Hag R = 000
7
+ Vex + ViXc ÖBR
(11)
2m
where B are atomic orbital indices within one unit cell, and Φau def (r 7:). ΦeR Tj R). The Hamiltonian matrix at given k point can be obtained via Fourier transform.
def Φr
Hap (k)
Σ R
eik.R Hap (R).
(12)
Similarly, the overlap matrix at a given k point is obtained as
(k)
Σ
.k (R),
(13)
R
where
SaB ( = (oaa (dn)).
(14)
The overall computational procedure follows an iterative self-consistent (SC) loop:
1. Initial Guess: Start with an initial electron density p (r). Initialize the number of itera-
tions to
2. Potential Construction: Compute the effective potential Vixe by (r))
3. Hamiltonian Assembly: Construct the Hamiltonian matrix H (5) using the current potential
using Eq. 11].
4. Eigenproblem Solution: Perform Fourier transformation and solve the generalized cigenvalue problem in Eq. (10p to obtain the KS eigenfunctions (Tn(() and eigenvalues Enk-
5. Density Update: Compute the updated density (r) from the new orbitals using Eq. (6
6. Convergence Check: Update > (+1, repeat steps 2-6 until the input and output densities agree within chosen convergence threshold.
This procedure can be summarized schematically
( (r) (0) (r) + (0) + ( (..))) (T) + vi [p] + (?).
(r) (r..)
Once self-consiseency is reached at iteration T, the final Hamiltonian matrix H can be used to compute down-sin stre-m physical quantities such as total energy, band structure, orbital energies, and derived electronic, magnetic, or transport properties.
Despite the remarkable success Kohn- Sham DFT in advancing fields such as materials science, energy, and biomedicine over recent decades (Nagy 1998) Jones) 2015), it still faces significant computational challenges, especially when applied to large atomic systems under limited computa- tional resources. The primary bottlenecks arise from two aspects. First, the matrix diagonalization in Eq. 10 scales as (NN*), where N is the number of atoms in the system. Second, the iterative nature of the SC procedure requires T rounds of self-consistent updates, which further amplifies the overall computational cost. This becomes particularly problematic when a high level of convergence accuracy is needed or when dealing with complex systems, often making it difficult to complete the calculations within reasonable time or resource constraints.
To address this challenge, recent approaches (Schütt et al., 2019 Unke al. 2021 Li al. 2022
Gong et al.] 2023) Yu al.] 22036b Li et al. 22025] I Luo et al. 2025 Yin et al. 2025) have adopted the deep graph learning paradigm predict the self-consistent Hamiltonians, These methods bypass the iterative and computationally intensive matrix diagonalization steps in traditional DFT algorithms
by directly predicting the final converged Hamiltonian matrix H(T) in single forward pass. As
15

---

Published as a conference paper at ICLR 2026
shown in Eq. II). the Hamiltonian matrix is inherently sparse: only pairs of atoms within cutoff radius contribute non-zero elements. Therefore, the total number of Hamiltomian matrix elements that need to be computed scales with the number of local atomic pairs in the system, leading to complexity of O)N where is the total number of atoms and E denotes the average number of neighhoring atoms within the cutoff radius per atom. Since the atomic orbital basis functions have finite spatial support, matrix elements vanish beyond a certain inter-atomic distance. In small systems where all atoms lie within each other's cutoff radius, E' N, and the total number of non- zero elements scales O( However, in sufficiently large systems, E saturates to a constant determined by local geometry, making the number of non-zero Hamiltonian elements scale linearly as O(N). Moreover, since most physical properties, such as transport, optical, and topological properties, depend only on the energy bands near the Fermi level, is unnecessary to solve for the eigenfunctions of all occupied states once the Hamiltonian is known. Since the Hamiltonian matrix is sparse and only limited number of bands near the Fermi level are needed, these eigenstates can be efficiently computed using methods like the shift-invert approach available in the ARPACK package (Lehoucq et al. 1998). with a computational complexity O(N) for large systems.
These deep-learning approaches have exploited the sparsity of the Hamiltonian, yielding a compu- tational cost that scales approximately linearly with the number of non-zero matrix elements and enabling efficient, scalable prediction of quantum properties in large atomic systems. As result, they offer a significant efficiency advantage over traditional DFT methods with computational com- plexity of (TT This efficiency makes them particularly promising for predicting electronic structures of complex atomic systems under limited computational resources, potentially accelerat- ing down-stream application areas like materials simulation and design.
B OVERVIEW OF BASIC CONCEPTS IN GROUP THEORY
This section reviews several fundamental concepts from group theory that form the basis of the sym- metry principles employed in this work. Readers interested in a more comprehensive introduction may consult the monograph of Dresselhaus et al. (2007).
Definition B.1. Group. set G endowed with a binary operation is called a group if the following axioms hold:
1. Closure: For any fig E G, the product f g remains in G.
2. Associativity: For all g. h E G, the equality (f - = f (g- holds.
3. Identity: There exists an element e E G such that e f f e = for every f E G.
4. Inverse: Each G has an inverse f e G' satisfying f f ff.
Definition B.2. Group Representation. A representation of group G tensor space T(V) is a homomorphism
p: G + GL(T(V)).
mapping each group element to an invertible linear operator acting on T(V). The mapping preserves the group structure, i.e.,
p(gige) = p(()) p(e)= =1
Definition B.3. Irreducible Representation. pet G + GL(V) be representation vector space V. It is called irreducible if no nontrivial subspace c V exists such that p(g)W C W for all E G. If such a proper invariant subspace exists, the representation is said to be reducible.
Definition B.4. Equivariant Map.
Let pV : G GL(T(V)) and Pw G GL(T(W)) be representations of group G. A function f T(() T(W) is equivariant if
u(v pw(g) f(v),
Vg E G, v e T(V).
Definition B.5. Invariant Map. Given a representation pv GL(T(V)), function f
T(V) + T(W) is invariant under group G if
f(v( f(v),
UET(V)
16

---

Published as a conference paper at ICLR 2026
Definition B.6.
The Group SO(3). The special orthogonal group SO(3) consists of all real 3
rotation matrices:
S3(3) (R
E
R
3x3
R 1, det 1}. R
Elements of SO(3) represent rotations in three-dimensional Euclidean space.
Definition B.7. Representations of SO(3). A representation of SO(3) is homomorphism
p SO(3) + GL(V).
The irreducible representations of SO(3) are labeled by a non-negative integer 1, which corresponds to the angular momentum quantum number quantum mechanics.
Definition B.8. Wigner-D Matrices. A standard family irreducible representations of SO(3) provided by the Wigner-D matrices:
D D'm'm R (1, m' m)),
where m) denotes the eigenstate angular momentum with quantum number / and magnetic index m. These matrices specify how angular momentum states transform under rotation R.
Definition B.9. The Group O(3). The orthogonal group (3) consists of all real 3 x 3 orthogonal matrices, including both proper rotations and reflections:
0(3) (R E 3x3
R RRR-1].
Elements of ((3) represent possible orthogonal transformations in three-dimensional Euclidean space, including both rotations (with determinant 1) and reflections (with determinant-11
Definition B.10. Irreducible Representations of the O(3) Group. The orthogonal group 0(3) contains both proper rotations (det +1), forming the subgroup SO(3), and improper rotations (det - 1), including reflections and spatial inversion. An irreducible representation of O(3) is homomorphism
T: 0 ~ GL(V).
Irreducible representations of 0()) are obtained by extending the irreducible representations O(3). For each angular-moment momentum degree there exist exactly two inequivalent irreducible representa- tions of O(3):
p(.+)
((.-),
corresponding respectively to even and odd parity under spatial inversion.
For any group element R E O(3), their actions are defined by
p(.+) (R) = т (R) (1) (R),
where (R) is the Wigner-D matrix giving the degree-l irreducible representation of O(3), and the parity factor (R) is
+1, det(RR +1,
Tt. (R)
+1 for ((.+t) det(R) = 11,
for p(l-), det(R) -1
Thus, [l++] is even and (1,-) is odd under inversion. These two parity-extended forms exhaust all irreducible representations of O(3).
Definition B.11. The Euclidean Group E(3). The Euclidean group E(3) is the group of all rigid
motions in direeddimessioal space. consists of all compositions of rotation or reflection and translation:
E(3) ((R, t) R E 0(3), e TR
The action of a group element (R, t) point R33 is given by
(R, t) Rx +t.
17

---

Published as a conference paper at ICLR 2026
Definition B.12. Direct-Product State. For vector spaces V1 and V2, their tensor product space V1 & V/2 consists of bilinear combinations of vectors from both spaces. A basis of V1 X V2 can be written as {|t) @ [j)]; }. and general element takes the form
(0)
Ct i 1
This construction increases dimensionality multiplicatively:
dim(V1 @ V2) dim(V1) dim(
A group action the tensor censor-product state acts each factor:
((v1) & (02)) = (g (v1)) @ (u2))
Definition B.13. Direct-Sum State. For vector spaces V1 and V2, the direct-sum space Vi ) V/2 consists of ordered pairs ([u1), (02)). A general vector takes the form
(0) = lui) @ (02),
and the dimensionality increases additively:
dim(V1 @ V2) = dim Vi1 + dim(V2).
The group acts independently on each component:
g (111) @ (02)) (g (2))) @ (g (0-)))))
Definition B.14. Physical Quantity as a Direct Product of Angular-Momentuu Degrees. Let Ip and by be two angular-moment degrees of freedom, with corresponding 0(3) irreducible repre- sentations (p) and (() A tensor
Q 4 RR (2 +)x
that is formed as the direct-product quantity of these two degrees transforms under any R E O(3)
Q
4 81 (R) = p(p (R) Q 4,0 ( (R)
This transformation rule expresses that the tensor carries product representation of the two angular- momentum degrees, each transforming according its respective 0(3) irrep with the appropriate parity.
Definition B.15. Clebsch-Gordan Decomposition for 0(3). Given angular-momentum de- grees lp and 4g, the tensor Q'r Q'r006 introduced in Definition B.14 can be decomposed as:
+
CGDecomp Q hh))
q'
by b
with components:
4m
Σ
Chilg 1000 mip my
mmp.mgg
where Chi Ma. the Clebsch-Gordae coefficients.
Definition B.16. Parametric Clebsch-Gordan Decomposition for O(3). To introduce learnable parameters while preserving equivaaiiie,, the above decomposition can be extended to para- metric form. Specifically, the decomposition can be written as:
4 +4
CGDecomp( Q Wl;
1-1l--b hgl
where
W Σ
Om Mp my
mp.
and W 1, H-w))=,,- is set of scalar (or channel-wise) weights that act each irreducible component. Since each un acts as scalar on the entire (21 + 1)-dimensional subspace labeled by / and does not mix the magnetic indices the map
{q }
remains 0(3) equivarai..... The narameeiii Clebsch-Gorie decomposition is recovered as the special case where wy for all l.
18

---

Published as a conference paper at ICLR 2026
C RELATED WORK
The foundation of deep learning- electronic-structure Hamiltonian prediction involves con- structing neural networks that respect E(3)-symmerry, which inherently includes equivariance the O(3) group, covering 3D rotations and inversions. -(3)-qquivariant graph neural networks typically construct and update features using group-theoretic operators that preserve equivariance, such as linear combinations of tensors, direct sums, tensor products, Clebsch-Gordan decompositions, and tensor contractions (Thomas et al. 2018 Schütt et al. 2018 Gasteiger et al. 2021 Batzner et al.
2022 Batatia et al. Geiger & Smidt 2022 Wang et al. 2024a).
However, since traditional non-linear activation functions, when applied directly to 0())- features, may break equivariance, a central research topic is to reconcile strong non-linear expres- siveness with striet 0(3) eq3i-vvuivre An early attempt to address this problem was the use of gated activation functions (Weiler et al., 2018). which first apply non-linear activations to 033)-invariant features and then use them as coefficients to scale the 0(3) -qui-equuaa features. Representative works adopting this mechanism include Allegro (Musaelian al., 2023) for force and energy pre- diction, as well as DeepH-EE (Gong et al.) 2023) and QHNet (Yu et al.. (023b) for Hamiltonian prediction. To further enhance non-linear expressiveness of (3)-nnttwrrss. Zitnick et al. (2022) and Passaro & Zitmick (2023) proposed eSCN (cquivariant Spherical Channel Networks). which applies non-linear operations to the coefficients obtained from the spherical decomposition of features. This approach has been widely used in Hamiltonian prediction (Wang et al. (024b] tasks. Neverthe- less, eSCN methods project features onto discrete basis functions through inner-product operations, which may degrade strict 50(3)-equivariance to a discrete sub-group. Furthermore, they use SO(2) convolutions in place of SO(3) convolutions, which could result in a loss of strict inversion equiv- ariance. These trade-offs may influence the physical consistency of the results. As introduced in detail in Appendix D Yin et al.] (2025) proposed the TraceGrad method, which effectively unifies strict 0(3) -quuiaarraa,, with strong non-linear expressiveness for Hamiltonian prediction; however, the backbone network it adopted is a simple graph neural network and has not yet evolved into a non-linear equivariant Transformer framework.
Despite these progresses, deep learning methods for Hamiltonian prediction still face substantial challenges on generalization performance, which can be summarized as follows. First, crystalline materials commonly found in nature can be composed of over 65 different elements from the first six rows of the periodic table, leading to an exceptionally large and heterogeneous input space for deep neural network models. Existing deep learning methods for Hamiltonian prediction typically em- play learnable embeddings to represent nodes (atoms) and edges (atom pairs). These embeddings are randomly initialized and learned directly from the dataset, without incorporating any explicit physical priors. As a result, they struggle to capture the fundamental physical relationships between different atoms and across different material systems, which are crucial for generalization. Second, as illustrated in Figure I the regression target, namely the self-consistent electronic-structure Hamil- tonian, is inherently high-dim -ighedimensioaal and complex, especially when considering SOC effects. For instance, a system containing several tens of atoms may involve nearly several thousands of non- zero Hamiltonian matrix elements that need to be accurately predicted. Most of the existing methods attempt to directly predict the entire self-consistent Hamiltonian matrix, namely HT) as formulated in Appendix A placing a heavy burden on the model due to the vast size of the output space, often resulting in optimization difficulties during training and limited generalization to unseen systems. In addition, most existing methods treat the real-space Hamiltonian as the sole regression target, which can lead to sub-optimal physical fidelity in down-stream applications, particularly in captur- ing low-energy band structures accurately. Although Li et al. (2025) designed a method for molec- ular systems to reduce the regression space of Hamiltonians and introduced basis transformation of the Hamiltonian matrix in the wavefunction loss function to improve the prediction accuracy of downstream physical quantities, their approach limited to molecular systems and not applicable to periodic crystalline materials, which have different mathematical formulations and physical prop- erties. For example, in the case of periodic crystalline materials, the predicted electronic-structue Hamiltonian may involve erroneous couplings between high-energy and low-energy subspaces in the k-space, which can affect the accuracy of downstream physical quantity predictions. These is- sues require entirely new considerations. Moreover, their formulation does not explicitly account for the inherent gauge freedom in Hamiltonian representations, Physically, adding global shift μu to the Hamiltonian matrix, where y is an arbitrary scalar and S is the overlap matrix, leaves down-
19

---

Published as a conference paper at ICLR 2026
stream physical quantities unchanged. This property necessitates a gauge-invariant metric for rigorous evaluation. However, as their approach currently lacks a mechanism to handle this gauge ambiguity, it may potentially lead to optimization instability and physically inconsistent predictions in crystal systems.
As result, constructing unified model that generalizes across diverse crystal prototypes remains challenging, and many existing approaches explicitly constrain their scope. For example, [Li et al. (2022), Gong al. (2023), and Xia al. (2025) each train and evaluate their methods within single material system (e.g., MoS,, BiySes, or a- a-HfO2), without assessing cross-material general- ization. More recently. DeepH-2 (Wang et al., 2024c) broadened coverage systems involving elements primarily from the first four rows of the periodic table; however, they reduced the orbital basis by omitting f-orbitals. While such choices help reduce computational and modeling com- plexity, they may limit broad applicability to the full diversity of real-world materials. [Zhong et al. (2024) developed Hamiltonian prediction model aimed a broader range of element types, while also highlighting the challenge of achieving consistently high accuracy across diverse crystal sys- tems. What's more, except for very few exceptions such as DeepH-E3 (Gong et al. 2023), most existing prediction models neglect the spin-orbit coupling (SOC) effect. Furthermore, open-source datasets with a broad and diverse collection materials dedicated to training and validating uni- versal Hamiltonian models across the periodic table remain scarce. Although the QH9 (Yu al.) 2023a) dataset is a well-known open-source Hamiltonian dataset, consists of molecular systems rather than periodic material systems, and includes only structures composed of H. O. N. and el- ements. To solve these challenges, this work presents an advanced unified deep learning framework together with large benchmark dataset for Hamiltonian prediction, targeting broader generalization across richer classes of materials.
D OVERVIEW OF THE TRACEGRAD PARADIGM
The TraceGrad (Yin et al. 2025) mechanism addresses a key challenge in conventional neural ar- chitectures, which struggle to preserve ((3)-qquivariance when applying non-linear transformations to higher-degree tensor features.. It offers a principled solution to achieving strong non-linear ex- pressiveness while strictly maintaining foundational idea, we first review the motivation and core mechanism of TraceGrad in this appendix. This background will help readers better understand the extensions and architectural developments presented in Section 2.2. where we adapt and generalize TraceGrad into a high-capacity Transformer framework tailored for Hamiltonian learning.
equivariine As our neural network builds upon this
D.1 THE EECUIVAARAANC-EXXPESSSSIIII
DILEMMA
In (eu)-equivariant neural networks, intermediate features at the k-th layer are represented in the direct-sum form irreducible components, i.c.,
h)
p())
p(A E RR21
(A)
and they must satisfy the transformation rule:
((R) ( ((.:)
(1.2) (R) p(M)
The main difficulty to construct a non-linear operator Inonlin(.) that provides genuine nonlinear
expressive power while still preserving strict 0(3) eqqivarriin The equivariance condition requires that the output after a non-linear update satisfies:
plc+1)1 (R) (l.+t) (R) p(K+1M
R R
where
f(K+ = Gnonlin
((()))
However, applying standard element-wise nonlinear activation functions such SiLU Softmax to (()) with 1 > 1 breaks the required equivariance condition. On the other hand, avoiding non-linear operations, significantly limits the expressive power of the model, restricting its fitting and gen- eralization performance. Reconciling these requirements preserving strict equivariance while
20

---

Published as a conference paper at ICLR 2026
allowing strong nonlinear expressiveness therefore poses fundamental challenge. This challenge is precisely what the TraceGrad method is designed to overcome.
This challenge is precisely what the TraceGrad method is designed to overcome. The TraceGrad approach addresses this by proposing a unified representation learning framework that integrates 03)-eqquvvriant and O(3)-invariant physical quantities and neural representations. It constructs -n)--invrriant trace quantities as supervision signals to learn high-quality non-linear O(3)-invariant features. These features then leveraged to induce non-linear -qquivaara representations through a gradient operator, effectively achieving strict unification of -3)-eqqivvreiane and non- linear expressiveness.
D.2 CONSTRUCTING 0(3)-INVARIANT TRAINING LABELS
For an Hamiltonian block H, TraceGrad defines an 0(3--invariant trace quantity:
T = tr H (H)))
which satisfies the equivariance condition:
T(R) =
VR E O(3).
These invariant quantities are directly derived from the equivariant targets and serve as additional supervision for learning -(3)-invariant neural representations, without the need for extra labeling efforts.
D.3 BUILDING NON-LINEAR 0(3)-INVARIANT FEATURES FROM EQUIVARIANT FEATURES
Given an -quivvaarr feature f(k) Dier. f(A)) TraceGrad first forms tensor products
f()), & f(k) and applies extended, parametric Clebsch-Gordan decomposition to extract the degree-O component:
(k) CGDecompox (P(k) p(k); W)
Σ
W CGDecomm(F(h). @ p(
L.I,EE 1,
where W (W:) are learnable scalar (or channel-wise) weights, and the scalar (I = 0) component of the Clebsch-Gordan decomposition. Collecting all channels gives an invariant vector (k) [u] (k)
indicates that we only take
..... uc (.) As each degree-O component is 0(3)-invariant, u is
0(3)-invariant
An arbitrary differentiable non- non-linear neural network Snoulin is then applied:
z Snoulin
(k))
yielding O(3)-invariant features (k) with non-linear expressiveness, since invariance is preserved under any non-lineat function of invariant inputs.
D.4 INDUCING NON-LINEAR EQUI-EQUIAA FEATURES VIA GRADIENTS
TraceGrad uses the gradient of the invariant scalar with respect to the equivariant feature to yield an -quiviauir and non-linearly enriched representation. For single scalar channel ze the gradient is defined as:
_(k
Oze (k)
OF(K)
Mathematically, !! is O(3))))equiiii equivvaqui,,,,,nt,, meaning that for any rotation R O(3),
v( (k) (R) = T (k)
where T(R)) is the representation matrix. Summing over all channels, we obtain an -(3)-equivariant feature, now enriched with the non-linear expressive power of (k),
(k)
c=l
21

---

Published as a conference paper at ICLR 2026
In practice, TraceGrad combines the original equivariant feature and its induced counterpart in residual fashion:
p(k+1) = p(k) + (())
Since (()) and v-k) are equivvraai,,,,,, the new feature f(++ also remains 0(3) equivarraat
p(++1) ( = ((R)
Thus, by applying the gradient operation in this residual fashion, we ensure that both the original and the updated features maintain ((3)-equivariance throughout the layers. The model stacks K such modules to build deep encoder of -(u-equuiariian non-lineat representations.
D.5 JOINT DECODING AND TRAINING OBJECTIVE
TraceGrad uses two decoding branches: (i) an 03)-equivvriian decoder that maps the final equiv- ariant features f(K) the target Hamiltonian block H; and (ii) O(3)-invariant decoder that maps the collection of invariant features {z { k== the trace quantities T tr(H (H)
The two branches are trained jointly with a coupled loss function that balances the error on both the equivariant block predictions and the invariant trace predictions. A adaptive factor adjusts the relative importance of each loss component, ensuring an effective balance during training without backpropagating gradients through the scalar factor.
DETAILS ON THE CONSTRUCTION OF INITIAL NODE AND EDGE FEATURES E
E. INITIAL NODE FEATURES
To construct the initial node features, extract the on-site Hamiltonian block H a from the full zeroth-step Hamiltonian ( This block encodes the local electronic environment of atom (where 1 S N). To convert this block into vector-form representation compati- ble with the input form of equivariant neural networks, we apply the inverse transformation of the Wigner-Eckart layer (Gong et al. 2023), which transforms the SOC Hamiltonian block into di- rect sum of vector-form representations aligned with the symmetry of E(3). The transformation is applied to ( follows:
(0) for each atom
(node-init)
Eckart Inv Bow_WWgger..m Eckart (H na
(15)
E.2 INITIAL EDGE FEATURES
To construct the initial edge features, we first extract the Hamiltonian block H (0)
corresponding to
the interaction between atoms and from the zeroth-step Hamiltonian H))) We then apply the
inverse transformation of the Wigner-Eckart Eckart layer (Gong et al. 2023) to convert H ab
into direct-
sum state:
(0) Inv_ Miv_Wigner_Eokat H (0) ab
(16)
Next, we apply a spherical harmonics transformation to the normalized displacement vector Fab = (ra ra) encode the directional information between atoms a and b. The spherical harmonics where and rs represent the position vectors of atoms Fab
function is defined as Yau Y a and b, respectively.
Fak.
Next, we introduce a parameterized Clebsch-Gordas decomposition (as defined in Definition of Appendix B). which is applied to the tensor product of the combined edge features and the spherical harmonics functions. The resulting edge feature is computed as:
p (edgo-init)
= CG GGG_Decompp
((rioodd((iiiiiii + h
(0) ab @ 1 ((node-imit) & Yaui W),
(17)
ab
22

---

Published as a conference paper at ICLR 2026
where W are the weights derived from Gaussian expansion of the displacement vector magnitude Tabl This weights are defined as:
(-(1rab [[rabbldde) dx) 20
W Gaussian = exp
1 k K,
(18)
where de (for 1 < A S K) corresponds to set of reference distances, and σ% controls the width of the Gaussian function. These weights modulate the Clebsch-Gordan decomposition, enabling the incorporation of distance information between atoms and b.
Eq. (17) integrates the initial node features, the interaction captured by the zeroth-step Hamilto- nian, and the directional and distance information from the displacement vector into unified edge descriptor, i.c., g(odge-initt The use of tensor products and parameterized Clebsch-Gordan decompo- sition allows to efficiently combine these various types of information in form that is compatible with the input form equivariant neural networks. This enables the model to effectively capture both local atomic environments and interatomic interactions in expressive, physically informed manner.
F INTRODUCTION OF RECIPROCAL SPACE DIACTRONIC-STRUCTTOR HAMILTONIANS INTO DEEP LEARNING PARADIGM
The self-consistent Hamiltonian H obtained through the procedure described in Section A is inherently defined in real space. Its matrix elements H are constructed over localized atomic orbital basis functions centered at atoms, and are truncated beyond spatial cutoff. While real- space representations are efficient for representing local interactions, many physical phenomena such as band structures, effective low-energy models, and quasiparticle dynamics are most naturally described in reciprocal space.
To obtain a reciprocal-space
Hamiltonian, we perform a Fourier transformation of the real-space matrix elements. For a periodic system with lattice vectors (R), the Bloch Hamiltonian H (k) at wavevector k is defined as:
Ha. (k) Σ e k. H RR),
(19)
R
where i the imaginary unit (62 = -1), and (R) denotes the real-space Hamiltonian matrix element between orbital in reference unit cell and orbital 3 in cell displaced by lattice vector R. These elements are directly taken from the converged real-space Hamiltonian H²2) defined over the localized atomic orbital basis, with each pair of orbitals uniquely associated with a displacement vector R. For simplicity, we omit the superscript (T)) in Eq. 199 with the understanding that all real-space matrix elements originate from H
Diagonalizing (k) at each wavevector k in the Brillouin zone yields the system's electronic band structure:
Hkk = (k)
(20)
Let (k) E C= denote a Hermitian matrix that approximates H (k). It can also be solved through generalized eigenvalue equation to obtain the eigenvalues and wave functions.
H (k) Skk))k)
(21)
where Enk and Enk diagonal matrices of eigenvalues, and t'nk and Dink are the corresponding eigenvectors. Define A (H(k) H (k) ((k)). Assume spectral gap 8 separates the generalized eigenvalues of H (k) and (k). x(-) denotes the condition number a given matrix. l1 represents the spectral norm, || (k) = Σi, [AH;;(k)), Then, the difference in eigenvalues and the
angle 0 between the eigenspace of (k) and (kk satisfy:
1. Eigenvalue Differences:
Enk - Enkl < k(((())
[[((k)]]2
23

---

Published as a conference paper at ICLR 2026
2. Eigenspace Angle:
x(S(k)) [ [5(k)]2
sin 0
where 0 is the angle between the eigenspaces corresponding to Enk and Enk. The theorem (Golub & Van Loan 2013) highlights that due to the non-orthogonality of the orbital basis set, the errors in band energies and wave functions can be amplified by the condition number factor s(S(k)) [[S(K]]|a As
result, even small error may cause the band eigenvalues and wave functions deviate significantly from the true results, manifesting as the appearance of ghost states in the band structure.
To mitigate the amplification of perturbations in the predicted results caused by the condition num- ber, a feasible approach is to perform a basis transformation for the Hamiltonian matrix H (k) by introducing projection operators (fk) formed from the complete of eigenstates fnkk thereby transforming (k) into a diagonal representation. From a physical perspective, the low-energy subspace near the Fermi level governs essential material properties such as optical, thermal and transport behaviors. Accordingly, the projected Hamiltonian H (k) can be decomposed into three parts of the projection space:
Low-energy subspace Hrr(k): P(k) are spanned by Np eigenvectors (Vink) with ener- gies below the cutoff energy, the (()) are projected into P(k) space.
High-energy subspace Hgg (k): Q(k) are spanned by the remaining No eigenvectors above the cutoff energy, the H (k) are projected into Q(k) space.
Coupling subspace P (k): the off-diagonal coupling between P and Q, encoded in the cross blocks of the full Hamiltonian.
Let P(k) & NxNr and Q(k) E NNNN the matrices whose columns are orthonormal eigen- vectors spanning the low- and high-energy subspaces, stacking the bases as
u(k) = [P()) 2(k) E CC (NN
and assuming Np + NQ- = N (())(((K)(())) = 1), the Hamiltonian in the (P, basis is obtained by single similarity transform:
Hkk = u(k) H(K)U(() = Hpr(k) (r)(k)
(Q) (kq
For the ground-truth Hamiltonian, when transformed by its own eigenbasis 4(k), the cross block vanishes, i.e., (k) = O. In contrast, when predicted Hamiltonian is projected onto the sub- spaces defined by the ground-trute eigenbasis, the mismatch between the predicted and exact eigen- vectors may produce spurious non-zero entries, H (k) A 0. These unphysical couplings manifest artifacts such ghost states, and thus provide meaningful signal for penalization during train- ing.
Because the eigenvalues of H(k) directly define the band structure, reciprocal-space supervision provides a natural training signal. We therefore assign distinct loss terms to the three components.
The low-energy block Hk)(k) governs the states near the Fermi level and thus dominates observ- able physics; accurate supervision this block is crucial. The high-energy block Hoq (k) does not directly determine low-energy phenomena, but maintaining its fidelity is important: otherwise emrors in Q may propagate indirectly through erroneous PQ couplings. Finally, the cross block Hro (k) should ideally vanish; we enforce this by adding an explicit penalty on H (k) which suppresses unphysical couplings between P and Q, thereby eliminating ghost states and restoring the intended decoupling of subspaces.
G DETAILS ON TRAINING LOSS FUNCTIONS
We elaborate the details of Eq. in the following equation:
24

---

Published as a conference paper at ICLR 2026
loss R = ER (R(AR( ( Xc) loss (R) + y (1oss lOSST; Ac) -lossT(R))
loss [ = MSE (R). RR p)):
(22)
lossy R MAE T (R)) T [R. ()),
1oss (R)
(losS loss Ac) Ac No. Grad
loSsT (R)
where RR is a hyper-parameter, R denotes the lattice vector connecting the reference unit cell and a neighboring unit cell. (R)) and T(R) denote the predicted Hamiltonian and its corresponding trace quantity in real space, respectively. Here, compute H(R) as:
HR)- H (R) AH(R)
where (H(R) is the predicted correction term of the Hamiltonian.
The ground truth Hamiltonians are denoted as H HR) = (R). However, rather than directly
using these ground truth values to supervise H()), we construct augmented supervision targets by introducing additional term:
= - (R) S(R),
(23)
where μ is a scalar coefficient, (R) denotes the real-space overlap matrix, and ( denotes the real-space zeroth-stee step Hamiltonian matrix. Following the gauge-error formulation of Wang al. (2024)). adding a shift term (S) to the Hamiltonian leaves all down-stream physical observables unchanged. In practice, is chosen the solution that minimizes the overall loss, established in Wang et al. (2024c). This removes the gauge freedom inherent in the Hamiltonian representation, facilitating more stable and efficient convergence of the neural network.
The corresponding trace quantity used as the supervision signal is computed as:
T (R. v1) = tr (AH (R. u)) AH (R. k)))
(24)
= (H RR, 11) (0) (R)) (AH* (R. p) ( (R))')
where (0) (R) denotes the real-space zeroth-step Hamiltonian matrix.
Inspired by Yin 2025), the scaling factor >(loss H lossy, Ac) Eq. designed to har- monize the contributions from the two loss terms, ensuring stable optimization. Here, Ac is hyper- parameter that controls the overall strength of the balancing mechanism. The term No. No. Grad(.) en- sures that gradients are dropped during the computation of this coefficient, preventing interference lossT(R)- By applying this balancing strategy, better numerical stabil-
with the back-propagation ity and balanced learning performance across both the Hamiltonian and trace quantity supervision branches can achieved.
We elaborate the details of Eq. (2) as follows. Let H(k) denote the predicted full Hamilto- nian in reciprocal space, obtained from the Fourier transform of H(R) using Eq. 19 Simi- larly, let Ho*k,, μ)) denote the ground-truth Hamiltonian in reciprocal space, obtained from the Fourier transform of H (R, Both Hamiltonians are projected by the ground-truth eigenbasis u(k) [P)),, ]. yielding block- partiiaatii forms:
((k) = u()) (())((k) = HpP(K) HpQ(k)]
(Q)(k) HQq
H u)) (k, (u)
Hog (k,
H (k, n1) (k)t T (K)
Hg (k, (k) 11)
For the exact Hamiltonian, the off-diagonal block ideally vanishes, i.c., H for the predicted Hamiltonian, spurious non-zero entries generally appear in (pQ(k), manifesting unphysical ghost states.
HP (k, R)) = 0, whereas
25

---

Published as a conference paper at ICLR 2026
The loss is then defined block-wise:
loss(k) Ex Ap (SP(Hpp((K Hp n))
Ao MSE (k), H (k, ni)
(25)
+ Arq MSE ( Hp (k, mi)
where Ap, Xe, APQ are tunable terms.
controlling the relative importance of the three
The overall loss function combines the losses from both R-space and k-space:
loss =loss R) + loss(k)
ER(AR (1 Ac) lossH RR) + (1osSH lOSST: Ac) -loss (Ros)(RR))
(26)
+ Ex/Ap losss (k) + do loss@ + XPQ loss (k))
where the value of j is determined by alossall 8y
0. It can be solved analytically by:
AR
((((R)
H (R)
2
+ n² S(R)a as
N
R
2uRee
(H(R) H [R)(R)
AP
Hpp(k) (k)p(() aBl
+ +1²0 1d00
k,a9
2uRe [Flp((k) H (k)pp (k)lāsdas)
(27)
(Qq(() -H@0 (k)
agl l' + p200s
No
k.a
22uRe (Hee(k) Hot (k) (( ]
XPR
Σ
[Hpp((()- - -Ht% 19) ( =
Np k,a
which obtains:
A
A
An Σ Re (H(R)
A1 NR R.o8
H (R))
AP
Σ Hpp((k)
-Hop (k))an
Np k.a
(28)
da
(k) -Hor (k)laa
No
k.n
AR
Ap Np
Xg No
A2
S(R) + R.03
NR
where * denotes the complex conjugate operation, NR, Np, No, and Npo denote the total number of Hamiltonian matrix elements corresponding to real space, P space, Q space, and their coupling space respectively. (k) and H (k) are computed from the ground-truth Hamiltonian H**(k)
QQ
by:
Ht (k)
u(k) H (k)
=
0
Hig (k)
26

---

Published as a conference paper at ICLR 2026
It is important to clarify that, in the analytical derivation of μ in Eq. the real-space contribution be written in simplified form as:
Σ [(A())) apl | ++ + 22uR (B)) NR
An
Ag-lossiy (R)
R.as
rather than explicitly retaining the trace supervision term lossT(R) like:
AR (11 (1->c) loss (R) + y(losHH loss Ac) losst (R))
This simplification purely at the algebraic and notational level and does not imply that losst (R) omitted. In fact, balancing factor 7 (loss lOSST, Ac) A Nc-No_Grad
[osssr
guarantees [o.)
that this weighted combination of loss (R) and lossy (R) numerically equivalent loss (R), in which a fixed fraction of the contribution has been substituted by lossy-(R in stable and adaptive manner. other words, lOssT serves as surrogate for controlled fraction of loss while after normalization the effective value of the entire term remains consistent with loss (R). Therefore, in the derivation of μ1, it is sufficient and mathematically consistent to retain only loss (R), while the beneficial regularization effect of losst is still fully incorporated through the design of 7())
H DATASET DETAILS
To construct Soarrial--HM--SO--- the first-principles calculations are performed using the Atomic-
Orbital Based Ab-initio Computation at USTC (ABACUS) at. Lin et al., (2023) package. The
Erikeeof (PBE) eechange-correlatinn functional (Perdew et al.) 1996) and the optimized nom--coneerving Vanderbilt (ONCV) fully relativistic pseudopotentials (Hamann 2013)
from the PseudoDojo library (van Setten al. 2018) used. Table 2 summarizes the valence electron configurations used in the pseudopotentials and the corresponding numerical atomic orbital (NAO) basis for each element. self-consistent calculations, the energy cutoff for wave functions is set to 120 Ry and the charge density was converged to threshold of Ix10 6 The I l'-centered Monkhorst- Pack 6 k-point mesh is employed for self-consistent calculations.
The crystal structures were obtained from the Materials Project database, from which total of ap- proximately 17,000 nonmagnetic compounds were randomly selected. Among them, 12,000 struc- tures were used for training, 2,000 for validation, and 3,000 for testing. The statistical distributions of atomic species and atomic counts in the training, validation, and test sets are illustrated in Fig- ardd Furthermore, the occurrence frequencies of different elements across the three subsets are presented Figures 7c
We visualize representative crystal structures in Fig. highlighting the diversity and broad coverage of our curated dataset
HAM-SOC. These samples span a wide range chemistries, crystal symmetries, and atomic complexities, illustrating the richness of the dataset and its suitability for training universal Hamiltonian prediction models.
Table 2:
The valence electron configurations for pseudopotentials and corresponding NAOs of the elements used in this study.
Element Number
Element Name
Valence Electrons
NAOs
Cutoff Radius
001
H
1s_
2s1p
a.u.
002
He
2
2slp
a.u.
003
Li
12
4slp
a.u.
004
Be
1s225²
4slp
a.u.
005
B
2s22p'
2s2pld
a.u.
006
C
2s22p²
2s2pld
a.u.
007
N
2x22p²
2s2pld
a.u.
008
O
2x²2p"
2s2pld
a.u.
009
F
2s22p"
2s2pld
a.u.
010
Ne
2s
2s2pld
a.u.
011
Na
2s 2522p
4s2pld
a.u.
27

---

Published as a conference paper at ICLR 2026
Element Number
Element Name
Valence Electrons
NAOs
Cutoff Radius
012
Mg
2p22p6362
4s2pld
a.u.
013
AI
3s"
2s2pld
a.u.
014
Si
3p
2s2pld
a.u.
015
P
3s"3p"
2s2pld
a.u.
016
S
3s"3p'
2s2pld
a.u.
017
Cl
35"
2s2pld
a.u.
018
Ar
3s"3p"
2s2pld
a.u.
019
K
3s 3s
4s2pld
a.u.
020
Ca
323p'443
4s2pld
a.u.
021
Sc
3: 3p"
4s2p2d1f
a.u.
022
Ti
35 3p"
4s2p2d1f
a.u.
023
V
38 3s23d
4s2p2d1f
a.u.
024
Cr
38 3d333d
4s2p2d1f
a.u.
025
Mn
38 3d33dd
4s2p2d1f a.u.
026
Fe
38 3p333d
4s2p2d1f a.u.
027
Co
38 "d
4s2p2d1f a.u.
028
Ni
38 3d
4s2p2d1f
a.u.
029
Cu
38 3s23pd
4s2p2d1f a.u.
030
Zn
3s 3p
4s2p2d1f
a.u.
031
Ga
3d 4p2
2s2p2d1f
a.u.
032
Ge
3d 4p2
2s2p2d1f
a.u.
033
As
4s2
2s2pld
a.u.
034
Se
4s"
2s2pld
a.u.
035
Br
4824p
2s2pld
a.u.
036
Kr
4824pb
2s2pld
a.u.
037
Rb
4s"
4s2pld
a.u.
038
Sr
4s"4p"5s²
4s2pld
a.u.
039
Y
4s2
4s2p2d1f a.u.
040
Zr
4d2
4s2p2d1f
a.u.
041
Nb
4d"4
4s2p2d1f
a.u.
042
Mo
4d24p"5$2dd
4s2p2d1f
a.u.
043
Tc
4s"4p"5524d
4s2p2d1r
a.u.
044
Ru
4s"4p555"4d
4s2p2d1f
045
Rh
4s24p5552dd"
4s2p2d1f
046
Pd
4d"
2s2p2d1f
a.u.
047
Ag
4s"4p"5""dd
4s2p2d1f
a.u.
0448
Cd
4p
4s2p2d1f
a.u.
049
In
4d 5s"5p"
2s2p2d1f
a.u.
050
Sn
4d 5p"5pp
2s2p2d1f
a.u.
051
Sb
4d 5522p
2s2p2d1f
a.u.
052
Te
5s"5p
2s2p2d1f
a.u.
053
I
5s"5p"
2s2pld
a.u.
054
Xe
5s 5p
2s2pld
a.u.
055
Cs
5s 5p
4s2pld
a.u.
056
Ba
5s 5p
4s2p2d1f
a.u.
072
Hf
5s 5s
4s2p2d2f
a.u.
073
Ta
5s 5d2
4s2p2d2F
a.u.
074
W
5525p"
4s2p2d2r
a.u.
075
Re
5s25p'
4s2p2d1f
a.u.
076
Os
5s"5p"
4s2p2d1f
a.u.
077
Ir
5s"5p"
4s2p2d1f a.u.
078
Pt
5s os
4s2p2d1f
a.u.
079
Au
5625p"ds25d"
4s2p2d1f
a.u.
28

---

Published as a conference paper at ICLR 2026
Element Number
Element Name
Valence Electrons
NAOs
Cutoff Radius
080
Hg
4s2p2d1f
a.u.
081
T1
6s 6p
2s2p2d1f
a.u.
082
Pb
6s 6p
2s2p2d1f
a.u.
083
Bi
6s
2s2p2d1f
a.u.
I IMPLEMENTATION DETAILS
Our NextHAM framework is implemented based on PyTorch 2.2.0, E3NN 0.5.6, and CUDA 12.1. The training was conducted GPU cluster equipped with NVIDIA A800 GPUs, each with 80 GiB memory.
For the input of the neural network, we adopt distance of 8.0 Á to define the neighboring range in the atomic graph. The angular relations between atoms are represented using spherical harmonics with degrees 0 < 1 < 5, while the interatomic distances are encoded through Gaussian basis expansion (Gong et al. 2023) with a preset base number of 64. The Transformer network consists of 4 stacked basic blocks. Each block contains E(3)-ymmetrry layer normalization module, an E(3)-symmetry feed-forward module, an E(3)-symmetry multi-head graph attention module, and a TraceGrad module. For the first three blocks, the internal node features f(made) and edge features (e(odge) p'(odge) and ab (edge)) are represented in a direct-sum state with a total of 392 channels. In the ab
final block, apply tensor product and decomposition to ab p'(edge) and (edge) to lift the representation to higher angular momentum degrees, constructing tensor representations that correspond to the atomic orbital basis sets up to 4s2p2d1f. For the TraceGrad module, the constructed 033)-invariant feature =ab has dimension of 256. On the output side, to map the network outputs from the direct-sum E(3)-symmetric tensors into Hamiltonian matrices, we employ the conversion modules provided by Gong et al. (0023), thereby ensuring the exact symmetry of the predicted results with SU(2) symmetry. We employ an ensemble of four sub-models to predict the electromic-structure Hamiltonian. The first sub-model by atomic pairs with interatomic distances in the range [0, 1.0 A), where the case of distance equal to zero corresponds to the on-site Hamiltonian (i.e., the Hamiltonian formed by an atom with itself). The second sub-model handles atomic pairs with distances in the range [1.0 A, 2.0 A), the third sub- model covers the range [2.0 A, 4.0 A), and the fourth sub-model addresses the range [4.0 A, 6.0 A). For atomic pairs with distances greater than 6.0 Á, we found that their self-consistent Hamiltonian almost identical the zeroth-step Hamiltonian numerically. Therefore, for these distant atoms, we bypass the neural network correction step and use the zeroth-step Hamiltonian as the final result.
responsible for predicting the Hamiltonian submatrices formed
In the training stage, each card assigned to one of the sub-models. In our training strategy, elec- tronic states S 10 eV above the Fermi level are included in the low-energy subspace P, while the remaining states are divided to the high-energy subspace Q. We train the model for a total of 100 epochs on the training set and evaluate the checkpoint that achieves the best performance idation set for testing. The hyper-parameters for loss functions are set as Ào = 0.2, A = 0.99955, Ap 0.0002, Ae 0.0001, and λpQ = 0.00015, determined according the performance on the validation set. We adopt the Adam optimizer with an initial learning rate of 5 x 10-1. A warm-up phase of 5 epochs linearly increases the learning rate from x 10 to the base value, followed by cosine decay to a minimum learning rate of 1 x 10-5 by the end of training. To mitigate stochastic variations, we fix the random seed to throughout model training and inference.
the val-
J BAND STRUCTURE RESULTS
We examine the accuracy and physical reliability of the band structures predicted by our method, by comparing the results obtained from three different Hamiltonians on representative testing samples spanning diverse elements and structures, as illustrated in Fig. [9] The red curves correspond to the ground-trute truth bands derived from the self-consistent Hamiltonian H H(T; the blue curves correspond to the bands obtained from the zeroth-step Hamiltonian H();; and the orange curves
29

---

Published as a conference paper at ICLR 2026
80
Dataset
train
62.5
60
Percentage (%)
test
40
28.3
24.8
22.9
20
11.3
1.4
3.0
2.2
1.8
2
>5
Figure 5: Bar charts of elemental species distributions the training, validation, and test sets.
80
Dataset
train
60
59.5
val
Percentage (%)
test
47.4
40
20.5
21.9
21.0
20
15.4
12.2
12.1
12.8
1-5
6-10
11-30
>30
Figure 6: Bar charts of atomic count distributions in the training, validation, and test sets.
represent the bands obtained from the predicted Hamiltonian of our full method, = H + AH. The results show that the zeroth-step Hamiltonian ()) provides only rough sketch of the band structure: it approximately captures the overall positions and qualitative trends of the bands, but suffers from noticeable deviations in curvature and energy levels. In contrast, after applying neural corrections, the predicted Hamiltonian H yields band structures that align almost perfectly with the DFT ground truth, showing no significant deviations. This striking agreement demonstrates the practical value our method for materials science and technology, where obtaining accurate band structures is central problem.
K EFFICIENCY COMPARISON BETWEEN OUR METHOD AND DFT
We evaluate the efficiency of NextHAM against the conventional DFT workflow on the same Linux server equipped with Intel(R) Xeon(R) Silver 4114 CPUs@ 2.20 GHz and NVIDIA A800 (80 GiB) GPUs. All DFT computations are executed on the CPU, while the neural inference of NextHAM evaluated on both CPU and GPU. On the CPU, both DFT and NextHAM are run with four CPU cores in parallel. On the GPU, use four A800 cards: each card executes sub-mode model, and the outputs are aggregated on a single card. The testing batch size is fixed (no batching). We report the minimum, mean, and maximum wall-clock times across all testing samples.
neural-network
30

---

Published as a conference paper at ICLR 2026
He
870
2007
1202
309
Ne
Mg
Ar
s4V
240
300
404
59V
444
109
Ra
Sr
Zr
Nb
Mo
Tc
Fl
Rh
Pd
Ag
Ir
Sn
Te
Xe
(a) Training
No
Mg
Al
S
CI
Rb
No
Mn
Tc
Ra
Rh
Pd
Ag
Cd
Sn
Sb
T+
X=
(b) Validation set
B4
No
Sc
TI
Cr
Mn
Fa
Co
Ni
Cu
ZA
Ga
Ge
Aa
Se
Br
Kr
100
179
163
132
131
141
130
Hr
In
W
Rie
Os
Pt
Au
Hg
14
P0
A2
Rn
10
12
14
16
18
(c) Testing set
Figure 7: Statistical charts of element occurrence frequencies in the training, validation, and test sets.
31

---

Published as a conference paper at ICLR 2026
Figure 8: Representative crystal structures sampled from the Materials-HAMM-SOE dataset. The ex- amples cover diverse chemical compositions, structural patterns, and atomic configurations, demon- strating the dataset's 's broad coverage across the periodic table. Such diversity ensures that the bench- mark provides comprehensive foundation for training and evaluating universal Hamiltonian pre- diction models.
Table 3 summarizes the runtime results:
For DFT, the entry O @ CCU (stage 1) includes reading the structural inputs from disk and structing the zeroth-step Hamiltonian H))) from scratch. This stage performs no diagonalization. The entry SC@ CPU (stage 2) measures the self-consistent (SC) loop that starts from ())) and iter- ates to the converged (2)) with repeated matrix diagonalizations. Writing the final results to disk also included this stage. The entry Total: ( @CPU SC@CPU is the total runtime for the
DET workflow, i.e., the sum of the stages.
32

---

Published as a conference paper at ICLR 2026
10.0
7.5-
Only H
Full Method
5.0
(-Er
2.5
-7.5
-10.0
(a)
Ground
Ground Truth
Only
5.0
E-E: (ev)
2.5-
0.0
-25
2
AG
MA
10.0
Graund Truth
Ground Truts
7.5
FUl Method
E-Er (eV)
5.0
-7.
-10.0
XSRA
(c)
7.5
Only 10
5.0
(ev)
2.5
0.0
-10.0
MLA
811
(d)
33

---

Published as a conference paper at ICLR 2026
10.0
7.5-
Only H
Rull Method
5.0
(eVr
2.5
-7.5
10000
SX
Ground Tuth
Only
5.0
E-E, (ev)
2.5
0.0
25
6
M
10.0
Ground Tuuhh MIRE
Ground Truth Full Nethidd
7.5
E-Er (eV)
5.0
-7.
10.0
7.5
5.0
(ev)
255
0.0
-10.0
(h)
34

---

Published as a conference paper at ICLR 2026
7.5
Onty Hill
Full Nethoo
5.0
E-Er
2.5
-7.5
-10.0
Ground
Ground Tuth
5.0
E-E, (ev)
2.5
0.0
25
M
L H
H M
10.0
Ground TTUCH OnTY HITH
Ground Tuuhh
Full Method
7.5
E- (eV)
5.0
-7.
10.0
kk)
7.5
Onty
5.0
(ev)
2.5
0.0
-10.0
(
35

---

Published as a conference paper at ICLR 2026
10.0
Ground Truch Only
Ground Tuth Method
E-Er (ev)
0.0
-2.5
5.0
UX
10.0
Glound
Ground Fall Mathod
7.5
Only bill
5.0
(ev)
2.5
-5.0
-7.-
-10.0
(n)
Truth
Ground Thuts
Orly
E-E iev)
25-
0.0
25
6
H
(0)
Figure 9: Comparison of band structures obtained from Hamiltonians of representative testing sam- ples. For each subfigure, the left and right panels show different comparisons. both panels, the red solid curves correspond to the ground-truth bands derived from the self-consistent Hamiltonian Hot (T), In the left panel, the black dashed curves represent the bands from the zeroth-step Hamiltonian ()), In the right panel, the blue dashed curves represent the bands from the predicted Hamiltonian of our full method, H ( + AH.
For NextHAM, the stage I, i.e., ()) @ CPU has the same meaning as in DFT: the cost of construct-
ing ()))) from the initial electron density. The stage 2, i.e., NN@
NN@ GPU, covers the
full inference workflow after (0) is available: loading H (0) into the model's input tensors, running the neural-network forward pass to predict AH, processng the outputs into DFT-compatible
Hamiltonian format, and writing the results disk. The rows Total: H @CUU NN @ CPU and
36

---

Published as a conference paper at ICLR 2026
Table 3: Runtime on the testing set of Materials-HAM-OOC (min/mma//mean seconds per sample). All stage timings include the data I/O associated with that stage. Note that the total times are computed per sample as the sum of the corresponding stages; therefore their min/max need not equal the sum of the per-stage minima/maxima.
Method
Stage
Min (s)
Max
Mean
H @ CPU
3.14
742.43
55.46
DFT
SC@CPU
16.01
28397.45
2251.64
Total: CCPU
21.86
28617.18
2307.11
H @CPU
3.14
742.43
55.46
NN @ CPU
5.15
26.92
12.62
NextHAM
NN @ GPU
1.16
8.95
3.01
Total: H (0) @
12.69
755.84
68.08
Total: @CCUU NN@GPU
4.84
744.66
58.47
Total: O @CPU+ NN@ report and-to-end runtimes of NextHAM with neural inference on CPU or GPU, respectively.
From Table 3 we could observe that, NextHAM is substantially faster than the conventional DFT wall-clock time drops from 2307.1 (DFT total) to
pipeline. Using GPU inference, the 58.47s (97.4% time reduction). Even with CPU inference, the mean time is 68.08 (97.0% time reduction). In the worst case. the total runtime decreases from 28617.18 to 744.66 with GPU inference (97.3% time reduction), and to 7 with CPU inference (97.3% speedup).
Within the DFT workflow, the self-consistent (SC) stage constitutes the overwhelming majority of the runtime, accounting for 97.6% of the mean total (2251.64s out of 2307.11 and 99.2% of the observed maximum (28397.45 out of 28617.18 This observation consistent with its algorithmic structure: each SC iteration entails dense matrix diagonalizations with computational complexity ( leading to an overall cost of O(TN*). where N denotes the atom number in cell and I" denotes the number of SC iterations. Since T may be very large and is strongly problem dependent, with no reliable a priori bound on convergence, wall-clock times are both substantial and difficult to predict, and the worst-case runtime can be prohibitive. By contrast, NextHAM avoids the iterative SC loop entirely. As discussed in previous sections, constructing H scales with the number of non-zero Hamiltonian elements and O(N2) for small systems, crossing over toward ((N) for sufficiently large ones; the neural inference follows the same scaling and produces a result in a single forward pass. This one-shot computation makes the runtime more predictable and markedly lower in both mean and worst-case scenarios. Moreover, neural inference benefits strongly from hardware parallelism: switching from CPU to GPU significantly reduces the mean inference time.
is worth noting that our testing set does not contain very large systems, and the number of non-zero Hamiltonian entries typically scales N2 (many atoms fall within each other's cutoff). Even in this less favorable sparsity regime, NextHAM already delivers the large speedups reported in Table 3 For substantially larger systems, the neighbor count of each atom saturates and the total number of non-zero elements grows only O(N), both ()) construction and neural inference become near-linear, while DFT remains O(TN3) with an a priori unknown iteration count T'. Hence the efficiency advantage of NextHAM over DFT should increase further scale the system becomes larger. We point out that in our current CPU implementation the construction of ()) accounts for a large portion of the runtime. Fortunately, this step requires no matrix diagonalization and can be carried in highly parallel fashion. In future work, we plan exploit GPU-based parallel algorithms for H preparation, which is expected to dramatically reduce this overhead and further amplify the efficiency advantage M.XtHAM. We leave these works as future work plans.
Overall, the combination of favorable scaling, Lngle-passs prediction (no SC iterations), and effi- cient GPU parallelization enables NextHAM to deliver large speedups across the board, opening a practical path to high-throughput materials simulations.
37

---

Published as a conference paper at ICLR 2026
Table 4: Comparison of Gauge MAE computed in real space (R-space) for different ablation terms and the full method on the testing set of Materials--AM-SOC. Metrics are averaged over non-zero elements only; entries set to zero due the truncation distance are masked out. All values are in meV.
Method
Gauge MAE (meV)
Ablation @ Input
1.720
Ablation @ Output
2.974
Ablation @ TraceGrad
1.789
Ablation @ Ensemble
1.862
Ablation @ Loss-k
1.615
Ablation @ Loss-PQ
1,496
Full Method
1.417
L ABLATION STUDIES
We conduct fine-grained ablation studies for our framework by comparing the following settings. All ablation variants are implemented by removing single component from the Full Method of NextHAM, while keeping all other settings identical, so as validate the effect each component:
Ablation @ Input: In this ablation term, we replace the zeroth-step Hamiltonians in our input descriptors with conventional atom (node) and atomic-pair (edge) embeddings. Specifically, for an atom of chemical element Ze, maintain learnable 32-dimensional embedding vector e, ez. E R:3 randomly initialized and updated during network training. The embedding of an atomic pair (a, b) is the concatenation of the two element embeddings, eab (ez. ez, R64
Ablation @ Output: In this ablation term, the residual learning scheme, which the network pre- dicts the correction term AH (2) H)),, removed. Instead, the neural network is trained to directly regress the full seff-consistent Hamiltonian H(?), following the setting commonly adopted in existing deep learning approaches for Hamiltonian prediction. This ablation allows us to examine the effectiveness of using AH as the output target in reducing the complexity of the regression space and improving generalization.
A Alltion @ TraceGrad: In this ablation term, remove the TraceGrad mechanism. Concretely, the supervision from the trace quantity is omitted in the loss function, and the gradient-based mech-
anism that delivers non-linearity from O(3)-invariant features (edge) to induce 0(3) - fea- (elge)
tures via prpepp is also discarded.
Ablation @ Ensemble: In this ablation term, we remove the ensemble mechanism based on dis- tance ranges. Instead of training multiple sub-models specialized for different interatomic distance intervals and aggregating their outputs, a single neural network used to predict all Hamiltonian correction terms across all distance ranges.
Ablation @ Loss-k: In this ablation term, we remove the k-space loss terms and train the neural network using only the real-space loss, is commonly used in most of the existing deep learning approaches for Hamiltonian prediction. This setting allows us to assess the contribution of the k- space supervision in improving the physical fidelity of the predicted Hamiltonians and the resulting band structures, particularly in eliminating ghost states.
Ablation @ Loss-PQ: This variant retains the k-space supervision the intra-subspace blocks (P and Q) but removes the cross-we cross-subspart subspace coupling penalty, i..., we set Arq = 0. This ablation isolates the role of the PQ term.
Full Method of NextHAM.
We train all of the ablation terms under the same number of epochs, optimizer, and scheduler as the full method (see Appendix M. then evaluate them on the testing set. The R-space errors are summarized in Table 4 Beyond R-space, because k-space is directly tied to downstream quan-
38

---

Published as a conference paper at ICLR 2026
tities (e.g.. band structures), we visualize band predictions for Ablation @ Loss-k (R-space only), Ablation @ Loss-PQ (setting APQ = 0) versus the Full Method in Fig. 10
From Table 4 the Full Method achieves the lowest Gauge MAE. The Full Method duces the error by 17.6%. 52.3%. 20.7%, 23.8%. 12.2%. and 5.2% compared with Abla- tion @ Input, Ablation @ Output, Ablation @ TraceGrad, Ablation Ennsembe,,,,, Ablation @ Loss- k. and Ablation @ Loss-PQ, respectively. As shown in Fig. [0] Ablation@ Loss-,,, which removes the k-space supervision and relies solely on real-space loss, produces band structures with frequent ghost states: in many cases, while most k-points are predicted reasonably well, some k-points ex- hibit abrupt and severe deviations from the ground truth hallmarks of non-physical artifacts. This phenomenon mainly arises from the error amplification effect analyzed in Appendix [ where the large condition number of the overlap matrix can magnify small real-space errors into significant k- space deviations. Importantly, such sparse but catastrophic failures cannot be effectively captured by real-space loss alone. Ablation @ Loss-PQ, which augments the training with k-space supervision on the intra-subspace blocks (P and Q), demonstrates better performance than Ablation @ Loss-k, but still fails to completely suppress ghost states. The reason is that unphysical couplings between the low-energy subspace P and the high-energy subspace Q remain unpenalized, and these cou- plings are precisely what give rise to unphysical artifacts in the band structures. In contrast, the Full Method introduces an important penalty on the PQ cross block, which has clear physical signifi- cance: for the exact Hamiltonian, P and Q are strictly decoupled, and any spurious PQ couplings in the predicted Hamiltonian are the direct source of ghost states. By explicitly enforcing this de- coupling, the PQ loss term addresses the root cause of the artifacts. As result, the full method produces band structures in excellent agreement with first-principles DFT and free of ghost states. This comparison clearly demonstrates the necessity of our k-space loss design, in particular the PQ penalty, for ensuring the physical reliability of predicted band structures.
These results collectively indicate that injecting the physically informed zeroth-step Hamiltonian as an input prior improves generalization, and predicting AH = HTT) - H) reduces the effec- tive regression space and eases optimization. They further confirm the effectiveness of the Trace- Grad mechanism: supervising with the trace quantity and propagating non-linearity from invariant to equivariant features enhances representation quality. Notably, this observation aligns with the findings of Yin et al. (2025) on simpler GNN backbones, and our results demonstrate that Trace- Grad remains effective within a Transformer-based framework. Moreover, the ensemble strategy, which partitions the regression space by interatomic distance and aggregates multiple specialized sub- models, yields mcasurable capacity gains over a single monolithic predictor. highlighting the benefit of distance-dependenn specialization. In addition, k-space supervision provides complemen- tary guidance that enhances physical fidelity, while explicitly penalizing the cross-subspace coupling (PQ) significantly suppresses band structure errors and eliminates unphysical artifacts. In summary, all validated components contribute both individually and synergistically to the overall performance of our method.
M COMPARISON WITH RELATED WORK
In the field of decctroni-sttuuctie Hamiltonian prediction for periodic material systems, one of the most representative methods is DeepH-E3 (Gong et al.) 2023). As a pioneering effort, DeepH-E3 has played crucial role in introducing equivariant networks the calculation of electronic structures of materials. We aim to perform a clear comparison with it to demonstrate the innovations and advantages of our approach compared with DeepH-E3.
First, DeepH-E3 uses randomly initialized element (node) and element pair (edge) embeddings trained from scratch as input descriptors. However, the sparse nature of these embeddings poses a problem. Since elements are not uniformly distributed in nature, many element pairs appear in- frequently, resulting in poorly trained embeddings that struggle with generalization. What's more, this leads to the need to maintain an embedding cost of (MM and O(M²) for nodes and edges, respectively, where M is the number of elements included in the dataset. As the number of elements increases, this cost grows rapidly, leading to significant memory overhead.
In contrast, our approach addresses these issues by replacing the node and edge embeddings with the zeroth-step Hamiltonian H(), which eliminates the space complexity and sparsity problems. ())) captures crucial information about the system's electronic structure, embedding the funda-
39

---

Published as a conference paper at ICLR 2026
5-
8
(b)
400
E-E-980
Fush
(e)
mental characteristics of different elements into unified representation space, thereby offering richer physical context. This enables robust generalization across a wide range of material sys- tems, making approach better suited for constructing truly universal model. What's more, the space complexity our input descriptor is independent of the number of distinct elements in the dataset, thus avoiding the exponential cost of embeddings. This of easily accessible DFT
40

---

Published as a conference paper at ICLR 2026
5.
Dovng
1991
2.2
3
E- E-E--00
8
41

---

Published as a conference paper at ICLR 2026
DOLITO
VEL0D EAB
2-5.000
SEOUIGG PAL
2-5. 5,15,000
2-25.00
(m)
VIUNO FAR
(n)
(0)
Figure 10: Comparison of band structure performance representative testing samples. For each subfigure: all panels, the red solid curves correspond to the ground-truth bands derived from the ground-truth self-consistent Hamiltonians. In the left panel, the black dashed curves represent the band structure results of the ablation term Ablation @ Loss-k, which exhibit ghost states, i.c., abrupt and severe deviations from the ground truth certain k- k-points. the middle panel, the green dashed curves correspond to the results of A Ablation@) Loss-PQ, where such artifacts are mitigated but not fully removed. the right panel, the blue dashed curves denote the predictions our full method, which successfully eliminates ghost states and achieves excellent agreement with the ground truth.
42

---

Published as a conference paper at ICLR 2026
Table Comparison of R-space errors for DeepH-E3 and our method the testing set of Materials- HAM-SOC. All values are in meV.
Method
Gauge MAE
DeepH-E3
12,605
Our Method
1.417
initial state tensor as descriptor highlights emerging direction in Al for Science: embedding into machine learning models, which enhances predictive
physically grounded information directly power and provides general principle that can be broadly applied across scientific ML tasks beyond electromic-structue prediction.
Second, the architecture of DeepH-E3 is relatively simple and lacks sufficient expressive power for building a universal model, which reflected in two aspects. First, it uses a non-attentive struc- ture, with feature fusion performed through simple average pooling. This may not be well-suited to handle highly complex, diverse structures. Second, it only employs the gating mechanism as the non-linear activation function, without fully exploring the non-linear expressiveness. In con- trast, our approach builds upon the TraceGrad paradigm with advanced framework to effectively induce non-linearity from invariant quantities and representations, and develops an E(3)-symmetry Transformer architecture with high non-linear expressiveness. This architecture is capable of dy- namically weighting features to adapt to different elements and geometric configurations, making more suitable for building universal large-scale models.
In addition, our network adopts a delta-learning approach, predicting AH = H H instead of the Hamiltonian itself. This significantly reduces the complexity of the regression target, making it easier for the network to finely fit and generalize complex systems, especially those with heavy atoms and a large number of orbitals. contrast, DeepH-E3 directly predicts the entire Hamiltonian, which becomes challenging when dealing with more complex scenarios.
Third, DeepH-E3 only uses the R-space loss function for training, while we propose a joint loss func- tion combining both R-space and k-space. As shown in Section Appendix the inclusion of k-space loss improves the physical reliability of downstream quantities, such band structure, and prevents ghost states, which appear as abrupt discontinuities at isolated k-points, caused by the am- plification of small R-space errors in k-space (Appendix) [F) By addressing these issues, our method is better suited for handling a wide range of materials and elements, making it more adaptable and reliable across various scientific applications.
We apply DeepH-E3 to our Materials- HaMisal-------OOCCCCC dataset, and follow the same -aaiiiii-aaiittiinnn testing pipeline on our dataset as described in Appendix H. the feature layers near the output layer and the output itself, construct tensor representations that correspond our dataset's 's atomic orbital basis sets, which extend to 4s2p2d1f, while leaving the rest of the DeepH-E3 unchanged. These modifications make the comparison more relevant and fair. Unfortunately, even with these adjustments, DecpH-E3 achieves high R-space егтоr on our testing set, failing to converge to reasonable error range, as shown in Table In contrast, our method achieves low error value. This result confirms that DeepH-E3 is not suited for the task of universal electronic structure Hamil- tonian prediction, as we have previously analyzed. The research in the DeepH-E3 paper focuses on specialized scenarios, where the training and testing sets consist of perturbations from molecular dynamics simulations the same material. In contrast, our task involves predicting the Hamiltonian across broad range of materials, which requires more generalizable approach that DeepH-E33's specialized design cannot handle effectively.
Furthermore, compare the performance in specialized scenarios, we conduct experiments on the Monolayer Graphene (MG) and Monolayer MoS₂ (MM) datasets, which are released by the DeepH series (Li et al. 2022 Gong al. 2023). introduced in Table
For both datasets, we use the same training, validation, and testing sets as those used in DeepH- E3. To ensure fairness, we retrain our method under identical conditions, without pre-training and measure errors using the classic MAE (Mean Absolute Error) metric, in alignment with DeepH- E3. In addition DeepH-EF3, we also compare method with the original TraceGrad work Yin
43

---

Published as a conference paper at ICLR 2026
Table 6: Overview of the Monolayer Graphene (MG) and Monolayer MoS₂ (MM) datasets. m.
number of samples in the current dataset; number of atoms per unit cell in the current dataset.
Statistic Types
MG
MM
270
300
Training
75
90
100
Validation
a
75
100
Testing
72
75
Table 7: Comparison of MAE values among DeepH-E3, the original TraceGrad work, and MextHAMolddonn All values are meV.
MAE
Dataset
DeepH--E3
Original TraceGrad NextHA dua
MG
0.251
0.175
0.102
MM
0.406
0.285
0.163
et al. 2025), which extends the DeepH-E3 backhone network by constructing non-linear equivariant representations from invariant ones.
The version of our method used in these experiments NextHAMezt.dow Since the datasets do not provide the (0) (zeroth-step Hamiltonian) label, we modify NextHAM by removing the components related to H Specifically, do not use H))) as an input descriptor. Instead, we use randomly initialized node and edge embeddings, which are trained jointly with the network, similar those used DeepH-E3, represent the elements and their pairwise interactions. The output directly predicts H rather than AH H H and because the datasets do not include wavefuncction-reltted data, we retain only the R-space loss function during training, omitting the k-space loss function. What remains is the E(3)-equivariant graph Transformer network architecture developed upon the TraceGrad paradigm. The comparisons are presented Table from which the results DecpH-E3 and the original TraceGrad work from Yin et al. 2025
cut-down version, denoted as
These results show that our method, even after cutting out some modules, still dramatically outper- forms both DeepH-E3 and the TraceGrad work in prediction accuracy for these material systems. While the original TraceGrad work explores effective methods constructing non-linear equivari- ant features, it remains simple graph network based average pooling. In contrast, our approach extends the TraceGrad paradigm as a more advanced E(3)-symmetry Transformer architecture that can dynamically weight and adapt non-linear equivariant features, which is more expressive than the simple graph network. In addition to demonstrating strong generalization across a variety of materials in previous sections, these results show that method excels in specialized systems as well, making it powerful tool for both broad and focused applications in computational materials science.
STATEMENTS REGARDING THE USAGE OF LLMs
No large language models were used for this work.
44


Reviews:
[{"review_id": "jtGtg2NUUi", "reviewer": "Reviewer_ts7f", "summary": "This paper proposes NextHAM, a deep-learning framework for predicting electronic-structure Hamiltonians of materials with near-DFT accuracy.\n\nThe method introduces a physically informed prior, the zeroth-step Hamiltonian $H^{(0)}$, derived from the initial electron density (superposition of atomic charge densities). The model learns only the correction $\\Delta H = H^{(T)} - H^{(0)}$, effectively simplifying the regression space.\n\nNextHAM uses a Transformer-based E(3)-equivariant architecture extended with the TraceGrad mechanism for non-linear expressiveness under strict SO(3)-equivariance, and jointly optimizes both real-space and reciprocal-space Hamiltonians to reduce “ghost-state” artifacts.\nA new dataset, Materials-HAM-SOC, is also released, covering 17 000 crystalline materials across 68 elements, computed with ABACUS + PYATB including explicit spin–orbit coupling.", "questions": "One point is unclear to me. In NAO-based electronic structure calculations, the reciprocal-space Hamiltonian $H(\\mathbf{k})$ is obtained through the Fourier transform of the real-space Hamiltonian $H(\\mathbf{R})$. However, the authors mention applying an 8 Å cutoff in their model architecture. This implies that for interatomic distances beyond 8 Å, the predicted matrix elements vanish, i.e., $H(\\mathbf{R}) = H^{(0)}(\\mathbf{R})$. If so, the Fourier transform would only include short-range interactions, and the long-range components in H(\\mathbf{k}) would be incorrect. How, then, does the model reconstruct a physically meaningful $H(\\mathbf{k})$ and corresponding band structure? Did I misunderstand the implementation, or is there an additional correction or extrapolation beyond the cutoff?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Strong Physical–ML Integration: The introduction of the zeroth-step Hamiltonian as both input and output prior is conceptually sound and reduces the learning complexity.\n2. Extends TraceGrad to an equivariant Transformer with enhanced edge-centric attention\n3. Training target in both real and reciprocal spaces\n4. Large-Scale and Diverse Datasets: Materials-HAM-SOC is the first open benchmark with SOC-included Hamiltonians across nearly the full periodic table, offering a valuable resource to the community.", "weaknesses": "1. Marginal Conceptual Novelty:\nThe introduction of the “zeroth-step Hamiltonian” is interesting, but it can be viewed as a variant of the superposition of atomic potentials or the initial DFT guess—concepts that have already appeared in physics-informed ML models. Hence, the main contribution seems to lie in the integration and refinement of known components rather than in introducing a fundamentally new idea.\n2. Venue Suitability:\nThe contribution of this work feels more aligned with a scientific journal in computational materials science than with the core themes of ICLR. The paper emphasizes methodical accuracy, dataset curation, and physical completeness, but it lacks the conceptual abstraction or general ML innovation typically expected at this venue.\n3. Limited Evidence for Claimed Efficiency:\nThe paper claims substantial computational speed-ups, yet it provides no quantitative scaling analyses (e.g., runtime versus system size) or benchmarks against established ML Hamiltonian predictors such as DeepH-2. The reported improvements remain qualitative. I understand that the focus here is on SOC-inclusive Hamiltonians—a domain rarely addressed in prior work—but even so, direct numerical comparisons with existing open-source models would considerably strengthen the experimental section.\n4. Unclear Dataset Generation Strategy:\nThe dataset construction choices appear somewhat arbitrary. For instance, all DFT calculations use a 6×6×6 k-point grid, but no justification is given for this selection. Was this chosen to maintain consistent k-space density, convergence accuracy, or computational feasibility? A systematic benchmark or discussion of how these parameters were determined—either within this work or relative to existing datasets—would improve the scientific rigor.\n5. Lack of Physically Meaningful Evaluation:\nThe reported metrics (e.g., MAE of Hamiltonian matrix elements or visual comparison of band structures) show numerical accuracy but do not necessarily reflect physically interpretable performance. While these are useful from a machine-learning perspective, it would be more convincing to assess the model’s impact on derived physical quantities such as effective masses, band velocities, or even carrier mobilities. These quantities would offer a clearer picture of how well the learned Hamiltonians reproduce observable electronic behavior.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "gv9UjcH1gR", "reviewer": "Reviewer_Gv72", "summary": "This paper proposes NextHAM, a neural E(3)-equivariant model for predicting materials' electronic structure Hamiltonians. The method incorporates zeroth-step Hamiltonians from the initial DFT charge density as both informative input descriptors and initial estimates for the target. The architecture employs a Transformer designed for strict E(3)-equivariance and high non-linear expressiveness. A novel training objective combining real and reciprocal space losses is introduced to mitigate error amplification and \"ghost states\" caused by the large condition number of the overlap matrix. Evaluated on a newly curated benchmark, Materials-HAM-SOC, NextHAM demonstrates excellent accuracy in predicting Hamiltonians and band structures, with spin-off-diagonal blocks reaching sub-μeV scale.", "questions": "* Could the authors provide more details on how the sub-matrices of the zeroth-step Hamiltonian are transformed into initial node and edge descriptors? The feature engineering process is crucial for reproducibility.\n\n* The term \"universal\" in the title is a strong claim, implying robust generalization across diverse chemical and conformational spaces. However, the current experiments primarily evaluate on a static benchmark. Could the authors provide additional evidence, such as out-of-distribution generalization tests, to substantiate this claim?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "* The proposed NextHAM method is technically sound and demonstrates state-of-the-art performance on the newly introduced benchmark.\n* The release of the Materials-HAM-SOC dataset is a valuable contribution that will facilitate future research in machine learning for materials science.", "weaknesses": "* Overall, the technical novelty is limited since the core technical components build heavily upon established concepts:\n\n * The delta-learning strategy (predicting residuals to a zeroth-order Hamiltonian) is a well-established paradigm in ML-based Hamiltonian and potential energy surface prediction [1, 2].\n\n  * The TraceGrad branch for non-linear expressiveness is a direct adaptation of a prior architecture [3] to this task.\n\n  * The primary novel contribution appears to be the reciprocal space ($k$-space) loss. However, its performance gain over the strong baseline (which already uses the zeroth-step Hamiltonian as an output estimate) seems marginal (Table 4). Furthermore, the authors do not provide sufficient empirical evidence or analysis to demonstrate how this loss concretely alleviates the \"ghost states\" issue.\n\n* The paper lacks comparisons with several relevant recent methods in the field (e.g., those methods for material systems mentioned in section 2), making it difficult to assess NextHAM's relative advancement. A well-defined, chemically meaningful accuracy threshold is also absent. While the reported metrics (e.g., sub-μeV) are low, it remains unclear whether this precision is sufficient for reliable downstream property predictions (e.g., band gaps, densities of states) in practical computational materials science.\n\n\n[1] Δ-Machine Learned Potential Energy Surfaces and Force Fields. JCTC 2022\n\n[2] Efficient and Scalable Density Functional Theory Hamiltonian Prediction through Adaptive Sparsity. ICML 2025\n\n[3] Tracegrad: a Framework Learning Expressive SO(3)-equivariant Non-linear Representations for Electronic-Structure Hamiltonian Prediction. ICML 2025", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "6vmK2ovPKY", "reviewer": "Reviewer_hRHV", "summary": "This paper focuses on predicting the hamiltonian matrix of materials. There are four core contributions that I understood:\n\n- Using H0 as an initial guess and predicting the difference\n- Incorporating the TraceGrad architecture (although I'm not sure if this is intended to be a novelty, since it seems to be already published). \n- Incorporating the coupling between occupied and unoccupied states of the reciprocal Hamiltonian into the loss\n- A new dataset of material hamiltonians", "questions": "What is the computational complexity of this loss function?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "I think that predicting the difference between H0 and the final Hamiltonian is an interesting approach. However, as pointed out by the authors in the appendix, it comes at a formidable computational cost (55s on average for structures which seem to have between 5 and 20 atoms, going by Fig. 7).\n\nI also appreciate that this network accounts for SOC (the only other one which does so to my knowledge is DeepHE3)", "weaknesses": "My primary concerns are (1) I think there have been simpler ways to do what the authors are doing with an otherwise complex architecture and loss function, (2) without either performance comparisons or applications (eg, scf iteration reduction), there is no way to judge the utility of their model, and (3) I question the universality and utility of this dataset of 12,000 structures, seemingly containing fewer than 10 instances of several atomic elements. I elaborate on these points below:\n\n(1):\n\nA lot of the components of this model (use of the trace as a supervision signal, initializing the features with H0, predicting the refinement of H) seem to be ultimately reducing the numerical range of the data (as mentioned in the paper). Other works have tackled the same problem by simply scaling the trace out of the data [1, 2], similar to what is commonly done in MLIPs. I'm not yet convinced that the architectural complications introduced has a benefit over simply scaling the data and using a regular equivariant network like DeepHE3 (which also treats SOC) or QHNet.  \n\nOther overly complex sections of the architecture include the sub-models for various radii. I think this kind of hard-coding defeats the purpose of either learning the correct attention mechanism, or simply learning the coulomb decay from data. Does this improve generalizability, compared to a model with the same number of parameters?\n\nLoss function: I understand that the near-ill-conditioning of the overlap matrix can amplify small errors in matrix elements. However, I believe this is effectively treated by Ref [3] in the wavefunction alignment loss. Can the authors discuss the computational cost and scaling of their loss function, and compare to using the wavefunction alignment loss? Could the effective separation between occupied and unoccupied subspaces be recovered by using a simpler subspace loss? \n\n(2):\n\nI'm not sure how to evaluate the performance of this model. Matrix-element MAE errors are meaningless when introducing a new dataset, especially one which covers multiple atomic elements since heavier elements will have larger-valued elements. The authors mentioned that off-diagonal matrix element errors are \"sub-meV\" - but these matrix elements are typically an order of magnitude lower than diagonal matrix elements already, so this is expected. MAE matrix element errors are only useful for comparing on benchmarks with other models. \n\nExamining the provided bandstructures in the appendix, and comparing them to those computed by methods like Wannierization, I think the errors seem to be fairly high. In many cases, a rigid shift from the bandstructures computed from H0 looks like it could recover the learned solution.\n\n(3):\n\nSeveral key details about the dataset are missing, such as the basis set used, and the distribution of the number of atoms per structures (the authors mention it does not contain large systems, but this is not quantified). My current understanding based on Fig 7 is that there are 5-20 atoms per structures, and the corresponding Hamiltonian matrices have sizes on the order of 100s. \n\nIn general, Hamiltonian datasets can contain fewer structures due to the increased amount of information-per-structure. However, I am questioning the utility of 12,000 training structures distributed across 68 elements. For example, looking at Fig 6a in the appendix, there seems to be on the order of 1 atom of several noble gases in the training data. I don't know if this can really be considered a representative dataset of \"68 atomic elements\". I think it would ultimately be more useful to focus on a smaller range of elements with better sampling across them.\n\n[1] Equivariant Electronic Hamiltonian Prediction with Many-Body Message Passing, https://arxiv.org/html/2508.15108v1\n[2] GEARS H: Accurate machine-learned Hamiltonians for next-generation device-scale modeling, https://arxiv.org/abs/2506.10298\n[3] Enhancing the Scalability and Applicability of Kohn-Sham Hamiltonians for Molecular Systems, https://arxiv.org/abs/2502.19227", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "P3esh9EBeQ", "reviewer": "Reviewer_Yiie", "summary": "This paper presents NextHAM, a deep learning framework for predicting electronic structure Hamiltonians. Leveraging zeroth-step Hamiltonians as informative priors, NextHAM employs an E(3)-equivariant transformer for expressive modeling and incorporates a dual-space loss function to ensure both stability and accuracy. Additionally, the paper introduces a new dataset, Materials-HAM-SOC, which includes spin–orbit coupling effects, to facilitate benchmarking.", "questions": "* I greatly appreciate the authors’ effort in performing extensive DFT calculations to provide the open-source community with a new dataset of material Hamiltonians. The paper mentions that “17,000 material structures [were] sampled from the Materials Project.” I am curious about the selection criteria: many of these materials exhibit weak SOC effects. What is the rationale for performing the more computationally expensive SOC-inclusive calculations on such systems? What benefits does this bring, especially when SOC contributions are negligible?\n* NextHAM uses the zeroth-order Hamiltonian $H_0$ as model input. The paper states: “These two types of sub-matrices can naturally serve as the initial descriptors for nodes and edges, respectively, in the graph neural network.” While I agree this is a natural choice, I would like to understand more concretely how these equivariant matrices are transformed into node and edge features. Moreover, have the authors considered alternative approaches to constructing $H_0$-based features, such as those proposed in [1] and [2]?\n* In the introduction section, the authors note that prior works often “neglect spin–orbit coupling (SOC) effects.” However, to the best of my knowledge, DeepH-E3 [3] already incorporates a method to handle SOC. This makes the introductory claim somewhat inaccurate. More importantly, does NextHAM introduce any specific architectural or algorithmic modifications to better capture SOC effects?\n* The paper frames the use of $H_0$ as a way to inject prior knowledge into the model. In the Hamiltonian prediction literature, other forms of prior knowledge have also been explored—for instance, enforcing self-consistency constraints [2,4,5]. It would be valuable to include a discussion comparing different strategies for incorporating physical priors: what are their respective strengths, limitations, and trade-offs in terms of accuracy, computational cost, and generalizability?\n\n[1] Qiao Z, Christensen A S, Welborn M, et al. Informing geometric deep learning with electronic interactions to accelerate quantum chemistry[J]. Proceedings of the National Academy of Sciences, 2022, 119(31): e2205221119.\n\n[2] Wang Z, Liu C, Zou N, et al. Infusing self-consistency into density functional theory hamiltonian prediction via deep equilibrium models[J]. Advances in Neural Information Processing Systems, 2024, 37: 89652-89681.\n\n[3] Gong X, Li H, Zou N, et al. General framework for E (3)-equivariant neural network representation of density functional theory Hamiltonian[J]. Nature Communications, 2023, 14(1): 2848.\n\n[4] Zhang H, Liu C, Wang Z, et al. Self-consistency training for density-functional-theory hamiltonian prediction[J]. arXiv preprint arXiv:2403.09560, 2024.\n\n[5] Li Y, Tang Z, Chen Z, et al. Neural-network density functional theory based on variational energy minimization[J]. Physical Review Letters, 2024, 133(7): 076401.", "limitations": "", "rating": 4, "confidence": 5, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "* The paper is well-structured and clearly written, offering an accessible and concise overview of both the problem and the proposed solution.\n* The appendix includes a thorough introduction to the relevant DFT background, making the work particularly approachable for readers less familiar with the field.\n* The authors open source their code. Moreover, the authors provide a new dataset of material Hamiltonians, which represents a valuable contribution to the open-source community in this domain.", "weaknesses": "* The authors propose using the zeroth-step Hamiltonian as input and training the model to predict the correction term—a strategy that essentially follows the well-established delta-learning paradigm in machine learning. This approach has been previously employed in the context of machine-learned density functional theory (DFT), for instance in [1]. While the ablation study demonstrates that this technique effectively reduces prediction errors, it does not introduce significant novelty.\n* Although the related work section discusses equivariant approaches, it omits several key references. While this paper focuses on Hamiltonian prediction, the earliest and most influential applications of equivariance in materials science were in machine-learned force fields—foundational works that paved the way for modern equivariant networks. These should be cited; for example, [2-10]. In addition, several important studies [11, 12] on using machine learning to predict Hamiltonians are also missing.\n* The evaluation results are primarily demonstrated on the authors’ newly constructed dataset, Materials-HAM-SOC. However, to provide a more comprehensive and convincing assessment of the model’s true performance, comparisons with existing methods on established public benchmarks are essential.\n* I greatly appreciate the authors’ inclusion of a detailed introduction to DFT-related background in the appendix, which enhances accessibility. That said, since the proposed model heavily relies on TraceGrad, the current manuscript provides surprisingly little explanation of it. For the sake of completeness and self-containment, I recommend adding a more thorough description of TraceGrad in the appendix, so readers do not need to consult external sources to understand key technical details.\n\n[1] Chen Y, Zhang L, Wang H, et al. DeePKS: A comprehensive data-driven approach toward chemically accurate density functional theory[J]. Journal of Chemical Theory and Computation, 2020, 17(1): 170-181.\n\n[2] Schütt K T, Sauceda H E, Kindermans P J, et al. Schnet–a deep learning architecture for molecules and materials[J]. The Journal of chemical physics, 2018, 148(24).\n\n[3] Batzner S, Musaelian A, Sun L, et al. E (3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials[J]. Nature communications, 2022, 13(1): 2453.\n\n[4] Thomas N, Smidt T, Kearnes S, et al. Tensor field networks: Rotation-and translation-equivariant neural networks for 3d point clouds[J]. arXiv preprint arXiv:1802.08219, 2018.\n\n[5] Musaelian A, Batzner S, Johansson A, et al. Learning local equivariant representations for large-scale atomistic dynamics[J]. Nature Communications, 2023, 14(1): 579.\n\n[6] Gasteiger J, Groß J, Günnemann S. Directional message passing for molecular graphs[J]. arXiv preprint arXiv:2003.03123, 2020.\n\n[7] Gasteiger J, Becker F, Günnemann S. Gemnet: Universal directional graph neural networks for molecules[J]. Advances in Neural Information Processing Systems, 2021, 34: 6790-6802.\n\n[8] Wang Y, Wang T, Li S, et al. Enhancing geometric representations for molecules with equivariant vector-scalar interactive message passing[J]. Nature Communications, 2024, 15(1): 313.\n\n[9] Wang Z, Liu G, Zhou Y, et al. Efficiently incorporating quintuple interactions into geometric deep learning force fields[J]. Advances in Neural Information Processing Systems, 2023, 36: 77043-77055.\n\n[10] Batatia I, Kovacs D P, Simm G, et al. MACE: Higher order equivariant message passing neural networks for fast and accurate force fields[J]. Advances in neural information processing systems, 2022, 35: 11423-11436.\n\n[11] Schütt K T, Gastegger M, Tkatchenko A, et al. Unifying machine learning and quantum chemistry with a deep neural network for molecular wavefunctions[J]. Nature communications, 2019, 10(1): 5024.\n\n[12] Unke O, Bogojeski M, Gastegger M, et al. SE (3)-equivariant prediction of molecular wavefunctions and electronic densities[J]. Advances in Neural Information Processing Systems, 2021, 34: 14434-14447.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)