Denoising Hamiltonian Network for Physical Reasoning
Authors: Congyue Deng
Brandon Y. Feng
Cecilia Garraffo
Alan Garbarz
Robin Walters
William T. Freeman
Leonidas Guibas
Kaiming He
Venue: Accepted by TMLR ()

Abstract: Machine learning frameworks for physical problems must capture and enforce physical constraints that preserve the structure of dynamical systems. Many existing approaches achieve this by integrating physical operators into neural networks. While these methods offer theoretical guarantees, they face two key limitations: (i) they primarily model local relations between adjacent time steps, overlooking longer-range or higher-level physical interactions, and (ii) they focus on forward simulation while neglecting broader physical reasoning tasks. We propose the Denoising Hamiltonian Network (DHN), a novel framework that generalizes Hamiltonian mechanics operators into more flexible neural operators.  DHN captures non-local temporal relationships and mitigates numerical integration errors through a denoising mechanism. DHN also supports multi-system modeling with a global conditioning mechanism. We demonstrate its effectiveness and flexibility across three diverse physical reasoning tasks with distinct inputs and outputs.

Published in Transactions on Machine Learning Research (02/2026)
Denoising Hamiltonian Network for Physical Reasoning
Congyuc Deng
congyyue@stanford.ce edu
Stanford University, Massachusetts Institute of Technology
Brandon Y. Feng
bronfeng@mii edu
Massachusetts Institute of Technology
Cecilia Garraffo
syarafff@@eaaharrvrrdd edu
Horvand-Smithsonin Center for Astrophysics
Alan Garbarz
alan@dff uba.ar
Universidad Buenos Aires and Instituto de Física de Buenos Aires CONICET
Robin Walters
T. walters Saallers@noorthhsseeeee. edu
Northeastern University
William T. Freeman
billfümit.cm edu
Massachusetts Institute of Tecknology
Leonidas Guíbas
guibas @stannfottte woraaaoorr..oo edu
Stanford University
Kaiming He
kaiming@mill edu
Massachusetts Institute of Technology
Reviewed on OpenReview: https: openneviee. net/ net/forum?
forum? id= Kd= LEg 7Hu
Abstract
Machine learning frameworks for physical problems must capture and enforce physical constraints that preserve the structure of dynamical systems. Many existing approaches achieve this by integrating physical operators into neural networks. While these methods offer theoretical guarantees, they face two key limitations: (i) they primarily model local
relations between adjacent time steps, overlooking longer-range higher-level physical interactions, and (ii) they focus on forward simulation while neglecting broader physical reasoning tasks. We propose the Denoising Hamiltonian Network (DHN), a novel framework that generalizes Hamiltonian mechanics operators into more flexible neural operators. DHN captures non-local temporal relationships and mitigates numerical integration errors through a denoising mechanism. DHN also supports multi-system modeling with global conditioning mechanism. We demonstrate its effectiveness and flexibility across three diverse physical reasoning tasks with distinct inputs and outputs.
1 Introduction
Physical rcasoning - the ability to infer, predict, and interpret the behavior of dynamic systems is fundamental to scientific inquiry. Machine learning frameworks designed to address such challenges are often expected to go beyond merely memorizing data distributions, aiming to uphold the laws of physics, account for energy and force relationships, and incorporate structured inductive biases that surpass those of purely data-driven models. Scientific machine learning addresses this challenge by embedding physical constraints directly into neural network architectures, often through explicitly constructed physical operators.

---

Published in Transactions on Machine Learning Research (02/2026)
Convolution kernel
Image processing image super-resolution
spatial translation equivariance for content consistency
image classification
image completion
"cat"
Physical reasoning sparse-data Interpolation
Denoising Hamiltonian block temporal translation invariance for conservation laws
physical property inference
trajectory completion
mass1
mass2 length
Figure 1: Denoising Hamiltonian Network (DHN) generalizes Hamiltonian mechanics into neural operators. It enforces physical constraints while leveraging the flexibility of neural networks, opening pathways for broader applications in physical reasoning.
However, these methods face two key limitations. (i) These methods primarily learn local temporal updates, predicting state transitions from one time step to the next, without capturing long-range dependencies or abstract system-level interactions. (ii) They focus predominantly on forward simulation, forecasting a system's 's evolution from initial conditions, while largely overlooking complementary tasks such as super-resolution, trajectory inpainting, or parameter estimation from sparse observations. To address these limitations, we aim to design more general neural operators that both follow physical constraints, and unleash the expressivity of neural networks to learn high-level information from data.
To motivate our approach, we draw inspiration from the success of CNNs. Classical image-processing systems utilized handcrafted convolution filters for low-level tasks such as edge detection. The breakthrough came with deep convolutional neural networks (CNNs), which generalized these filters by making them learnable, stacking them with nonlincaritics, and expanding them into high-dimensional feature channels. This transition enabled progress from local processing high-level tasks like object recognition and image generation.
Building on this inspiration, we introduce the Denoising Hamiltonian Network (DHN), framework that generalizes Hamiltonian mechanics into more flexible neural operators (Fig. D Similar to CNNs absorbing parameters from classical image processing (e.g., filter size, stride) into learnable and tunable components, DHN reformulates parameters of numerical integration as architectural hyperparameters, cnabling more adaptive and expressive physical modeling. For example, state discretization becomes a learnable block size, and small, fixed time steps are replaced by variable strides, analogous to how CNNs adapt their receptive fields.
More concretely, DHN enforces physical constraints while leveraging the flexibility of neural networks with three key innovations: (i) First, DHN extends Hamiltonian neural operators to capture non-local temporal relationships by treating groups of system states as tokens, allowing it to reason holistically about system dynamics rather than in isolated steps. (ii) Second, DHN introduces a denoising objective analogous to variational integration but learned through data. mitigates integration errors without relying on additional off off-trajcctory data for state optimization. Additionally, with different noise patterns, DHN supports ficxible training and inference across various task contexts. (iii) Third, introduce global conditioning facilitate multi-system modeling, enabling DHN to model heterogeneous physical systems under unified framework.
To evaluate DHN's versatility, we test it across three distinct reasoning tasks: (i) trajectory prediction and completion, lesting long-horizon stability and accuracy; (ii) physical parameter inference from state observations, examining whether the learned representations capture global physical properties; and (iii) trajectory interpolation via progressive super-resolution, assessing the ability reconstruct physically consistent trajectories from sparse observations. These tasks together also demonstrate that DHN provides unified framework for diverse physical reasoning problems beyond standard next-state prediction.
2

---

Published in Transactions on Machine Learning Research (02/2026)
In summary, this work moves toward more general network architectures that embed physical constraints beyond local temporal relationships, opening pathways for broader applications in physical reasoning beyond conventional forward simulation and next-state prediction.
2
Related Work
Machine learning approaches for physical modeling span from fundamental equations of motion to high- dimensional operator learning. Our work extends Hamiltonian neural networks (HNNs) into flexible, sequence-based paradigm that enables multi-task inference and generative conditioning.
Hamiltonian Neural Networks (HNNs) Scientific machine learning aims to embed physical laws into neural architectures. Hamiltonian Neural Networks (HNNs) GGryydanus et al. 2019) enforce symplectic structure and energy conservation in learned dynamics, inspiring various extensions: Lagrangian Neural Networks (LNNs) (Cranmer et al.) 2020), Symplectic ODE-Nets (Zhong et al. 2019), and Dissipative SymODEN (Zhong et al. 2020), which introduce damping terms. Constraints have also been incorporated into HNNs Finzi et al. 2020b), and some models infer Hamiltonian dynamics directly from image sequences (Toth et al.] 2019). Despite their strengths in forward simulation, standard HNNs typically model one system at a time and rely on uniform-step integration, limiting their use in trajectory completion, sparse-data interpolation, or super-resolution
Physics-informed and operator-based methods Beyond Hamiltonnian-specffic formulations, broader line of work embeds physical structure directly into neural models, most commonly partial differential equation (PDE) constraints. Another approach embeds partial differential equation (PDE) constraints directly into neural models. Informee Neural Networks (PINNs) (Raissi et al. 2019 enforce PDE-based losses for solving forward and inverse problems, while Fourier Neural Operators (FNOs) Li et al.. 2020) learn mappings between function spaces using global Fourier transforms. Neural ODEs (Cen al. 2018 Dupont al. 2019 parameterize continuous-time dynamics with learnable differential equations. As the term "physics-informedd neural network" has expanded to cover wide range of models and applications, much of the literature has focused on fitting a single system or solving specific PDE instance Bonev et al. (2023), , with emphasis on the fitting accuracy. In contrast, our work targets a different regime of physical learning: given training data from many systems of similar form but varying parameters, can a model form high-lovel understandings that can generalize to previously unseen systems while still preserving meaningful physical guarantees?
Symmetry--cosstrained neural networks From a broader perspective, the conservation laws enforced in Hamilronian-based models can be interpreted as symmetry constraints, grounded in Noether's theorem Noether (1971), which establishes direct connection between continuous symmetries and conserved quantities Greydanus et al. 2019); Cranmer et al. (2020). This viewpoint places Hamiltonian neural networks within the more general Tramework of equivariant learning, where network architectures are designed to respect prescribed group actions on inputs and outputs Cohen & Welling (2016); Kondor & Trivedi (2018). Historically, convolutional neural networks provided one of the earliest and most successful examples of such guarantees by enforcing translation equivariance, a principle that has since been extended beyond Euclidean domains to graphs, manifolds, and other structured data types LeCun et al. (2002); Bronstein et al. (2017).
More recently, research on equivariant architectures has expanded et al. 2019 Weiler et al. (2021); Deng al. (2021): Thomas et al. (2018): Geiger & Smidt (2022 and importantly, has begun to explore relaxations of exact equivariance, allowing models to balance inductive bias with expressive flexibility in settings where symmetries may be approximate, partial, or system-dependent
wider range of symmetry groups Cohen
Finzi et al. (2020a): Romero & Lohit (2022); Kaba & Ravanbakhshi (2023): Pertigkiozoglou el al.] (2024). Our
work aligns with this latter direction: rather than enforcing strict equivariance with respect to predefined conservation laws, DHN integrates physically motivated constraints while retaining the capacity to generalize across physical systems.
System identification and multi-system modeling Learning from heterogencous physical systems requires system identification, traditionally performed via parametric models (Ljung 1999) or hybrid PDF-
3

---

Published in Transactions on Machine Learning Research (02/2026)
block size=1 stride-1
block size=4
stride=2
H*
H*
block-wise
discrete Hamiltonian
JC
discrete
J[
Hamiltonian
autograd
HH
autograd
discrete Hamllton's equation of motion
I+S
I+S+
Figure 2: Block-wise Hamiltonian. Left: Classical HNN with the discrete Hamiltonian, which can be viewed as a special case with block size = and stride 8 = 1. Right: discrete (right) Hamiltonian block with = 4, 2. Dark blue and dark red indicate network inputs and outputs. Light colors are for placeholder states.
constrained approaches [Raissi et al. 2019 While Hamiltonian methods implicitly encode system parameters through energy landscapes, conventional HNNs often require training separate models per system. We introduce generative conditioning mechanism via a learned latent code, enabling a single model to generalize across multiple systems while preserving inductive biascs from Hamiltonian dynamics.
3 Method
We aim design neural operators that respect physical constraints while leveraging the flexibility and expressivity of neural networks. To this end, we extend Hamiltonian operators to block-wise form that learns more flexible state relations with generalized conserved quantities (Sec. 3.2 We then introduce denoising mechanism that replaces the state optimization in variational integration with a learned process; using different masking patterns, it also enables adaptive inference strategies across tasks within unified framework (Sec. 3.3 Finally, we incorporate global conditioning to extract high-level information from physical trajectories (Sec. 3.4].
3.1 Preliminaries
Learning with Hamiltonian mechanics Let's start with phase-space coordinates (q, p), where is the generalized coordinates and p is the generalized momerda or conjugate momenta. Tf q represents the particle positions in Euclidean coordinates, then p corresponds to their linear momenta. If q represents angular positions in spherical coordinates, p corresponds to the associated angular momenta. We consider the inne-invariant Hamiltonian, which is scalar function H(q.p) p) satisfying
dp dg = UPH dt =-
(1)
dt
Eq. 1 is known as Hamilton's equations of motion and describes system evolution by defining a trajectory in phase space along the vector field (V,H, V,. This field, called the symplectic gradienl, governs the dynamics such that movement along induces the most rapid change in the Hamiltonian, whereas motion in the symplectic dircction preserves the system's energy structure.
Hamiltonian Neural Networks (HNN) Greydanus al. (2019) treat the Hamiltonian black-box function H(g, 0) parameterized by neural network and optimize the network parameters to minimize the loss function
LHNN = 4
dg dt
dp
+
(2)
4

---

Published in Transactions on Machine Learning Research (02/2020)
ranioom noise
noised states
H
noīse scale 0 "equation of motion"
P
J
noise scale "variational integration"
autograd
Denoising Hamiltonian block. The model integrates the Hamiltonian equation of motion (pink arrow) and state optimization (blue arrow) into a unified framework (left) by learning state denoising conditioned on different noise scales (right).
Figure 3:
Starting with an initial state (go.Po), one can compute the trajectory (gt, Pt) by integrating the symplectic gradient (V Vy,l(eeePss Pc; V V,Hlh.PPe Pr: f)) over time
Discrete Hamiltonian Aside from the continuous Hamiltonian H and its discretizations, one can also directly define the discrete Hamiltonian with discrete mechanics and duality theory in convex optimization Gonzalez (1996). The discrete "right" Hamiltonian H gives the equation of motion in the form
9++1 V V ((+1), Pt = V (ge, P(+1).
(3)
The "right" means that q is forward and p is backward in time. This formulation serves as a first-order discrete approximation the contimuus Hamiltonian H. by
a++1 4t + AIV HAVPH(1ePP P(+1), = Pc+1 + AIV , H(ge.P(41)
(4)
Fig. 2 left illustrates discrete right. Hamiltonian network for computing the state relations between time steps to and f In the following sections, we describe our network design primarily using the right Hamiltonian H but similar equations can also define the left Hamiltonian H Additional details can be found in the Appendix.
Our motivations Exemplified by HNN, physical networks generally learn the state relations between adjacent time steps and t+1 modeled by an update rule (#+1.Pt+11) to forward modeling, the discretization in Eq. 3 is more accurate and better preserves the symplectic structure of the system under temporal integrations. However, the implicit nature of these update rules poses inference-time challenges, determining new system states requires solving optimization problem, which is difficult only with data points on the simulation trajectory but without additional reference points outside the trajectory. Our solution to incorporate the optimization process into the network, leading to the denoising Hamiltonian network (Sec. 3.3 that unifies the state optimization at each time step and the Hamiltonian-modeled state relations across time steps.
Compared
3.2 Block-wise Discrete Hamiltonian
We define state blocks as a stack of (g.p) states concatenated along the time dimonsion Q ... "..., . .Ptoo, with b being the block size. We also introduce the stride s as a hyperpa- rameter that can be flexibly defined, replacing the fixed time interval At Eq. 4 This approach enables the network to capture broader temporal correlations while preserving the underlying Hamiltonian structure. We define our block-wise discrete (right) Hamiltonian by relating two overlapping blocks of system states, each of size b, with shift stride of
Q.+:+b (pH+(Q) P!tath), pi+ = VQ P1+2-b).
P
(5)
Fig. illustrates a block-wise discrete Hamiltonian of a block size b 4 and a stride Classical HNNs be viewed special case of block size b = 1 and stride = Physical interpretations of the block-wise
5

---

varying magnitudes Published in Transactions on Machine Learning Research (02/2026)
known state
unknown state
Autoregression
Forward simulation
Super-resolution
Trajectory interpolation
Random masking
Representation learning
global parameters
Figure 4: Different masking patterns (left) adapt to different reasoning tasks (right), Blocks with pinks and blues surrounded by dotted lines are the denoising Hamiltonian blocks.
Hamiltonian with b > 1, > 1 and theoretical derivations of its conserved quantities can be found in the Appendix. Similar to the equation-motion loss for HNN (Eq. block-wise discrete Hamiltonian network Hg can be trained with the equation-of motion loss
Leom ( = V (Q,+), Pltteth Q + Hg (Q,+), Pltett) pftb|l -
(6)
3.3 Denoising Hamiltonian Network
Masked modeling and denoising Following our motivations introduced in Sec. 3.1 we want the Hamiltonian blocks to not only model the state relations across time steps, but also learn the state optimization per time step for inference. To achieve that, we adopt masked modeling strategy He et al. (022) by training the network with part of the input states masked out (Fig. 3
Concretely, taking the blocked input state Qi++b as an example, we define known and unknown states in this block with a binary mask M = [m... ........ me+o], with for unknown states and for known states. Rather than simply masking out the unknown states, we perturb them with noise sampled and learn to denoising them with iterative refinements (Fig. 3 left). With a sequence of increasing noise levels ag < 01
aN 1, randomly sample Gaussian noises gr+b = [Ee, Se+b) and noise scales ALtb [aa..... "a...,,,,0 ,, Applying the noise the unknown states, we get the final perturbed state Qi+b with
g= (1 (1- ag)gg + 0ggs if m = 0
if m, = 1
d. -
(for t,... +1+b).
(7)
Similarly, we can get the perturbed state petb and logether with Q,+b define the denoising loss
Ldeuoise ( V H. (Q+tb, PPts P.tetb) Q (Q @Âthh Piteth PA- -PAttl
(8)
This encourages the network H¹ to learn a conditioned state optimization, with the known states with mask values m, as conditions and the unknown ones with m, = 0 as the states denoise (Fig. right).
Combining Eq. f and Eq. the final loss function for the nor is Leom (0) Ldenoise (0).
To note, unlike the denoising diffiusin model that learns multi-modal distributions, this denoising process has a fixed target and thus docsn't require many denoising steps (we set it to be 10 in our experiments). More details for training and inference are the Appendix.
Different masking patterns By designing distinct masking patterns during training, we enable flexible inference strategies tailored to different tasks. Fig. 4 shows three types of different. masking patterns:

---

Published in Transactions on Machine Learning Research (02/2026)
input block
input block
input trajectory
codebook
random noisc
Latent
query
20
query Pi)
input states
Encader
noise scale embed.
embed.
DHN
DHN
Decoder-only transformer
output (0..P)
output (01.P)
block Hamiltonian X
Autoencoder
Autodecoder
Figure Network architecture. Left: Decoder-only transformer. We use latent code for each trajectory to serve the query token for the Hamiltonian value output. Per-state noise scales encoded and added to the positional embeddings. Dark purples (in all shades) indicate trainable modules variables. Right: Autodecoder. Instead of encoding the input trajectory with an encoder, we maintain a codebook for the entire dataset with a learnable latent code for each trajectory. Dark purples (in all shades) indicate trainable modules or variables.
auforegression by masking out the last few states a block, which resembles physical simulation in terms of next-state prediction with forward modeling; super-resolution by masking out the states in the middle of a block, which can be applied data interpolation; and more generally, arbitrory-order masking including random masking, which adapt to the different tasks.
3.4 Network Architecture
Decoder-only transformer For each Hamiltonian block, the network inputs are a stack of Q+ of different time steps, a stack p p and we also introduce a global latent code for the entire trajectory as conditioning. We employ a decoder-only transformer Radford et al. (209 Jin et al. (2024), which resembles GPT-like decoder-only architocture but without causal attention mask, as shown Fig. loft. We apply self-attention to all input tokens [Q] ph""h_ 222 as a sequence of length 2b + Analogous to the CLS token in Vision Transformers (ViT) Dosovitskiy (2020). we employ global latent code 2 serves as the query token for outputting the Hamiltonian value H. We also encode the per-state noise scales into the network by adding their embeddings to the positional embedding.
Autodecoding Rather than relying on an encoder network to infer the global latent code from the trajectory data, we adopt autodecoder framework Park et al. (2019), maintaining learnable latent code z for each trajectory (Fig. 5 right). This approach allows the model to store and refine system-specific embeddings efficiently without requiring separate encoding process. During training, we jointly optimize the network weights and the codebook. After training, given a novel trajectory, we freeze the network weights and only optimize the latent code for the new trajectory.
4 Experiments
We test our model with three different tasks corresponding to the three different masking patterns, illustrated in Fig. They are (i) next-state prediction (autoregression) for forward simulation (Sec. 4.1. (ii) representation learning with random masking for physical parameter inference (Sec. 4.2), and (iii) progressive super-resolution for trajectory interpolation (Sec. 4.3h. These tasks highlight DHN's adaptability to diverse physical reasoning challenges, testing ability to generate, infer, and interpolate system dynamics under varying observational constraints. More concretely, will demonstrate the following:

---

Published in Transactions on Machine Learning Research (02/2026)
Single pendulum
Double pendulum
Total energy per time step (of example trajectory)
Average error on à over time
Total energy error per time step
Average error on y over time
1.4
Cora Blott coa
Outt 18:ac
Dut aec
Dutt Elick obe Dus
Out blask ober
bias
HSM Sanael tukel Leaptoog
Id6s Siemuniiuule 1909 Larpprol
H40 (srmesiitsle)
Hin Enre ditle
1801 Laagirog)
1901 hunging
Trrs
Toma 1top
Time
Time 68pp
Figure 6: Forward modeling: fitting known trajectories. The results of our method are shown in pink, and the results of HNN with different numerical integrators are shown in different shades of blue. The 2nd column shows the error of total energy for the single pendulum system, calculated with state (9t, Pc) at each time step analytically. The 3rd column shows the total energy predicted by the network over time steps (which wasn't 't supervised during training) on one example trajectory.
Task flemibility. A single DHN framework supports a range of physical reasoning tasks (Sec. 4.1-4.3). Taken logether, these experiments illustrate the breadth of capabilities enabled by our formulation, beyond performance on any single task in isolation.
Physsccally meaningful latent representations. DHN Iearns latent representations that capture highor-level physical properties, rather than only low-level local relations between nearby states (Sec. 4.2).
Balanced inductive bias and expressivily. Across all tasks, DHN achieves improved performance by balancing explicit physical constraints with the flexibility of black-box operator designs.
Setups We evaluate our model with two settings: the single pendulum and the double pendulum. Both settings comprise a dataset of simulated trajectories. The single pendulum is a periodic system where the total energy each state can be directly computed from (q,p), and thus we use it to evaluate the models' energy conservation ability. The double pendulum is chaotic system where small perturbations can lead to diverged future states. More detailed experimental setups in the Appendix.
Unlike prior works Toth et al. (019)) that generate data using a fixed set of system parameters while varying initial conditions, we introduce variation by altering the string lengths of the pendulums while keeping initial states fixed. This modification evaluates whether models can generalize to a broader class of parameterized dynamical systems rather than fit into a single-instance system. For both settings, we split the datasct into 1000 training trajectories and 200 lesting trajectories. Each trajectory is discretized into 128 time steps. More details can be found in the Appendix.
4.1 Forward Simulation
We start with the forward simulation task, where the model predicts the future states of a physical system step-by-step given the initial conditions. We implement this by applying masking strategy within each DHN block, where the last few tokens are masked during training, requiring the model to iteratively refine and denoise them (Fig. 4 top). For one DHN block of block size b and stride s, the mask is applied to the last b 8-8 tokens. At inference time, given the known states at time steps [0. ,4], we apply the DHN block to the time steps [t - + 1....
.t............................ + s], where we use the known states [t b 1....... ,f] to predict the unknown states [t + 1,... . + s). We experiment with block sizes = 2, 4, 8 with strides b/2.
Fitting known trajectories We first evaluate the model's capability to represent known physical trajecto- ries with forward modeling. In this experiment, we train the model to fit 1000 training trajectories, and we test by giving the first time steps of cach trajectory and using the model to predict the future 120 stops. As all models are only trained with states of nearby time steps (pairs of adjacent time steps for the basclines, and blocks of b states for DHN), small fitting errors can accumulate over time in forward modeling. Beyond
8

---

Published in Transactions on Machine Learning Research (02/2026)
Single pendulum
Double pendulum
Total energy per time step (of one example trajectory)
Average error q over time
Total. energy error time step
Average error over time
Cas thace
thane tibe
1.28
Due sleck ven-]
Due -
Deny Hlick wee-dl
Dura drack veenn))
Hes foog
Laaptng
190 Larptool
03
Jaaphtg
62
033
180
Thme
Tort
2.-
OLDS (2
12
Oc's Block axm-2l
Durn tbact sibenut
Bostuit OHE (iciiw)))
sapted (we (tokloo))
Facuet jos actee3)) Tanalanee
Tux doroe
Oaa biick 2apm
Tar
23
180 120
20
09 180 120
205 150
Figure 7: Forward modeling: completion on novel trajectories. Top row: Comparison between our method (shown in pink) and HNN with different numerical integrators (shown in blue). Bottom row: Comparison between our method (shown in pink) and vanilla networks with different architectures (shown in yellow). The vanilla networks directly predict the next state (4++1, from the current state (4t, with one feedforward step. Note that the y-axis scales between the two rows are different.
accumulated prediction errors inherent to the network, inaccuracies also arise from numerical integration approximations, which can amplify deviations over time.
Fig. shows the results of our model with different block sizes, compared to HNN Toth et al. 2019 with different numerical integrators. Left and right are the mean squared error (MSE) on the q predictions at each time step for the single and double pendulum systems, respectively. The middle plots show the averaged total energy error and the evolution of total energy on one example trajectory. Although HNN is a symplectic network with guaranteed energy conservation, the numerical integrator can still induce uncontrollable energy drifts. This additional numerical error is inevitable with forward methods. As mentioned in Sec. 3.1 while this can be addressed by variational integration methods, such implicit state optimizations require off-trajectory data, whereas DHN achieves similar elfects through a denoising mechanism without this overhead. With block size b = 2, our model conserves the total energy stably. Increased block sizes can cause energy fluctuations at long time ranges, but this fluctuation doesn't 't show an obvious inclination of energy drift.
Completion novel trajectories We then evaluate our models novel trajectories with partial observations. Concretely, we give the first 16 time steps in each testing trajectory and use them optimize for the per-trajectory global latent codes with the network weights frozen, as described in Sec. 3.4 A After optimizing these latent codes, we use them to predict the next 112 time steps. This task evaluates DHN's ability to infer system dynamics from sparse initial obscrvations and accurately forecast future states.
Fig. shows our results compared to HNN (lop row) and various baseline models without physical const (bottom row). Our DHN with small block sizes shows more accurate state prediction with better energy conservation compared to both baselines. Large block sizes can cause error explosion long time ranges it is hard for our simple 2-layer network to fit very complex multi-state relations.
Higherrdimennioonl system Figure g left illustrates the experimental setup. We consider a one- dimensional spring chain with 10 particles, where a transverse wave propagates from right to left. All particles share the same mass except for one "weird" particle, whose index and mass are randomly sampled. This design produces a family of related systems with localized variations physical properties. DHN is evaluated on the forward simulation task, using the same training and testing protocol as in the previous experiment in this section (completion on novel trajectories).
9

---

Published in Transactions on Machine Learning Research (02/2020)
The spring chain system
Average error on over time
Initial displacenent
(block 52t=8)
(blec Roshet blocis-2
(block 920-2)
0.6
HHN Hooward Euler)
Transfomer
INN [Leapting)
0.4
0.2
State evolution
Wave traveling
100
120
60
100
Time
Figure 8: Traveling wave on spring chains. Left: Illustration of the system. Blue dots are normal particles of identical mass, and the pink dot is the "weird particle", whose index and mass are randomly sampled. Right: Averaged q prediction errors in MSE.
Figure g right reports the average mcan squared error of positions q over time. DHN with larger block sizes (4 or 8) achieves the lowest error, particularly at longer time horizons. Predieting multiple steps in parallel reduces the number of autoregressive rollouts and consequently mitigates temporal error accumulation. Notably, HNN with Leapfrog integration performs worse than with forward Euler, despite Leapfrog being a higher-order integrator. Since the test set consists of previously unseen physical systems, this indicates that the primary performance bottleneck arises from model generalization across systems rather than numerical integration accuracy. In addition, DHN consistently outperforms unconstrained neural baselines, highlighting the benefit of incorporating physical structure.
The 10-particle spring chain is chosen deliberately for its simple and nearly linear spatial structure, which can be effectively modeled by an MLP. In our architecture, the transformer primarily captures temporal relationships, while spatial interactions are handled by a lightweight network. Although DHN is not inherently limited by system dimensionality, practical performance depends on how well spatial structure is represented. For more complex multi-particle or high-dimensional systems, stronger spatial inductive biases are likely beneficial, for example, permutationninvaaiann modules such as NointNet,, graph neural networks for arbitrary connectivity, and fully spatiotemporal transformers that jointly model spatial and temporal interactions. These directions offer natural path toward scaling DHN to more general physical systems.
4.2 Representation Learning
Next, we test the model's ability to effectively encode and distinguish the parameters of different physical systems. Denoising and random masking are well-established techniques in self-supervised learning, producing state-of-the-art representations in language modeling Devlin (2018) and vision Vincent et al. (2008); He et al. (2022). Here, we apply the random masking pattern (Fig. bottom) and study whether similar paradigms can enhance representation ion learning in dynamic physical systems.
To quantify the quality of the learned representations, we follow the widely adopted self-supervised representa- tion learning paradigm in computer vision Chen et al. (020)): Dord et al. (2018); He et al. (2020); Kolesnikov
et al. (209)) with feature pre-training and linear probing- Specifically, we pre-train the autodecoder alongside the codebook using the training set, then freeze the learned representations and train simple lincar regression layer on top to predict system parameters. This approach assesses whether DHN's latent codes capture meaningful physical properties. We experiment with the double pendulum and predict the length ratio 12/4, because this physical quantity is dinensionless and therefore invariant under normalizations in data preprocessing.
Fig. left shows the linear probing results of our DHN with different block sizes (with s = b/2), compared to the HNN and vanilla networks. Our model achieves much lower MSE compared to the bascline networks. As illustrated in Fig. 2 HNN can be viewed as a special case of our Hamiltonian block with kernel size and stride being 1, which is of the most locality. The block sizes and strides introduce allow the model to
10

---

Published in Transactions on Machine Learning Research (02/2020)
0.300
0.299
0.275
0.317
0.252
0242
0.30
0.33
Ours (b=2,6-1)
0.25
Ours (b=4,5-2)
0.208
Ours (bm8,ym4)
2
01.855
0.111
0.20
0.155
HNN
Reshict blocks-1)
0168
0.289
ResNet blocks-
0.150
0.15
Transformer
0.125
0.10
Ours
HNN
Vanilla
0.100
block size=2
block size-4
block
Figure 9: Linear probing on latent codes (MSE 4). We predict 12/4 by applying a linear regression layer to the global latent code. Left: Comparison with baselines. Right: DHN results with different block sizes and strides. Better results are achieved with block size b and stride s around b/2.
observe the system at different scales. In this double pendulum system, block size of is the best temporal scale for inferring its parameters.
Fig. g right shows DHN results with varying block sizes and strides. An explanation is in the "overlap" between the input and output states of b- b time steps. The generalized energy conservation in DHN relies on overlapping regions having identical inputs and outputs, enforced during training via the state prediction loss. Larger overlaps strengthen self-coherence but reduce focus on inter-state relations, while smaller overlaps (with larger strides) encourage learning from distant states but weaken self-coherence, affecting stability. At the extreme, full overlap with zero stridee reduces training to enforcing self-coherence alone with identical inputs and outputs. HNN is the special case at another extreme with zero overlap. The results suggest. that the optimal block sizes and strides are around b/2, leading to moderate overlaps.
4.3 Trajectory Interpolation
To demonstrate the flexibility of the DHN block, we show trajectory interpolation (super-resolution) with the masking pattern in Fig. f (e.g., linear quadratic) are insufficient to recover the underlying structure of the data. In computer vision, super-resolution methods go beyond simple pixel interpolation to recover semantic content and fine-grained details from low-resolution images (e.g., Ledig et al. (2017); Lim et al. (2017)). In this work, we consider an analogous setting for physical trajectories: given sparsely observed temporal states, we aim to reconstruct high-resolution, physically mcaningful trajectorics rather than merely filling in missing values mumerically.
middle. Learning-baat based super-resolution useful when naive interpolation methods
We conduct 4x super-resolution by repeatedly applying 2x super-resolutions, as shown in Fig. 10 left. Each trajectory is associated with a global latent code, shared across all three stages. During training, both the network weights and these latent codes are optimized jointly with all stages (0, 1, 2). At inference time, given a novel trajectory with known states only at the sparsest level (stage 0), we freese all network weights in the DHN blocks and optimize for the global latent code with stage O. After this test-time optimization (autodecoding). we apply the DHN blocks of stages (1, 2) to predict the unknown states in between the known states.
We evaluate the models with two test settings: (i) trajectorics with the same initial states as the training ones, and (ii) trajectories of unseen initial states. To set this up, we crop all training trajectories to time steps [0, ........... 64].. For each trajectory in the test set, we divide it into two segments: time steps [O,-.. 64] and (65....,123) the former having the same initial state as the training set and the latter having different. initial states.
We compare our model to a Convolutional Neural Network (CNN) for super-resolution. Fig. 10 shows our results. For the trajectories with the same initial state as the training data, both models show good interpolation results with lower MSEs. The baseline CNN shows slightly better results, as has no regularization in itself and can easily overfit the training trajectories. For testing trajectories with unseen
11

---

Published in Transactions on Machine Learning Research (02/2026)
known state
predicted state
Single pendulum
Double pendulum
0.680
25.020
stage 0
Ours
MSE (x200)
CNN
MSE (x100)
20
CNN
0.4
stage
10
0.2
6.801
stage 2
0.0
0.029
0.
0.000
0.035
Diff. init.
0.016
init.
Diff. init.
Figure 10: Interpolation via progressive super-resolution (MSE.). Left: The three stages for 2x super-resolution repeated twice. Right: Result comparison between DHN (Ours) and CNN-basod super- resolution nelwork (CNN). Results are MSE multiplied by 100.
initial states, the CNN struggles to generalize, its interpolations rely heavily on the training distribution. In contrast, DHN demonstrates strong generalization, as its physically constrained representations enable it to infer plausible intermediate states even under distribution shifts.
5 Discussions and Conclusions
Balancing flexibility with physical constraints is key to advancing physics- based learning. Inspired by unified architectures in vision and NL. (e.g., CNNs, transformers), we ask: can a single model handle tasks ranging from global parameter inference to local state prediction, while preserving physical consistency? This motivates three broader questions beyond DHN:
What defines physical reasoning in deep learning? Beyond next-state prediction, it encompasses parameter estimation, system identification, and discovering high-level relationships in dynamical systems. We aim for unified physical learning frameworks that adapt to diverse tasks while enforcing fundamental physical principles.
What is physical simulation? Simulation docsn't always require gencrating trajectories step by step from an initial state. Even used sequentially, a full trajectory can be generated any order. For example, video generation with denoising models produces all frames at once, and Diffusion Policy Chi et al. (2023 predicts chunks of future actions ahead of time, even though they're 're executed step by step.
What physical attributes should neural network possess? While PDEs describe systems through local temporal relations, directly using them in networks imposes the same locality. But networks aren't 't limited to this. Instead of strictly following physical operators, we can extend them, preserving core properties while relaxing others, to design more flexible architectures.
We believe that physics-based based learning is on the verge of a major transformation, similar to the rise of self-supervised lcarning in vision and NLP. By reframing physical reasoning as a reconstruction problem, predicting system states from partial corrupted inputs, we move toward unified framework that blends deep learning flexibility with the rigor of physical laws. We also provide more discussions in the Appendix.
Limitations and future work While our current work provides increased flexibility in Hamillonian--assel network designs, it incurs a higher computational cost due to intensive gradient evaluations inherited from HNN. In addition, current experiments focus on small-scalc systems with simple temporal dynamics. Scaling to complex spatial-tempon systems may benefit from hierarchical or allention-based architectures inspired by modern vision models.
Broader Impact Statement
This work aims to advance scientific studies by developing AI tools for physics-based reasoning. By incorpo- rating physical constraints into neural networks, we seek to improve the interpretability and reliability of learning- basod models in scientific contexts. However, as with other machine learning approaches, applying neural networks scientific problems requires caution, as they can produce hallucinations that potentially lead to incorrect conclusions if not properly validated. While incorporating physical constraints can mitigate
12

---

Published in Transactions on Machine Learning Research (02/2026)
such risks, it does not eliminate the need for rigorous verification. It remains essential that results from AT-assisted analyses are evaluated critically and supported by both physical principles and empirical evidence.
Acknowledgments
We thank Rell the cat for her photo in Figure I We also thank Tianwei Yin, Tianyuan Zhang, Shivam Duggal, Yichen Li, Carolina Cuesta-Lázaro, and Katherine L. Bouman for their helpful discussions. C. Deng and L. Guibas are in part supported by the Toyota Research Institute University 2.0 Program and Vannevar Bush Faculty Fellowship. B. Y. Feng and W. T. Freeman in part supported by the NSF Award 2019786 (The NSF AI Institute for Artificial Intelligence and Fundamental Interactions) and the NSF CIF Award 1955864 (Occlusion and Directional Resolution in Computational Imaging). C. Garraffo is funded by AstroAT at the Center for Astrophysics at Harvard & Smithsonian. A. Garbarz is supported by UBA and CONICET and through the grants PICT 2021-00644, PIP 112202101 00685CO, and UBACYT 2002022040010BBA R. Walters supported by NSF 2134178.
References
Boris Bonev, Thorsten Kurth, Christian Hundt, Jaideep Pathak, Maximilian Baust, Karthik Kashinath, and Anima Anandkumar. Spherical fourier neural operators: Learning stable dynamics on the sphere. In International conference on machine learning, pp. 2806 2623.222. PMLR, 2023.
Michael M Bronstein, Joan Bruna, Yann LeCun, Arthur Szlam, and Pierre Vandergheynst. Geometric deep learning: going beyond euclidean data. IEEE Signal Processing Maguzine, 34(4):18 42, 2017.
Tian Qi Chen, Yulia Rubanova, Jesse Bettencourt, and David Kristjanson Duvenand. Neural ordinary differ- ential equations. In Neural Information Processing Systems, 2018. URL https://api. http:/ppp..emmaticccchhhlolaa
:tt//CorpuuIID:4311444
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference on machine learning, pp. 1597 1607. PMLR, 2020.
Cheng Chi, Zhenjia Xu, Siyuan Feng, Eric Cousineau, Yilun Du, Benjamin Burchfiel, Russ Tedrake, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion. The International Journal of Robolies Research, pp. 027836419441233666, 2023.
Taco Cohen and Max Welling. Group equivariant convolutional networks. In International conference machine learning, pp. 2990-2999. PMLR, 2016.
Taco S Cohen, Mario Geiger, and Maurice Weiler. A general theory of equivariant cmns on homogeneous spaces. Advances in neural information processing systems, 32, 2019.
M. Cranmer, Sam Greydanus, Stephan Hoyer, Peter W. Battaglia, David N. Spergel, and Shirley Ho. Lagrangian neural networks. ArXiu, abs/2003.04630, 2020. URI. https://api. semanticscholar. org/
2D
Congyue Deng, Or Litany, Yueqi Duan, Adrien Poulenard, Andrea Tagliasacchi, and Loonidas Guibas. Vector neurons: A general framework for (0(3)-equvvaiiant networks. In ICCV, 2021.
Jacob Devlin. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint Xrx:::1810.044 0400,, 2018.
Alexey Dosovitskiy. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint (rXiv:2010.1 11929, 2020.
Emilien Dupont, A. Doucet, and Yee Whye Teh. Augmented neural odes. ArXiv, abs/ 1904.01681, 2019. URL http:s//pp....mmmt.ccccco....... www.aaatttccooola...ooooooroaaaa
13

---

Published in Transactions on Machine Learning Research (02/2026)
Mare Finzi, Samuel Stanton, Pavel Izmailov, and Andrew Gordon Wilson. Generalizing convolutional neural networks for equivariance to lie groups on arbitrary continuous data. In International conference on machine learning, pp. 3165 3176. PMLR, 2020a.
Marc Finzi, Ke Alexander Wang. and Andrew Gordon Wilson. Simplifying hamiltonian and lagrangian neural networks via explicit constraints. ArXiv, abs/2010.13581, 2020b. URL https://api. http://///pp..sseaaatccccciccii semanticcoaa
Mario Geiger and Tess Smidt. e3nn: Euclidean neural networks. arXiv preprinl arXiv:2307.09452, 2022.
Oscar Gonzalez, Time integration and discrete hamiltonian systems. Journal of Nonlineor Science, 6:449-467, 1996.
Sam Greydanus, Misko Dzamba, and Jason Yosinski. Hamiltonian neural networks. In Neural Information Processing Systems, 2019. URL
Seeeaaat..cccool....roor/raaaplp 1747997777
Ernst Hairer, Marlis Hochbruck, Arieh Iserles, and Christian Lubich. Geometric numerical integration. Oberwolfach Reports, 311)8803-882, 2006.
Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVE conference on computer vision and pattern recognition, pp. 17299-733 2020.
Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF conference on computer vision and pottern recognition, pp. 16000-16009, 2022.
Jonathan Ho, Ajay Jain, and Picter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840 6851, 2020.
Haian Jin, Hanwen Jiang, Hao Tan, Kai Zhang, Sai Bi, Tianyuan Zhang, Fujun Luan, Noah Snavely, and Zexiang Xu. Lvsm: A large view synthesis model with minimal 3d inductive bias. ar preprint 20u:2410.172422 2024.
Sékou-Oumar Kaba and Siamak Ravanbakhsh. Symmetry breaking and equivariant neural networks. arxiv preprint 2Xtt:2312.09016 2023.
Alexander Kolesnikov, Xiaohua Zhai, and Lucas Beyer. Revisiting self-supervised visual representation learning. In Proceedings of the LEEE/CVE conference on computer vision and pattern recognition, pp. 1920-1929, 2019.
Risi Kondor and Shubhendu Trivedi. On the generalization of equivariance and convolution in neural networks to the action of compact groups. In International conference on machine learning, pp. 2747-2755. PMLR, 2018.
S Lall and Matthew West. Discrete variational hamiltonian mechanics. Journal of Physics A: Mathematical and general, 39(19)55509, 2006.
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278-233) 2002.
Christian Ledig, Lucas Theis, Ferenc Huszár, Jose Caballero, Andrew Cunningham, Alejandro Acosta, Andrew Aitken, Alykhan Tejani, Johannes Totz, Zehan Wang, al. Photo-realistic single image super-resolution using generative adversarial network. In Proceedings of the IEFE conference on computer vision and pattern recognition, pp. 4681-4690, 2017.
Zong-Yi Yi Li, Nikola B. Kovachki, Kamyar Azizzadencsheli, Burigede Liu, Kanshik Bhattacharya, Andrew M. Stuart, and Anima Anandkumar. Fourier neural operator for parametric partial differential equations. Ar Xio, abs/2010.08895, 2020. URL htt.:tpppp/ppp...eattcccccoo..oo 2ttp/CooppuIDD:227555555
semanticscholar
14

---

Published in Transactions on Machine Learning Research (02/2026)
Bee Lim, Sanghyun Son, Heewon Kim, Seungjun Nah, and Kyoung Mu Lec. Enhanced deep residual networks for single image super-resolution. In Proceedings of the IEFF conference computer vision and poltern recognition workshops, pp. 136 144, 2017.
Lennart Ljung. System identification: theory for the user. 1999. URL httpss///pp...eeatttssso........ :orpusID: 538211555
Jerrold E Marsden and Matthew West. Discrete mechanics and variational integrators. Acta numerica, 10: 357-514, 2001.
Emmy Noether. Invariant variation problems. Transport theory and statistioal physics, 13): 207, 1971.
Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding. arXiv preprint arXiv: 180::1800000 01118807.0377 03748, 2018.
Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, and Steven Lovegrove. Deepsdf: Learning continuous signed distance functions for shape representation. In Proceedings of the IEEE/CVF conference on compuler vision and pollern recognition, pp. 165 174, 2019.
Stefanos Pertigkiozoglon, Evangelos Chatzipantazis, Shubhendu Trivedi, and Kostas Danillidis. Improving equivariant model training via constraint relaxation. Advances in Neural Information Processing Systems, 37:83497-8833 83520, 2024.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAT blog, 1(8):9, 2019.
Maziar Raissi, Paris Perdikaris, and George Em Karniadakis. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlincar partial differential equations.
J. Compul. Phys.,, 378:686 707, 2019. URL https://app..semmtiicceleeeee www.ontnnnchllaa.....aaa :ttg//oopus:::::::
David W Romero and Suhas Lohit. Learning partial equivariances from data. Advances in Neurol Information Processing Systems, 35:36466-36417 2022.
Yang Song, Jascha Sohl-Dickstein, Diederik Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. (core-based generative modeling through stochastic differential equations. arXiv preprint Xrxiv:2011.10 13456, 2020.
Nathaniel Thomas, Tess Smidt, Steven Kearnes, Lusann Yang, Li Li, Kai Kohlhoff, and Patrick Riley. Tensor field networks: Rotation-and translation--qqiivarant neural networks for 3D point clouds. 1iv:18011 08219, 2018.
Peter Toth, Danilo Jimenez Rezende, Andrew Jaegle, Sébastien Racanière, Aleksandar Botev, and Irina Higgins. Hamiltonian generative networks. ArXiv, abs/11909133) 2019. URL https://api semanticscholar
: 203593936
Pascal Vincent, Hugo Larochelle, Yoshua Bengio, and Pierre-Antoine Manzagol. Extracting and composing robust features with denoising autoencoders. In Proceedings of the 25th international conference on Machine learning, pp. 1096 1103, 2008.
Maurice Weiler, Patrick Forré, Erik Verlinde, and Max Welling. Coordinate independent convolu- tional networks isometry and gauge equivariant convolutions on riemannian manifolds. ar preprint arXiv:2106.060: 2021.
Yaofeng Desmond Zhong, Biswadip Dey, and Amit Chakraborty. Symplectic ode-net: Learning hamiltonian dy- namics with control. ArXiv, htss/199.120777 2019. URL https://app... http:///aap...mmnnnccccchooll worp:/cccccccco.oooooooooooooooe Tang/CorpusDD 202889233
Yaoleng Desmond Zhong, Biswadip Dey, and Amit Chakraborty. Dissipative symoden: Encoding hamiltonian
dynamics with dissipation and control into deep learning. ArXiv, abs/2002.08800 2020. URL https:
//api www.semtiicccollaa.......o hrt.rb/a....rrg/ooooppp022211111
15

---

Published in Transactions on Machine Learning Research (02/2026)
A Additional Discussions and Motivations
f f= mä
1
Vo + at
2 =mv² = mggh
Global solution analytical solution
Local relation PDE numerical solver most general
(IIl) Direct non-local relation use global physical principles certain cases
simple systems
Figure 11: How can we solve for a physical state? (T) Closed-form analytical solutions for simple systems. (II) For more complex physical systems, most physical PDEs only model local relations of close-by by time steps. (IIT) For cerlain physical systems, states can be directly related even if they are not close time.
Our goal is to design more general neural operators that both follow physical constraints and unleash the flexibility and expressivity of neural networks as optimizable black-box functions. We start by asking the question: What "physical relations" can we model beyond nent-state prediction?
Fig. compares three classical approaches to modeling physical systems without machine learning:
Case (I): Global analytical solution. For simple systems with regular structures, one can derive a closed-form solution directly.
Case (II): PDE + numerical integration. In more complex settings where no closed-form solution exists, the standard practice is to formulate the system's 's dynamics as a PDE and solve it step- syt-bb-step over time via numerical methods. This local integration approach underlies most physics-constraiig neural network designs that incorporate the PDE operators into the network.
Case (III): Direct global relation. In some complex systems, states that are temporally far apart can be related directly using global conservation laws (e.g., energy conservation). This is akin high-school physics problems: can compute an object's velocity conditions alone, without solving for a full trajectory. While this is less general than PDE-based approaches, it suggests promising avenue: leveraging global physical principles within a black-bor neural network could extend this technique to more complex, real-world dynamical systems beyond simple textbook problems.
certain position from initial
Prior works such HNN Greydamus et al. 2019 draw inspiration from the intuition of Case (III) to leverage global conservation laws, but are still realized in the form of local, step-by-step constraints between adjacent time steps, as in Case (II). To move step forward, we aim to design network architectures that enforce global conservation principles in a truly global manner, allowing black-box neural networks to capture complex, long-range, and indirect physical relationships. This approach seeks to fully exploit the expressive power of neural nelworks while remaining faithful fundamental physical laws.
B Discrete Left Hamiltonian H
The discrete right Hamiltonian H gives the equation of motion in the form
a: H (9+1,,p)) P++1 - (g)+1.P().
(9)
Tt can be first-order
approximation of the continuous Hamiltonian H by P+41).
ge 4++1 A/V AIV,H((g,2)) Pt41), Pt+1 Pt
(10)
When the states are extended to blocks, the biock-wise discrcte left Hamiltonian defined as
b - (Qttett p!tt),
Pts = H (Qttetb p!+b).
(11)
16

---

Published in Transactions on Machine Learning Research (02/2026)
Fig. 12 below illustrates the relation between discrete left and right Hamiltonians in both classical forms and our block-wish extensions. Both the left and right Hamiltonians take each other's outputs as inputs.
discrete right Hamiltonian
block-wise discrete right Hamiltonian
4
9
p p
P
L
4 Fo
1
C0
11
C2
L3
C4
Es
t4
discrete left Hamiltonian
block-wise discrete left Hamiltonian
Figure 12: Discrete left. and right Hamiltonian blocks. They describe the same formulation of a physical system with different discretizations (that both preserve the system's 's symplectic structures). They take each other's outputs as inputs and vice versa.
C Physical Interpretations for DHN
In this section, discuss whether extending the discrete Hamiltonian to block sizes and strides greater than still allows for explicit physical interpretations. Specifically, we address the following two questions:
(i) What is the conserved quantity with the block-wise Hamiltonian? For a discrete Hamiltonian block of size b, the conserved quantity is the sum of the total energy independent states. More specifically, the states within a discrete Hamiltonian block can interpreted as those of identical physical systems, each starting different time. Fig. provides an ration of this concept.
System started at iz
System started at tz
System started at fz
System started at fy
5
fa
fg
C
Discrete Hamiltontan block L = 140. 91 42 qal
Figure 13: Physical interpretations of block-wiss wise discrete Hamiltonian. The figure shows an example of block size b = 4. The states within discrete Hamiltonian block can be interpreted those of identical physical systems, each starting at different time step.
17

---

Published in Transactions on Machine Learning Research (02/2026)
Consider the case where the block size is b 4. Suppose we have four identical physical systems, each initialized at different times: lo. L,, 12, 13. By time 13, these systems will have evolved for 0, 1, 2, and 3 time steps, respectively. If we take their states at 83 and stack them together, we obtain a state block that effectively represents four consecutive states spanning four time steps within a single system. Importantly, the Tour states at (3 remain independent, as the lour duplicated systems not interact with one another. Thus, the conserved quantity in this framework is the total energy summed across all these identical, non-interacting systems. Fig. 14 provides detailed illustration of this.
40
46
17
422
System start at
System start at f
System start at t
System start t
q
input
output
Block-wise discrete (right) Hamiltonian
equation of motion
output
input
Eo
F
C3
fs
E6
8y
to
E10
412
System start at (
P
System start P
System start at P
System start at P
Figure 14: Detailed explanations of the block-wise discrete Hamiltonian. The figure shows an example of block size = 4 and stride 8 == 2.
(ii) What are the relarations compared to the classical discrete Hamiltonian? When extending the classical discrete Hamiltonian to a block-wise formulation, certain physical constraints are relaxed. The two main relaxations are as follows:
First, instcad conserving the energy of each individual state, the block-wise Hamiltonian conserves the total energy summed over b states. This allows for different energy distributions across the b states, making the constraint weaker than enforcing per-state energy conservation.
Second, as discussed in Sec. 4.2 when the stride s is smaller than the block size b, there is an overlap of -- 8 between network inputs and outputs. In theory, exact energy conservation (in the generalized form) requires that the overlapping states remain identical. However, in practice, this self-consistency loss is rarely minimized exactly zero. The extent which minimized depends on factors such as network expressivity, architocture, and hyperparameters and s, which in affect how well energy conservation is maintained.
18

---

Published in Transactions on Machine Learning Research (02/2026)
Despite these relaxations, the model still enforces form of physical consistency across the trajectory. Rather than strictly conserving per-state energy; it shifts toward preserving higher-level conserved quantities. This relaxation also opens the door to developing more a abstract notions of physical consistency on latent embeddings instcad of the raw observed states.
C.1 Formal Derivations of the Conserved Quantity
Here provide formal derivations of the (generalized) energy conservation in block-wise Hamiltonian. We show that after extending the discrete Hamiltonian into our block-wise discrete Hamiltonian with block size and stride it still has a conserved quantity that relates to the system energy.
The discrete Hamiltonian can be derived two ways: on the one hand, it can be directly derived from discrete systems, and the energy conservation (time invariance) is expressed as its symplectic structure (intuitively, it means that across a long time, the energy fluctuate around a constant value, instead of staying constant. precisely); on the other hand, it can be derived as the first-order approximation of the continuous Hamiltonian, and the energy conservation also refers to approximately conserving the energy of continuous system. Here, for our block-wise Hamiltonian, the symplectic structure is highly relevant to numerical integrations for state updates from to L + 1, while in our case with stride > 1 and different masking patterns, our state update is much more flexible.
analyze its conserved quantity with the second interpretation, because
First-order approximation of the continuous Hamiltonian Given time steps t = 0,... ,T suppose we have a physical trajectory with states Po): (9I, (QT,PT). The total action with the discrete right Hamiltonian is
T-1 S+ S+= Σ (pe41 9-+1 - H (ge, P(+1)).
(12)
On the other hand, the action of the continuous Hamiltonian is
s= f (v(7) (() ((T) ((()))
(13)
Here we assume that the discrete time steps are uniformly sampled between [0, 1] with interval At 1/T. To derive numerical integrators using discrete Hamillonians, it important to recognize that the discrete
action must approximate the continuous action Lall & West (2006) Marsden & West (2001) et al. 2006), that is, St x S. This means that the right Hamiltonian should satisfy
Hairer
((t))a
# (ge, Po41) R P++1 4-+1
(p(T)-Q(T) H((((), dr.
(14)
With first-order approximation, we have
H+
PPt1 (ge + AIV, AIV,H(q.Pt+H At(p+ V VpH((g.)4s)) - H(gp = PP41 x qe AtH((:.P(+1) + 0
(15)
With time invariance, the continuous Hamiltonian H conserves the total energy of the system, and the discrete right Hamiltonian H + is an approximation of that.
Block-wise approximation Consider a discrete right Hamiltonian block of block size and stride 8, with inputs are Q++b = (qt.... (b+b) and pl++-b [pets.... :P4s+4] and satisfying the equation of motion as Eq. 5 for any The total action approximation generalizes to
8+b Σ k=
(k+s)
H H+ (Q) pt+8+b) x
pets Ikts
(p(7) - ((T) P(((())
(16)
19

---

Published in Transactions on Machine Learning Research (02/2026)
And similar to Eq. 15
have
1+b Σ (pu+s The + &AiH + O(A12).
H (Q Pitsth)
(17)
Omitting the higher-order terms ( this can be viewed as a summation of identical systems starting at time steps = 0.... b, and their continuous Hamiltonians are discretized with time intervals sât. In this analogy, the total conserved quantity is an approximation of
+ +At) p(r + kA()).
(18)
k=0
D Denoising Inference
As mentioned in Sec. 3.3 at training time, we apply noises with randomly sampled scales to different unknown states. However, at inference time, we progressively denoise the unknown states with sequence of decrcasing noise scales that are synchronized on all unknown states. Fig. illustrates the iterative denoising process at inference time with a pair of DHN blocks H+ and H-
p
p
4
Figure 15: Iterative denoising at inference time. A pair of DHN blocks, 11 'I' with block size b and stride s, jointly applied to stack of + states to denoise the unknown blocks. Progressive denoising is conducted by iteratively applying block-wise H+ and H and gradually decreasing the noise scales.
Consider a block of states (Q) p(+o+b) with some unknown slots indicated by binary masks (M/+++b
N , N((++b) We iterative apply H H+ (Q) pttsth and H T= ,p*h)) to infer the unknown states conditioned on the known states. We sample the initial state predictions from a Gaussian distribution
and then apply iterative denoising with decreasing noise levels 1 aN ON-1 > ai an 0, in
Algorithm [ This is similar to the diffusion models Ho et al. (020 Song et al. 2020
E Network Architecture Details
As mentioned in Sec. 3.4
we employ a decoder-only transformer to learn the block-wise Hamiltonian. Its
detailed architecture shown Fig. [6] The network is very lightweight, with only two transformer blocks, plus input emboddings and the (lincar) output projection. The two transformer blocks are identical, with input/ aaput/oute /utput feature dimension 128, FFN hidden feature dimension 120x4, and 4 heads in their multi-head attentions.
20

---

Published in Transactions on Machine Learning Research (02/2026)
Algorithm 1: Iterative denoising with the Hamiltonian blocks
Known states (Qknown) ( + (P (Kooww)) t binary state mask (M,+s , " .Nttoto). Result: The complete block Q++s+b plto+b with the masked states predicted. Sample initial state predictions Qi +++++b pttttb ~ I..
Data:
for n = N 1, do
Sample noise E NO I), and apply H+ denoising:
(Qprad)ite +totb +- - -1 Vpp (Qttb, pitetb) + anE, (Ppred): + (1 (1 P(tsth) + anE
Combine the predicted states with the known states:
Qitstb + (1 - 1--M Mitstttt(@s0 (Qrea)
(Qknown)i=st) b+s+b + Mi+ 1 1
pt+ + (1 - M1t (PPrtt)))
M, (b (Pknow))
Sample noise E NO, I), and apply H denoising:
(Q pftt)) + anE, (Prraa)14: sto+b +- -(1 an) V (Q p/+b) + an
(Qprea) ( +- (1 (1- an VPH
Combine the predicted states with the known states:
0,4b (1 MM+ (CQrea)) + M1+h (Qknowa)t"
1ts+b + M1+e+b ( 14++b pt+++b +- - M,te+b (Pprot) [+s
(Pknown)its*
end
F
Experimental Setups
F.1
Physical Systems and Parameter Randomization
Here we elaborate the details of the two settings experiment with: the single pendulum and the double pendulum, as illustrated in Fig, 17 In both settings, we first define the generalized coordinate g and the system's Lagrangian L(g. Q). The generalized momenta is then defined by p V VgC. We set the gravitational acceleration 0.981.
Single pendulum this system, the varied parameter is the string length 1, randomly sampled between [0.5, 1.0] for each trajectory. The mass of the ball is set to be m = 1. The generalized coordinate is defined as 8, with initial value f = x/2 for all trajectories. The Lagrangian of the system is
L= ml -mg((1 mmo((1-cos))
(19)
Here (q.p) are the standard angular position and angular momentum in spherical coordinates.
Double pendulum In this system, the varied parameter the string length l2, randomly sampled between [0.5, 1.5] for each trajectory. The remaining fixed paramcters are I = 1, m, = my = 1. The generalized coordinate defined as = (0,, 02), with initial values 0, 02 = \/2 for trajectories. The Lagrangian of the system is
C = 2
(m1 + m20 + mm2208 + m2h1tz0 02 cos(A: - 02) + (m1 - m2)gh1 cos 01 + mzglz cos fg. (20)
21

---

Published in Transactions on Machine Learning Research (02/2026)
states (noised)
noise scales
global z
state emb
noise scale emb
emb
positional embedding
transformer block
transformer block
linear
H
Figure 16: Detailed architecture of the decoder-only transformer. Learnable modules or variables are shown in dark purple. The final output of the network is scalar Hamiltonian value, decoded from the position the global latent code in the transformer (shown yellow).
Single pendulum
Double pendulum
Initial state
Initial state
0
4
1 l2
Figure 17: Physical systems for the experiments. Circles with dotted lines and swallower colors the initial states, which identical to all training and testing trajectories. Circles with solid lines and darker colors illustrate the intermediate states along the simulated trajectory.
22

---

Published in Transactions on Machine Learning Research (02/2026)
F.2 Network Training and Computational Resources
At the training stage, optimize for both the network weights and the latent codes. We use Adam optimizer with a learning rate 10 and weight decay 10 and train for 200 epochs with a batch size of 32.
At the test-time optimization stage, we freeze the network weights and only optimize for the latent codes. e use an Adam optimizer with a learning rate 10-2 and weight decay 10-1, As all latent codes are optimized independently, we optimize for 1000 epochs with a batch size of 100.
All networks are trained on single Titan RTX 24GB GPU.
23


Reviews:
[{"review_id": "oDPSrBAtY6", "reviewer": "Reviewer_kx7j", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper presents a physics-inspired reasoning framework for problem sets that enforce dynamical structures. The authors tackle issues stemming from current methodologies' obsession with local temporal prediction via simulation; instead, they propose a general-purpose neural operator with architectural innovations. The proposed denoising Hamiltonian network (DHN) is a unified model that extends Hamiltonian neural operators to capture non-local temporal dynamics, equipped with a denoising objective analogous to variational integration. The article demonstrates its performance on trajectory fitting for simulated data (pendulums) and relevant inference tasks on 1D or 2D degree-of-freedom (DOF) physical systems. The authors successfully demonstrate that the proposed architecture retains the scalability and robustness required for use in general physics-related tasks.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The paper is mostly sound. The key component of DHN is its use of discretized Hamiltonian dynamics with consecutive state blocks, which enables the capture of broader temporal correlations through a vectorized procedure. Another methodological improvement over traditional Hamiltonian neural networks is the masked modeling and denoising training scheme, which injects random perturbations into the input states (the “position” slots). By encouraging the model to correctly identify the original states, the authors claim this approach mitigates the cumulative errors that arise from numerical integration, a novel concept for PINN studies. Lastly, the authors seem to have incorporated a transformer-based decoding architecture to handle variable inputs and various masking schemes. I believe the experiments are standard and were appropriately conducted, but I did not go over every detail in the appendix.", "audience_yn": "Yes", "audience_explanation": "I find this work intriguing and believe it is novel in this field. Physics-based neural architectures that incorporate internal differential equation solvers inevitably struggle with the intrinsic numerical errors that arise during trajectory acquisition. The authors actively incorporated temporal architectures, such as transformers, which can effectively learn denoising spatiotemporal patterns within an erroneous learning environment. That said, the work would attract a broader audience if the proposed method demonstrated certain features, such as generality or scalability (as implied in the introduction), through more varied general experiments.", "requested_changes": "* I think the introduction (Section 1) needs significant improvement to clarify the exact motivation and expected outcomes of the suggested improvements.\n* The main argument seems to focus on the error-prone nature of local temporal prediction, an issue DHN claims to solve by examining global sequences via a masked training regime. However, the experiments show that leapfrog methods also demonstrate comparable improvements over forward Euler. This suggests that theoretical numerical analysis also has the potential to solve the outlined issue without the excessive memory and computational constraints of DHN. The authors are encouraged to disclose their computational costs in detail and discuss how theoretical work on Hamiltonian structures could benefit their work in the future.\n* I believe the breadth of experiments in the current version is quite limited, even though the network architecture itself implies the universality of its solutions. The authors are encouraged to include other simulations besides pendulums or conduct more intensive ablation studies on the representation learning capabilities of the DHN framework. I also request the authors to test the DHN is scalable to multiple DOFs (>10), unless elaborate on why it is not scalable to multi-dimensional problems.", "broader_impact_concerns": "There is no concern.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "sZIOR9OErp", "reviewer": "Reviewer_SNKY", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This work proposes Denoising Hamiltonian Networks (DHNs) that incorporate a denoising loss in addition to regular Hamiltonian loss over block data. The benefits include better generalization over multiple trajectories in a given system and improved long-range modeling.", "claims_and_evidence_yn": "Yes", "claims_explanation": "From my understanding, the claim is that a Hamiltonian neural network (HNN) with denoising can improve generalization and long-range modeling of physical systems. These metrics are improved by the proposed denoising Hamiltonian neural network (DHN) in the experimental results. However, the experiments do not clearly show that the improvements actually come from the architectural/training choices, and not the nature of the data used. Therefore, I feel that, currently, the claims are partially supported.", "audience_yn": "Yes", "audience_explanation": "I believe this work will interest the ML research community that focuses on physical reasoning using machine learning.", "requested_changes": "The following are a mix of requested changes and questions about the work. The important ones have a hash symbol (#) next to their numbering. My major concerns are with writing -- specifically, in comparing the contributions of this work against recently proposed works and clearly explaining the experimental design choices.\n\n**# C1. Related works**: The absence of recent related works in this submission makes it hard to gauge the contribution of the paper. Only three references from 2021-present are included in this work, and none of them are related to the core topic of this work. Although the submission is evaluated on the basis of claim-and-evidence, it is difficult to know how original the primary claim in this paper is without a sufficient literature review. The lack of recent related works particularly impacts the introduction section. The introduction section currently includes no references, and the second paragraph that discusses the limitations of current methods is on weak grounds with references. If the limitations listed in the second paragraph cannot be backed with references, then the existence of these limitations must be proven in the work.\n\n**# C2. Role of denoising**: Why is denoising required? Sec. 3.3 (first paragraph) says \"we want the Hamiltonian blocks to...also learn the state optimization per time step for inference.\" I didn't understand why the Hamiltonian blocks must learn state optimization for inference and how denoising helps to achieve that goal. Is there an ablation on what happens without $\\mathcal{L}_{\\text{denoise}}$?\n\n**# C3. Noisy samples for HNN**: This question is related to C2. Were noisy samples used in training HNNs (baselines)? I know HNNs do not have a denoising model, but I was wondering if noisy training data implicitly improves generalization. If it doesn't improve the performance, then this experiment will also strengthen the cause for $\\mathcal{L}_{\\text{denoise}}$.\n\n**# C4. Role of overlapping regions**: The baselines are trained on \"pairs of adjacent time steps,\" while DHNs are trained on overlapping blocks. How much difference will this make to the performance? Please correct me if it's not possible to compare these two settings.\n\n**C5. Parameter estimation experiments**: There are neural network-based methods for parameter estimation in PDEs from a few measurement samples [R6, R7]. Are these baselines valid for the experiments in Sec. 4.2?\n\n**C6. Why is $z$ included?** Why is the global latent code $z$ included in the architecture? Can the authors clarify how it \"serves as a query token for outputting the Hamiltonian value?\"\n\n**C7. Super-resolution experiment baseline**: How is the CNN used for super-resolution? And why is a CNN without a physical prior used for trajectory interpolation, instead of cubic Hermite splines or physics-informed learned models?\n\n**C8. Usage of the term \"neural operator\"**: In the second paragraph of the introduction, it is written \"we aim to design more general neural operators that both follow physical constraints, and unleash the expressivity of neural networks...\" It is not clear how \"neural operator\" is different from \"neural networks\" in the submitted work. The term \"neural operator\" is generally used in contemporary works [R1-4] with its meaning introduced by [R5].\n\n**References**:\n\n[R1] Ziming Liu, Yixuan Wang, Sachin Vaidya, Fabian Ruehle, James Halverson, Marin Soljačić, Thomas Y. Hou, Max Tegmark, \"KAN: Kolmogorov-Arnold Networks\", ICLR 2025.\n\n[R2] Boris Bonev, Thorsten Kurth, Christian Hundt, Jaideep Pathak, Maximilian Baust, Karthik Kashinath, Anima Anandkumar, \"Spherical Fourier Neural Operators: Learning Stable Dynamics on the Sphere\", ICML 2023.\n\n[R3] Makoto Takamoto, Timothy Praditia, Raphael Leiteritz, Dan MacKinlay, Francesco Alesiani, Dirk Pflüger, Mathias Niepert, \"PDEBENCH: An Extensive Benchmark for Scientific Machine Learning\", NeurIPS (D&B) 2022.\n\n[R4] Steven L. Brunton, J. Nathan Kutz, \"Promising directions of machine learning for partial differential equations\", Nature Computational Science, 2024.\n\n[R5] Nikola Kovachki, Zongyi Li, Burigede Liu, Kamyar Azizzadenesheli, Kaushik Bhattacharya, Andrew Stuart, Anima Anandkumar, \"Neural Operator: Learning Maps Between Function Spaces\", JMLR 2023.\n\n[R6] Masanobu Horie, Naoto Mitsume, \"Physics-Embedded Neural Networks: Graph Neural PDE Solvers with Mixed Boundary Conditions\", NeurIPS 2022.\n\n[R7] Xuyang Li, Hamed Bolandi, Talal Salem, Nizar Lajnef, Vishnu Naresh Boddeti, \"NeuralSI: Structural Parameter Identification in Nonlinear Dynamical Systems\", ECCV 2022 Workshop on Computer Vision for Civil and Infrastructure Engineering.", "broader_impact_concerns": "No concerns about the ethical implications.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "Jiz9zSkpCw", "reviewer": "Reviewer_3Acm", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper explores what physical attributes a neural network should possess when modeling dynamical systems. While partial differential equations describe systems through strictly local temporal relations, neural networks need not be bound by this locality. The Denoising Hamiltonian Network extends traditional Hamiltonian neural operators to capture non-local, block-wise temporal interactions while maintaining core physical constraints such as energy conservation. By introducing learnable block sizes, variable strides, and a denoising objective, Denoising Hamiltonian Network generalizes physical reasoning beyond forward simulation, enabling multi-scale temporal modeling and robustness against numerical integration errors. This framework bridges the gap between rigid physical formalism and the flexibility of deep networks, offering a new paradigm for physics-based learning architectures.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The claims made in the submission are partially supported by accurate and convincing evidence. The paper provides solid experimental results on representative tasks such as forward trajectory prediction and super-resolution interpolation, demonstrating that the proposed Denoising Hamiltonian Network (DHN) can capture non-local temporal dynamics and outperform baselines in long-term stability and reconstruction accuracy. The methodology is clearly presented, and the empirical trends align with the theoretical motivation for block-wise and denoising-based physical reasoning.", "audience_yn": "Yes", "audience_explanation": "Yes, the findings of this paper would likely interest a portion of TMLR’s audience, particularly those working on temporal modeling, scientific machine learning, and sequence prediction. The model in this paper introduces a block-wise operator that generalizes beyond traditional one-step (one-tick) prediction, enabling reasoning over multi-tick temporal blocks. This represents a conceptual shift similar to what is now emerging in broader machine learning fields — moving from local, incremental updates toward non-local, span-level modeling that captures higher-order dependencies.", "requested_changes": "1. Include missing examples: The paper mentions parameter estimation and trajectory inpainting as representative reasoning tasks but does not provide corresponding experimental demonstrations. Including at least one clear example—such as parameter estimation on a pendulum or spring system—would make the claims of general applicability more convincing and illustrate DHN’s versatility beyond forward prediction and interpolation.\n2. Improve figure clarity and discuss figure implications: Figures 6 and 7, which are central to illustrating DHN’s comparative performance and stability, are difficult to interpret due to small fonts and overly similar color choices. The visual distinctions between models and baselines should be enhanced, and key trends or takeaways should be clearly annotated within the figures. A more legible design would help readers better appreciate DHN’s improvements in long-term trajectory reconstruction and error reduction", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)