Deconstructing the Inductive Biases of Hamiltonian Neural Networks
Authors: Nate Gruver
Marc Anton Finzi
Samuel Don Stanton
Andrew Gordon Wilson
Venue: ICLR 2022 Spotlight (2022)

Abstract: Physics-inspired neural networks (NNs), such as Hamiltonian or Lagrangian NNs, dramatically outperform other learned dynamics models by leveraging strong inductive biases. These models, however, are challenging to apply to many real world systems, such as those that don’t conserve energy or contain contacts, a common setting for robotics and reinforcement learning. In this paper, we examine the inductive biases that make physics-inspired models successful in practice. We show that, contrary to conventional wisdom, the improved generalization of HNNs is the result of modeling acceleration directly and avoiding artificial complexity from the coordinate system, rather than symplectic structure or energy conservation. We show that by relaxing the inductive biases of these models, we can match or exceed performance on energy-conserving systems while dramatically improving performance on practical, non-conservative systems. We extend this approach to constructing transition models for common Mujoco environments, showing that our model can appropriately balance inductive biases with the flexibility required for model-based control. 

Published as a conference paper at ICLR 2022
DECONSTRUCTING THE INDUCTIVE BIASES OF HAMILTONIAN NEURAL NETWORKS
Nate Gruver, Mare Finzi, Samuel Stanton, Andrew Gordon Wilson New York University
ABSTRACT
Physics-inspired neural networks (NNs), such Hamiltonian Lagrangian NNs, dramatically outperform other learned dynamics models by leveraging strong in- ductive biases. These models, however, challenging to apply to many real world systems, such as those that don't conserve energy or contain contacts, a common setting for robotics and reinforcement learning. In this paper, we exam- physics-inspired ine the inductive biases that make physics inspired models successful in practice. We show that, contrary to conventional wisdom, the improved generalization of HNNs is the result of modeling acceleration directly and avoiding artificial com- plexity from the coordinate system, rather than symplectic structure or energy con- servation. We show that by relaxing the inductive biases of these models, match or exceed performance on suergy-conserving systems while dramatically improving performance approach to constructing transition models for common Mujoco environments, showing that model can appropriately balance inductive biases with the flexi- bility required for model-based control.
practical, non-conservative systems. We extend this
Expectation
Reality
Hazzhoeia
Handhars
Symptkent
Scceal Ondier Ruse
Sympleene
100%
100%
Performance
75%
Perfornance
75%
50%
Secoedi
50%
Bias
25%
000
25%
Bac
Figure 1: The common perception in physics-informer machine learning is that increased perfor- mance is the result of complex biases. We find, however, that simpler implicit biases (such as second-order structure) often account for almost all of the improvement over baselines.
INTRODUCTION
The inductive biases of convolutional neural networks, such as translation equivariance, locality, and parameter sharing, play key role in their generalization properties. Other biases have similarly guided the development of deep models for other domains like graphs and point clouds. Yet since models often encode multiple biases simultaneously. can be challenging to identify how each contributes to generalization in isolation. By understanding which inductive biases are essential, and which are merely ancillary, we can simplify our models, and improve performance. For example, Wu et al. (2019) demonstrate that Graph Convolutional Networks can be radically simplified to mere logistic regression by removing nonlinearities, leading to dramatic gains in computational efficiency, while retaining comparable accuracy.
Hamiltonian Neural Networks (HNNs) (Greydanus et al., 2019) have emerged as a leading approach for modeling dynamical systems. These dynamics models encode physical priors and outperform alternative Neural ODE approaches (Chen et al., 2018). the spirit of Wu et al. (2019), we seek to identify the critical components of HNNs. Since Lagrangian models (LNNs) (Lutter al., 2019a;

---

Published as a conference paper at ICLR 2022
Cranmer et al., 2020) share the same structure and inductive biases as HNNSs we focus HNNs where energy conservation and symplecticity are more explicit.
HNNs encode number of inductive biases that help model physical systems:
1. ODE bias: HNNs model derivatives of the state rather than the states directly.
2. Second-order (SO) bias: HNNs model changes in position through changes in velocity.
3. Energy conservation bias: HNNs conserve their learned energy function.
4. Symplectic bias: HNNs dynamics are symplectic: phase space areas are conserved, and the vector field has a Hamiltonian structure.
In this paper we theoretically and empirically examine the role of these biases. In contrast conven- tional wisdom, we find that the generalization benefit of HNNs is not explained by their symplectic or
properties, but rather by their implicit second-order bias. We highlight these findings in Figure 1. Abstracting and extracting out this second -oddr bias, we show how to improve the performance of Neural ODEs, empowering applications when HNNs assumptions are violated as is often the case. Code for our experiments can be found at: http://gthhu..com/nnwveeeeeeon
2 RELATED WORK
Physics-inspired models and energy conservation Since Greydanus et al. (2019), many re- searchers have sought extend the HNN approach to make more general or applicable to systems that break energy conservation. Jin et al. (2020), Li et al. (2020), Tong et al. (2021), and Xiong et al. (2020) propose methods for creating neural networks that preserve symplectic structure di- rectly. Zhong et al. (2020), and later Desai et al. (2021), Li et al. (2021), and Lee et al. (2021) propose models with additional capacity for changes to the system energy. In our analysis, we com- pare against approaches that add additional capacity for changes in energy, although approach is fundamentally different - we attempt to remove unnecessary biases rather than add complexity.
Physics-inspired models for control A large and rapidly expanding body of work explores how to use physics-based biases in dynamics models with controls or contacts (Lutter et al., 2019b; Zhong et al., 2019; Gupta et al., 2019; 2020; Chen al., 2019; Hochlehnert et al., 2021; Chen al., 2020; Zhong et al., 2021b) intended for application in model-based planning. Especially relevant to our evaluations, Alvarez al. (2020) models MuJoCo (Todorov al., 2012) trajectories with numerically integrated neural network but does not explore other physics-inspired inductive biases.
Analyzing physics-based inductive biases Karniadakis et al. (2021) and Liu (2021) propose conceptual frameworks for deep learning with physics-based inductive biases, but present minimal empirical analysis common design decisions. Zhong et al. (2021a) compare many approaches to physics-inspired deep learning, with results that parallel some findings here. The contribution of our work, however, is not simply benchmarking but also an actionable theory of HNNs' success. In spirit, our work is similar Botev et al. (2021), which also critically examines the role of physics- based priors in dynamics models, though their focus is on learning latent space dynamics, while we focus on temporal models in observation space.
BACKGROUND
We consider dynamical systems described by ordinary differential equations (ODEs) which deter- mine how the system evolves over time. Hven with high order derivatives, these systems can be arranged into the form dz di F(z,t) for z R If the dynamics F are dime-independent, then the ODE can be understood as vector field, specifying at each point where the next state will be.
Neural ODEs (NODEs) (Chen et al., 2018) parametrize a vector field with neural network and learn the dynamics directly from observed trajectories. A NODE dynamics model Fg rolled out from the initial condition 2n with ODE integration, &, ODESolve(2,, Fo, t), and fit to trajectory data by minimizing an L2 loss L(0) = Ei= || 12 between the predicted and observed trajectories,
i, and 2t-
2

---

Published as a conference paper at ICLR 2022
Like NODEs, HNNs also model dynamical systems parameterized vector field,
dz = JV Ho with J -- 0
0 1
!
(1)
dt
where Hg is a neural network with scalar output (Greydanus et al., 2019).
This differential equation expresses Hamiltonian dynamics where and are the canonical positions and momenta,
d
9 P
aA
and
dp ai
dt
By
It is common practice to also explicitly define H = - V. where T and V are the kinetic and potential energy (Zhong al., 2019; Gupta et al., 2019; Finzi al., 2020). The assumption that the Hamiltonian is separable holds for mechanical systems and allows for further simplification.
1 20 (4) Volg).
Ho p)
where positive definite mass matrix M and scalar V are outputs of the neural network.
4 BREAKING DOWN HNN PERFORMANCE
In the following section investigate what extent commonly held beliefs about HNN prop- erties actually explain the ability of HNNs to generalize. To separate out the different properties of these models, we select synthetic environments from Finzi et al. (2020) and Finzi al. (2021) that are derived from time independent Hamiltonian, where energy is preserved exactly. We use a link pendulum in angular coordinates. and kSpringPendulum, a pendulum connected with & spring links in Cartesian coordinates.
We compute relative error between predicted states, é, and ground truth, 2, as || 2112/ |I ||2 and between the predicted and ground truth Hamiltonian as f | Following Finzi al. (2020), evaluate the performance of the model by computing the geometric mean of the relative error of the state over rollouts length 20 times the size used at training, which more faithfully predicts downstream performance compared to other metrics like MSE (Finzi et al., 2020). The geometric mean of a function f over time T is given by exp(+ f log ((()dt).
4.1 ENERGY CONSERVATION
HNNs are commonly believed to be superior for energy conservation than comparable models (Greydanus et al., 2019; Cranmer et al., 2020). While there is some empirical evidence to sup- port this claim, precise mathematical explanation for why this should be the case has not been established. Surprisingly, we find that conditioned on the trajectory reconstruction error, HNNs are no better at conserving the true energy of the system than unconstrained NeuralODE models.
Up to the numerical accuracy of an ODE solver, HNNs do conserve their own learned energy func- tion Á (different from the true energy of the system H), due to basic properties of Hamiltonian mechanics,
dff()) VĀT dē CATJVĀ=O
dt
where the last equality follows from the antisymmetry of J. If the HNN has fit the data well, we might expect H to be close to the true Hamiltonian H, and if Á is conserved then it seems that Il should be too. As we show in Appendix B.2, the problem with this argument is that we have no guarantees that H and H close even if we have fit the data well with low error in the rollouts or the dynamics. Since the dynamics error and the training rollout error depend only on gradients VÍ, while the gradients may be close, the differences between the two sealar functions can grow arbitrarily large.
In Figure 2 (left) we show that the degree of energy conservation is highly correlated with the rollout error of the model, regardless of the choice of architecture. While HNNs have better rollout
3

---

Published as a conference paper at ICLR 2022
2 link
3 link
link
10"
10a
Energy Vinlation
10-2
Chain
10"2
Energy Erro
10
10-4
10a
HNN NODE
Spring
10-2
10-4
10-2
10P
Rollout Error
10
10-1
10
10-2
HNN NODE + SO
NODE
Rollout Time T
The degree of energy violation ( - = test rollouts as a function of rollout relative error (l - =||/||@||[=||| across different environments and random seeds. Both HNNs and NeuralODEs mance, whether or not the model is Hamiltonian has little impact on the energy violation. Right: Energy violation test trajectories plotted as a function of the time T of the rollout, with the shaded regions showing 1 standard deviation in log space taken across 5 random seeds and the test trajectories.
Figure 2:
Left:
scattered around the line
y. Conditioned on the rollout perfor-
performance than NeuralODEs, they are on the same regression line with Rollout Error x Energy Violation. The differences in energy violation are best predicted by the rollout error and not the architecture.
In Appendix B. we derive a bound that helps explain this behaviour. Given that the trajectories are a bounded region of the phase space and that there a fixed amount of error in the dynamics model, the energy violation grows at most linearly in time and the dynamics error. In Figure 2 (right) we demonstrate empirically that energy is not conserved as time progresses for HNNs. In fact, the energy error of both NODE and HNN models grow linearly our bound suggests. Although energy conservation may be helpful for generalization, the evidence does not indicate that HNNs inherently better conserving energy than NODEs, suggesting that the superior generalization of HNNs cannot be attributed to superior energy conservation.
4.2 SYMPLECTIC VECTOR FIELDS
defining property of Hamiltonian mechanics is the fact that the dynamics are symplectic. If energy conservation does not explain the effectiveness of HNNs, the symplectic property of HNN dynamics may be the cause. Informally, one the consequences symplectic dynamics that every part of the state space is equally attractive and repulsive. There are no sources where all nearby trajectories flow out from, or sinks where all nearby trajectories flow into, only saddle points and centers where the inflow and outflow is balanced. Symplectic integrators (Leimkuhler and Skeel, 1994) make use of this property for more stable integration of Hamiltonian systems over very long timespans, and it is intuitive that enforcing this property in learned dynamics would have benefits also, at the very least for reducing the size of the hypothesis space.
More formally, symplecticity is the property that the matrix (the symplectic form) is preserved by the dynamics (Equation 1). This condition can be expressed as constraint on the Jacobian DF of the vector field F(z) dt the Jacobian, symplecticity is the condition DFT JDF = 0 or equivalently (JDF)" JDF since JT = -J The significance of this condition is that areas occupied by states in phase space (which have units of energy) are preserved by symplectic transformations, One consequence that volume of solutions in phase space will continue to occupy the same volume over time and will not be compressed or expanded. other words, the vector field has 0 divergence: Tr(DF) = 0, which can derive from the above expression.
da (here the derivative D maps from a function its Jacobian). terms of
The energy violation considerably larger than merely the numerical error associated with the solver.
4

---

Published as a conference paper at ICLR 2022
Ralban 10
Chain
0.01
Chain
Chain
0.011
Spring
Spring
Spring 3
Cloin
Choin 2
Cloin
Spring
Speing
Spring
[e-10 Ic-08 1e-060.0001
0001
200
Symplectic Erno
Figure 3: Left: Test rollout error a function of the regularization weighting in the loss. Even at an optimally chosen symplectic regularization strength, the benefit to model generalization is negligible. Right:
Test rollout regularized models. For systems with more than a couple degrees of freedom, symplecticity error is negatively correlated with the quality of predictions.
plotted against the final value of the symplecticity error for the
On IR" it can shown that symplectic vector fields can expressed as the dynamics of some Hamiltonian system, and vice versa:
== JVH = (JDF)" = JDF
(5)
To show the forward direction, one can simply substitute in Hamiltonian dynamics. is clear that the symplecticity property satisfied: using J² = II, JDF JD(JHH) J2 H - H and
-v² H is symmetric matrix. The reverse direction is less obvious; however, by Poincaré's s lemma if vector field F satisfies (JDF)² JDF R", then there exists Hamiltonian function H such that F H, which is shown in subsection B.3.
The equivalence of Hamiltonian dynamics and symplecticity allows separate the unique proper- ties of HNNs from other inductive biases that result indirectly from modeling F' through H. Follow- ing Ghosh et al. (2020), we can create regularizer SymplectieBrror("" = [[DD)"""JDDFP DDF that directly measures the degree to which the symplectic property is violated. By parametrizing NeuralODE and regularizing the symplectic error, we can enforce Hamiltonian structure while still directly modeling ăi dz F rather than H. Alongside unregularized NeuralODE, we can isolate and evaluate the benefit of this Hamiltonian structure bias with a direct comparison.
Surprisingly, we find that the Hamiltonian structure bias, as enforced by the symplectic regularizer, provides no real benefit the model's ability to generalize over the long test rollouts (Figure left). The achieved symplectic error Figure (right) is not positively correlated with the final test rollout error of the model, and in some cases even negatively correlated. Even when the symplectic error is very low and the symplecticity condition is enforced, there is no consistent improvement the rollout generalization.
4.3 SECOND-ORDER STRUCTURE
the superior performance HNNs over NeuralODEs does not come from their better energy conservation properties, nor from the symplectic structure of the predicted vector field, what the true cause?
In previous work, authors have used slightly different implementations of HNNs. One subtle im- provement over the original work (Greydanus et al., 2019) comes from explicitly splitting the Hamil- tonian as = T + V = T M(g) + V(g) and modeling the mass matrix M()) and the po- tential V(g) with separate neural networks rather than using a single neural network for H (Zhong et al., 2019). This splitting enforces a strong assumption about the functional form of the Hamil- tonian that applies to mechanical systems that makes it casier to learn and extrapolate. Through Hamiltons equations F JVH. this splitting in fact specifying the relationship between position and momentum d4 3p
8H M-)( and that forces can only affect
5

---

Published as a conference paper at ICLR 2022
NODE Models
HNN Models
Rodion Etror
10-1
10-2
Chain
Chain 5
Chain
Spring
Spring
Spring
Chain 2
Chain 3
Chain Spring
Spring 3 Spring 4
Without
Figure 4: Left: NODE model with and without second-orde structure (encoding dg/dt v). Right: HNN models with and without second-order structure. Models with the SO bias significantly outperform those that do not. Error bars show standard error across seeds.
Complex Coordinates
Simple Coordinates
10
Rollont Ervor
10-1
Rolkont
10-3
10-3
Chain
Chain
Chain
Chain 4
Spring
Spring
Spring
Spring
NODE SO
HNN
Figure 5: Left: Log rollout error for NODEs with second order bias and HNNS trained chain pendu- lums, where the analytic form of the Hamiltonian is simpler than the vector field. Right: Mechanic- sNNs and HNNs trained on spring pendulums, which have Hamiltonians and vector fields similar complexity. HNNs outperform NODE with second order bias on systems that use non-Cartesian coordinates. Error bars show standard error across seeds.
In fact, we can see that this assumption essentially leads to the second-order differential equation
M (q)p)
da
(d dt
(4) (2
dq .M -g) dV dy (6)
A(a,
d12
dt
dq di
dg
This second-order (SO) structure,
is a direct by-product the separable Hamil- tonian inductive bias, but more general, applying to both conservative and non-conservative phys- ical systems.
v A(g, v)
We can isolate the effect of this bias by directly observing its effect on both HNNs and NODEs. For HNNs the bias is made explicit in separable Hamiltonians, but not in the general case, when H p) the direct output of the network instead of (q).. We can design a NODE with second- order structure (NODE SO) by setting z [q. p] with = Mr and de
di v, di Ás(g, p), or
equivalently just the second order equation d'g 2 order structure on the test predictions of NODEs and HNNs. It is clear that this bias explains the superior performance
Ao(a, de Figure 4 shows the effect of second
HNNs much more than other biases that are frequently given more credit. In fact, when we add this bias to NODE models, we see that their performance more closely matches HNNs than vanilla NODES without creating a conservative or symplectic vector.
4.4 FUNCTIONAL COMPLEXITY
Adding second order structure to NODEs is always helpful, and matches the HNN performance for many of the systems. However, we see that there is still a gap for some systems, and curiously in each of these cases the system is described in non-Cartesian coordinate system, such with joint angles and Euler angles. Recall that with the symplecticity bias, we only found no benefit for enforcing that there exists a function Á such that f = JV A bias while parametrizing F; however, for HNNs this function not only exists, it is directly parametrized by the neural net. If the function

---

Published as a conference paper at ICLR 2022
Rollout Error
10-3
Chain
Chain
Chain 3
Spring
Spring
Spring
NODE
NODE
HNN
SymODEN
Figure 6: Comparing the performance on damped systems. The NODE + SO matches the perfor- mance of SymODEN with fraction of the parameters and compute. HNNs without forcing terms encode the wrong inductive biases and thus fit the data poorly. Error bars denote standard error across 5 seeds.
H happens to be simpler function to express and learn than HH, then representing the solution in this way can be beneficial.
For systems expressed in Cartesian coordinates like the spring pendulum, the mass matrix M (g) M is constant, and so the gradients of H = p Mp + V()) are simple to express and learn. How- ever, for systems with constraints such as the Chain pendulum, where states are typically expressed in angular coordinates, the mass matrix M()) will have complicated form and that complexity will be magnified when taking the derivative. As an example, consider the Hamiltonian that an HNN must learn for the 2 link chain pendulum versus the vector field that NODE + SO model must learn for this system (derived in Finzi et al. (2020), Appendix F.2). For the spring pendulum the functional complexity of the Hamiltonian and vector fields is comparable, while for the chain pendulum, the vector field contains many more terms.
Parameterizing such system via its Hamiltonian simplifies the learning problem, and enables neural network to converge more rapidly towards plausible solution. This observation aligns with the insight in Finzi et al. (2020), which shows that changing the coordinate system Cartesian dramatically simplifies the learning problem, at the expense of needing to enforce the constraints to the configuration space more directly.
Figure 5 shows the relative performance of the NODE+SO across the chain pendulum and spring pendulum environments. As we would now expect, the gap between the NODE+SO and HNN vanishes (and even favors NODE+SO) when complexity of the Hamiltonian and vector field are comparable.
4.4.1 NON-CONSERVATIVE SYSTEMS
Perfectly energy-conserving systems are useful for analyzing the limiting behaviour of physics- informed networks, but in the vast majority of real world applications, we do not model closed systems. Energy is changed through contact with the environment (as in friction or drag) or actor applying controls. In these cases, HNNs can be generalized by adding a forcing term
dg
dp
T
0p
DH
+
g(q.p)
u
0g
in SymODEN (Zhong et al.. 2019; 2020) and Desai al. (2021), where is the control input, which can be fixed as constant in systems with drag or friction.
Though SymODEN can accommodate controls and damping, we show that simply using the bias of second-order dynamics is sufficient to achieve nearly the same performance with much less com- plexity. We demonstrate the matching performance our n-body pendulum systems, augmented dp 8H Ae (Figure 6).
with drag, di 3q

---

Published as a conference paper at ICLR 2022
Test
Train
10
I
Rollout Err
10-1
Rollout
10-1
10-2
HalfCheetah
Hopper
Swimmer
HalfCheetah
Hopper
Swimmer
NODE
NODE + SO
SymODEN
Figure 7: HNNs perform very poorly on complex dynamics like OpenAl Gym Mujoco control systems. Biasing the model towards Hamiltonian dynamics makes it difficult to fit the training data. Simply imposing second-order structure on a NODE much more effective. Error bars show standard error across 4 seeds.
5 MODELING MUJOCO TRANSITION DYNAMICS
HNNs are typically evaluated on relatively simple systems, like those considered in the previous sections. principle, we would expect these results to extend more complex systems that are governed by similar physical laws. In practice, however, there is little evidence to suggest that applying HNN methods to complex systems is easy or effective.
The MuloCo physics simulator (Todorov et al., 2012) is one such complex system that we would expect to benefit from HNN inductive biases. Gym Mujoco systems are heavily used in model-based reinforcement learning (MBRL) literature, but the dynamics models commonly used are surprisingly simple (often just MLPs trained predict the next change state) (Chua et al., 2018; Wang and Ba, 2019).
Given how much benefit other applications of deep learning have derived from specialized archi- tectures, such CNNs for computer vision (LeCun et al., 1989), RNNs for NLP (Mikolov al., 2010), or WaveNets for audio (Oord et al., 2016), we would expect analogous improvements to be possible in MBRL. Algorithms for MBRL are infamously sensitive to choice of prediction horizon, and one possible explanation is poor generalization caused by weak inductive biases (Janner et al., 2019; Pan al., 2020; Amos et al., 2021). Improving model design for mechanical systems has the potential to improve both the sample efficiency of MBRL algorithms and their robustness.
We train NODES and HNNs trajectories from several OpenAl Gym Mujoco environments (Brockman et al., 2016). Crucially, we compare NODEs endowed with second-order structure (NODE SO) against pre-existing NODE and HNN models, as we did in Section 4.4. Note that with fixed step size integrators, NODEs are equivalent to discrete transition models that predict the next state or delta directly with an MLP, and therefore our NODE baseline is representative of models commonly used in MBRL. See Appendix C for implementation details.
In Figure show that NODE + SO significantly outperforms baseline methods. Surprisingly HNNs underperform NODEs all the systems we consider. Although the HNN could in principle learn the dynamics, in practice the bias towards Hamiltonian dynamics makes fitting the training data very difficult and provides tangible benefit to generalization. This outcome is notably different from what we observe in toy tasks, where HNNs can fit con-conservative systems (c.g., pendulums with drag) with little difficulty.
In spirit, our results parallel the findings Wu et al. (2019) in the different setting of graph CNNs. We able to distill the inductive biases of HNNs into NODE + SO without losing performance on systems that HNNs perform well on, and, even more importantly, these reduced systems are much more capable of scaling to complex systems and larger datasets.
8

---

Published as a conference paper at ICLR 2022
CONCLUSION
In this paper, we deconstructed the inductive biases of highly performing HNN models into their component parts, a NeuralODE, symplecticity, conservation of the learned energy function, and second order structure. Contrary to conventional wisdom, the success of HNNs is not from their en- engy conservation or symplecticity, but rather from the assumption that the system can be expressed as single second order differential equation. Stripping away the other components of an HNN, we left with model that is simpler, more computationally efficient, and less restrictive in that can be directly applied to non-Hamiltonian systems. As consequence, we are able to apply the result- ing model constructing transition models for the challenging Mujoco locomotion environments, with promising performance.
ACKNOWLEDGEMENTS
The authors would like to thank Greg Benton and Pavel Izmailov for helpful comments original submission. This research is supported by an Amazon Research Award, Facebook Research, Google Research, NSF I-DISRE 193471, NIH RO1DA048764-01A NSF IIS- 118-1910266, and NSF 1922658 NRT-HDR: FUTURE Foundations, Translation, and Responsibility for Data Science.

---

Published as a conference paper at ICLR 2022
REFERENCES
Victor M Martinez. Alvarez, Rares Rosca, and Cristian G Fälcujescu. Dynode: Neural or- dinary differential equations for dynamics modeling in continuous control. arXiv preprint arXiv:2009.04275 2020.
Brandon Amos, Samuel Stanton, Denis Yarats, and Andrew Gordon Wilson. On the model-based stochastic value gradient for continuous reinforcement learning. In Learning for Dynamics and Control, pages 6-20. PMLR, 2021.
Aleksandar Botev, Andrew Jaegle, Peter Wirnsberger, Daniel Hennes, and Irina Higgins. Which priors matter? benchmarking models for learning latent dynamics. 2021.
Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojciech Zaremba. Openai gym, arXiv preprint arXiv:1606.01 2016.
Ricky TQ Chen, Brandon Amos, and Maximilian Nickel. Learning neural event functions for ordi- nary differential equations. arXiv preprint arXiv:201 1,09902, 2020.
Tian Qi Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. Neural ordinary dif- ferential equations. In Advances in neural information processing systems, pages 6571-6583, 2018.
Zhengdao Chen, Jianyu Zhang, Martin Arjovsky, and Léon Bottou. Symplectic recurrent neural networks. arXiv preprini arXiv: /909. 13334, 2019.
Kurtland Chua, Roberto Calandra, Rowan McAllister, and Sergey Levine. Deep reinforcement learn- ing in handful of trials using probabilistic dynamics models. arXiv preprint arXiv: 1805.1211,, 2018.
Miles Cranmer, Sam Greydanus, Stephan Hoyer, Peter Battaglia, David Spergel, and Shirley Ho. Lagrangian neural networks. arXiv preprint wrXii:2003.046300 2020.
Shaan Desai, Marios Mattheakis, David Sondak, Pavlos Protopapas, and Stephen Roberts. Port- hamiltonian neural networks for learning explicit time-dependent dynamical systems. arXiv preprint arXiv:2107.08024, 2021.
Mare Finzi, Ke Alexander Wang, and Andrew Gordon Wilson. Simplifying hamiltonian and la- grangian neural networks via explicit constraints. arXiv preprint arXiv:2010.13580 2020.
Mare Finzi, Max Welling, and Andrew Gordon Wilson. practical method for constructing equiv- ariant multilayer perceptrons for arbitrary matrix groups arXiv preprint 2rXiv:2104.094:3 2021.
Arnab Ghosh, Harkirat Singh Behl, Emilien Dupont, Philip HS Torr, and Vinay Namboodiri. Steer:
Simple temporal regularization for neural odes. arXiv preprint arXiv:2006. 10711, 2020.
Samuel Greydanus, Misko Dzumba, and Jason Yosinski. Hamiltonian neural networks. 2019.
Jayesh K Gupta, Kunal Menda, Zachary Manchester, and Mykel J Kochenderfer. A general frame- work for structured learning of mechanical systems. arXiv preprint arXiv:1902.08700) 2019.
Jayesh K Gupta, Kunal Menda, Zachary Manchester, and Mykel Kochenderfer. Structured mechani- cal models for robot learning and control. In Learning for Dynamics and Control, pages 328-337. PMLR, 2020.
Tuomas Haarnoja, Aurick Zhou, Kristian Hartikainen, George Tucker, Sehoon Ha, Jie Tan, Vikash Kumar, Henry Zhu, Abhishck Gupta, Pieter Abbeel, et al. Soft actor-critic algorithms and appli- cations. arXiv preprint urXiv:1812.05905 2018.
Andreas Hochlehnert, Alexander Terenin, Steindór Samundsson, and Mare Deisenroth. Learning contact dynamics using physically structured neural networks. In International Conference Artificial Intelligence and Statistics, pages 2152-2160. PMLR, 2021.
Michael Janner. Justin Fu, Marvin Zhang, and Sergey Levine. When to trust your model: Model- based policy optimization. arXiv preprint arXiv: 1rXiv:1906.08253 2019.
10

---

Published as a conference paper at ICLR 2022
Pengzhan Jin, Zhen Zhang, Aiqing Zhu, Yifa Tang, and George Em Karniadakis. Sympnets: Intrin- sic structure- prructur-pprseerving symplectic networks for identifying hamiltonian systems. Neural Net- works, 132:166-131 1796,,17900090 2020.
George Em Karniadakis, loannis Kevrekidis, Lu Lu, Paris Perdikaris, Sifan Wang, and Liu Yang. Physics-informed machine learning. Nature Reviews Physics, 366):422-440 2021.
Yann LeCun, Bernhard Boser, John Denker, Donnie Henderson, Richard Howard, Wayne Hubbard, and Lawrence Jackel. Handwritten digit recognition with a back- pooo-ppopggttton network. Advances in neural information processing systems, 2, 1989.
Kookjin Lee, Nathaniel Trask, and Panos Stinis. Machine learning structure preserving brackets for forecasting irreversible processes. Advances in Neural Information Processing Systems, 34, 2021.
Benedict J Leimkuhler and Robert D Skeel. Symplectic numerical integrators in constrained hamil- tonian systems, Journal of Computationall Physics, 12(():117-1255 1994.
Shuo-Hui Li, Chen-Xiao Dong, Linfeng Zhang, and Lei Wang. Neural canonical transformation with symplectic flows. Physical Review X, 10(2):22000, 2020.
Ziming Li, Bohan Wang. Qi Meng, Wei Chen, Max Tegmark, and Tie- Yan Liu. Machine-learning non non-conservative dynamics for new-physics detection. arXiv preprint arXiv:2 00026, 2021.
Ziming Liu, Yunyue Chen, Yuanqi Du, and Max Tegmark. Physics-augmented learning: A new
paradigm beyond physics-informed learning, arXiv preprint arXivv2209.13901, 2021.
Michael Lutter, Christian Ritter, and Jan Peters. Deep lagrangian networks: Using physics model prior for deep learning. arXiv preprint wwtXiv:997.00444 2019 9a.
Michael Lutter, Christian Ritter, and Jan Peters. Deep lagrangian networks: Using physics model prior for deep learning. arXiv preprint
2019b.
Tomas Mikolov, Martin Karafiát, Lukas Burget, Jan Cernocký, and Sanjeev Khudanpur. Recurrent neural network based language model. In Interspeech, volume 2, pages 1045-1048. Makuhari, 2010.
Aaron van den Oord, Sander Dieleman, Heiga Zen, Karen Simonyan, Oriol Vinyals, Alex Graves, Nal Kalchbrenner, Andrew Senior, and Koray Kavukcuoglu. Wavenet: A generative model for raw audio. arXiv preprint arXiv: /ct.::360993999 2016
Feiyang Pan, Jia He, Dandan Tu, and Qing He. Trust the model when it is confident: Masked model-based actor-critic. arXiv preprint arXiv:2010.04893, 2020.
Emanuel Todorov, Tom Erez, and Yuval Tassa. Mujoco: A physics engine for model-based control. In 20/2 JEEE/RSJ International Conference Intelligent Robots and Systems, pages 5026-5033. IEEE, 2012.
Yunjin Tong. Shiying Xiong, Xingzhe He, Guanghan Pan, and Bo Zhu. Symplectic neural networks in taylor series form for hamiltonian systems. Journal of Computational Physics, 10325, 2021.
Tingwu Wang and Jimmy Ba. Exploring model-based planning with policy networks. arXiv preprint arXiv:1906.08666 2019.
Felix Wu, Amauri Souza, Tianyi Zhang, Christopher Fifty, Tao Yu, and Kilian Weinberger. Sim- plifying graph convolutional networks. In International conference on machine learning, pages 6861-6871. PMLR, 2019.
Shiying Xiong, Yunjin Tong, Xingzhe He. Shuqi Yang, Cheng Yang, and Bo Zhu. Nonseparable symplectic neural networks. arXiv preprint arXiv:2010.12636 2020.
Yaofeng Desmond Zhong, Biswadip Dey, and Amit Chakraborty. Symplectic ode-net: Learning hamiltonian dynamics with control. arXiv preprint 1909./2000//000 2019.

---

Published as a conference paper at ICLR 2022
Yaofeng Desmond Zhong, Biswadip Dey, and Amit Chakraborty. Dissipative symoden: En- coding hamiltonian dynamics with dissipation and control into deep learning. arXiv preprint arXiv:2002.08880, 2020.
Yaofeng Desmond Zhong, Biswadip Dey, and Amit Chakraborty. Benchmarking energy-conserve neural networks for learning dynamics from data. In Learning for Dynamics and Control, pages 1218-1229. PMLR, 2021a.
Yaofeng Desmond Zhong, Biswadip Dey, and Amit Chakraborty. Extending lagrangian and hamil- tonian neural networks with differentiable contact models. Advances in Neural Information Pro- cessing Systems, 34, 2021b.
12

---

Published as a conference paper at ICLR 2022
A APPENDIX OUTLINE
This appendix is organized as follows. Section B, we present proofs for the energy conservation properties of HNNs and Neural ODEs, well as more detailed description of symplecticity. In Section C, we described the details of our experiments on Mujuco, including our data preprocessing, training, and architectural decisions. Lastly, in Section D, we provide additional experimental results requested by reviewers of our original submission. These include comparison of alternative loss functions and comparison additional rigid body systems.
B MATHEMATICAL DETAILS
B. ENERGY CONSERVATION FOR NEURAL ODES
Let F JVH be the ground truth dynamics of time independent Hamiltonian, and f be the dynamics learned by a neural network through an learned Hamiltonian H for HNNs or otherwise. Given some initial condition 20, let & denote the solution to Fz)) at time starting from 20 and 27 be the solution to the ground truth dynamics from zg.
Suppose the error in the dynamics model e(z) F(z) F()) is bounded Vs ||<(=)]] < and that we are only considering a bounded region of the state space (such as the states of a pendulum with bounded energy).
Since energy is conserved (zx) = Hzz) H(o)) we can write the the energy error
H(2r) (2T)) H(Tr) H ffa
VH(=) dz
Since the value is independent of the path, we may consider the path given by the approximated dynamics &. Noting that dł/dt = ((t)= = (&e) + e(2)), we have
T
dt 1' (
H(2r) H H(rr)
(2(2)
dt
(H((2)) JVH(22) + V e((()]dt
VH())) (H())))(())d d()
Bounding the maximum value of the integrand along the path, we have that
(H())) H (=T)l supup ||V
which grows only linearly with time.
This linear bound on the energy error is in stark contrast with the state error which could grow exponentially according to the Lyapunov exponents, even if the dynamics error is arbitrarily small. Advancing the ground truth and learned dynamics forward to some small time t 6, &
+ (zo) yields error || ze ||e](20) - e=())) elle(=)]] < eô. And yet, this error even if propagated by the ground truth dynamics will grow exponentially (2 zrll =s x ( ze = ex <lle(2)]l
B.2 HNN ENERGY CONSERVATION
simple but erroneous argument for why HNNs approximately conserve the true energy goes as follows:
We would like to know if HNNs achieve better energy conservation given the same levels of error in the predicted dynamics. For HNNs, F. = H, and we can see that the dynamics error e(z) can also be written as e(z) JV H).
If we could convert bound on the derivatives ||e|| I| H)ll < å (since J is an orthogonal matrix) into bound on the learned Hamiltonian itself F(z) f[)=) H)2) and F < A
13

---

Published as a conference paper at ICLR 2022
holding globally for some constant c, then would have constraint the energy enror that doesn't grow with time. Expanding the difference in initial and final energy, the constant cancels out and we have
H(²r) - ((0) = (((()))) (1 E(TT) + E(oo)
= + E(a)).
using the fact that the learned energy function H is conserved over the model rollout &, there was a constraint EE| < A then
H((T) ((o) 2A.
Unfortunately, even the gradients are close and is small, that does not imply that A small. Small differences in gradient add up very large differences in the values of the two functions. While the dynamics may well approximate the data, and achieve low rollout error, there is no reason believe that at a given point in phase space the learned Hamiltonian should have a value that is close to the true Hamiltomian.
B.3 SYMPLECTICITY
Symplecticity is the requirement that dynamics satisfy (JDF) JDF, Defining G JF, the requirement is simply that the antisymmetric part of the jacobian is 0, DG DG= - 0.
Unpacking Poincare's 's lemma requires some familiarity with differential geometry concepts such as differential forms and exterior derivatives, and so we will assume them but for this section only. Poincare's lemma states that on a contractible domain (such as IR") differential k-form w is closed du 0 (the exterior derivative of is 0) then it is also exact derivative of another differential (k- - 1)-form v). While F is a vector field, G JF is a differential 1-form (dual vector field). If DG is symmetric, then it is also closed: dG Σ, a,G.rr Adx! = Gd)x' A dx 0 since (a,G, O,G.)
dv (it is the exterior
2
0 is just another way of expressing DG DG Therefore by Poincare's lemma, G = de for some O-form (scalar function) ó Therefore = - = )d(-)) since J-1 = J. As the exterior derivative of scalar function just the gradient, we can define // o and see that there exists a scalar function Hl such that JVH.
C MUJOCO EXPERIMENT DETAILS
Data collection: for each control task we trained a standard soft actor-critic RL agent to convergence (Haarnoja al., 2018). Note that had to modified versions of the Gym environments since the standard environments preprocess observations in ad-hoc ways. For example, Hopper clips the velocity observations to [-10, 10jd and truncates part of the position. 2 Our versions of the environments simply return [g. v] as the observation. Then we randomly split the episodes in the replay buffer into train and test. The training data was 40K 3-step trajectories (i.e. two transitions) randomly sampled from the training episodes, The test data was 200 200-step trajectories randomly sampled from the test episodes. This data-collection strategy is important to the experiment because random controls typically do not cause the agent to cover the entire itate-action space. Similarly many control policies are highly cyclical, so it is important to separate train and test splits at the episode level.
Training:
we trained each model for 256 epochs using Adam with batch size of 200 and weight decay (A = le-4. We used a cosine annealing learning rate schedule, with 7max 2e-4, Mmiin le-6.
Model Architecture Each network was parameterized as 2-layer MLP with 128 hidden units. Each model used the Euler integration rule with integration steps per transition step. The step size was determined by the integration step size of the underlying environment, h At/8.
NODE + SO: given the state and controls u, a standard NODE takes dz/dt /tt = f(z, u, 0). However, if 2 (g. e| (that is, if both position and velocity observed), then already have good estimate
14

---

Published as a conference paper at ICLR 2022
of dg/dt in the observation itself, namely Hence propose only using the network to model acceleration du/dt dt = f(z, u, 0), and to model dg/dt implicitly. It is important to note that we cannot take dq/dt dt
because is observed before the control is applied. Instead we take dg/dt: = v/2 (e + x dv/dt) averaging the velocity at time (before the control is applied) and the predicted velocity time 1 (after the control is applied), given Euler integration step of size h on v.
This integration rule can be viewed as an approximate RK2 step on @t. where U++-1 is approximated via a learned Euler step on This approach has two benefits. In the first place constrains the predicted velocity and acceleration be consistent across time. Second the model is able to take an approximate RK2 step at the cost of a single forward pass (instead of 2). The latter is important because integration error can accumulate over long rollouts, even if the model fits the dynamies very well.
D ADDITIONAL EXPERIMENTAL RESULTS
D.1 COMPARISON OF LOSS FUNCTIONS
In the experimental results presented in the body the paper were obtained training on 12 loss between integrated and ground truth trajectories. As noted in feedback the early version of the paper, this practice goes contrary to prior work using 11 loss for stability (Finzi et al., 2020). In Figure we show the result of changing the loss.
Rollout Error
10-2
Chain (1l loss)
Chain (12 loss)
Spring (11 loss)
Spring (12 loss)
NODE
NODE + SO
HNN
Switching from 12 to I1 loss can improve rollout error slightly, but doesn't 't impact the ordering of the models. The other elements of the experimental setup are identical to above. Error bars show one standard deviation.
Figure 8:
D.2 ADDITIONAL SYSTEMS
To extend the comparison of NODE and HNN models, we trained models three additional sys- tems presented by Finzi et al. (2020). Figure shows the rollout error of NODE, NODE + SO, and HNN models. One the gyroscope system, observe similar result to the above. In the magnet pendulum and rotor systems, the results are slightly more counterintuitive, with the NODE model outperforming the more sophisticated alternatives. We suspect the small difference in performance the models due to the challenge of stably training HNNs in complex systems (with magnet pendulum having complex dynamics and rotor having complex coordinate system).
15

---

Published as a conference paper at ICLR 2022
Rollout Error
10-3
10-1
Gyroscope
Magnet
Rotor
NODE
NODE + SO
HNN
Figure 9: On the additional systems from Finzi et al. (2020), we can observe the effect of second order structure, compared with NODE and HNN baselines. As before, second order structure seems to account for much of the difference between NODE and HNN models. Error bars show one standard deviation.
16


Reviews:
[{"review_id": "pjHWOHKzR-1", "reviewer": "Reviewer_7KKB", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "########################################################################## \n\n Pros:\n\n-The presentation is clear and easy to follow, even for readers without an extensive mathematical background or a prior familiarity with Hamiltonian dynamics.\n\n-The inductive biases chosen to study are well-motivated. The experiments and analysis used to examine each inductive bias are well-thought-out and the conclusions are supported by the provided evidence.\n\n-The distilled architecture for modeling dynamical systems without the “bells and whistles” of HNNs, NODE + SO, is a solid contribution. This may be surprising to some and seems promising for being used as a baseline in future work. I also found the analysis on the relative energy violation of NODEs vs. HNNs in Section 4.1 compelling and illuminating.\n\n##########################################################################\n\nCons:  \n\n-A takeaway of the experiment in Section 4.2 (Symplectic vector fields) is missing. It would be helpful to add what (if anything) can be concluded by the finding that the symplectic regularizer alone is insufficient to help improve test rollout error. \n\n-For the Mujoco experiment in Section 5, only quantitative results are provided. Given that the page limit is 9 pages but only 8 were used, it seems reasonable to also provide qualitative visualizations that compare rollouts of NODE, NODE + SO, and SymODEN. Providing GIFs/videos would also be helpful for the reader.\n\n-A discussion in Section 5 on why the NODE and NODE + SO Swimmer test time performance differs by a large margin is missing (also, qualitative visualizations could be helpful to explain this).\n\n#########################################################################\n\nQuestion for authors:\n\nI didn’t quite understand the implications of the Mujoco experiment for the MBRL research community. Is the recommendation that NODE + SO could/should be used as a drop-in replacement for the “simpler dynamics models that predict the next change in state”?  Would it not make sense then to add a baseline to this experiment to compare directly against a simple dynamics model used in a popular Mujoco MBRL framework?\n\n#########################################################################\n\nSome suggestions for improvement: \n\n-In Section 4.1, the line after the equation references “where the *last line* follows” → should be changed to “where the last equality follows”.\n\n-I would suggest defining “symplectic” earlier in the paper, perhaps in the introduction. Currently, it is introduced at the beginning of Section 4.2, although it is discussed at the end of the introduction.\n\n-The last sentence of the introduction, “Extracting the second-order bias, we show how to improve the performance of dynamics models” is unclear to me. Perhaps it could be re-written to more clearly highlight the specific contribution, which to my understanding is: a Neural ODE that models dynamics with second-order information (NODE + SO)?\n", "summary_of_the_paper": "This is a well-written analysis of Hamiltonian Neural Networks (HNNs), a class of physics-inspired deep neural networks. The work is motivated by a desire to apply HNNs to non-toy datasets as well as to gain an understanding of the key inductive biases that explain the majority of their performance. Through controlled experiments on synthetic trajectory data, they explore energy conservation, a symplectic bias, complexity of state representation, and second-order structure bias. The key finding is that the second-order structure in HNNs is the main explainer of its performance. A simpler model that combines a Neural ODE with second-order structure is introduced, which can be seen as a distilled HNN. The simpler model achieves stronger performance on Mujoco rollout prediction compared to the HNN.", "summary_of_the_review": "I think this paper deserves acceptance. In my opinion, it clearly adds new knowledge to the body of work on HNNs, is written exceptionally well, and introduces a useful framework (NODE + SO) for modeling dynamics in non-trivial environments that has the potential to benefit the model-based reinforcement learning (MBRL) community. A few minor requests are provided to strengthen the paper.\n\n==========================================================================\n\nUpdate after rebuttal: I will be maintaining my initial score and hope that the authors will update their paper for the final camera-ready version to include the promised improvements.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "anWI6YdvXsm", "reviewer": "Reviewer_nLbj", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "## Strengths. \nThe submission is well-motivated and the investigation on HNNs inductive biases is thoroughly performed.\nUp to my knowledge, the contributions of this submission are indeed novel.\nThis submission is definitely relevant for the community as it gives practical insights on commonly encountered inductive biases in physically informed machine learning models.\nThe writing is also good, making it easy for the reader to follow the narrative.\n\n## Weaknesses.\nOne weakness perhaps is the lack of further investigation into the failure of HNNs on the MuJoCo physics simulator in Section 5 cf my comments at the end of this review.\n\n## Clarity.\nI found the submission well written, with the motivation, ideas and empirical findings well conveyed to the reader. As a consequence, reading this submission was very pleasant.\n\nI would perhaps suggest to add a causal graph---at the end of Section 1 or the begging of Section 4---showing the relationship between the inductive biases (e.g. symplectic dynamics <=> Hamiltonian system, etc). Then it would be easy to understand which assumptions are the strongest etc.\n\n## Relation to prior work.\nRelated work is discussed properly up to my knowledge.\n\n## Additional feedback.\n- Would suggest numbering more equations (at least for the submission).\n- \"Although energy conservation may be helpful for generalization, the evidence does not indicate that HNNs are better at controlling energy violation than NODEs\" -> Perhaps I am missing something, but Figure 2 seems to show that although both HNN and NODE (energy) error grows somewhat linearly wrt rollout time, the HNN's energy error is about ~x10 smaller than NODE's one. Yet, it is indeed interesting to highlight that empirically the energy is not preserved.\n- Section 4.2: \n    - It is a bit unclear what the vector field F is at first, although it becomes clear a bit later with Equation 1.\n    - I would suggest to add a Figure showing phase portraits for a symplectic and a non symplectic system, to give additional intuition to readers that may not be familiar with this concept.\n    - Figure 3: Why not having similar plots that in Figure 2? Is the empirical symplecticity growing linearly with rollout time?\n- Section 4.3: \n    - Would suggest recalling the Hamiltonian dynamics $\\frac{d}{dt}[q p]$.\n    - $\\frac{dp}{dt}=A_\\theta(q, p)$ potentially slightly confusing as the second-order structure above is on $v$ with $\\frac{dv}{dt}=A(q, v)$. Would advise writing both in terms of $v$ or $p$, or at least reminding the relationship between $p$ and $v$.\n- Section 4.4.1:\n    - Is the drag term added to all models?\n    - What if the drag term is not known?\n    - \"we show that simply using the bias of second-order dynamics is sufﬁcient to achieve nearly the same performance with much less complexity\" -> How come with much less complexity? In what sense?\n- Section 5:\n    - The subpar performance of HNNs is definitely interesting, but with no further investigation on the underling reasons we are left a bit unsatisfied. Does the MuJoCo physics simulator indeed conserve energy? Is it an Hamiltonian system? Would be interested to have a similar plot than Figure 2, perhaps the true energy is actually not well learnt.\n", "summary_of_the_paper": "## Summary and contributions.\nMotivated by the success of physics-inspired neural networks, authors investigate the different inductive biases (implicitly or explicitly) encoded in Hamiltonian neural networks (HNNs). Authors identify four biases: the ODE, second order, energy conservation and symplectic biases. They claim and empirically show that, as opposed to what was previously argued, HNNs' performance is mostly due to the implicit second-order bias. Indeed, the authors show that explicitly splitting the Hamiltonian into a momentum and potential term, leads to the system's position $q$ being parametrised as the solution of a second-order differential equation (which involves the mass matrix and the potential function).", "summary_of_the_review": "Overall this submission is well motivated, thoroughly executed and well written, hence my recommendation for acceptance.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "Xcm_f58KWsE", "reviewer": "Reviewer_wu5x", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The main contribution and novelty of this work lie in the insight it provides about the role of various, often intertwined, inductive-biases present in the networks that encode Hamiltonian and Lagrangian dynamics into their network structure. Furthermore, through carefully motivated and executed experiments, it shows that incorporation of second-order structure into the learning framework is the most important factor in improving the performance of Hamiltonian/Lagrangian dynamics based neural networks. \n\nFollowing are some more specific concerns/comments: \n\n-- While showing the relationship between the accuracy of energy conservation and rollout error in Figure 2, the authors have not provided a clear definition of *energy violation*. Please clearly define this performance metric in terms of $H$ and $\\hat{H}$. \n\n-- On a related note, the derivation and motivation of the inequalities in Section A.2 are not very clear. The constant present in the inequality \"$|\\hat{H} - H - const|<\\Delta$\" represents the amount of mismatch between the learned and the predicted Hamiltonian that can be attributed to the change of reference in potential energy. However, this quantity has not been connected with the rest of the derivation. Also, the last sentence of Section A.1 does not immediately follow the previous discussions. Can one use Gronwall–Bellman inequality) to make this claim about exponentially growing state error? It would be much appreciated if these theoretical discussions were made more precise. \n\n-- This paper shows that HNN/SymODEN performs poorly in control-related tasks from MuJoCo (e.g., HalfCheetah or Hopper). This is not unexpected since HNN/SymODEN tries to conserve energy. At the same time, Neural ODE (with or without Second-Order bias) does not have any such inclination, and energy may not be a conserved quantity in the ground-truth data (presence of contacts and collisions can often change the total energy of the system). Therefore, this does not appear to be a fair comparison. On the other hand, recent work by Zhong et al. (*Extending Lagrangian and Hamiltonian Neural Networks with Differentiable Contact Models*, arXiv:2102.06794) has extended Hamiltonian dynamics based networks to systems with contacts and collisions. I would encourage the authors to use this approach as one of the baselines while running the experiments for Section 5. \n\n-- Finzi et al. (*Simplifying Hamiltonian and Lagrangian Neural Networks via Explicit Constraints*, NeurIPS 2020)have shown that the $L_1$ loss functions exhibit more robustness in these problems. Can the authors confirm if the conclusions of this paper hold when the loss function is changed from $L_2$ to $L_1$? \n\n-- Also, it is not clear how the relative performance of the different models changes with an increase/decrease in the neural network parameters. I would strongly encourage the authors to explore these directions.", "summary_of_the_paper": "Incorporating physics-informed inductive bias, especially Hamiltonian and Lagrangian dynamics, into deep neural networks has been the focus of a fast-growing body of work over the last few years. This paper has carried out a formal analysis of these methods to identify which specific aspect(s) of these energy-conserving networks contribute(s) the most to their superior performance in prediction and generalization. Through theoretical and empirical analyses, this paper concludes that incorporating the second-order structure (i.e., using a neural network to model the acceleration) and reducing functional complexity through a change of coordinates, not energy conservation or the symplectic structure, play the most critical role in improving the performance. The experiments are thorough and emphasize the message precisely.", "summary_of_the_review": "This paper has looked into the existing solution approaches to a relevant and interesting problem and focused on understanding how those solution approaches work. This analysis provided some insight which, in turn, were exploited to introduce a new architecture that can match/outperform the existing methods with much simpler network architecture. The idea is interesting, the experiments are mostly well carried out, and the paper is reasonably well-written. However, as explained in the *main review*, some aspects should be further improved to increase the concreteness and overall clarity of the paper.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "Njmt4LisbsC", "reviewer": "Reviewer_SW9u", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strenghts: \n\nThis paper tackles an interesting and useful question of introspecting and analyzing different inductive biases. I think this work presents an important analysis which could have useful downstream applications. The experiments and analysis are well written and thorough, and the break down of how each type of inductive bias might help or hurt is interesting and easy to follow. The experiments related to the MuJoCo benchmarks help ground the need for this type of analysis, and open the door to some interesting future work. \n\n\nWeaknesses: \n\nI think the weakenesses in this paper are quite minor. It would be good to see more empirical analysis on more complex tasks such as the MuJoCo benchmark tasks instead of the kChainPendulum and kSpringPendulum tasks, especially since we see that the biases of HNNs have difficulty with modeling the dynamics for the MuJoCo tasks. A discussion or experimentation on where each of the inductive biases are actually useful would also be interesting. ", "summary_of_the_paper": "This paper presents an in-depth study of the inductive biases in physics-inspired neural networks, especially Hamiltonian Neural Networks (HNNs). The authors break down the biases in HNNs into multiple categories, such as energy conservation of the network, second-order structure of the output, symplectic vector fields produced by the networks, and the role of the complexity of the coordinate system. The authors show through experiments on a set of pendulum-based tasks, that HNNs are not in fact better at conserving energy than competing approaches that dont explictly model this such as Neural ODEs. They also show that Regularizing NODEs with a symplectic field bias does not help in generalization. Through experimentation, they show that a second order output structure similar to that of HNNs helps NODEs quite a bit and outperforms SymODEN. Finally, the authors use these findings to obtain good results in modeling trajectory dynamics with NODEs and a second order output to model dynamics in MuJoCo tasks. ", "summary_of_the_review": "The paper presents a thorough and well written analysis of inductive biases in HNNs. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper examined physics-inspired inductive biases in neural networks, in particular Hamiltonian and Lagrangian dynamics. The work separated the benefits arising from incorporating energy conservation, the symplectic bias, the coordinate systems, and second-order dynamics.  Through a set of experiments, the paper showed the most important factor for improved performance in the test domains was the second-order dynamics, and not the more common explanation of energy conservation or the other factors. The increased generality of this approach was demonstrated with better predictions on Mujoco tasks that did not conserve energy.

All reviewers liked the insights provided by the paper.  They agreed that the paper clearly laid out several hypotheses and systematically tested them.  The reviewers found the experiments thoughtful and the results compelling.  The reviewers also pointed out several aspects of the document that could be improved, including additional formalism clarifications (reviewer nLbj), baseline algorithms (reviewer wu5x), and domains (reviewers 7KKB,SW9u).  The reviewers found the author's response satisfactory but were disappointed that a revised paper was not ready to read. The reviewers want the final paper to include the modifications that were promised in the author response.

All four reviewers indicated to accept this paper which contributes novel insights that simplify and generalize physics-inspired neural networks. The paper is therefore accepted.
Decision: Accept (Spotlight)