Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: Incorporating physics-informed inductive bias, especially Hamiltonian and Lagrangian dynamics, into deep neural networks has been the focus of a fast-growing body of work over the last few years. This paper has carried out a formal analysis of these methods to identify which specific aspect(s) of these energy-conserving networks contribute(s) the most to their superior performance in prediction and generalization. Through theoretical and empirical analyses, this paper concludes that incorporating the second-order structure (i.e., using a neural network to model the acceleration) and reducing functional complexity through a change of coordinates, not energy conservation or the symplectic structure, play the most critical role in improving the performance. The experiments are thorough and emphasize the message precisely.
main_review: The main contribution and novelty of this work lie in the insight it provides about the role of various, often intertwined, inductive-biases present in the networks that encode Hamiltonian and Lagrangian dynamics into their network structure. Furthermore, through carefully motivated and executed experiments, it shows that incorporation of second-order structure into the learning framework is the most important factor in improving the performance of Hamiltonian/Lagrangian dynamics based neural networks. 

Following are some more specific concerns/comments: 

-- While showing the relationship between the accuracy of energy conservation and rollout error in Figure 2, the authors have not provided a clear definition of *energy violation*. Please clearly define this performance metric in terms of $H$ and $\hat{H}$. 

-- On a related note, the derivation and motivation of the inequalities in Section A.2 are not very clear. The constant present in the inequality "$|\hat{H} - H - const|<\Delta$" represents the amount of mismatch between the learned and the predicted Hamiltonian that can be attributed to the change of reference in potential energy. However, this quantity has not been connected with the rest of the derivation. Also, the last sentence of Section A.1 does not immediately follow the previous discussions. Can one use Gronwall–Bellman inequality) to make this claim about exponentially growing state error? It would be much appreciated if these theoretical discussions were made more precise. 

-- This paper shows that HNN/SymODEN performs poorly in control-related tasks from MuJoCo (e.g., HalfCheetah or Hopper). This is not unexpected since HNN/SymODEN tries to conserve energy. At the same time, Neural ODE (with or without Second-Order bias) does not have any such inclination, and energy may not be a conserved quantity in the ground-truth data (presence of contacts and collisions can often change the total energy of the system). Therefore, this does not appear to be a fair comparison. On the other hand, recent work by Zhong et al. (*Extending Lagrangian and Hamiltonian Neural Networks with Differentiable Contact Models*, arXiv:2102.06794) has extended Hamiltonian dynamics based networks to systems with contacts and collisions. I would encourage the authors to use this approach as one of the baselines while running the experiments for Section 5. 

-- Finzi et al. (*Simplifying Hamiltonian and Lagrangian Neural Networks via Explicit Constraints*, NeurIPS 2020)have shown that the $L_1$ loss functions exhibit more robustness in these problems. Can the authors confirm if the conclusions of this paper hold when the loss function is changed from $L_2$ to $L_1$? 

-- Also, it is not clear how the relative performance of the different models changes with an increase/decrease in the neural network parameters. I would strongly encourage the authors to explore these directions.
summary_of_the_review: This paper has looked into the existing solution approaches to a relevant and interesting problem and focused on understanding how those solution approaches work. This analysis provided some insight which, in turn, were exploited to introduce a new architecture that can match/outperform the existing methods with much simpler network architecture. The idea is interesting, the experiments are mostly well carried out, and the paper is reasonably well-written. However, as explained in the *main review*, some aspects should be further improved to increase the concreteness and overall clarity of the paper.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper presents an in-depth study of the inductive biases in physics-inspired neural networks, especially Hamiltonian Neural Networks (HNNs). The authors break down the biases in HNNs into multiple categories, such as energy conservation of the network, second-order structure of the output, symplectic vector fields produced by the networks, and the role of the complexity of the coordinate system. The authors show through experiments on a set of pendulum-based tasks, that HNNs are not in fact better at conserving energy than competing approaches that dont explictly model this such as Neural ODEs. They also show that Regularizing NODEs with a symplectic field bias does not help in generalization. Through experimentation, they show that a second order output structure similar to that of HNNs helps NODEs quite a bit and outperforms SymODEN. Finally, the authors use these findings to obtain good results in modeling trajectory dynamics with NODEs and a second order output to model dynamics in MuJoCo tasks. 
main_review: Strenghts: 

This paper tackles an interesting and useful question of introspecting and analyzing different inductive biases. I think this work presents an important analysis which could have useful downstream applications. The experiments and analysis are well written and thorough, and the break down of how each type of inductive bias might help or hurt is interesting and easy to follow. The experiments related to the MuJoCo benchmarks help ground the need for this type of analysis, and open the door to some interesting future work. 


Weaknesses: 

I think the weakenesses in this paper are quite minor. It would be good to see more empirical analysis on more complex tasks such as the MuJoCo benchmark tasks instead of the kChainPendulum and kSpringPendulum tasks, especially since we see that the biases of HNNs have difficulty with modeling the dynamics for the MuJoCo tasks. A discussion or experimentation on where each of the inductive biases are actually useful would also be interesting. 
summary_of_the_review: The paper presents a thorough and well written analysis of inductive biases in HNNs.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This is a well-written analysis of Hamiltonian Neural Networks (HNNs), a class of physics-inspired deep neural networks. The work is motivated by a desire to apply HNNs to non-toy datasets as well as to gain an understanding of the key inductive biases that explain the majority of their performance. Through controlled experiments on synthetic trajectory data, they explore energy conservation, a symplectic bias, complexity of state representation, and second-order structure bias. The key finding is that the second-order structure in HNNs is the main explainer of its performance. A simpler model that combines a Neural ODE with second-order structure is introduced, which can be seen as a distilled HNN. The simpler model achieves stronger performance on Mujoco rollout prediction compared to the HNN.
main_review: ########################################################################## 

 Pros:

-The presentation is clear and easy to follow, even for readers without an extensive mathematical background or a prior familiarity with Hamiltonian dynamics.

-The inductive biases chosen to study are well-motivated. The experiments and analysis used to examine each inductive bias are well-thought-out and the conclusions are supported by the provided evidence.

-The distilled architecture for modeling dynamical systems without the “bells and whistles” of HNNs, NODE + SO, is a solid contribution. This may be surprising to some and seems promising for being used as a baseline in future work. I also found the analysis on the relative energy violation of NODEs vs. HNNs in Section 4.1 compelling and illuminating.

##########################################################################

Cons:  

-A takeaway of the experiment in Section 4.2 (Symplectic vector fields) is missing. It would be helpful to add what (if anything) can be concluded by the finding that the symplectic regularizer alone is insufficient to help improve test rollout error. 

-For the Mujoco experiment in Section 5, only quantitative results are provided. Given that the page limit is 9 pages but only 8 were used, it seems reasonable to also provide qualitative visualizations that compare rollouts of NODE, NODE + SO, and SymODEN. Providing GIFs/videos would also be helpful for the reader.

-A discussion in Section 5 on why the NODE and NODE + SO Swimmer test time performance differs by a large margin is missing (also, qualitative visualizations could be helpful to explain this).

#########################################################################

Question for authors:

I didn’t quite understand the implications of the Mujoco experiment for the MBRL research community. Is the recommendation that NODE + SO could/should be used as a drop-in replacement for the “simpler dynamics models that predict the next change in state”?  Would it not make sense then to add a baseline to this experiment to compare directly against a simple dynamics model used in a popular Mujoco MBRL framework?

#########################################################################

Some suggestions for improvement: 

-In Section 4.1, the line after the equation references “where the *last line* follows” → should be changed to “where the last equality follows”.

-I would suggest defining “symplectic” earlier in the paper, perhaps in the introduction. Currently, it is introduced at the beginning of Section 4.2, although it is discussed at the end of the introduction.

-The last sentence of the introduction, “Extracting the second-order bias, we show how to improve the performance of dynamics models” is unclear to me. Perhaps it could be re-written to more clearly highlight the specific contribution, which to my understanding is: a Neural ODE that models dynamics with second-order information (NODE + SO)?

summary_of_the_review: I think this paper deserves acceptance. In my opinion, it clearly adds new knowledge to the body of work on HNNs, is written exceptionally well, and introduces a useful framework (NODE + SO) for modeling dynamics in non-trivial environments that has the potential to benefit the model-based reinforcement learning (MBRL) community. A few minor requests are provided to strengthen the paper.

==========================================================================

Update after rebuttal: I will be maintaining my initial score and hope that the authors will update their paper for the final camera-ready version to include the promised improvements.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: ## Summary and contributions.
Motivated by the success of physics-inspired neural networks, authors investigate the different inductive biases (implicitly or explicitly) encoded in Hamiltonian neural networks (HNNs). Authors identify four biases: the ODE, second order, energy conservation and symplectic biases. They claim and empirically show that, as opposed to what was previously argued, HNNs' performance is mostly due to the implicit second-order bias. Indeed, the authors show that explicitly splitting the Hamiltonian into a momentum and potential term, leads to the system's position $q$ being parametrised as the solution of a second-order differential equation (which involves the mass matrix and the potential function).
main_review: ## Strengths. 
The submission is well-motivated and the investigation on HNNs inductive biases is thoroughly performed.
Up to my knowledge, the contributions of this submission are indeed novel.
This submission is definitely relevant for the community as it gives practical insights on commonly encountered inductive biases in physically informed machine learning models.
The writing is also good, making it easy for the reader to follow the narrative.

## Weaknesses.
One weakness perhaps is the lack of further investigation into the failure of HNNs on the MuJoCo physics simulator in Section 5 cf my comments at the end of this review.

## Clarity.
I found the submission well written, with the motivation, ideas and empirical findings well conveyed to the reader. As a consequence, reading this submission was very pleasant.

I would perhaps suggest to add a causal graph---at the end of Section 1 or the begging of Section 4---showing the relationship between the inductive biases (e.g. symplectic dynamics <=> Hamiltonian system, etc). Then it would be easy to understand which assumptions are the strongest etc.

## Relation to prior work.
Related work is discussed properly up to my knowledge.

## Additional feedback.
- Would suggest numbering more equations (at least for the submission).
- "Although energy conservation may be helpful for generalization, the evidence does not indicate that HNNs are better at controlling energy violation than NODEs" -> Perhaps I am missing something, but Figure 2 seems to show that although both HNN and NODE (energy) error grows somewhat linearly wrt rollout time, the HNN's energy error is about ~x10 smaller than NODE's one. Yet, it is indeed interesting to highlight that empirically the energy is not preserved.
- Section 4.2: 
    - It is a bit unclear what the vector field F is at first, although it becomes clear a bit later with Equation 1.
    - I would suggest to add a Figure showing phase portraits for a symplectic and a non symplectic system, to give additional intuition to readers that may not be familiar with this concept.
    - Figure 3: Why not having similar plots that in Figure 2? Is the empirical symplecticity growing linearly with rollout time?
- Section 4.3: 
    - Would suggest recalling the Hamiltonian dynamics $\frac{d}{dt}[q p]$.
    - $\frac{dp}{dt}=A_\theta(q, p)$ potentially slightly confusing as the second-order structure above is on $v$ with $\frac{dv}{dt}=A(q, v)$. Would advise writing both in terms of $v$ or $p$, or at least reminding the relationship between $p$ and $v$.
- Section 4.4.1:
    - Is the drag term added to all models?
    - What if the drag term is not known?
    - "we show that simply using the bias of second-order dynamics is sufﬁcient to achieve nearly the same performance with much less complexity" -> How come with much less complexity? In what sense?
- Section 5:
    - The subpar performance of HNNs is definitely interesting, but with no further investigation on the underling reasons we are left a bit unsatisfied. Does the MuJoCo physics simulator indeed conserve energy? Is it an Hamiltonian system? Would be interested to have a similar plot than Figure 2, perhaps the true energy is actually not well learnt.

summary_of_the_review: Overall this submission is well motivated, thoroughly executed and well written, hence my recommendation for acceptance.
review_title: null

Meta-review: This paper examined physics-inspired inductive biases in neural networks, in particular Hamiltonian and Lagrangian dynamics. The work separated the benefits arising from incorporating energy conservation, the symplectic bias, the coordinate systems, and second-order dynamics.  Through a set of experiments, the paper showed the most important factor for improved performance in the test domains was the second-order dynamics, and not the more common explanation of energy conservation or the other factors. The increased generality of this approach was demonstrated with better predictions on Mujoco tasks that did not conserve energy.

All reviewers liked the insights provided by the paper.  They agreed that the paper clearly laid out several hypotheses and systematically tested them.  The reviewers found the experiments thoughtful and the results compelling.  The reviewers also pointed out several aspects of the document that could be improved, including additional formalism clarifications (reviewer nLbj), baseline algorithms (reviewer wu5x), and domains (reviewers 7KKB,SW9u).  The reviewers found the author's response satisfactory but were disappointed that a revised paper was not ready to read. The reviewers want the final paper to include the modifications that were promised in the author response.

All four reviewers indicated to accept this paper which contributes novel insights that simplify and generalize physics-inspired neural networks. The paper is therefore accepted.
Decision: Accept (Spotlight)