SyMetric: Measuring the Quality of Learnt Hamiltonian Dynamics Inferred from Vision
Authors: Irina Higgins
Peter Wirnsberger
Andrew Jaegle
Aleksandar Botev
Venue: NeurIPS 2021 Poster (2021)
Keywords: Hamiltonian
continual dynamics
metric
Hamiltonian Generative Networks

Abstract: A recently proposed class of models attempts to learn latent dynamics from high-dimensional observations, like images, using priors informed by Hamiltonian mechanics. While these models have important potential applications in areas like robotics or autonomous driving, there is currently no good way to evaluate their performance: existing methods primarily rely on image reconstruction quality, which does not always reflect the quality of the learnt latent dynamics. In this work, we empirically highlight the problems with the existing measures and develop a set of new measures, including a binary indicator of whether the underlying Hamiltonian dynamics have been faithfully captured, which we call Symplecticity Metric or SyMetric. Our measures take advantage of the known properties of Hamiltonian dynamics and are more discriminative of the model's ability to capture the underlying dynamics than reconstruction error. Using SyMetric, we identify a set of architectural choices that significantly improve the performance of a previously proposed model for inferring latent dynamics from pixels, the Hamiltonian Generative Network (HGN). Unlike the original HGN, the new SyMetric is able to discover an interpretable phase space with physically meaningful latents on some datasets. Furthermore, it is stable for significantly longer rollouts on a diverse range of 13 datasets, 
 producing rollouts of essentially infinite length both forward and backwards in time with no degradation in quality on a subset of the datasets.

SyMetrie: Measuring the Quality
of Learnt Hamiltonian Dynamics Inferred from Vision
Irina Higgins
Peter Wirnsberger
DeepMind London Arinah@deepmind.comm
DeepMind London pewi werridddeemid.ccom com
Andrew Jaegle
Aleksandar Botev DeepMind
DeepMind
London drewj arexjaaggleddespmind.. com
London botev@deepmind.com com
Abstract
A recently proposed class of models attempts to learn latent dynamics from high-dimensional observations, like images, using priors informed by Hamiltonian mechanics. While these models have important potential applications in areas like robotics or autonomous driving, there currently no good way to evaluate their performance: existing methods primarily rely on image reconstruction quality, which does not always reflect the quality of the learnt latent dynamics. this work, we empirically highlight the problems with the existing measures and develop set of new measures, including a binary indicator of whether the underlying Hamiltonian dynamics have been faithfully captured, which we call Symplecticity Metric or SyMetric. Our measures take advantage of the known properties Hamiltonian dynamics and are more discriminative of the model's ability to capture the underlying dynamics than reconstruction error. Using SyMetric, we identify set of architectural choices that significantly improve the performance a previously proposed model for inferring latent dynamics from pixels, the Hamiltonian Generative Network (HGN). Unlike the original HGN, the new HGN++ is able to discover an interpretable phase space with physically meaningful latents on diverse
some datasets. Furthermore, is stable for significantly longer rollouts range of 13 datasets, producing rollouts of essentially infinite length both forward and backwards in time with no degradation in quality subset of the datasets
Introduction
If you want understand how the world will change, good place start is with simple physical principles, such as the laws of motion. One way to achieve good performance in predicting dynamics forward and backward in time is building model using the Hamiltonian formalism from elassical mechanics. This formalism aims to model physical systems which conserve energy. Apart from real physical problems, other important dynamical systems can also be modelled using this formulation, including GAN optimisation [35] 38, 34 36), multi-agent learning in non- -on-tranniiive zero-sum games (5) 633), or the transformations induced by flows in generative models (41). While many naturally observed dynamics do not preserve energy - for example, energy is often added or dissipated they can
'The code for reproducing all results is available on deepmind- hts.aidddrroeearcccceeeeeeeeeeee http:/scssssssssred_nooeee models
https://ggttubb.com//eeeg
35th Conference on Neural Information Processing Systems (NeurIPS 2021).

---

still be modelled using the Hamiltonian formalism by augmenting it with additional terms capturing
the deviations from the energy conserved state [37 23 43 55 56
Recently, a body of work has emerged that brings these well-established principles of modelling dynamics from physics such as the conservation of energy, and numerical formulations from the theory of differential equations to neural network architectures [49 20
9 3
58
g 4 31
50
Most of these papers, however, learn dynamics directly from an abstract state, such as the positions and momenta of the physical objects.
43
56
12
17
24
14
32
45
57
48
21..
This may be suitable choice for some applica- tions, for example in molecular simulations, but in situations where inference at test time has to
Mass swriipppigggggg dolaset
be made from gigh-dimensional observations, like images - a setup common to many robotics, reinforcement learning or autonomous driving challenges it is important to be able to have models that can both correctly infer the abstract state space and faithfully capture the underlying Hamiltonian dynamics without access to the ground truth state space information at inference time. Currently only handful of methods exist for learning dynamics with physical priors from 3 58). These models learn in a completely unsupervised manner using pixel observations of the dynamies, however, currently there is no good way to know whether these models have indeed managed recover the underlying dynamics faithfully through learning. In this paper we empirically demonstrate that reconstruction based measures typically used in the field are poor indicators of how well the model has learnt the underlying dynamics and propose a better method for evaluating such models. In summary, our contributions are follows:
Oissersantona:
Canorial
phase space:
mf p
(1..P)
H(Q. Pw
Equatinns of molio:
p/m
0 Q=
P-1
pixels [4 45
Figure I: Illustration of canonical transformation between the Cartesian phase space (g.p) and the action-angle (Q, P) representations for the mass- spring Hamiltonian system. Both coordinate spaces preserve the underlying dynamics.
1. We introduce performance measures for identifying whether model trained with the Hamil- tonian prior has managed to capture the true dynamics of mneryy-conserving system when learning from high-dimensional observations.
2. We use these measures to identify a set of hyperparameters and architectural modifications that significantly improves the performance of Hamiltonian Generative Networks (HGN) [49], an existing state of the art model for recovering Hamiltonian dynamics from pixel observations, both in terms of long time-scale predictions, and interpretability of the learnt latent space.
2 Preliminaries
Hamiltonian dynamics The Hamiltonian formalism a mathematical formulation of Newton's 's
cquations of motion for describing energy- corrervvoneev dynamics (see (25] 29 ) It describes the continuous time evolution a system in an abstract phase space with state s = (q.p) E R22", where
qe e Rd is vector of positions, and p E 8" is the corresponding vector of momenta. The time evolution of the system in phase space is then given by the Hamiltonian equations of motion:
OH. 3p
0H
P
3g
where the Hamiltonian 2R maps the state s (q.p) to scalar representing the energy of the system. Based on the above equations of motion, the energy conservation property is easy to see, since dHf 0H 0H
di 3g q + op p p=00 O.
Hamiltonian dynamics as a symplectic map The time evolution of Hamiltonian systems is de- scribed using symplectic map. To define symplectic map, first need to define symplectic matrix.
0
I
A matrix M is called symplectic if M AM = A, where A =
T and E is the identity
matrix. Then, differenniable map F(s) : 2m is called symplectic if its Jacobian matrix J 0F
0s

---

is symplectic everywhere, i.e. J"AJ = A. Indeed, that Hamiltonian dynamics are governed by symplectic map da AH as (see Section 16.3 in [8] for the derivation).
dt
Canonical transformations Physical systems can be equivalently described in any number of arbitrary coordinate systems. For example, the dynamics of a simple mass-spring system can be described equally well using either canonical Cartesian phase space coordinates or the action-angle coordinates (see Fig. D. A canonical transformation is a mapping that moves between equivalent state spaces in way that preserves the form of the Hamiltonian dynamics. For example, there is a canonical transformation between the Cartesian phase space coordinates or the action-ang -angle coordinates shown in Fig. since both preserve the correct underlying Hamiltonian dynamics.
More formally, us consider a Hamiltonian system with phase-space state (q.p). Hamiltonian H(q.P) and equations of motion given by [ [ We can transform the state of this system according to new variable Q Q(q. p) and P P[q. p).
general, such transformation cannot be expected to preserve the equations of motion in the new variables (Q. However, if there exists new Hamiltonian H( that describes the same dynamics as the ones described by the original Hamiltonian H(q.p). where Q = 8h
and p 8h then such transformation is called canonical.
BP
aQ
Canonical transformations are also symplectic maps (2). We provide an illustration of a canonical transformation in Fig. and refer Stewart (46) for more details.
3 Measuring the quality of learnt Hamiltonian dynamics
To measure whether a model has learnt the underlying Hamiltonian dynamics faithfully, we propose measuring whether there exists a canonical transformation between the ground truth phase space and subspace or submanifold within the latent space discovered by the model. Note that direct comparison between the inferred state space and the ground truth phase space (e.g. I.2 distance) impossible because physical systems can be equivalently described in any number of arbitrary coordinate systems and there is no reason why the models should converge to the phase space arbitrarily chosen as the "ground truth" when learning from pixels.
As discussed in Sec. 2 symplectic maps (or canonical transformations) are typically defined over two spaces of the same dimension. our case, however, the latent space of the model is most often overparametrised to have more dimensions than the ground truth state. Saying this, if the model has learnt to capture the underlying Hamiltonian dynamics, the dynamics in its latent space should "mimic" the dynamics in the ground truth state space, in the following sense: given latent space trajectory (t) E Rm2m and ground state trajectory s(t) R 2m where m 2 n there should exist map F 2m + 2 such that the energy landscape defined by the learnt Hamiltonian H.m (S) scaled version of the energy landscape defined by the original Hamiltonian H., (FS)), i.e. Em Hm (S) = cH (L((((S)) (F(S)) cEn, (where is constant, and E is the energy of the system, see Sec. A.5 Sec.A..5j in the Appendix for more motivation for the constant). One can think of the map F as the pseudo-inverse inverse of symplectic map between the space 2 of the ground truth state and another space 2 which is embedded in the higher dimensional space R28 of the model latent space. This pseudo-inverse preserves the Hamiltonian dynamics of the ground truth state space in the model latent space. we expand the time derivative of the projection of the learnt state space back to the ground truth state space s(t) ((S())), see that
08(1) OF OS(1) Øt OS Øt aS aS
OF
8H.m
Am
Oa Am OF 81.,
OF
(2)
OS 0s
0Hn
=c[JA JT Ös
Hence, if we find a mapping F where c.J JT A, holds, then it suggests the dynamics in the latent space do in fact "mimic" the dynamics in the ground truth state space, and hence the model has recovered the underlying Hamiltonian dynamics of the system.
"These can also depend on time but we are not considering time- -eeeddeenddnt transformations here. Hence, we will only consider restricted canonical transformations in this paper.
3

---

To measure how well the model has learnt to mimic the original dynamics, we first learn the mapping F: SS+ s that explains as much variance in the original state space s as possible. We then check whether this mapping is symplectic by measuring how much J J deviates from A, up to constant
scaling factor, where J= af is the Jacobian of the learnt mapping, and A = as
0
11
is a 2k: 2
-1 0
anti-symmetric matrix. We next discuss each of the two steps in more detail.
Learning to explain the ground truth dynamics We first Iearn the mapping F' between the model state space S and the ground truth phase space A naive implementation would use MLP to approximate F. however one has to be very careful not to overfit to the training data. In the low data regimes found that even small MLPs trained with L1 regularization often overfit by learning an unnecessarily complicated map that ends up not being symplectic even if symplectic map exists (see Sec. A.3] in the Appendix). Hence, we make sure use least 1000x more datapoints for training the MLP than the number of its parameters.
Given that learning the map F relies on the availability of ground truth phase space data, which may be hard to obtain in certain domains, we also propose a more data efficient way of learning F that only requires labeling 100 training trajectories. We do so by going back to the basics of machine learning and avoiding overfitting by controlling the expressivity of the mapping F by training a Lasso regularised linear regression over polynomial expansion over the learnt state space features S, similarly to DiPietro et al [1]. We start with the expansion order of and progressively increase it up to an upper bound determined by the hyperparameter K, until enough variance in s is explained. In our experiments we use k = 5 (see Alg.[] iin Appendix for more details). To limit the exponential computational cost of computing higher order polynomial expansions of larger latent spaces, where possible we first filter "uninformative" state dimensions defined as those dimensions that have average KI. from the prior less than the threshold K [gg (S.) (S.))p((.). ([S))] 0.01 as in Duan et al [16]. however this step is not necessary. and we have successfully applied our approach without this step in practice to a 1024-dimensional latent space.
The MLP and polynomial regression (PR) instantiations of learning the mapping F have different trade-offs. As discussed above, MLP requires orders of magnitude more ground truth state data (in our experiments at least 640x) than PR, however because it is exposed to more data, it can be more accurate than PR, especially in terms of estimating how much variance in the ground truth dynamics can be explained well by the latent dynamics. The MLP can be arbitrarily expressive, while PR calculations get exponentially more expensive the expressivity of F is increased. PR, however, has the benefit of interpretability.
Having learnt F' using either the MLP or PR method, we calculate its goodness of fit, RP, which measures how much information about the ground truth phase space dynamics can be recovered from the learnt state space according to R² = ((F())-a)* where is the mean of the ground truth state variables, F(S) are the ground truth state variables predicted from the model state variables S, and the summation is over the available data points [13.. R2 serves useful estimate of how much information about the ground truth dynamics is preserved in the latent trajectories. We use it instead of estimation error based alternatives (e.g. MSE, which worked equally well in practice), because it is not sensitive to the magnitude of the data, and hence allows for a more direct comparison between scores obtained from different models trained on different datasets.
Measuring symplecticity of the mapping We check the symplecticity of the learnt mapping F by calculating A, J ATm.T As the next step we could measure directly whether this product is equal to the anti-symmetric matrix A,, however depending product will instead be equal to AT, (we are going to drop the subscript for easier readability where the dimensionality can be inferred from context), which may happen if the Hamiltonian learnt by the model is related to the original Hamiltonian through a negative constant in Eq 2 (also see Sec. AA3 in the Appendix). Hence, we instead calculate ÅÂT = JAJT JAT J and measure how far it deviates from the identity matrix:
the model it is possible that the
Sym=MME(44)
where e is a normalisation factor as per Sec. 3 (see See. A.5 in the Appendix for more motivation), and Il E 2 Note that e will be different between different models, since it depends on the learnt

---

HGN++
HGN
Dolaset
MSE
VPT
Sym
R²
MSE
VPT
8ym
R2
Mass-ggrnn
0.05
0,18
937.0
0.0%-0.0*
0.99%/1.0
25.07
197.67
5.75
0.68A0.0
0,8840.85
Mass-srrigg
567.5
0.03"-0.04
0.96-19995
34
0.78/000
Pendulum
1.95
10.5
85.25
L.76/0.61
0.54/0.96
25.99
294,22
1.25
0.430.0
0,79/0.81
Pendulum +
26.94
80.89
23.0
0.500.45
-0.0H0.5
23.43
194.52
5.75
0.37/50.0
0.39/0,55
Matching
640.5
0.35-0.25
0.990898
32.46
562.98
5.75
0.55/9118
0.880,95
Rock puper sciesoes
0.999.96
955.31
0:23/0.12
0.5540.88
Double pendulum
32.24
154.83
5.5
0.28/0.24
0.49.99
26.33
196.52
0.38L.1
0.200.64
Double pendulum rc
54.7
87.19
5.0
0.14/0.48
0,0590911
26,08
99.28
2,5
0.12
0.0/6.6
Two-body
0.04
447.0
0 0.2200115
0.918896
22.33
152.42
12.75
0.21/0.14
0.3/0.98
Two-body +o
0.22/0.19
0.49995
0.32/0.78
3D room circle
114.43
370.05
1.0
0.11/max
0.16/0.97
39.55
1044.6
1.25
0.11/max
0.1600.55
MD 4 particles
55.08
218,47
0.75
0.07/0.06
0.040.94
23.41
1711.71
4.0
0.91.9.66
0.040.8
MD- purticles
364.73
447.82
0.0
0.02/0022
0.090.64
5.78
396.71
1,02950.00
0.0/0:4
Table I: The reconstruction (Rec) MSE refers to the MSE measured over the first = 60 timesteps. i.e. the same trajectory length that was used for training, but on test data. The extrapolation (Ext) MSE was computed across the subsequent T timesteps, immediately following those used for training. All MSE values are reported as multiples of 10". VPT scores are calculated as the average over forward and backward extrapolation. SyMetrie, Sym and R² results correspond to MLP//R implementations. * indicates models with SyMetric
Hamiltonian. We estimate it by finding constant that minimises Sym for each model separately. Since for symplectic maps the right-hane hand side in 3a.3 to be zero everywhere, we need to evaluate across all avaliable data. In practice we sample points across k trajectories to evaluate the Jacobian 1/mean (max ((AAT))) ()).
and average Sym score over. The normalisation factor is calculated
where the absolute maximum value of ÁÄT is taken at every evaluation point, and their average is taken across all the points from a single trajectory. We calculate different constant for each trajectory to ensure that on datasets where different trajectories are sampled from different Hamiltonians (e.g. mass-spring +C or pendulum +c), Sym is still meaningful. This does not affect Sym for datasets generated from a single Hamiltonian, where in practice the constant is the same across all trajectories.
3.1 Aggregating into a single indicator: SyMetric
R2 and Sym measure two orthogonal properties of the learnt dynamics: 1) whether they capture enough information about the ground truth dynamies; and 2) whether the captured dynamics mimic those of the ground truth faithfully. Indeed, these scores can be at the opposite scales of the range in one model. For example, model may "cheat" and learn to produce perfect trajectories in pixel space by learning q [q. ft).2 p [0,0,0] and H = p2. In this case Hamiltonian dynamics would increase t and. given a powerful enough decoder that can memorise observations indexed by t, such a model would score perfectly on any reconstruction based metric, including VPT. Likewise, given powerful enough function approximator for F, this model can also have a perfect R² score. However, this case F would not be symplectic, and hence Sym would be high and serve as the only indicator that the model has not actually recovered the underlying Hamiltonian dynamics as hoped. Alternatively, degenerate F (e.g. the identity function) would be symplectic and result in the perfect Sym = 0 score, however this case if the dynamics learnt by the model do not actually mimic the ground truth dynamics, it will be captured by the low R2. In other words, if R22 is low, then the learnt latent space is degenerate and does not preserve all the information about the ground truth dynamics. In this case, the value of Sym is irrelevant and the model has failed to learn well. If R²2 is high but Sym is low, then the learnt dyn dynamics are expressive enough to mimic those of the ground truth, but they are likely to have overfit to the training trajectories and will fail when it comes extrapolation, since they do not capture the true underlying dynamics. Hence, in order for a model to capture the underlying Hamiltonian dynamics well, the mapping between the latent space and the ground truth phase space needs to be both informative (high R²) and symplectic (low Sym).
While our general recommendation is the practitioners to use the R² and Sym scores for comprehen- sive model evaluation, we also propose a way to aggregate the two measures into single "at-a-glance" score for capturing the main insight: whether model has discovered the true dynamics not. We call
5

---

this aggregate score the Symplecticity Metrid or SyMetric. We define SyMetric to be binary according
if R2 > <ym<<
SyMetric
0, otherwise.
We found that SyMetric parameters a 0.9 and <= -0.55 worked well for both MLP and PR implemen- tations.
4
Models
HGN HGN 49 is a generative model that aims to learn Hamiltonian dynamics from pixel observa- tions æ. It consists of the encoder network that learns to embed sequence of images (Zo) to lower-dimensional abstract phase space 8 g~ ( (...... which consists of abstract positions and corresponding momenta = (q.p); the Hamiltonian network that maps the inferred phase space to scalar corresponding to the energy of the system H-, (st) E IR; an integrator that takes in the current state s its time derivative calculated using the Hamiltonian network, and the timestep At to produce the state 8 Ar; and the decoder network that maps the position coordinates of the phase space back to HGN is trained using the variational objective
the image space po (x:)) = do
1
(acpi))DDx((((ss0 pp)a) (x. (x) ( (xL((o(s((X))p(s))
(5)
T+1
where X = [T) and p(s) is the isotropic unit Gaussian prior.
sweep over hyperparameters. investigated architectural changes (see Sec. i.I Appendix for more details) and used our proposed new measures described in the Sec. for model selection. We evaluated a range of learning rates, activation functions, kernel sizes for the convolutional layers, hyperparameter settings for the GECO solution for optimising the variational objective 422 and the inference and reconstruction protocols used for training the model. We also considered larger architectural changes, like whether to use the spatial broadcast decoder [52]: whether to separate networks for inferring the position and momenta coordinates the encoder; whether to explicitly encourage the rolled out state at time /++ N be similar the inferred state at time ++ N as in (3); whether infer the phase space directly or project it through another neural network as in the original HGN work; whether to train the model to predict rollouts forward in time, or both forward and backward in time in [24]: and finally whether to use 2D (convolutional) or a ID (vector) phase-space and corresponding Hamiltonian. We found that a combination of 3 kernel sizes and leaky ReLU activations in the encoder and decoder, Swish activations (39] in the Hamiltonian network, as well as a ID phase-space inferred directly from images used in combination with a Hamiltonian parametrised by an MLP network significantly helped to improve the performance of the model. Furthermore, changing the GECO-based training objective a B- --E--bbaeed [22] one, training the network for prediction rather than reconstruction and training it explicitly to produce rollouts both forward and backward in time further improved its performance.
HGN++ In order to improve the performance of HGN,
5 Datasets
We compare the performance of models 13 datasets instantiating different types of Hamiltonian dynamics introduced in Botev et [7]. including those of mass-spring, pendulum, double-pendulum and two-body toy physics systems, molecular dynamics, dynamics of camera movements in 3D environment and learning dynamics in two- two-player non-transitive zero-sum games known to exhibit Hamiltonian- cyclic behaviour 53 All of these (apart from 3D room) were instantiated in two different versions of increasing difficulty. See Sec. A.2 in Appendix for more details.
While we abuse the notation by calling our proposed performance measure a "metric", we could not resist the
pun.

---

Baseline measures
common way to evaluate performance of models of this class is by comparing the mean squared error (MSE) (or equivalent) between the reconstrueted and the original pixel observations test subset of the data. This has two problems. First, it is well known that pixel reconstruction errors can often be misleading 44 30
15 - e.g. low reconstruction errors may be obtained from a perfect reconstruction of the static background while failing to reconstruct the dynamics of interest, if these belong to a relatively small object. Furthermore, each absolute value of the reconstruction error may mean different things across datasets depending on the visual complexity and other particularities of the data, which makes model comparison across datasets hard. Furthermore, good predictive performance on short sequences used for training does not necessarily mean that the model has faithfully captured the underlying dynamies or imply good performance on long time horizon predictions.
51
Reconstruction and extrapolation errors To demonstrate the problems that arise when using observation- reconstruction quality for evaluating the model's s ability to learn dynamics, we calculate the "reconstruction" pixel MSE-tho the most commonly used measure of model performance (e.g. employed by [49, [9] 45 3 58), where the model is evaluated on how well it can reproduce the same trajectory length T as was used for training. albeit using test data. We also calculate MSE over extrapolated trajectories, where we continue roll out the model for total of 27 steps and measure MSE over the last T timesteps, to check whether measuring extrapolation even over short time periods might predict the model's ability to extrapolate further in time better than the "reconstruction" MSE. In all our experiments T' 60, and for more fair comparison across datasets in all cases MSE is normalised by the average intensity of the ground truth observation proposed in Zhong et al [57]: where æy is the ground truth and i; is the reconstructed observation.
MSE= x
/ //|1az [
Valid Prediction Time We validate our proposed measures by comparing their predictions to another estimate of the model's performance, namely the Valid Prediction Time (VPT) (as in Jin et al (6)). VPT is motivated by the intuition that those models that have truly captured the underlying dynamics of energy conserving non-chaotic systems should in principle be able to produce forward and backward rollouts over significantly longer time intervals than those used for training without significant detriment to the reconstruction quality. Note, however, that VPT may also produce false positive false negative results as will be discussed in Secs. and 7 and hence is not perfect. It is, however, the best alternative measure we could come up with for validating our proposed measures. To calculate VPT we generate a small number of very long trajectories of between 256 and 1000 steps for our datasets (we use 60 step trajectories for training) and measure how long the model's trajectory remains close to the ground truth trajectory in the observation space. It corresponds to the first time step at which the model reconstruction significantly diverges from the ground truth:
VPT = argmin (MS((2,,
where x, the ground truth, 2; is the reconstructed observation at time and is threshold parameter. For our datasets we found a threshold of X = 0.025 a reasonable choice based on visual inspection of the rollouts. VPT scores are averaged across 10 trajectories.
7 Results
We first trained different variations of the HGN model as described in Sec.ee......oo the mass-spring dataset and found that particular set of hyperparameters and architectural choices consistently resulted in models that were identified by SyMetric as having learnt the underlying Hamiltonian dynamics faithfully. We refer to this improved version of HGN HGN++ We then trained this model, as well as the original version of HGN, on all 13 datasets. Table [ebbonstattttteeee that HGN++ is indeed a significant improvement on HGN. While HGN is practically unable to produce good rollout trajectories beyond the 60 timesteps that it was trained on. HGN++ can extrapolate well for at least 267 steps forward and 178 steps backward on average across all datasets (vs 11 and 0 steps by HGN) as evidenced by the VPT scores. Backward extrapolation is achieved by feeding A/ to the integrator at test time. Furthermore, on both the easy and hard (+c) versions of the mass-spring dataset, HGN++ is able to faithfully recover the underlying Hamiltonian dynamics, as identified by the SyMetric score of 1.
7

---

A
0
Mhower mode
leay
800
HD particia
Pondalim
vatt-pring
200.
0
Model
Figure 2: A: 119 HGN models trained with different hyperparameters on 13 datasets ordered by average forward and backward VPT score. Arrows point at the models that achieved SyMetric = 1. B: Example trajectories of single phase space coordinate in the ground truth phase space (orange) and the projection of the model latent space into the ground truth phase space using F (blue). The models highlighted by SyMetric to have discovered the true Hamiltonian dynamics (trained mass-spring and mass-spring +c datasets) appear to mimic the ground truth dynamics well, while the other models have qualitatively different dynamics despite sometimes scoring similarly by VPT. Red vertical lines indicate the time step where VPT indicates divergence in pixel space.
Given the radically different learning ability of HGN and HGN++ SyMetric is not the only performance measure that differentiates between them. Indeed, MSE results are also indicative of the superior performance of HGN++. However, not all seeds of HGN++ always converge to the same level of performance, and indeed slight modifications in hyperparameters often affect the model's 's learning ability in ways that have negligible effect on MSE, yet can drastically affect whether the models able to uncover the true Hamiltonian dynamics or not. When we look at the top 20% of model seeds in terms of their VPT performance across all datasets, and compare the different measures in terms of how well they differentiate between the better and worse performing seeds based on how well they correlate with VPT (averaged forward and backward), reconstruction MSE has 0.07 absolute Spearman correlation, extrapolation MSE is better at 0.33, however R has stronger correlation of 0.59(0.41) and Sym has correlation of 0.37(0.45) MLP(PR). Hence, when it comes doing more precise model or hyperparameter selection and differentiating between models that are not obviously failing, it appears that the currently widely used reconstruction MSE measure is very poor, and using either R2² or S Sym is more informative.
Fig. 2 orders the 119 models in terms of their average VPT scores. We see that the mass-spring dataset is clearly solved, with a number of HGN++ seeds with slightly different hyperparameters achieving close to perfect 1000-step extrapolations both forward and backward in time after being trained on just 60 steps. The figure also demonstrates that all of these models are picked out by SyMetric as having discovered the true Hamiltonian of the system. However, the next step change down in VPT measures, we see inconsistent predictions from VPT and SyMetric. A number of model seeds trained on the matching pennies and two-body datasets are scored highly by VPT, but are not highlighted by SyMetric to have discovered the true Hamiltonian, while a HGN++ seed trained on the mass-spring to data with similar VPT is highlighted as having done so. Which measure is correct: VPT or SyMetric? When we visualise an example trajectory from these models, mapping the learnt latent space to the ground truth through F and visualising both (see Fig. (B), we see that SyMetric is in fact correct. The latent dynamics of models trained on matching pennies and two-body datasets are qualitatively different from those exhibited by the ground truth system, while the dynamics from the model trained on mass mass-spring +c appear to be mimicking those of the ground truth.
In order further verify the validity SyMetric, Fig. BA-B visualises four trajectories in the ground truth phase space and in the latent space learnt by the HGN++ seed highlighted as have learnt the underlying Hamiltonian dynamics on the mass-spring +c dataset. It is clear that latent Q3 and its corresponding P3 were able to capture the ground truth state dynamics well. Indeed, when we visualise the extrapolated rollouts of the four sampled trajectories Fig. BC, we see that they still look good even after 1000 steps, thus verifying that the model is indeed able to produce good rollouts and has appeared to have recovered the underlying ground truth system well. So what causes the relatively low VPT scores recorded for this model? It appears that this discrepancy arises due to a slight error in the estimation of the initial conditions by the model. When we visualise the 1000-step rollout in the ground truth state against the projection of the model's latent state into the ground truth phase space, it is clear
8

---

Ground truhh
B
Model
Trijstory
Gg
9
Termstop
q= 0.94
Arrpliuude
g1
GM
p- P,
25
35
100 26 ar
Figure 3: Visualisation of four trajectories of the mass-spring +0 dataset in the ground truth phase space. B: Same trajectories visualised in the informative latents of a trained HGN++ with the perfect SyMetric score. both the Q, and its corresponding P. are informative, the trajectories are plotted as a scatter plot. Otherwise they are plotted as points in the corresponding latent dimension. C: 1000 step rollouts produced by the model in B of the four trajectories in A-B. The reconstructions of good quality even after 1000 steps. D: Same 1000 step rollouts as in C but shown in the ground truth phase space and the projection of HGN++ from into the ground truth phase space as produced by the regression step of SyMetric. Due to slight errors in the inference of the initial state, the trajectories diverge after about 300 steps, resulting in a suboptimal VPT score. E: Latent traversals of the informative dimensions of HGN++ shown in B. Each plot shows 200 step rollout trajectory of Q3 when setting the value of the traversed latent dimension to 22 0 or 2. The inset images demonstrate the corresponding effect the reconstruction of the first frame of the trajectory (from left to right). Latents Q3. Qs. Q 10 and Q13 appear to have an interpretable meaning, which is indicated in the left top corner of the corresponding subplot.
that the two trajectories slightly out of phase with each other, which results in divergence around the time step captured by the VPT score (indicated by the red vertical line in Fig. 3D) Hence, although the model was able capture the underlying Hamiltonian faithfully as captured by SyMetric score, it appears that its VPT scores were handicapped by the slight errors in the inferred initial conditions.
Furthermore, the HGN++ shown in Fig. 3 had seven informative dimensions at the end of training. Two of them - Q2 and P3 ended up learning the dynamics that mimic the ground truth phase space. The other 5 latents did not have an informative momentum component, indicating that the model used these latents as constants. When we investigated what these latents learnt to represent by traversing their values and visualising the resulting effect on the dynamics in Q2, notice that many of these dimensions are physically meaningful Qs and Qı0 learnt to represent variations in colour, while Q1s learnt to represent the mass parameter of the Hamiltonian (see relevant equation in Appendix Tbl. Hence, it appears that HGN++ was able to address another criticism of the original HGN [9, i.e. lack of interpretability of its learnt latent space.
8 Conclusions
In this paper we have introduced set of novel performance measures R². Sym and their binary combination, SyMetric, which serve as better proxies than the existing measures of whether a model with Hamiltonian prior has captured the underlying dynamics of a Hamiltonian system from high- dimensional observations, such as pixel images. We have validated our measures both quantitatively
- by comparing them to the VPT score, another proxy of the models performance which measures the ability of the model to produce faithful trajectories in extreme extrapolation regimes in the high- dimensional observation space; and qualitatively by visualising
latent space the model. We have shown that we were able to use SyMetric to identify set of hyperparameters and architectural modifications that improved the previously published HGN model terms of extrapolation perfor- mance and interpretability of its latent space, in effect "solving" 2/13 datasets considered here. We

---

hope that SyMetric can be used by the community to make further progress towards building models with Hamiltonian priors that can learn dynamics from pixels.
While SyMetrie requires ground truth phase space information in addition to pixel observations, this information is only required for relatively small number of trajectories (we used 100 in the PR case), and for relatively short trajectory lengths (60 steps, as was used for training) -he amount of data that would not be sufficient to train the models. This is in contrast the significantly longer trajectories (256-1006 steps) that were necessary to compute VPT scores - the only observation-space measure to achieve comparable results. By avoiding the need to generate long trajectories, we hope that SyMetric can be good option for evaluating models that learn from observation where collecting such long trajectories is expensive, e.g. due to the computational costs of running a simulation or other costs corresponding to collecting experimental data.
Sy Metric has the drawback of relying on multiple measures: R2 and Sym. As part of future work hope to replace this with single step by using function approximators for F that are symplectic by design (e.g. (4), thus producing single measure equivalent to R², which will only be high there exists a symplectic mapping between a subspace of the latent space of the model and the ground truth phase space, Furthermore, we hope extend SyMetric to models that incorporate the Lagrangian prior in their dynamics (e.g. 58). which should be possible due to the equivalence of the Hamiltonian and the Lagrangian formalisms, and the ability to map between them through the Legendre transform.
Finally, would be interesting to investigate in detail how SyMetric compares to VPT on systems that known exhibit chaotic dynamics (e.g. the MD datasets). This type of problem poses significant challenge to our models because tiny perturbations to the initial conditions grow exponentially causing trajectories to diverge within few hundred training steps (e.g. see [19)). While this poses fundamental challenge to our training procedure and all evaluations metrics, we believe that the significantly shorter trajectories required for SyMerric may be advantageous.
10

---

References
[1] J. L. F. Abascal and C. Vega. A general purpose model for the condensed phases of water: Tip4p/2005. J. Chem. Phys., 123:234505, 2005.
[2] R. Abraham and J.E. Marsden. Foundations of Mechanics. AMS Chelsea publishing. AMS
Chelsea Pub//Amrrica Mathematical Society, 2008.
[3] Christine Blen-Blaanchete Sushant Veer, Anirudha Majumdar, and Naomi Ehrich Leonard. LagNetViP: Lagrangian neural network for video prediction. In Proceedings of AAAI Confer- ence Artificial Intelligence, 2020.
[4] Omri Azencot, N Benjamin Erichson, Vanessa Lin, and Michael W Mahoney. Forecasting sequen- tial data using consistent Koopman autoencoders. Proceedings of International Conference on Machine Learning (ICML), 2020.
[5] James P Bailey and Georgios Piliouras. Multi-agent learning in network zero-sum games is Hamiltonian system. In Proceedings of the International Conference on Autonomous Agents and Multiagent Systems (AAMAS), 2019.
[6] David Balduzzi, Sebastien Racaniere, James Martens, Jakob Foerster, Karl Tuyls, and Thore Graepel. The mechanics of n-player differentiable games. In International Conference Machine Learming, pages 354-363, 2018.
[7] Alexandar Botev, Andrew Jaegle, Peter Wimsberger, Daniel Hennes, and Irina Higgins. Which priors matter? Benchmarking models for learning latent dynamics. In Proceedings Neural Information Processing Systems (NeurIPS), 2021.
[8] Zhengdao Chen, Jianyu Zhang, Martin Arjovsky, and Léon Bottou. Symplectic recurrent neural networks. In Proceedings of International Conference on Learning Representations (ICLR), 2020.
[9] Anshul Choudhary, John F Lindner, Elliott G Holliday, Scott T Miller, Sudeshna Sinha, and William Ditto. Forecasting Hamiltonian dynamics without canonical coordinates. Nonlinear Dynamics, pages 1-10, 2020.
[10] Wendy D. Comelll, Piotr Cieplak, Christopher I. Bayly, Ian R. Gould, Kenneth M. Merz, David M. Ferguson, David C. Spellmeyer, Thomas Fox, James W. Caldwell, and Peter A. Kollman. A second generation force field for the simulation of proteins, nucleic acids, and organic molecules. 1995.
J. Am. Chem. Soc., 117(19):5179-51)
[11] Miles Cranmer, Sam Greydanus, Stephan Hoyer, Peter Battaglia, David Spergel, and Shirley Ho. Lagrangian neural networks. ICLR Deep Differential Equations Workshop, 2020.
[12] Shaan A. Desai, Marios Mattheakis, and Stephen J. Roberts. J.operts. Variational integrator graph networks for learning meegyy-conservnn dynamical systems. Physical Review E., 104(3), 2020.
[13] Jay L. Devore. Probability and Statistics for Engineering and the Sciences. Spinger, 2008
[14] Daniel M. DiPietro, Shiying Xiong, and Bo Zhu. Sparse symplectically integrated neural networks.
In Proceedings of Neural Information Processing Systems (NeurIPS). 2020.
[15] Alexey Dosovitskiy and Thomas Brox. Generating images with perceptual similarity metrics based on deep networks. In Proceedings of Neural Information Processing Systems (NeurIPS), 2016.
[16] Sunny Duan, Loic Matthey, Andre Saraiva, Nicholas Watters, Christopher P Burgess, Alexander Lerchner, and Irina Higgins. Unsupervised model selection for variational disentangled represen- tation learning. In Proceedings of International Conference on Learning Representations (ICLR), 2020.
[17] Marc Finzi, Ke Alexander Wang, and Andrew Gordon Wilson Simplifying Hamiltonian and Lagrangian neural networks via explicit constraints. Proceedings of Neural Information Processing Systems 2020.

---

[18] Jason Frank.
Symplectic flows and maps and volume preservation. https://webspaceessieeee www.:ee..scinnc........ / pddf 2008.
[19] Daan Frenkel and Berend Smit. Understanding Molecular Simulation. Academic Press, San
Diego, second edition, 2002.
[20] Samuel Greydanus, Misko Dzamba, and Jason Yosinski. Hamiltonian neural networks. In Proceedings of Neural Information Processing Systems (NeurIPS), 2019.
[21] Niklas Heim, Václav Smídl, and Tomáš Pevný. Rodent: Relevance determination in differential equations, arXiv preprint arXiv: 1912.00666.. 2019.
[22] Irina Higgins Loic Matthey, Arka Pal. Christopher Burgess, Xavier Glorot, Matthew Botvinick. Shakir Mohamed, and Alexander Lerchner. 3-VAE: Learning basic visual concepts with a constrained variational framework. In Proceedings of International Conference on Learning Representations (ICLR), 2017.
[23] William G. Hoover. Canonical dynamics: Equilibrium phase-space distributions. Physical Review A, 31:1695-1697,, 1985.
[24] In Huh, Eunho Yang, Sung Ju Hwang, and Jinwoo Shin. Time-reversal symmetric ODE network.
In Proceedings of Neural Information Processing Systems (NeurIPS), 2020.
[25] Maxim Jeffis. Harvard, Lecture Notes: Classical mechanics and symplectic geometry, 2020. http:ttpp://peelle..nn math.
[26] Pengzhan Jin, Zhen Zhang, Aiqing Zhu, Yifa Tang, and George Em Karniadakis. SympNets: Intrinsic cutuuture-preservinn symplectic networks for identifying Hamiltonian systems. Neural Networks, 132:166-179, 2020.
[27] J. Jones and Sydney Chapman. On the determination of molecular fields. -II. From the equation of state of a gas. Proc. R. Soc. Lond. A, 106(738):463 477, 1924.
[28) Rishabh Kabra, Chris Burgess, Loic Matthey, Raphael Lopez Kaufman, Klaus Greff, Malcolm Reynolds, and Alexander Lerchner. Multi-object datasets. https///thhb..cmmdeemmmmmmmmu object-datasets/, 2019.
[29] Lev Davidovich Landau and Evgenii Mikhailovich Lifshitz. Course of theoretical physics. Elsevier, 2013.
[30] Anders Boesen Lindbo Larsen, Søren Kaae Sønderby, Hugo Larochelle, and Ole Winther. Autoencoding beyond pixels using learned similarity metrie. In Proceedings of International Conference on Machine Learning (ICML), 2016.
[31] Yunzhu Li, Toru Lin, Kexin Yi, Daniel Bear, Daniel LK Yamins, Jiajun Wu, Joshua B Tenen- baum, and Antonio Torralba. Visual grounding of learned physical models. In Proceedings of International Conference on Machine Learning (ICML), 2020.
[32] Michael Lutter, Christian Ritter, and Jan Peters. Deep Lagrangian networks: as model prior for deep learning. In Proceedings of International Conference on Learning Representations (ICLR), 2019.
Using physics
[33] Andrew L Maas, Awni Y Hannun, and Andrew Y Ng. Rectifier nonlinearities improve neural network acoustic models. In Proceedings of International Conference on Machine Learning (ICMI.), 2013.
[34] Lars Mescheder, Andreas Geiger, and Sebastian Nowozin. Which training methods for GANs do actually converge? In Proceedings of International Conference on Machine Learning (ICML., 2018.
[35] Lars Mescheder, Sebastian Nowozin, and Andreas Geiger. The numerics of GANs. In Proceedings of Neural Information Processing Systems (NeurIPS), 2017.
12

---

[36] Luke Metz, Ben Poole, David Pfau, and Jascha Sohl-Dickstein. Unrolled generative adversarial networks. In Proceedings of International Conference on Learning Representations (ICLR), 2017.
[37] Shuichi Nosé. A unified formulation of the constant temperature molecular dynamics methods. Chem. Phys., (1()):11--51) 1984.
[38] Chongli Qin, Yan Wu, Jost Tobias Springenberg, Andrew Brock, Jeff Donahue, Timothy Lillicrap, and Pushmeet Kohli. Training generative adversarial networks by solving ordinary differential equations. In Proceedings of Neural Information Processing Systems (NeurIPS), 2020.
[39] Prajit Ramachandran, SarretZoph, and Quoc Le. Swish: a self--gate activation function. arXiv preprint arXivv
[40] Katharina Rath, Christopher G. Albert, Bernd Bischl, and Udo von Toussaint. Symplectic Gaussian process regression of maps in Hamiltonian systems. Chaos, ,1, 2021.
[41] Danilo Jimenez Rezende, Sébastien Racanière, Irina Higgins, and Peter Toth. Equivariant Hamiltonian flows. In NeurlPS workshop: Machine Learning and the Physical Sciences, 2019.
[42] Danilo Jimenez Rezende and Fabio Viola. Taming VAEs, arXiv preprint arXiv:1810.0059) 2018.
[43] Manuel A Rochrl, Thomas A Runkler, Veronika Brandtstetter, Michel Tokic, and Stefan Ober- mayer. Modeling system dynamics with thhysic-inffrmee neural networks based Lagrangian mechanics. In International Federation of Automatic Control (IFAC) World Congress, 2020.
[44] Oleh Rybkin. The reasonable ineffecctivenesss of pixel metrics for future prediction (and what to do about it). 2018. URL: http:ttts///btt..1////ppp
[45] Steindor Saemundsson, Alexander Terenin, Katja Hofmann, and Mare Deisenroth, Variational integrator networks for physically structured embeddings. In International Conference Artificial Intelligence and Statistics (AISTATS), 2020.
[46] lain W. Stewart. MIT, Lecture Notes: Advanced Classical Mechanics, 2016. URL: https //bit.1y/ooP7fi1
[47] Emanuel Todorov, Tom Erez, and Yuval Tassa. Mujoco: A physics engine for model-based control. In Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (ROSS,2012.
[48] Yunjin Tong, Shiying Xiong, Xingzhe He, Guanghan Pan, and Bo Zhu. Symplectic neural networks in Taylor series form for Hamiltonian systems. Journal of Computational Physics, 437, 2020.
[49] Peter Toth, Danilo Jimenez Rezende, Andrew Jaegle, Sébastien Racanière, Aleksandar Botev, and Irina Higgins. Hamiltonian generative networks. In Proceedingy of International Conference on Learning Representatiinss (ICIR), 2020.
[50] Silviu-Marian Udrescu and Max Tegmark. Symbolic pregression: Discovering physical laws from raw distorted video. Physical Review 103, 2021.
[51] David Warde-Farley and Yoshua Bengio. Improving generative adversarial networks with denois- ing feature matching. In Proceedings of International Conference on Learning Representations (ICLR), 2017.
[52] Nicholas Watters, Loie Matthey, Christopher Burgess, and Alexander Lerchner. Spatial broadcast decoder: A simple architecture for learning disentangled representations in VAEs. arXiv preprint arXiv: htt.::/9000000111 2019.
[53] Jinshan Wu. Hamiltonian formalism of game theory. arXiv preprint guant-ph 2005.
13

---

[54] Shiying Xiong, Yunjin Tong, Xingzhe He, Cheng Yang, Shuqi Yang, and Bo Zhu. Nonsep- arable symplectic neural networks, In Proceedings of International Conference on Learning Representations (ICIR), 2021.
[55] Yaofeng Desmond Zhong, Biswadip Dey, and Amit Chakraborty, Dissipative SymODENet: Encoding Hamiltonian dynamics with dissipation and control into deep learning. arXiv preprint arXiv:2002.08860. 2020.
[56] Yaofeng Desmond Zhong, Biswadip Dey, and Amit Chakraborty. Symplectic ODE-net: Learning Hamiltonian dynamics with control. In Proceedings of International Conference on Learning Representations (ICLR), 2020.
[57] Yaofeng Desmond Zhong, Biswadip Dey, and Amit Chakraborty. Benchmarking energy- conserving neural networks for learning dynamics from data. In Conference on Learning for Dynamics and Control (L4DC), 2021.
[58] Yaofeng Desmond Zhong and Naomi Leonard. Unsupervised learning of Lagrangian dynamics from images for prediction and control. Proceedings of Neural Information Processing Systems (NeurIPS), 2020.
14

---

Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] See Sec. g
(c) Did you discuss any potential negative societal impacts of your work? [N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full of assumptions of all theoretical results? [N/A]
(b) Did you include complete proofs of all theoretical results? [N/A]
3. If you experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either the supplemental material or as URL)? [Yes] See zip archive in supplementary materials
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they chosen)? [Yes] Both in main text and Appendix
(c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)? [No] Instead we provide the best seed results and full breakdown of performance by model seed as it more appropriate to showcase our proposed measure of model performance
(d) Did you include the total amount of compute and the type of resources used (e.g.. type of GPUs, internal cluster, cloud provider)? [Yes] See Appendix Sec.
4. If you using existing assets (e.g., code, data, models) curating/relcasing
assets...
(a) If your work uses existing assets, did you cite the creators? [Yes] See Sec. 5
(b) Did you mention the license of the assets? [Yes] See Appendix Sec. A.2
(c) Did you include any new assets either in the supplemental material
URL? [N/A]
(d) Did you discuss whether and how consent obtained from people whose data you'l 're using/curating? [N/A]
(c) Did you discuss whether the data you using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given participants and screenshots, if applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent participant compensation? [N/A]
15


Reviews:
[{"review_id": "LXMc2msREmp", "reviewer": "Reviewer_xPfe", "summary": "This paper mainly focuses on the application of Hamiltonian Dynamic in computer vision tasks. The authors propose a new metric to evaluate how the learned latent space coincide with the ground truth. Based on the novel metric, they further improve the previous HGN with multiple modifications in different aspects, leading to much better performance.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "1.\tThe authors provide a comprehensive analysis on the drawbacks of MSE as measurement for the corresponding task.\n2.\tRather than considering image-level metric, the authors take into account the canonical transformation for Hamiltonian dynamics, which is interesting and novel.\n3.\tThe proposed method is proven to be useful when considering HGN as baseline.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "1.\tThe main contribution of this paper is the new evaluation metric SyMetric. However, two things which I think are important for this metric are not explained in the paper.\na)\tThe final design of SyMetric is a binary indicator function. In this way, how to rank multiple models when only SyMetric is provided without $R^2$ and Sym?\nb)\tAlthough the authors conduct extensive experiments on a bunch of dataset, all results are based on the method of HGN and other variants. Is there any chance that the proposed metric cannot generalize to other type of methods? I suggest extra experiments on works like [8, 44, 2, 57] to further prove this.\n2.\tMaybe it is better to place the introduction of HGN++ after the new metric is explained for better understanding.\n3.\tThe authors mention that the design of HGN++ is selected using ‘proposed new measures’ (L106-107). However, it is not clear that:\na)\tWhich measurement is adopted during selection, Sym, $R^2$ or SyMetric? As I can understand, the HGN++ is designed so that one of these three metrics is consistently better. As explained in this paper, Sym and $R^2$ must be used together for comprehensive evaluation. However the proposed SyMetric is binary as mentioned above, thus unable to select among multiple choices. Moreover, the Sym of HGN++ is not consistently better than HGN among all datasets as shown in Tab. 1 which means the model selection may not be conducted based on Sym. \nb)\tWhat strategy is used in the model selection? How to find the best combination of the different modification?\n4.\tIs it safe enough to get the claim ‘SyMetric is in fact correct’ (L305) based on one sample visualization in Fig. 2B?\n", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "15 hours"}, "extra_text": {}}, {"review_id": "HAfIwbQTyet", "reviewer": "Reviewer_VPSk", "summary": "The paper proposes a distance measure to measure the symplecticness of the Hamiltonian Graph Networks. The experiments show that this measure is rarely equal to 1. However, it is only True for models that generally perform good on the task.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "I already reviewed the paper for ICML. The paper has been improved and the minor aspects were addressed. However, the main shortcomings have not been addressed (but most likely also cannot be addressed). The main shortcomings are:\n\n* the similarity measure is too complex to compute. It requires an additional function approximator to compute the measure. Furthermore, this fitting is sensible to the choice of the black-box function approximator.\n\n\n* the similarity measure is binary and only positive for two environments. Therefore, it is very hard to evaluate whether the distance measure is of any use or provides any intuition. It simply just shows that the currently learned latent codes are not that great.\n\nTo be honest, I do not think that there is any path to fix these two shortcomings without fundamentally looking for a new approach and evaluation. The approach needs to be improved such that the results increase my confidence in this measure as it is really hard to interpret. Due to these shortcomings, the approach lacks generality and I do not believe that the approach will make much of an impact on the community. However, the execution of the paper is very well done. The paper is well written, has nice figures and the evaluation is done thoroughly. Therefore, one cannot complain about the paper except the significance of the results. My personal take-away is that this paper provides a good documentation of performed research which leads to non-significant results. Therefore, I believe that the paper is valuable because of the good documentation, but its not significant to be presented at NeuRIPS. This conclusion is subjective and hence, please take it with a grain of salt. If the other reviewers think that the paper provides significant results please ignore my review.\n\n### Post Rebuttal Comment:\nThe comments of the authors clarified many points. I still cannot champion the paper for acceptance. As there is nothing wrong with the paper and its mostly about contribution and impact, I am happy with a weak accept. However, this rating is only marginally above the borderline. Therefore, I will not argue for the acceptance. \n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "N/A", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "1"}, "extra_text": {}}, {"review_id": "6gu7GgOIyT6", "reviewer": "Reviewer_sUUp", "summary": "The paper suggests novel measures for evaluating latent dynamics model by measuring how close the latent dynamics is to being governed by a symplectic map, a property of Hamiltonian dynamics. The paper analyses variants of the original Hamiltonian Generative Network [48] using the model and identifies better variants. The measure is also compared to related measures such as the MSE image reconstruction error, or the valid prediction time (VPT).\n", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "\nStrengths:\n- The proposed SyMetric measure is novel and can be interesting for theoretical analysis of learning latent dynamics in simulation environments.\n- The identified improvements over HGN are a secondary contribution.\n\nWeaknesses:\n- Sec 2, l74 ff is not well comprehensible. A differentiable map F is proposed whose symplectic property is defined. No relation to the Hamiltonian is made. The last sentence in this subsection in l. 78/79 uses the term \"symplectic map\" for ds/dt without defining it properly. Later in l. 93 suddenly canonical transformations are mentioned as being “symplectic maps” without explaining why. Since these notions are at the core of Sym measure, they need to be defined clearly so they can be taken up again when introducing the measure. Please clarify.\n- The Sym measure requires training a neural network for converting the learned latent state into the ground truth state representation. This is problematic for two reasons: a) the proposed measure is only of theoretical relevance, because it cannot be used in practice when ground truth is not available (otherwise learning a latent state representation would also be useless and the latent state mapping of the images could be directly learned onto the ground truth state representation). b) The analysis depends on the quality of the learned mapping which is unclear and can be prone to model errors. \n- Eq 2 has a sum ranging over t but it’s not used for any term inside the sum. Above the equation, the distribution p_\\theta(x_t) = d_\\theta( q_t ) is mentioned but never used again. What is the argument q_t of d_\\theta ? What is p_\\theta( x | s ) in eq 2 ? \n- The paper lists improvements over HGN but does not give insights why these changes make the model better.\n- l. 119, how can a 1D phase space make sense, shouldn’t it be at least 2D (1D position and 1D momentum)? Can a 1D phase space represent all the environments which are evaluated in Sec. 7?\n- l. 122 what does it mean to train the network on a “prediction” rather than a “reconstruction” task?\n- Eq. 3 what is the definition of \\hat{s} ?\n- l. 224. The R^2 measure is motivated as goodness of fit, but it does not follow the usual definition \\sum_i (F(S_i)-E(s_i))/E(s_i). Instead, it seems to compare the variance of mapping F(S) with the variance of the ground truth. The paper does not explain clearly, why this should be a meaningful measure. Why is a low number indicative of bad model performance? What happens if the learned mapping predicts larger or lower variance than the ground truth ? The motivation of thresholding for a low R^2 in l. 262 cannot be followed.\n- The proposed SyMetric measure is an ad-hoc combination of the R^2 and the Sym measure and relies on an empirical choice of two thresholds. What does it mean for the thresholds to “work well”? How to choose them subjectively?\n- What do the colors of the curves in the B plots mean? If the curves are prediction vs ground truth it seems the learned mass-spring +c clearly gets out of sync which is also detected by VPT, while the proposed SyMetric measure wrongly finds the learned model to mimic the ground truth dynamics well. Please discuss.\nOverall the technical description of the proposed measures seems flawed which make the method difficult to comprehend. The experiments do show issues of the proposed metric where the baseline VPT method seems sufficient and superior. The paper does not seem ready for publication yet. \n \n  \n\nFurther comments:\n- The description of HGN in sec. 3 should cite the original paper [48]\n- def of VPT: please use “\\operatorname{arg\\,min}” \n\n== Post-rebuttal comments ==\n\nThe author response addresses several of my concerns well. I still rate the paper borderline (\"marginally above acceptance threshold\"), since the approach requires to learn a mapping from the learnt latent state space to ground truth which could be difficult to assess in its quality itself and can invalidate the proposed measures if the mapping is \"off\". The paper should provide an adequate assessment of this potential problem, for instance, suggesting ways to analyze the accuracy of the learned mapping and its impact on the proposed measure. ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors mention several limitations of their work in the discussion section. Further discussion is needed according to the limitations discussed in sec \"weaknesses\" above.", "review_text": "", "extra_scores": {"ethical_concerns": "none apparent", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "8"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper has three reviewers, and the initial review are two borderline reject, and one borderline accept. After very long and detailed rebuttal and discussion process between authours and reviewers, two reviewers had raised their scores. In general, this paper has a theoretically grounded working solution to a valid and important problem in the field. The negative points/drawbacks come from that the results are not convincing and partially negative results which are very hard to impossible to support empirically. And the  approach needs to learn a mapping from the learnt state space to ground truth which is difficult to assess in quality itself and can invalidate the proposed measures if the mapping is "off". Thus, the initial meta reviewer would tend to borderline acceptance. 
Decision: Accept (Poster)