Neural Symplectic Form: Learning Hamiltonian Equations on General Coordinate Systems
Authors: Yuhan Chen
Takashi Matsubara
Takaharu Yaguchi
Venue: NeurIPS 2021 Spotlight (2021)
Keywords: Hamiltonian Neural Networks
Hamiltonian Mechanics
Symplectic Geometry

Abstract: In recent years, substantial research on the methods for learning Hamiltonian equations has been conducted. Although these approaches are very promising, the commonly used representation of the Hamilton equation uses the generalized momenta, which are generally unknown. Therefore, the training data must be represented in this unknown coordinate system, and this causes difficulty in applying the model to real data. Meanwhile, Hamiltonian equations also have a coordinate-free expression that is expressed by using the symplectic 2-form. In this study, we propose a model that learns the symplectic form from data using neural networks, thereby providing a method for learning Hamiltonian equations from data represented in general coordinate systems, which are not limited to the generalized coordinates and the generalized momenta. Consequently, the proposed method is capable not only of modeling target equations of both Hamiltonian and Lagrangian formalisms but also of extracting unknown Hamiltonian structures hidden in the data. For example, many polynomial ordinary differential equations such as the Lotka-Volterra equation are known to admit non-trivial Hamiltonian structures, and our numerical experiments show that such structures can be certainly learned from data. Technically, each symplectic 2-form is associated with a skew-symmetric matrix, but not all skew-symmetric matrices define the symplectic 2-form. In the proposed method, using the fact that symplectic 2-forms are derived as the exterior derivative of certain differential 1-forms, we model the differential 1-form by neural networks, thereby improving the efficiency of learning.

Neural Symplectic Form: Learning Hamiltonian Equations on General Coordinate Systems
Yuhan Chen
Takashi Matsubara
Kobe University Kobe, Japan 657-8501
Osaka University
Osaka, Japan aaakaauu.cc. ac
193x226x8stu. kobe-u.cce.o jc.jp jp
matsubara@sys.cm www..s...........aa...oaaa
Takaharu Yaguchi
Kobe University
Kobe, Japan 657-850
paguchi@pearl. kobe-u. ac.
Abstract
In recent years, substantial research on the methods for learning Hamiltonian equations has been conducted. Although these approaches are very promising, the commonly used representation of the Hamilton equation uses the generalized momenta, which are generally unknown. Therefore, the training data must be represented this unknown coordinate system, and this causes difficulty in ap- plying the model to real data. Meanwhile, Hamiltonian equations also have a coordinate-fres expression that is expressed by using the symplectic 2-form. this paper, propose model that learns the symplectic form from data using neural networks, thereby providing a method for learning Hamiltonian equations from data represented in general coordinate systems, which are not limited to the generalized coordinates and the generalized momenta. Consequently, the proposed method is capable not only of modeling the target equations of both Hamiltonian and Lagrangian formalisms but also of extracting unknown Hamiltonian structures hidden in the data. For example, many polynomial ordinary differential equations such as the Lotka-Volterra equation are known to admit non-trivial Hamiltonian structures, and our numerical experiments show that such structures certainly be learned from data. Technically, each symplectic 2-form is associated with a skew-symmetric matrix, but not skew-symmetric matrices define symplectic 2-form. In the proposed method, using the fact that symplectic 2-forms are derived the exterior derivative of certain differential I-forms, model the differential 1-form by neural networks, thereby improving the efficiency of learning.
Introduction
In recent years, the applications of deep learning to learn the fundamental equations of classical mechanics have been actively studied. Analytical mechanics, which is theory of classical mechanics.
classified into Lagrangian mechanics and Hamiltonian mechanics [ 3 18] The equations of motion of Lagrangian mechanics are called the Euler-Lagrange equation, for which several neural network models were proposed [7] In Lagrangian mechanics, the equations are described using the state variables and the time derivatives of them. This feature of Lagrangian mechanics makes it easy to prepare the data necessary for learning. On the other hand, Hamiltonian mechanics can describe more general equations which are not covered by Lagrangian mechanics. In the previous
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

neural symplectic form
gradient
observations general coordinate systems
dH
coordinate-free model for Hamiltonian systems
double pendulum
predator-prey (Lotka-Volterra) dynamics
exterior
dit = X
derivative
d0
shark
(xXg, v) = dil AV
fish
differential 1-form
etc.
Overview of the proposed method. Generally, the analytical representation of generalized momenta is unknown, so the data cannot presented in canonical coordinate system. The proposed method learns the Hamilton equation from data represented in arbitrary coordinate system by learning the symplectic 2-form as well as the energy function. In particular, to ensure that the learned symplectic 2-form is closed, our method Iearns the differential I-form that derives the symplectic 2-form. A universal approximation theorem is also provided.
Figure 1:
models for Hamiltonian equations, the equation
dt 1 -
)
Dq
(1)
typically assumed, where is the state variable, and p a variable called the generalized momentum. It is known that Hamiltonian equations can be written in this form by using these variables; more precisely, any Hamiltonian equation can be locally written as (T) by using a special coordinate system called the Darboux coordinate system [20]. However, this coordinate system depends on the generally unknown Hamiltonian (the energy function), and it is usually not possible to prepare data in this coordinate system.
On the other hand, Hamiltonian equations also have a coordinate-free representation using the symplectic 2-form 20 In this paper, we propose method to learn the symplectic 2-form from data by using neural networks, thereby introducing method to learn the Hamilton equation from data represented in general coordinate systems, not restricted to the generalized momentum. As explained below, general 2-forms are corresponding to skew symmetric matrices. Hence, a naive way to learn the symplectic 2-form is learning a skew symmetric matrix; however, actually, the equation learned in this way may not be Hamiltonian because not skew symmetric matrices are corresponding to symplectic forrms In the proposed method, instead of learning the 2-form directly, a 1-form that derives the symplectic 2-form is learned (see Figure
Because the proposed model can be applied to data in any coordinate system, it can be employed for extracting the unknown underlying Hamiltonian structure behind the data that are expected have such a structure. For example, many polynomial ordinary differential equations are known to have a Hamiltonian structure, but is not easy to find the structure analytically. The proposed method extracts the hidden structure in such cases from data. If the Hamiltonian structure can be found, it is possible to make predictions while preserving the energy conservation law. In addition to the energy, other hidden conserved quantities can be also extracted.
Main contributions of this paper include:
Symplectic geometric approach to learning symplectic 2-forms. The symplectic 2-form required describe the Hamilton equation corresponds to a skew-symmetric matrix, but conversely, not all kkee-rymmettic matrices correspond to symplectic 2-form. In this paper, we propose an efficient model by learning the symplectic 1-form that derives the symplectic 2-form with neural networks.
Learning the Hamilton equation from data in arbitrary coordinate systems. By using the coordinate-free representation of the Hamilton equation, possible to learn the Hamilton equation from data represented in general coordinate system, not restricted to the Darboux coordinate system. In this way, the proposed method can determine whether the given data can be explained by the hidden theory of classical mechanics or not. A universal approximation theorem is also provided.

---

Table 1: Comparison with other studies.
Skew Matrix Learning (Sec. 3)
Neural Symplectic
HNN
11
LNN
Form (proposed)
In the known Darboux coordinate In general coordinates on cotangent bundles On general symplectic manifolds
yes
yes
yes
yes
yes
yes
yes
yes
yes
Only symplectic forms
N/A
N/A
yes
2
Related Work
Neural Networks for Hamiltonian Mechanies Neural ordinary differential equation (NODE, [5) is neural network that models the time-derivative of the states, thereby defining an ordinary differential equation (ODE) in a general way. Due to the generality, this model does not admit the energy conservation law.
Hamiltonian neural network (HNN) is neural network that models the Hamiltonian H and defines the dynamics following the Hamiltonian mechanics, thereby ensuring the energy conservation law [. Although models of the form [I are often used in the previous studies, the Hamilton equation has this form only in the Darboux coordinates (20). The Darboux coordinate system, which is essentially the generalized momentum p. is defined using the Hamiltonian H, which is the target to be learned.
Hence, training data in the coordinate is usually unavailable. In addition, Hamiltonian equations are defined on general symplectic manifolds; however, in the existing studies, cotangent bundles are typically assumed as the symplectic manifold.
numerical integration of the Hamiltonian system known to destroy the symplectic structure and does not conserve the Hamiltonian H, unless the integrator is carefully designed (see, e.g.,
12 26)). 19 27 28
Several studies focus on the numerical integration that conserves the Hamiltonian H (6) 9
The HNN was extended to energy mergy-conserrvng partial differential equation (PDE) systems, such as the Korteweg- friction [19 29 In 15 discrete-time model is proposed for Poisson systems, which are extension of Hamiltonian systems, where the skew symmetric matrix can be degenerate. In particular, in 115, dynamics with state-dependent skew symmetric matrices are leaned by introducing coordinate transformations for learning the dynamics in the latent space. The proposed method is different from this study in that our method does not use the coordinate transformations and hence has advantage in interpretability. Another approach is employed in 16), where the symplectic map is modeled. SympNets [16] are also shown to be universal approximators, and, in [4], bound on the prediction error is provided. The proposed method can be combined with these discrete- time approaches.
Vries (KdV) equation [19], to dissipative systems, such as a pendulum with
Modeling using machine learning has also been performed in the field of quantum mechanics, for example, by Tkatchenko and coworkers (e.g. (22 25)). Some breakthroughs have been reported that have not been possible with conventional computational chemistry methods. The relationship with these studies needs to be investigated in the future.
Neural Networks for Lagrangian Mechanics Another branch of studies focus on Lagrangian mechanics. Lagrangian neural network (LNN) is neural network that models the Lagrangian L in general way 77, and deep Lagrangian network explicitly defines the kinetics energy with a trainable mass matrix 17 Lagrangian mechanics defines Lagrangian systems tangent bundles (18), where the state is the pair of the position q and velocity q. The systems have specific symplectic structures, which are equivalent to Hamiltonian systems in general coordinate systems cotangent bundles. Because the LNN does not assume equations of a specific form, it can learn a wider class of systems, including double pendulum, in addition to the systems that the HNN can learn (see Table J. Similarly to the HNN, numerical integrators that preserve the symplectic structure have been investigated 88] (23. Neural network architectures that ensure translational and rotational symmetries have also been investigated [1] 24].
As mentioned before, not all Hamiltonian systems are defined on cotangent bundles. For examples, the some polynomial equations including the Lotka-Volterra equation have different symplectic structure [13.. In fact, the Lotka- equation is actually Hamiltonian system, even though its
3

---

states are not position, velocity, nor generalized momentum. These equations are out of the scopes of the HNN and LNN (see T..
3 Methods
Coordinate-Fre Representation of Hamiltonian Equations In this paper, we propose neural network model based on coordinate-free representation of Hamiltonian equations. First, we describe this representation. Because detailed knowledge of geometry is required for precise description of this representation, the details are omitted here. For more details, see Appendix [A]
The model in this paper can be used on a general symplectic manifold, but for simplicity, we describe the case where the phase space is M = R2NN differential 2-form w on M skew-symmetric bilinear function that maps given two vectors into real number, depending on each point on M. The skew seemsyymeetric bilinear function defined by has the following matrix representation:
wi (1,, 22) 01 W,, for all U1 U2 E 1822,
where W., is a skew-symmetric matrix, and the subscript u denotes that w and hence its matrix representation W,, depend on u. A symplectic 2-form is differential 2-form that nondegenerate and closed. See Appendix A
for the precise meanings of these terms.
The following the coordinate-free form of Hamiltonian equations
[8]
du
di = XH, w(H, v) = dd( for all v E ERN.
(2)
Here, d/f is the Fréchet derivative of the Hamiltonian H, and XH is a vector field depending on H. This equation is satisfied regardless of the coordinate system in which the state variable is expressed. Therefore, by using this equation as a model, as long as the given data is described by the Hamilton equation, it is possible to learn both the symplectic 2-form and the Hamiltonian that define the Hamilton equation, no matter what coordinate system the data is given in.
Naive method: the skew matrix learning By replacing the symplectic 2-form with the matrix Wa. can be rewritten as
du dt
du WW VH.
XII, WTW.U= = H for all E 22
dt
natural model based on this representation would be a model in which W,, and H are modeled by multilayer perceptrons:
du W W di NNNN v (NN)(u),
(3)
where HNN a function of u defined by a multilayer perceptron, and W,,NN is skew-symmetric matrix depending on represented by another multilayer perceptron. We refer this model as the skew matrix learning. If u is represented the Darboux coordinate system, becomes (T) and hence the model is the same the Hamiltonian neural network.
Similarly Hamiltonian neural networks, this model has the energy conservation law. Theorem 1. Solutions
3 satisfy dHNN/ /tt 0.
See Appendix C for the proof.
Neural Symplectic Form Although the model in the previous section is natural, it is redundant model and inefficient for learning as explained below. As explained in Introduction, although a differential 2-form corresponds to skew-symmetric matrix, not all skew-symmetric matrices define a symplectic 2-form. Symplectic 2-forms have the characteristic feature of being closed. In this paper, propose a model in which the learned 2-form is guaranteed to be closed. We refer the learned 2-form as the neural symplectic form.
First, the necessary terminology is briefly explained. See Appendix A for details. A differential (-form on M 2N is function from M to R. A differential 1-form 0 on M = 2R2N is a field of linear functions each of which maps vector v E IR2N to (x) E R. depending each point

---

E M. In general, a linear function from R to R can be expressed as an inner product with vector, a differential 1-form can be expressed as a vector field depending on u. A differential operation called the exterior derivative d is defined for differential forms. The exterior derivative is graded linear map, i.e. linear map depending on an integer which transfers a differential k-form to differential (k + 1) ) and has the property that dd 0.
The differential form in Ker d is called closed form. Since a symplectic 2-form is a closed form, in order to learn the symplectic 2-form by neural networks, neural networks should be designed so that differential 2-forms represented by the neural networks are contained in Ker d. Meanwhile, due to the property of the exterior derivative, dd 0, it holds that Im C Ker d.
skew matrix learning
space of 1-fors
space 2-forms
((N)) degrees of freadom
space of 3-forms
exterior derivative
eaterior derivative
Im II Ker
neural symplectic form
Actually, according to the de Rham theorem, when the phase space is 22 these two spaces coincide: Im d Ker d. The difference be- tween these two spaces Im d/Ker d is called the cohomology space. The de Rham theorem states that the cohomology space is isomorphic to the homology space, which is roughly space of "spatial holes." Because 2N contains holes, the homology space must vanish, and hence Im = Ker d holds (see Figure 2 Even when the phase space has a hole, in many cases the space holes, and the model should be defined on that space. See Appendix B
O(N) degrees of freedom
Ker (under certain geometric condition)
Figure 2: The de Rham theorem ensures the Ker d Im d.
be embedded in large Euclid space without
Therefore, in this paper, instead of learning the symplectic 2-form directly, we propose a method to learn the symplectic 2-form by learning the differential 1-form of which exterior derivative gives the symplectic 2-form. The following is the mordinntee-free form of the proposed model: du e d0NN, dt
XHNNs WXH v) cHNN(() for all RRN
(4)
Unlike (3). in this model skew-symmn swwmmmmmttric matrices that do not correspond to symplectic forms are not explored. Therefore, this model can be trained much more efficiently than (3). In fact, suppose that the dimension of the phase space is 2N. Then, the number of components of the skew-symmetric matrix is N22N 1). while the differential 1-form is represented as vector field, the number of which components is 2N. Consequently, a model using the neural symplectic form can reduce the order of the number of functions to be learned from O(N²) to O(N) without sacrificing the expressive power. As per the above. the differential 1-form can be expressed as vector field. Hence the neural network for modeling the 1-form in the proposed model essentially models a vector field YNN, which represents the differential 1-form 0Nx. As shown in Appendix[A] the vector field YNN is transformed to the matrix Wr, representing the symplectic 2-form follows:
(Wa).. 0(YNN), a(Ynn),
Ou,
Ou;
In the actual model, YNN given by the neural network is differentiated by the automatic differentiation, and W. NN is obtained by substituting the derivatives into the above equation. Thus we have the model expressed in terms vectors and a matrix, without using the differential forms:
du = W (H)(
dt (W.)ss = 0(YNN)s a(YNN)s
(5)
Ouj Ou:
The proposed model has the energy conservation law and the universal approximation property. For the proofs, see Appendix C
Theorem 2. Solutions to 4.. equivalently, to Satisff dH /dt = 0.
Theorem 3. Suppose that the Hamilton equation to be learned can be represented in the form
using symplectic 2-form derived from a 1-form O E W and the Hamiltonian H. W
5

---

Suppose also that the phase space is compact. The model 4 with multilayer perceptrons with sufficiently smooth activation function has the universal approximation property in the sense that and H can be upproximated by the multilayer perceptrons with arbitrary accuracy.
W.PPP and Wad are Sobolev spaces; see
14
Learning dynamics by using the neural symplectic form The proposed model learns the sym- plectic form and the Hamiltonian by minimizing the squared error between the left- and right-hand sides of (4). assuming that time series data of state vectors {u(m)} and its time-derivatives { dt
du (n)
given;
Σ du (n)
( (a. o(YiN)) (u(m) o(YNN), (u(1)). du,
minimize
WT a(s))
dt
Ou:
the time-derivatives are not available, interpolated data should be used. For each state vector u("), first YNN(wi (u(=)) computed. Then, the derivatives of (Nv(u(*)) are calculated by the automatic differentiation to obtain (W,T)., Meanwhile, the gradient of the Hamiltonian is computed in the same way; first HNN (fx) is computed, and then V HNNN ((x) is obtained by the automatic differentiation.
In the standard Hamiltonian neural networks only the computation the gradient of the Hamiltonian is required. Hence the computational cost of the neural symplectic form is roughly 2.V times the computational cost of Hamiltonian neural networks, where 2N is the number of the state variables, because the number of components of (N)( 2N. When N large, such as many-body problems, the most computationally expensive part may be computation of the inverse of the matrix W Wl(n), which requires ((N3) operations. To avoid the computation of the inverse, one may be tempted to minimize
Σ WTT du (n) (n) di - HNNN(u (m))
However, this model does not work because this is trivially minimized by setting WT,,, O and HNN (constant.) Learning the inverse while maintaining the symplectic structure is future work.
4 Numerical Examples
Outline of experiments The proposed method with the neural symplectic form can model general Hamiltonian equations on general symplectic manifolds using data represented by general coordinate systems. We illustrate these advantages in the following numerical experiments with comparative methods. The details of the data in the numerical experiments along with additional results are provided in Appendix D
We performed the experiments using HNN, L
skew matrix learning and neural symplectic form. The energy function, the skew-symmetric matrix, and the 1-form for the neural symplectic 2-form were modeled by using a neural network that had two hidden layers of 200 units and the tanh activation function. We used 80 percent of collected data for training and the remaining for test. We trained each model 10 times using the Adam optimizer with a learning rate of 10 for 2000 iterations. Only for LNN, we the learning to 10 due an instability of learning; in our experiments the loss function of LNN sometimes did not converge monotonically, but oscillated when the learning rate was to 10-3 Hence we truncated the training process at the iteration where the loss function achieved the best score. This oscillation should be due to the non-uniqueness of Lagrangian; see Appendix Doo details.
The experiments were performed on NVIDIA A100 with double precision. After training, we evaluated the models using the squared time-derivative errors of the test subset. Using SeiPy odeint under the default setting, we predicted 10 orbits from random initial values and obtained the errors in the system energy.
We used the implementattonn of the LNN model in torchdyn [1] https://githuu http:://giiuuub..oo/D//ffffffl
boc.aha///.boaaarr/////////aaaae ipyy
ipynb (Apache 2.0 License)

---

Mass-Spring System general coordinate on a cotangent bundle, namely, a simple mass- spring system, depicted in Figure 3 The equation of motion of this system
First, we investigate a Hamiltonian system in
k1 Ivow
kz
4
m4 l
m2
d
92
U2
Figure 3: The mass-spring
dt
11
mi
(1)
11
my
92 91
12)
system.
1/2
(42
41 - 12)
which is Hamiltonian system with the energy function
pi
P lg - 4122 k2 Q1
-2)2
(qq,, Q2, P2)
2m
2mz
2
where 91 22 are the positions of the mass points, and P1, P2 the momenta, which are defined by Pi = 01, P2 = m2 81 = dqp/dtt 1/2 = dq2/dt Suppose that the exact values of my and m2 unknown, and only the positions qi and (2 and their derivatives are given. Although my and m2 may be estimated from the given states, for evaluation of the models, we tried to learn the dynamics from the given states directly.
The time-derivative errors and the energy errors are shown in Tables 2 and respectively. As explained in Appendix D if the state variables are represented by the positions and the velocities, the equation of motion this system cannot be written the standard form. Nevertheless, as this is a simple problem, all the models work well. Most of the predicted orbits shown in Figure 4 have good fit to the true one. Note that the errors shown in Table 2 are small enough. They are multiplied by 10 and shown in the physical scale; i.e., the errors variables but by using the state variables with the true scale. In the normalized scale, the error by LNN, for example, was 0.0095. This is also the case in the other experiments. Among the methods, NODE and the neural symplectic form performed better. The test time-derivative error of the skew matrix learning is a little larger. In terms of the energy error, the neural symplectic form and HNN are superior; however there is not much difference.
not measured by using the normalized state
Double Pendulum Next, consider another Hamiltonian system, namely, double pendulum shown in Figure 5 of which equation of motion is
d0
d02
Φ1.
d2:
dt
dt
dø ssin Og sin
02)
mi+ma sin 01) (110f (01 02) 1203) sin(O; 02)
m2
dt
1
[m) +m2 cos2 (0, 02))
m2
døz
g(mi +ma)
(sin 0 cos(O)
02)
sin 02) 11 (m +maa B? 1203 cos(O) -22)) sin(0, 02)
ma
1/2
dt
la
m2
cos 02))
m2
For the energy function and the Lagrangian of this system, see Appendix D The generalized momenta of this system are
P1 = (m1 + m2)(f + m2l12200 cos(01 - 02), P2 = m22202 + mzlil201 cos(A (f1 - 02).
Because the generalized momenta are not obvious, we assume that the data of 01, 02, Φ1, d2 are given instead of 01, 02 (h,P1, P1,P2, where Φ1, dh are the time derivatives of 0,, 02-
The test dime-derivative errors are shown in Table 2 Firstly, (6) cannot be written as the standard form (I) with certain Hamiltonian. Hence, the test time-derivative error of HNN cannot be completely zero when learned using states 01 (t), Φn (t), (t),), (2(). The result shown in Table 2 confirms this. The test derivaaive errors by NODE, the skew matrix learning and the neural symplectic form are much smaller than that by HÑN. In particular, the error by the skew matrix learning is larger than that by the neural symplectic form. This is due the non eoi-exise of the one-to-one correspondence between the skew matrix and the symplectic 2-form, which decreases the efficiency of learning. NODE performed very well too; however, as explained below, this model failed to predict long-term behaviors.
7

---

Table 2: Test time-derivative errors.
Neural
Skew Matrix Learning
NODE
HNN
LNN
Symplectic Form (proposed)
mass-spring
0.17 0.14
694.77 + 26.37
882.52 + 1753.93
102.07 + 68.00
0.52 0.71
double pendulum
8.65 = 1.38
15.69 + 0.44
269.52 + 136.85
8.47 + 1.22
4.02 : 2.08
2.02 5.55
227.03 + 2.31
N/A
1.05 + 0.76
0.46 + 0.30
The second best results are emphasized by bold and underlined fonts, respectively. Multiplied by 10 for the mass-spring system and the Lotka Volterra equation. The experiments were conducted 10 times cach. The results of LNN for double pendulum were computed using nine times except for failed onc.
Table 3: Energy errors.
Neural Symplectic Form (proposed)
Skew Matrix Learning
NODE
HNN
LNN
mass-spring
0.840 + 0.328
0.551 + 0.112
2.281 + 0.004
6.203 = 7.555
0.368 + 0.055
double pendulum (T=5)
17.740 +
1.070 0.694
(0.755 0.320)
3.931 7.266
6.400 + 0.971
10.804
double pendulum (T=30)
622982.133
11.240 + 12.297
N/A
N/A
7,300 + 3,925
+ 1814794.079
Lotka-Volterra
0.578 + 0.558
0.444 1 0.458
N/A
0.041 = 0.072
0.012 + 0.013
The best and second best results of the true energies are emphasized by bold and underlined Tonts, respectively. The differences of the true energy functions at t = T and t = O. where T = 30 for the Lotka-Volterra for the others. 10 orbits were simulated using randomly generated initial values. In the double pendulum test, the energy error of the HNN was best but this because the very small amplitudes of the predicted solutions; hence the result of NODE cosidered best.
equation and
(a) Ground truth
(b) NODE
(c) HNN
(d) LNN
(e) Skew Matrix (f) Neural Sym- Learning
plectic Form Figure 4: Example of the orbits predicted by the trained models for the mass-spring test. The horizontal axis represents time. Each component of u(t) (g1(t), on (2 v2 (t)) is represented: blue (q1), green (21), orange (42), and red (U2).
Examples of the predicted orbits are shown in Figure As expected, HNN failed to predict the orbits. In the result by the skew matrix learning, although the speed of oscillation appears be correct, the heights of the peaks are different from the true trajectory. Mcan- while, the prediction by the neural symplectic form achieved better agreement with the true one than other models.
4
my
lz
02
m2
The energy errors shown in Table 3 The emror of NODE and the skew matrix learning are small for the short term prediction. However, Figure 7 shows example of the long-term prediction results of these models and the proposed neural symplectic form. The results of the NODES obtained using the models trained with different seeds. Since NODE does not have a Hamiltonian
Figure 5: A double pendulum and the related constants and the variables.
structure, the results gradually decay or diverge. The prediction results by the skew matrix learning also show a gradual increase in the state variables, and the results collapse after exceeding a certain value. On the other hand, the result by the neural symplectic form oscillates stably, although the range of oscillation is little larger than the true orbit.
Volka-Volterra equation Systems differential equations with polynomial right-hand side are often Hamiltonian with hidden symplectic structure. For example, the Hamiltonian structure of the

---

boyeok
(f) Neural Sym- plectic Form Figure 6: Example of the orbits predicted by the trained models for the double pendulum test. The horizontal axis represents time. Each component of u(l) (01 (t), dh (1), (2)), tbe (L)) is represented: blue ) orange (+1), green (02), and red (1b2).
(a) Ground truth
(b) NODE
(c) HNNN
(d) LNN
(e) Skew Matrix
Learning
(a) Ground truth
(b) NODEI
(c) NODE2
(d) Skew Matrix (e) Neural Sym-
Learning
plectic Form
Figure 7: Example of the orbits during long period (t 30) predicted by the trained models for the double pendulum test. NODEI and NODE2 are results of two NODE models trained with a different random seed. The horizontal axis represents time. Each component of u(t) (01 (t), o1 (t), (t), (2))) is represented: blue (01), orange (01), green (02), and red (d12).
generalized Lotka-Voltere Volterra equation
dæ
xg 04j IT
1= + X1),
by
dt
is investigated in Hernández- Berrmeeee--ereejoo and Fairén [13]. The right-hand side of this equation is quite general because this equation has the form of z, (polynomial of the other state variables). For example, many mathematical compartment models in biology are of this form. In fact, the original Lotka-Voltere model proposed model of predator- prey dynamies.
We used a standard Lotka Volterra model, which included in Zp. the following form:
dx di
dx
= 1 + λ
421 + A2æ2.
dt
Provided that 11 * 0 and ø2 * this equation can be written Hamiltonian equation:
am Uz1
x1 x22
2
(2), -021 #1 A2 2n + a1222 + A In
di
22
87g
which is different from the standard Hamiltonian equation I In fact, this equation cannot be written as (T) globally; the Darboux coordinate system only locally exists in general. See Appendix D
We tested the models other than LNN, which is not applicable because this equation cannot be expressed naturally as a second-order differential equation. We again used simulated solutions for data with 1000 randomly generated initial conditions. Because the Lotka-Volterr model typically describes the population dynamics of certain species, we choose the initial conditions from the uniform distribution on [0, IJ. For other details, see Appendix D
The test time-derivative errors shown in Table 2 the other models. As seen from the table, the proposed neural symplectic form stably gave better results than the other models.
Again, the error by HNN larger than those by
Regarding the energy errors, the energies are well preserved as shown Table 3 In particular, the proposed method preserves the energy with the highest accuracy. Besides, predicted orbits shown Figure 8 The peaks of the orbits by HNN and the skew matrix learning are smaller than the true trajectory, and the orbit of the NODE is gradually decaying. On the other hand, the proposed neural symplectic form gives the almost identical orbits to the true one.
Thus the hidden symplectic structure of this equation is certainly extracted by the proposed method.

---

Ground truth
(b) NODE
HNN
(d) Skew
Matrix Neural Symplec- Form
Learning
Figure 8: Example of the orbits predicted by the proposed and the comparative models for the model. The horizontal axis represents time. 21(t), x2()) are represented: blue (x1) and orange (122).
Ground truth
HNN
Neural Symplectic Form
Figure 9: Example of the predicted images by the proposed method and HNN.
Learning from Images As an application of the proposed method, we learned the equation of motion from some images. One way to achieve this is to extract features from the images by using an autoencoder and learn the equation of motion that the features satisfy [IT] In this case, the extracted features are not supposed to be momenta. The proposed method is suitable for this application because is independent of the coordinate system.
In this experiment, we first pretrained an autoencoder. Then HNN and the neural symplectic form are applied to learn the dynamics of the features. The final test losses of HNN and the neural symplectic form were 0.161 and 0.060 respectively. The predicted pictures are shown in Figure 9 The last four pictures of HNN are noisy; this implies relatively large errors in the latent space. However, this is only a preliminary test; for example, the performance may depend on the architecture of the autoencoder. Further thorough investigation is needed for this application.
Concluding Remarks
In this paper, we proposed a method for learning Hamilton equations from data represented on general coordinate systems, which are not restricted to generalized momenta. The key ingredient is the neural symplectic form; we proposed to learn the symplectic 2-form by using neural networks from data, thereby lcarning this coordinate-fin ffre representation. In particular, the Hamilton equation can be represented using a state-dependent related the symplectic 2-form. In the proposed method, in order to learn the symplectic 2-form efficiently, the I-form that derives the symplectic 2-form is learned by the neural networks.
-ymmmerte matrix, but not all dew-syymettric matrices are
Meanwhile, the proposed method requires the inverse of the skew-symmetrie matrix, which may be computationally expensive for modeling large systems. To address this problem, perturbation theory of the inverse matrix may be applied. For example, it is known that for matrix M if the norm of A. small enough, (M + AM) M-1 + M-1AMM-1 holds. This should be investigated to reduce the computational costs in future work.
10

---

Acknowledgments and Disclosure of Funding
Funding in direct support of this work: JST CREST Grant Number JPMJCR1914, JST PRESTO Grant Number JPMJPR2IC7 and JSPS KAKENHI Grant Number 20K11693. The SQUID supercomputer in Osaka University was used for the experiments.
References
[1] R. Abraham and E. Marsden. Foundations of Mechanics. American Mathematical Soc., 2008.
[2] R. A. Adams and J. Fournier. Sobolev Spuces. Elsevier, June 2003.
[3] V. I. Arnol'd, Mathematical Methods of Classical Mechanics. Springer Science & Business Media, Apr.
2013.
[4] R. Chen and M. Tao. Data-driven prediction general hamiltomian dynamics via learning exactly- symplectic maps. Proceedings of the 38 th International Conference on Machine Learning, PMLR 139, Mar. 2021.
[5] T. Q. Chen, Rubanova, Bettencourt, D. Duvenaud, T. Q. Chen, Rubanova, J. Bettencourt, and D. Duvenaud. Neural Ordinary Differential Equations. In Advances in Neural Information Processing Systems (NeurIPS), 2018.
[6] Z. Chen, Zhang. M. Arjovsky, and I.. Bottou. Symplectic Recurrent Neural Networks. In International Conference on Learning Representations (ICLR), 2020.
M. Cranmer, S. Greydanus, S. Hoyer, Battaglia, D. Spergel, and Ho. Lagrangian Neural Networks.
ICLR 2020 Deep Differential Equations Workshop, 2020.
[8] S. Desai and S. Roberts. VIGN: Variational Integrator Graph Networks. arXiv, 2020.
[9] D. M. DiPietro, Xiong, and B. Zhu. Sparse Symplectically Integrated Neural Networks. In Advances in Neural Information Processing Systems (NeurIPS), 2020.
[10] Finzi, K. A. Wang, and A. Wilson. Simplifying Hamiltonian Lagrangian Neural Networks via Explicit Constraints. In Advances in Neural Information Processing Systems (NeurIPS), 2020.
[11] S. Greydanus, M. Dzamba, and J. Yosinski. Hamiltonian Neural Networks. In Advances in Neural Information Processing Systems (NeurIPS), 2019.
[12] E. Hairer, C. Lubich, and G. Wanner. Geometric numerical integration: Structure preeeturrp algoritims for ordinary differential equations. Springer Series in Computational Mathematics. Springer, Berlin, Germany, 2002 edition, Mar. 2013.
[13] B. Hernández-Bermet) and V. Fairén. Haniltonian structure and darboux theorem for families of general- ized
systems. J. Math. Phys., 39(1 1):6162 6174, Nov. 1998.
[14] T. Iwaniec, John Raymond French Distinguished Professor of Mathematics Tadeusz Iwaniec, G. Martin, and of Mathematics Gaven Martin. Geometric Function Theory and Non-linear Analysis. Oxford University Press, 2001.
[15] P. Jin, Z. Zhang. 1. G. Kevrekidis, and G. E. Karniadakis. Learning Poisson systems and trajectories of autonomous systems via Poisson neural networks. arXiv, 2020.
[16] P. Jin, Zhang, Zhu, Y. Tang, and G. E. Karniadakis. SympNets: Intrinsic structure- preeuurriggeeeiiiggggg
symplectic networks for identifying Hamiltonian systems. Neural Networks, 132:166-179, 179, 2020.
[17] M. Lutter, C. Ritter, and Peters. Deep Lagrangian networks: Using physics model prior for deep learning. International Conference on Learning Representations (ICLR), pages 1-17, 2019.
[18] J. E. Marsden and T. S. Ratiu. Introduction Mechanics and Synmetry: A Basic Exposition of Classical Mechanicul Systems. Springer Science Business Media, Mar. 2013.
[19] T. Matsubara, A. Ishikawa, and T. Yaguchi. Deep Energy-Based Modeling of Discrete-Time Physics. In Advances in Neural Information Processing Systems (NeurlPS). 2020.
[20] D. McDuff and D. Salamon. Introduction to Symplectic Topology. Oxford University Press, Mar. 2017.
[21] M. Poli, S. Massaroli, A. Yamashita, H. Asama, and J. Park. Torchdyn: A neural differential equations library. arXiv preprint arXiv:2009. 09340, 2020.
[22] M. Rupp, A. Tkatchenko, K.-R. Müller, and O. A. von Lilienfeld. Fast and accurate modeling of molecular atomization energies with machine learning. Phys. Rev. Lett., 108(5)008301, Feb. 2012.
[23] S. Sæeuundsoon K. Hofmann, A. Terenin, and M. P. Deisenroth. Variational integrator networks for physically meaningful embeddings. In Artificial Intelligence and Statistics (AISTATS), volume 108, pages 3078 3087, 2019.

---

[24] A. Sanzhez-Gonzalez, V. Bapst. K. Cranmer. and P. Battaglia. Hamiltonian Graph Networks with ODE Integrators. arXiv, 2019.
[25] K. T. Schütt, H. E. Sauceda, P.J. Kindermans, A. Tkatchenko, and K.-.. Müller. SchNet deep learning architecture for molecules and materials. J. Chem. Phys., 148(24):241722, June 2018.
[26] Tao. Explicit symplectic approximation of nonseparable hamiltonians: Algorithm and long time performance. Phys Rev E, 94(4-1):043303, Oct. 2016.
[27] S. Xiong, Y. Tong, X. He, C. Yang. S. Yang, and B. Zhu. Nonseparable Symplectic Neural Networks. In International Conference on Learning Representations (ICLR), 2021.
[28] Y. D. Zhong, B. Dey, and A. Chakraborty. Symplectic ODE-Net: Learning Hamiltonian Dynamics with Control. In International Conference on Learning Representations (ICLR), 2020.
[29] Y. D. Zhong, B. Dey, and A. Chakraborty. Dissipative SymODEN: Encoding Hamiltonian Dynamics with Dissipation and Control into Deep Learning. In ICLR 2020 Workshop on Integration of Deep Neural Models and Differential Equations (DeepDiffEg), 2020.
12


Reviews:
[{"review_id": "sJ188Ps5otE", "reviewer": "Reviewer_qdun", "summary": "This paper proposes to learn conservative physical dynamics from data by directly learning a differential 1-form. The dynamics is subsequently determined by computing the exterior derivative of the learned 1-form (yielding a symplectic 2-form) and providing a coordinate-free representation of the dynamics. In addition to inherently preserving the symplectic nature of the dynamics, this procedure does not require *a priori* knowledge of the generalized momentum coordinates, i.e. only velocities need be observed so masses of the degrees of freedom in the data are not needed. The neural symplectic form methodology is compared to an approach in which the symplectic 2-form is learned directly as a skew-symmetric matrix as well the existing Hamiltonian neural network and Lagrangian neural network approaches. ", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Learning the underlying Hamiltonian structure of observed trajectorial data requires methodologies that are sensitive to the conservation laws inherent in the physical structure of Hamilton's equations of motion. Without a doubt, conservation of energy is the most important property to preserve. This can be achieved by using a symplectic integration and learning with a symplectic integration scheme noticeably improves the quality of learning (cf. Chen et al. https://arxiv.org/abs/1909.13334 which should surely be cited). Indeed, a direct approach of this type requires rather complete information about the system, including knowledge of all generalized positions and momenta along a time series. This paper aims to relax this requirement by parameterizing a coordinate-free representation of the equations of motion, an elegant idea with significant potential impact.\n\nThe formulation of the algorithm relies on (1) representing the Hamiltonian $H$ as a neural network and (2) either representing the symplectic 2-form $W_u$ as a neural network or representing a 1-form as a neural network and computing its exterior derivative to compute $W_u$. These formulations constitute the core contribution of the paper. In both cases, the authors establish formally that the dynamics conserves energy (a comment in the proof of Theorem 1 could be added to indicate that the argument also applies to Theorem 2). The proof of the universal approximation theorem is rather perfunctory, but I do think it constitutes a reasonable addition to this work. The arguments for all three theorems are essentially standard results.\n\nThe motivation for preferring the neural symplectic form approach is motivated, in part, by invoking de Rham cohomology. I found this to be a bit of a distraction, likely to lead to confusion. The articulation beginning at line 153 struck me as a much clearer and conceptually better explanation of why one should prefer learning the 1-form. Of course, if this could be clearly tied to the de Rham cohomology, it could be worthwhile to discuss this. Fig. 2 uses a lot of space and did not add meaningfully to the manuscript in my opinion. I appreciated the discussion of the computational cost of the inverse of $W_u$---this is a significant limitation of the algorithm for large systems. \n\nI found the numerical experiments useful, concise, and sufficient to establish the efficacy of the method. \n\nIs learning $W_u$ directly as a 2-form distinct from learning a mass matrix in the case where the kinetic energy is assumed to be harmonic?\n\nIt would be useful for the authors to make some contact with the literature on learning potential energy functions (e.g., the work Tkatchenko and coworkers) because in that line of inquiry it has been concluded that learning with forces (learning the gradients of the Hamiltonian) is often superior to learning the potential energy. This strikes me as being analogous to learning the 1-form vs learning the 2-form and I'm not sure how to reconcile the two conclusions. ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "There are a few inherent limitations when it comes to scaling this problem up to large, many-body systems. First, it is hard to imagine a realistic setting in which one could observe all the positions of a complex system with sufficient spatio-temporal resolution. Secondly, most many-body systems will decorrelate over short times due to ergodicity and the onset of chaos. In my mind, this makes the problem a rather academic one without significant applications in physics per se. That said, discovering Hamiltonian structure in ODEs (as in the Lotka-Volterra example) provides a nice counterpoint to this criticism. ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2.5"}, "extra_text": {}}, {"review_id": "k2z-sPmuJtv", "reviewer": "Reviewer_MBYz", "summary": "This work addresses the problem of learning Hamiltonian systems from data without assuming that the generalized momentum is given. This assumption limits the applicability of standard Hamiltonian Neural Networks (HNNs) because the generalized momentum is defined in terms of the Hamiltonian itself, which means it cannot typically be provided as training data for unknown Hamiltonians. This work proposes to get around this limitation by exploiting a coordinate-free formulation of the Hamiltonian and incorporating the 2-form from symplectic geometry into the structure of a HNN (either directly as a learned skew-symmetric matrix or via a learned 1-form and an exterior derivative). The resulting architecture performs favorably compared to a standard HNN and Lagrangian Neural Network (LNN) on several simple physical systems.", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Positives:\n- By eliminating the requirement for labeled generalized momenta at input, the core technical contribution of the paper overcomes a key limitation of HNNs and does so in a simple and elegant way. \n- The paper does an admirable job exposing the relevant literature in a way that I feel will be largely interpretable by the NeurIPS audience, despite the fact that it draws heavily from symplectic geometry. In particular, I found the discussion of 1- and 2-forms and the exterior derivative very readable.\n\nNegatives:\n- My largest concern with this paper lies in the experimental sections. Experiments were conducted with very small (single hidden-layer) networks, which are smaller than typically used in this literature. This appears to be leading to artificially bad performance by HNN and LNN baselines, as discussed in L269-272 and section D. While it's suggestive that the proposed method works well with this network, it makes the results very hard to interpret in the context of the literature. This choice isn't motivated as far as I could tell. For the method to be interpretable, it should be run in settings where HNNs and LNNs are known to perform well.\n- The model state prediction should be analyzed quantitatively: how well does the model predict the state when integrated over time? It's hard to interpret the quality of the estimates without knowing how it fares over longer horizons than a single step, which is essentially what's quantified in tables 2 and 3.\n- How were the energy errors in Table 3 estimated? As I understand it, these models can only recover the energy up to a constant offset.\n- I believe the paper could be much more impactful if more difficult experimental settings where the proposed method can shine were explored. In particular, the proposed formulation appears to be particularly amenable to image- or other observation-based depictions of Hamiltonian systems, because the method does not depend on a representation being recovered in the Darboux coordinate system and does not require momentum to be estimated. Image-based Hamiltonian learning has been addressed in several prior works in the literature e.g. [1, 2, 3], and it would be very useful to know how the proposed method fares in those settings and how the present analysis sheds light on those experiments.\n- Several of the figures could be improved:\n  - Figure 1 is cluttered and hard to follow: this could probably be improved by moving many of the graphical elements to the text description, which should be expanded. I also found it hard to read the left-hand side of the figure, as the double pendulum and Lotka-Volterra systems look as if they're both being input to the system (rather than showing two example inputs).\n  - Figure 2 is very hard to follow and is essentially uninterpretable without referring to the main text.\n  - Figures 5, 6, and 7 are missing axis labels and do not refer to what system they evaluate.\n\nMinor:\n- L92: extra \"the\" at the beginning of the sentence\n- L115: (4) should be (2).\n- I found it a little hard to understand the structure Y_{NN} on a first read. I believe this would be easier to understand if its input and output dimensions were mentioned.\n- L190: \"If the time derivatives are not available, interpolated data should be used.\" Please rephrase this - I believe the implication is that finite-difference estimates of the temporal derivatives should be used.\n- L204: Nitpick: I believe a more idiomatic way to refer to this section in a machine learning paper would be \"Experiments\" or \"Numerical Experiments\".\n\nOverall: I believe this work makes a solid contribution to the literature on learning Hamiltonians and the weaknesses are addressable, so I weakly recommend acceptance. However, I think the paper would be much stronger if the experimental section were improved.\n\nI am willing to raise my score if the experimental limitations pointed above are addressed, and ideally if the authors can present results on more challenging datasets.\n\n[1] Greydanus et al - NeurIPS 2019 - Hamiltonian Neural Networks\n[2] Toth et al - ICLR 2020 - Hamiltonian Generative Networks\n[3] Allen-Blanchette et al - AAAI 2020 - LagNetViP: A Lagrangian Neural Network for Video Prediction", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes.", "review_text": "", "extra_scores": {"ethical_concerns": "None.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "Ym71zeraQzS", "reviewer": "Reviewer_XXhP", "summary": "This paper considers the learning of non-canonical Hamiltonian dynamics from time-series data. Canonical Hamiltonian systems in Euclidean spaces correspond to $\\dot{x}=J \\nabla H(x)$ where $J=[0,  I; -I, 0]$ is a constant matrix corresponding to the canonical symplectic form. Non-canonical Hamiltonian systems are in the same form, however $J$ can be $x$ dependent, and it has to be a skew-symmetric matrix satisfying Leibniz’s rule and Jacobi identity. This paper studies the Euclidean case, which has nice enough topology so that $J:\\mathbb{R}^{d}\\to \\mathbb{R}^{d\\times d}$ satisfying these conditions can be expressed using a vector field $Y: \\mathbb{R}^{d}\\to \\mathbb{R}^d$ instead, which is no longer constrained. The proposed method represents $H$ and $Y$ by neural networks and trains them to match the data. Some numerical investigations are provided.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The method is new and interesting. Although it needs a nice cohomology like that of the Euclidean space, the setup corresponds to sufficiently many important problems. The paper is well written and the informal descriptions are very helpful to general readers.\n\nWhile I like the continuous (in time) part of the paper, I hope the authors could agree that discretization matters very much when the method is actually implemented. This is something that the current version can improve upon. For example, the formulation of the method is summarized inbetween Line 189 and 190, but how is $du/dt$ estimated? After the continuous dynamics is learned, how is prediction done? In fact, these questions are central to many of the recent advances in this area, some of them already cited, such as [20, 21], but two important references are missed. Namely, [Jin et al. SympNets: Intrinsic structure-preserving symplectic networks for identifying Hamiltonian systems. Neural Networks 2020] and [Chen & Tao. Data-driven Prediction of General Hamiltonian Dynamics via Learning Exactly-Symplectic Maps. ICML 2021]. These aspects should be discussed for the sake of the readers.\n\nIn addition, [12] was cited but not sufficiently discussed. However, it is closely related work and their similarity and difference should be clarified.\n\nContinuing along the line of discretization, it was written that “A numerical integration of the Hamiltonian system is known to destroy the symplectic structure and does not conserve the Hamiltonian H.” This is incorrect. Symplectic integrators exist; see for instance [Hairer Lubich Wanner. Geometric Numerical Integration. 2006] for separable Hamiltonians (and much more) and [Tao. Explicit symplectic approximation of nonseparable Hamiltonians. 2016] for the rest of the picture. Regarding conserving the Hamiltonian, though, one cannot have simultaneous exact conservation of symplecticity and energy in constant step integrations, as the Ge-Marsden theorem gives.\n\nIn addition, I would like to understand two things:\n\nFirstly, the proposed method is better than skew matrix learning and much better than HNN, according to the mass spring experiment and the double pendulum experiment, but why? For these two problems, the coordinates are already canonical, so I was expecting the new method to degenerate to HNN and give comparable performance, but it is not the case. What are the learned symplectic 2-forms in these two cases? If it is because of different network architecture and/or training details, the comparison needs to be made fair.\n\nSecondly, how is the invertibility of $\\tilde{W}$ ensured?\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "N/A", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "OGD-f1oVaVN", "reviewer": "Reviewer_yYL7", "summary": "Authors present a ML-based approach to learning dynamics in the space of Hamiltonian systems. The approach is based on generic formulation of Hamiltonian mechanics in the language of differential geometry, effectively avoiding the requirement of knowing the appropriate generalized coordinates beforehand. The proposed approach is based on modeling the symplectic 2-form (known to be closed) as an exterior derivative of a parametrized (learned) 1-form. This method is termed \"neural symplectic form\" and provides a compact parametrization of the underlying governing equations.\n\nThe proposed model is trained by minimizing the MSE between estimated ground truth time derivatives of the dynamical system and predictions of the model on several datasets: mass-spring system; double pendulum; Lotka-Volterra equations (which are not captured by a traditional formulation of Hamiltonian mechanics with fixed symplectic form).\n\nThe model is compared to baselines (HNN, LNN and a naive implementation termed skew matrix learning) on test loss and errors of the conserved quantity in the predicted dynamics accumulated during simulation for a fixed time.\n\nAuthors find that the proposed model produces best results across most of the comparisons, especially on the Lotka-Volterra test case.\n\n", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The article present an original approach to modeling Hamiltonian systems that uses fewer variational parameters while accurately capturing the target space of dynamical laws. When combined with symplectic integrators, such systems admit conservation laws that are highly desirable for long-term modeling. In my opinion this is a significant direction worthy further exploration.\n\nThe work is written clearly, but is definitely more accessible to readers with elementary background in exterior calculus. One of the organization caveats that in my opinion should be addressed is the silent push of the NODE model to the appendix, which often perform on-par with the proposed method.\n\nThe results section raises a few questions:\n1) Some of the error-bars are comparable/larger than the mean values? Is it possible to include more experiments to make comparisons precise?\n2) While summarized errors are useful it would be valueable to include the time dependence of the errors as a function of time (for both energy and MAE/MSE)\n3) The networks used for modeling the system were MLP with a single hidden layer - it would be very reassuring to see that the results are not too sensitive to this choice, especially in a more expressive regime.\n\nOverall - interesting paper worth publishing assuming that the results section can be tuned up.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The work does not have negative societal impact.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5"}, "extra_text": {}}]

Author rebuttal:

Meta-review: All ratings were "accept". The reviewers raised various suggestions which the authors responded to thoroughly, with additional experiments that reinforced their results. This should be a solid contribution.
Decision: Accept (Spotlight)