Neural Symplectic Form: Learning Hamiltonian Equations on General Coordinate Systems
Year: 2022

Abstract:
In recent years, substantial research on the methods for learning Hamiltonian equations has been conducted. Although these approaches are very promising, the commonly used representation of the Hamilton equation uses the generalized momenta, which are generally unknown. Therefore, the training data must be represented in this unknown coordinate system, and this causes difficulty in applying the model to real data. Meanwhile, Hamiltonian equations also have a coordinate-free expression that is expressed by using the symplectic 2-form. In this paper, we propose a model that learns the symplectic form from data using neural networks, thereby providing a method for learning Hamiltonian equations from data represented in general coordinate systems, which are not limited to the generalized coordinates and the generalized momenta. Consequently, the proposed method is capable not only of modeling the target equations of both Hamiltonian and Lagrangian formalisms but also of extracting unknown Hamiltonian structures hidden in the data. For example, many polynomial ordinary differential equations such as the Lotka–Volterra equation are known to admit non-trivial Hamiltonian structures, and our numerical experiments show that such structures can certainly be learned from data. Technically, each symplectic 2-form is associated with a skew-symmetric matrix, but not all skew-symmetric matrices define a symplectic 2-form. In the proposed method, using the fact that symplectic 2-forms are derived as the exterior derivative of certain differential 1-forms, we model the differential 1-form by neural networks, thereby improving the efficiency of learning.

heading: 1 Introduction
text: In recent years, the applications of deep learning to learn the fundamental equations of classical mechanics have been actively studied. Analytical mechanics, which is a theory of classical mechanics, is classified into Lagrangian mechanics and Hamiltonian mechanics [1, 3, 18]. The equations of motion of Lagrangian mechanics are called the Euler–Lagrange equation, for which several neural network models were proposed [7, 17]. In Lagrangian mechanics, the equations are described using the state variables and the time derivatives of them. This feature of Lagrangian mechanics makes it easy to prepare the data necessary for learning. On the other hand, Hamiltonian mechanics can describe more general equations which are not covered by Lagrangian mechanics. In the previous
35th Conference on Neural Information Processing Systems (NeurIPS 2021).
models for Hamiltonian equations, the equation
d
dt
( q p ) = ( O I −I O )(∂H ∂q ∂H ∂p ) (1)
is typically assumed, where q is the state variable, and p is a variable called the generalized momentum. It is known that Hamiltonian equations can be written in this form by using these variables; more precisely, any Hamiltonian equation can be locally written as (1) by using a special coordinate system called the Darboux coordinate system [20]. However, this coordinate system depends on the generally unknown Hamiltonian (the energy function), and it is usually not possible to prepare data in this coordinate system.
On the other hand, Hamiltonian equations also have a coordinate-free representation using the symplectic 2-form [20]. In this paper, we propose a method to learn the symplectic 2-form from data by using neural networks, thereby introducing a method to learn the Hamilton equation from data represented in general coordinate systems, not restricted to the generalized momentum. As explained below, general 2-forms are corresponding to skew symmetric matrices. Hence, a naive way to learn the symplectic 2-form is learning a skew symmetric matrix; however, actually, the equation learned in this way may not be Hamiltonian because not all skew symmetric matrices are corresponding to symplectic 2-forms. In the proposed method, instead of learning the 2-form directly, a 1-form that derives the symplectic 2-form is learned (see Figure 1).
Because the proposed model can be applied to data in any coordinate system, it can be employed for extracting the unknown underlying Hamiltonian structure behind the data that are expected to have such a structure. For example, many polynomial ordinary differential equations are known to have a Hamiltonian structure, but it is not easy to find the structure analytically. The proposed method extracts the hidden structure in such cases from data. If the Hamiltonian structure can be found, it is possible to make predictions while preserving the energy conservation law. In addition to the energy, other hidden conserved quantities can be also extracted.
Main contributions of this paper include:
Symplectic geometric approach to learning symplectic 2-forms. The symplectic 2-form required to describe the Hamilton equation corresponds to a skew-symmetric matrix, but conversely, not all skew-symmetric matrices correspond to a symplectic 2-form. In this paper, we propose an efficient model by learning the symplectic 1-form that derives the symplectic 2-form with neural networks.
Learning the Hamilton equation from data in arbitrary coordinate systems. By using the coordinate-free representation of the Hamilton equation, it is possible to learn the Hamilton equation from data represented in a general coordinate system, not restricted to the Darboux coordinate system. In this way, the proposed method can determine whether the given data can be explained by the hidden theory of classical mechanics or not. A universal approximation theorem is also provided.

heading: 2 Related Work
text: Neural Networks for Hamiltonian Mechanics Neural ordinary differential equation (NODE, [5]) is a neural network that models the time-derivative of the states, thereby defining an ordinary differential equation (ODE) in a general way. Due to the generality, this model does not admit the energy conservation law.
Hamiltonian neural network (HNN) is a neural network that models the HamiltonianH and defines the dynamics following the Hamiltonian mechanics, thereby ensuring the energy conservation law [11]. Although models of the form (1) are often used in the previous studies, the Hamilton equation has this form only in the Darboux coordinates [20]. The Darboux coordinate system, which is essentially the generalized momentum p, is defined using the Hamiltonian H , which is the target to be learned. Hence, training data in the coordinate is usually unavailable. In addition, Hamiltonian equations are defined on general symplectic manifolds; however, in the existing studies, cotangent bundles are typically assumed as the symplectic manifold.
A numerical integration of the Hamiltonian system is known to destroy the symplectic structure and does not conserve the Hamiltonian H , unless the integrator is carefully designed (see, e.g., [12, 26]). Several studies focus on the numerical integration that conserves the Hamiltonian H [6, 9, 19, 27, 28]. The HNN was extended to energy-conserving partial differential equation (PDE) systems, such as the Korteweg–De Vries (KdV) equation [19], to dissipative systems, such as a pendulum with friction [19, 29]. In [15], a discrete-time model is proposed for Poisson systems, which are extension of Hamiltonian systems, where the skew symmetric matrix can be degenerate. In particular, in [15], dynamics with state-dependent skew symmetric matrices are leaned by introducing coordinate transformations for learning the dynamics in the latent space. The proposed method is different from this study in that our method does not use the coordinate transformations and hence has an advantage in interpretability. Another approach is employed in [4, 16], where the symplectic map is modeled. SympNets [16] are also shown to be universal approximators, and, in [4], a bound on the prediction error is provided. The proposed method can be combined with these discrete-time approaches.
Modeling using machine learning has also been performed in the field of quantum mechanics, for example, by Tkatchenko and coworkers (e.g. [22, 25]). Some breakthroughs have been reported that have not been possible with conventional computational chemistry methods. The relationship with these studies needs to be investigated in the future.
Neural Networks for Lagrangian Mechanics Another branch of studies focus on Lagrangian mechanics. Lagrangian neural network (LNN) is a neural network that models the Lagrangian L in a general way [7], and deep Lagrangian network explicitly defines the kinetics energy with a trainable mass matrix [17]. Lagrangian mechanics defines Lagrangian systems on tangent bundles [18], where the state is the pair of the position q and velocity q̇. The systems have specific symplectic structures, which are equivalent to Hamiltonian systems in general coordinate systems on cotangent bundles. Because the LNN does not assume equations of a specific form, it can learn a wider class of systems, including a double pendulum, in addition to the systems that the HNN can learn (see Table 1). Similarly to the HNN, numerical integrators that preserve the symplectic structure have been investigated [8, 23]. Neural network architectures that ensure translational and rotational symmetries have also been investigated [10, 24].
As mentioned before, not all Hamiltonian systems are defined on cotangent bundles. For examples, the some polynomial equations including the Lotka–Volterra equation have a different symplectic structure [13]. In fact, the Lotka–Volterra equation is actually a Hamiltonian system, even though its
states are not position, velocity, nor generalized momentum. These equations are out of the scopes of the HNN and LNN (see Table 1).

heading: 3 Methods
text: Coordinate-Free Representation of Hamiltonian Equations In this paper, we propose a neural network model based on a coordinate-free representation of Hamiltonian equations. First, we describe this representation. Because detailed knowledge of geometry is required for a precise description of this representation, the details are omitted here. For more details, see Appendix A.
The model in this paper can be used on a general symplectic manifold, but for simplicity, we describe the case where the phase space isM = R2N . A differential 2-form ω onM is a skew-symmetric bilinear function that maps given two vectors into a real number, depending on each point u onM. The skew-symmetric bilinear function defined by ω has the following matrix representation:
ωu(v1, v2) = v > 1 Wuv2, for all v1, v2 ∈ R2N ,
where Wu is a skew-symmetric matrix, and the subscript u denotes that ω and hence its matrix representation Wu depend on u. A symplectic 2-form is a differential 2-form that is nondegenerate and closed. See Appendix A for the precise meanings of these terms.
The following is the coordinate-free form of Hamiltonian equations [18]
du dt = XH , ω(XH , v) = dH(v) for all v ∈ R2N . (2)
Here, dH is the Fréchet derivative of the Hamiltonian H , and XH is a vector field depending on H . This equation is satisfied regardless of the coordinate system in which the state variable u is expressed. Therefore, by using this equation as a model, as long as the given data is described by the Hamilton equation, it is possible to learn both the symplectic 2-form and the Hamiltonian that define the Hamilton equation, no matter what coordinate system the data is given in.
Naive method: the skew matrix learning By replacing the symplectic 2-form with the matrix Wu, (4) can be rewritten as
du dt = XH , X > HWuv = v · ∇H for all v ∈ R2N ⇔ du dt = W−>u ∇H.
A natural model based on this representation would be a model in which Wu and H are modeled by multilayer perceptrons:
du dt = W−>u,NN∇HNN(u), (3)
where HNN is a function of u defined by a multilayer perceptron, and Wu,NN is a skew-symmetric matrix depending on u represented by another multilayer perceptron. We refer this model as the skew matrix learning. If u is represented in the Darboux coordinate system, (3) becomes (1), and hence the model is the same as the Hamiltonian neural network.
Similarly to Hamiltonian neural networks, this model has the energy conservation law. Theorem 1. Solutions to (3) satisfy dHNN/dt = 0.
See Appendix C for the proof.
Neural Symplectic Form Although the model in the previous section is natural, it is a redundant model and inefficient for learning as explained below. As explained in Introduction, although a differential 2-form corresponds to a skew-symmetric matrix, not all skew-symmetric matrices define a symplectic 2-form. Symplectic 2-forms have the characteristic feature of being closed. In this paper, we propose a model in which the learned 2-form is guaranteed to be closed. We refer the learned 2-form as the neural symplectic form.
First, the necessary terminology is briefly explained. See Appendix A for details. A differential 0-form onM = R2N is a function fromM to R. A differential 1-form θ onM = R2N is a field of linear functions each of which maps a vector v ∈ R2N to θu(v) ∈ R, depending on each point
u ∈ M. In general, a linear function from R2N to R can be expressed as an inner product with a vector, so a differential 1-form can be expressed as a vector field depending on u. A differential operation called the exterior derivative d is defined for differential forms. The exterior derivative is a graded linear map, i.e. a linear map depending on an integer k, which transfers a differential k-form to a differential (k + 1)-form and has the property that dd = 0.
The differential form in Ker d is called a closed form. Since a symplectic 2-form is a closed form, in order to learn the symplectic 2-form by neural networks, neural networks should be designed so that differential 2-forms represented by the neural networks are contained in Ker d. Meanwhile, due to the property of the exterior derivative, dd = 0, it holds that Im d ⊂ Ker d. Actually, according to the de Rham theorem, when the phase space is R2N , these two spaces coincide: Im d = Ker d. The difference between these two spaces Im d/Ker d is called the cohomology space. The de Rham theorem states that the cohomology space is isomorphic
to the homology space, which is roughly a space of "spatial holes." Because R2N contains no holes, the homology space must vanish, and hence Im d = Ker d holds (see Figure 2.) Even when the phase space has a hole, in many cases the space can be embedded in a large Euclid space without holes, and the model should be defined on that space. See Appendix B.
Therefore, in this paper, instead of learning the symplectic 2-form directly, we propose a method to learn the symplectic 2-form by learning the differential 1-form of which exterior derivative gives the symplectic 2-form. The following is the coordinate-free form of the proposed model:
ω̃ = dθNN, du
dt = X̃HNN , ω(X̃HNN , v) = dHNN(v) for all v ∈ R2N (4)
Unlike (3), in this model skew-symmetric matrices that do not correspond to symplectic forms are not explored. Therefore, this model can be trained much more efficiently than (3). In fact, suppose that the dimension of the phase space is 2N . Then, the number of components of the skew-symmetric matrix is N(2N − 1), while the differential 1-form is represented as a vector field, the number of which components is 2N . Consequently, a model using the neural symplectic form can reduce the order of the number of functions to be learned from O(N2) to O(N) without sacrificing the expressive power.
As per the above, the differential 1-form can be expressed as a vector field. Hence the neural network for modeling the 1-form in the proposed model essentially models a vector field YNN, which represents the differential 1-form θNN. As shown in Appendix A, the vector field YNN is transformed to the matrix W̃u representing the symplectic 2-form as follows:
(W̃u)i,j = ∂(YNN)i ∂uj − ∂(YNN)j ∂ui .
In the actual model, YNN given by the neural network is differentiated by the automatic differentiation, and Wu,NN is obtained by substituting the derivatives into the above equation. Thus we have the model expressed in terms of vectors and a matrix, without using the differential forms:
du dt = W̃−>u ∇HNN(u),
(W̃u)i,j = ∂(YNN)i ∂uj − ∂(YNN)j ∂ui . (5)
The proposed model has the energy conservation law and the universal approximation property. For the proofs, see Appendix C. Theorem 2. Solutions to (4), or equivalently, to (5) satisfy dHNN/dt = 0. Theorem 3. Suppose that the Hamilton equation to be learned can be represented in the form (2) using a symplectic 2-form ω derived from a 1-form θ ∈ W d,p and the Hamiltonian H ∈ W 1,p.
Suppose also that the phase space is compact. The model (4) with multilayer perceptrons with a sufficiently smooth activation function has the universal approximation property in the sense that θ and H can be approximated by the multilayer perceptrons with arbitrary accuracy.
W 1,p and W d,p are Sobolev spaces; see [2, 14].
Learning dynamics by using the neural symplectic form The proposed model learns the symplectic form and the Hamiltonian by minimizing the squared error between the left- and right-hand sides of (4), assuming that time series data of state vectors {u(n)} and its time-derivatives {dudt
(n)} are given; minimize ∑ n ∣∣∣∣∣dudt (n) − W̃−>u(n)∇HNN(u(n)) ∣∣∣∣∣ 2 , (W̃u(n))i,j = ∂(YNN)i ∂uj (u(n))− ∂(YNN)j ∂ui (u(n)).
If the time-derivatives are not available, interpolated data should be used. For each state vector u(n), first YNN(u(n)) is computed. Then, the derivatives of YNN(u(n)) are calculated by the automatic differentiation to obtain (W̃−>u )i,j . Meanwhile, the gradient of the Hamiltonian is computed in the same way; first HNN((n)) is computed, and then ∇HNN((n)) is obtained by the automatic differentiation.
In the standard Hamiltonian neural networks only the computation of the gradient of the Hamiltonian is required. Hence the computational cost of the neural symplectic form is roughly 2N times the computational cost of Hamiltonian neural networks, where 2N is the number of the state variables, because the number of components of YNN(u(n)) is 2N . When N is large, such as in many-body problems, the most computationally expensive part may be computation of the inverse of the matrix W̃u(n) , which requires O(N3) operations. To avoid the computation of the inverse, one may be tempted to minimize ∑
n
∣∣∣∣∣W̃>u(n) dudt (n) −∇HNN(u(n)) ∣∣∣∣∣ 2 .
However, this model does not work because this is trivially minimized by setting W̃> u(n) = O and HNN = (constant.) Learning the inverse while maintaining the symplectic structure is future work.

heading: 4 Numerical Examples
text: Outline of experiments The proposed method with the neural symplectic form can model general Hamiltonian equations on general symplectic manifolds using data represented by general coordinate systems. We illustrate these advantages in the following numerical experiments with comparative methods. The details of the data in the numerical experiments along with additional results are provided in Appendix D.
We performed the experiments using HNN, LNN1, skew matrix learning and neural symplectic form. The energy function, the skew-symmetric matrix, and the 1-form for the neural symplectic 2-form were modeled by using a neural network that had two hidden layers of 200 units and the tanh activation function. We used 80 percent of collected data for training and the remaining for test. We trained each model 10 times using the Adam optimizer with a learning rate of 10−3 for 2000 iterations. Only for LNN, we set the learning rate to 10−4 due to an instability of learning; in our experiments the loss function of LNN sometimes did not converge monotonically, but oscillated when the learning rate was set to 10−3. Hence we truncated the training process at the iteration where the loss function achieved the best score. This oscillation should be due to the non-uniqueness of Lagrangian; see Appendix D for details.
The experiments were performed on NVIDIA A100 with double precision. After training, we evaluated the models using the squared time-derivative errors of the test subset. Using SciPy odeint under the default setting, we predicted 10 orbits from random initial values and obtained the errors in the system energy.
1We used the implementation of the LNN model in torchdyn [21] https://github.com/DiffEqML/ torchdyn/blob/master/docs/tutorials/09_lagrangian_nets.ipynb (Apache 2.0 License)
Mass-Spring System First, we investigate a Hamiltonian system in a general coordinate on a cotangent bundle, namely, a simple massspring system, depicted in Figure 3. The equation of motion of this system is
d
dt q1q2v1 v2  = 
v1 v2
− k1m1 (q1 − l1) + k2 m1 (q2 − q1 − l2) − k2m2 (q2 − q1 − l2)  , which is a Hamiltonian system with the energy function
H(q1, q2, p1, p2) = p21
2m1 + p22 2m2 + k1(q1 − l1)2 2 + k2(q2 − q1 − l2)2 2 ,
where q1, q2 are the positions of the mass points, and p1, p2 are the momenta, which are defined by p1 = m1v1, p2 = m2v2, v1 = dq1/dt, v2 = dq2/dt. Suppose that the exact values of m1 and m2 are unknown, and only the positions q1 and q2 and their derivatives are given. Although m1 and m2 may be estimated from the given states, for evaluation of the models, we tried to learn the dynamics from the given states directly.
The time-derivative errors and the energy errors are shown in Tables 2 and 3, respectively. As explained in Appendix D, if the state variables are represented by the positions and the velocities, the equation of motion of this system cannot be written in the standard form. Nevertheless, as this is a simple problem, all the models work well. Most of the predicted orbits shown in Figure 4 have a good fit to the true one. Note that the errors shown in Table 2 are small enough. They are multiplied by 103 and shown in the physical scale; i.e., the errors are not measured by using the normalized state variables but by using the state variables with the true scale. In the normalized scale, the error by LNN, for example, was 0.0095. This is also the case in the other experiments. Among the methods, NODE and the neural symplectic form performed better. The test time-derivative error of the skew matrix learning is a little larger. In terms of the energy error, the neural symplectic form and HNN are superior; however there is not much difference.
Double Pendulum Next, we consider another Hamiltonian system, namely, a double pendulum shown in Figure 5, of which equation of motion is
dθ1 dt = φ1, dθ2 dt = φ2,
dφ1 dt
= g(sin θ2 sin(θ1 − θ2)− m1+m2m2 sin θ1)− (l1θ 2 1 cos(θ1 − θ2) + l2θ22) sin(θ1 − θ2)
l1( m1+m2 m2 − cos2(θ1 − θ2))
,
dφ2 dt =
g(m1+m2) m2 (sin θ1 cos(θ1 − θ2)− sin θ2)− ( l1(m1+m2)m2 θ 2 1 + l2θ 2 2 cos(θ1 − θ2)) sin(θ1 − θ2)
l2( m1+m2 m2 − cos2(θ1 − θ2))
.
(6)
For the energy function and the Lagrangian of this system, see Appendix D. The generalized momenta of this system are
p1 = (m1 +m2)l 2 1φ1 +m2l1l2φ2 cos(θ1 − θ2), p2 = m2l22φ2 +m2l1l2φ1 cos(θ1 − θ2).
Because the generalized momenta are not obvious, we assume that the data of θ1, θ2, φ1, φ2 are given instead of θ1, θ2, p1, p2, where φ1, φ2 are the time derivatives of θ1, θ2.
The test time-derivative errors are shown in Table 2. Firstly, (6) cannot be written as the standard form (1) with a certain Hamiltonian. Hence, the test time-derivative error of HNN cannot be completely zero when learned using states θ1(t), φ1(t), θ2(t), φ2(t). The result shown in Table 2 confirms this. The test time-derivative errors by NODE, the skew matrix learning and the neural symplectic form are much smaller than that by HNN. In particular, the error by the skew matrix learning is larger than that by the neural symplectic form. This is due to the non-existence of the one-to-one correspondence between the skew matrix and the symplectic 2-form, which decreases the efficiency of learning. NODE performed very well too; however, as explained below, this model failed to predict long-term behaviors.
Examples of the predicted orbits are shown in Figure 6. As expected, HNN failed to predict the orbits. In the result by the skew matrix learning, although the speed of oscillation appears to be correct, the heights of the peaks are different from the true trajectory. Meanwhile, the prediction by the neural symplectic form achieved a better agreement with the true one than other models.
The energy errors are shown in Table 3. The errors of NODE and the skew matrix learning are small for the short term prediction. However, Figure 7 shows an example of the long-term prediction results of these models and the proposed neural symplectic form. The results of the two NODEs are obtained using the models trained with different seeds. Since NODE does not have a Hamiltonian
structure, the results gradually decay or diverge. The prediction results by the skew matrix learning also show a gradual increase in the state variables, and the results collapse after exceeding a certain value. On the other hand, the result by the neural symplectic form oscillates stably, although the range of oscillation is a little larger than the true orbit.
Lotka–Volterra equation Systems of differential equations with a polynomial right-hand side are often Hamiltonian with a hidden symplectic structure. For example, the Hamiltonian structure of the
generalized Lotka–Volterra equation
dxi dt = xi( m∑ j=1 aijΠ l k=1x bjk k + λi), (7)
is investigated in Hernández-Bermejo and Fairén [13]. The right-hand side of this equation is quite general because this equation has the form of xi(polynomial of the other state variables). For example, many mathematical compartment models in biology are of this form. In fact, the original Lotka–Volterra model is proposed as a model of a predator-prey dynamics.
We used a standard Lotka–Volterra model, which is included in (7), of the following form:
dx1 dt = a12x1x2 + λ1x1, dx2 dt = a21x1x2 + λ2x2.
Provided that x1 6= 0 and x2 6= 0, this equation can be written as a Hamiltonian equation:
d
dt ( x1 x2 ) = ( O x1x2 −x1x2 O )( ∂H ∂x1 ∂H ∂x2 ) , H(x1, x2) = −a21x1 − λ2 lnx1 + a12x2 + λ1 lnx2,
which is different from the standard Hamiltonian equation (1). In fact, this equation cannot be written as (1) globally; the Darboux coordinate system only locally exists in general. See Appendix D.
We tested the models other than LNN, which is not applicable because this equation cannot be expressed naturally as a second-order differential equation. We again used simulated solutions for data with 1000 randomly generated initial conditions. Because the Lotka–Volterra model typically describes the population dynamics of certain species, we choose the initial conditions from the uniform distribution on [0, 1]. For other details, see Appendix D.
The test time-derivative errors are shown in Table 2. Again, the error by HNN is larger than those by the other models. As seen from the table, the proposed neural symplectic form stably gave better results than the other models.
Regarding the energy errors, the energies are well preserved as shown in Table 3. In particular, the proposed method preserves the energy with the highest accuracy. Besides, predicted orbits are shown in Figure 8. The peaks of the orbits by HNN and the skew matrix learning are smaller than the true trajectory, and the orbit of the NODE is gradually decaying. On the other hand, the proposed neural symplectic form gives the almost identical orbits to the true one.
Thus the hidden symplectic structure of this equation is certainly extracted by the proposed method.
Learning from Images As an application of the proposed method, we learned the equation of motion from some images. One way to achieve this is to extract features from the images by using an autoencoder and to learn the equation of motion that the features satisfy [11]. In this case, the extracted features are not supposed to be momenta. The proposed method is suitable for this application because it is independent of the coordinate system.
In this experiment, we first pretrained an autoencoder. Then HNN and the neural symplectic form are applied to learn the dynamics of the features. The final test losses of HNN and the neural symplectic form were 0.161 and 0.060 respectively. The predicted pictures are shown in Figure 9. The last four pictures of HNN are noisy; this implies relatively large errors in the latent space. However, this is only a preliminary test; for example, the performance may depend on the architecture of the autoencoder. Further thorough investigation is needed for this application.

heading: 5 Concluding Remarks
text: In this paper, we proposed a method for learning Hamilton equations from data represented on general coordinate systems, which are not restricted to generalized momenta. The key ingredient is the neural symplectic form; we proposed to learn the symplectic 2-form by using neural networks from data, thereby learning this coordinate-free representation. In particular, the Hamilton equation can be represented using a state-dependent skew-symmetric matrix, but not all skew-symmetric matrices are related to the symplectic 2-form. In the proposed method, in order to learn the symplectic 2-form efficiently, the 1-form that derives the symplectic 2-form is learned by the neural networks.
Meanwhile, the proposed method requires the inverse of the skew-symmetric matrix, which may be computationally expensive for modeling large systems. To address this problem, the perturbation theory of the inverse matrix may be applied. For example, it is known that for a matrix M if the norm of ∆M is small enough, (M + ∆M)−1 ' M−1 + M−1∆MM−1 holds. This should be investigated to reduce the computational costs in future work.
Acknowledgments and Disclosure of Funding
Funding in direct support of this work: JST CREST Grant Number JPMJCR1914, JST PRESTO Grant Number JPMJPR21C7 and JSPS KAKENHI Grant Number 20K11693. The SQUID supercomputer in Osaka University was used for the experiments.

Review summaries:
Authors present a ML-based approach to learning dynamics in the space of Hamiltonian systems. The approach is based on generic formulation of Hamiltonian mechanics in the language of differential geometry, effectively avoiding the requirement of knowing the appropriate generalized coordinates beforehand. The proposed approach is based on modeling the symplectic 2-form (known to be closed) as an exterior derivative of a parametrized (learned) 1-form. This method is termed "neural symplectic form" and provides a compact parametrization of the underlying governing equations.  The proposed model is trained by minimizing the MSE between estimated ground truth time derivatives of the dynamical system and predictions of the model on several datasets: mass-spring system; double pendulum; Lotka-Volterra equations (which are not captured by a traditional formulation of Hamiltonian mechanics with fixed symplectic form).  The model is compared to baselines (HNN, LNN and a naive implementation termed skew matrix learning) on test loss and errors of the conserved quantity in the predicted dynamics accumulated during simulation for a fixed time.  Authors find that the proposed model produces best results across most of the comparisons, especially on the Lotka-Volterra test case.  

This work addresses the problem of learning Hamiltonian systems from data without assuming that the generalized momentum is given. This assumption limits the applicability of standard Hamiltonian Neural Networks (HNNs) because the generalized momentum is defined in terms of the Hamiltonian itself, which means it cannot typically be provided as training data for unknown Hamiltonians. This work proposes to get around this limitation by exploiting a coordinate-free formulation of the Hamiltonian and incorporating the 2-form from symplectic geometry into the structure of a HNN (either directly as a learned skew-symmetric matrix or via a learned 1-form and an exterior derivative). The resulting architecture performs favorably compared to a standard HNN and Lagrangian Neural Network (LNN) on several simple physical systems.

This paper considers the learning of non-canonical Hamiltonian dynamics from time-series data. Canonical Hamiltonian systems in Euclidean spaces correspond to $\dot{x}=J \nabla H(x)$ where $J=[0,  I; -I, 0]$ is a constant matrix corresponding to the canonical symplectic form. Non-canonical Hamiltonian systems are in the same form, however $J$ can be $x$ dependent, and it has to be a skew-symmetric matrix satisfying Leibniz’s rule and Jacobi identity. This paper studies the Euclidean case, which has nice enough topology so that $J:\mathbb{R}^{d}\to \mathbb{R}^{d\times d}$ satisfying these conditions can be expressed using a vector field $Y: \mathbb{R}^{d}\to \mathbb{R}^d$ instead, which is no longer constrained. The proposed method represents $H$ and $Y$ by neural networks and trains them to match the data. Some numerical investigations are provided.

This paper proposes to learn conservative physical dynamics from data by directly learning a differential 1-form. The dynamics is subsequently determined by computing the exterior derivative of the learned 1-form (yielding a symplectic 2-form) and providing a coordinate-free representation of the dynamics. In addition to inherently preserving the symplectic nature of the dynamics, this procedure does not require *a priori* knowledge of the generalized momentum coordinates, i.e. only velocities need be observed so masses of the degrees of freedom in the data are not needed. The neural symplectic form methodology is compared to an approach in which the symplectic 2-form is learned directly as a skew-symmetric matrix as well the existing Hamiltonian neural network and Lagrangian neural network approaches.