Neural Symplectic Form: Learning Hamiltonian Equations on General Coordinate Systems

1 Introduction . In recent years , the applications of deep learning to learn the fundamental equations of classical mechanics have been actively studied . Analytical mechanics , which is a theory of classical mechanics , is classified into Lagrangian mechanics and Hamiltonian mechanics [ 1 , 3 , 18 ] . The equations of motion of Lagrangian mechanics are called the Euler–Lagrange equation , for which several neural network models were proposed [ 7 , 17 ] . In Lagrangian mechanics , the equations are described using the state variables and the time derivatives of them . This feature of Lagrangian mechanics makes it easy to prepare the data necessary for learning . On the other hand , Hamiltonian mechanics can describe more general equations which are not covered by Lagrangian mechanics . In the previous 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . models for Hamiltonian equations , the equation d dt ( q p ) = ( O I −I O ) ( ∂H ∂q ∂H ∂p ) ( 1 ) is typically assumed , where q is the state variable , and p is a variable called the generalized momentum . It is known that Hamiltonian equations can be written in this form by using these variables ; more precisely , any Hamiltonian equation can be locally written as ( 1 ) by using a special coordinate system called the Darboux coordinate system [ 20 ] . However , this coordinate system depends on the generally unknown Hamiltonian ( the energy function ) , and it is usually not possible to prepare data in this coordinate system . On the other hand , Hamiltonian equations also have a coordinate-free representation using the symplectic 2-form [ 20 ] . In this paper , we propose a method to learn the symplectic 2-form from data by using neural networks , thereby introducing a method to learn the Hamilton equation from data represented in general coordinate systems , not restricted to the generalized momentum . As explained below , general 2-forms are corresponding to skew symmetric matrices . Hence , a naive way to learn the symplectic 2-form is learning a skew symmetric matrix ; however , actually , the equation learned in this way may not be Hamiltonian because not all skew symmetric matrices are corresponding to symplectic 2-forms . In the proposed method , instead of learning the 2-form directly , a 1-form that derives the symplectic 2-form is learned ( see Figure 1 ) . Because the proposed model can be applied to data in any coordinate system , it can be employed for extracting the unknown underlying Hamiltonian structure behind the data that are expected to have such a structure . For example , many polynomial ordinary differential equations are known to have a Hamiltonian structure , but it is not easy to find the structure analytically . The proposed method extracts the hidden structure in such cases from data . If the Hamiltonian structure can be found , it is possible to make predictions while preserving the energy conservation law . In addition to the energy , other hidden conserved quantities can be also extracted . Main contributions of this paper include : Symplectic geometric approach to learning symplectic 2-forms . The symplectic 2-form required to describe the Hamilton equation corresponds to a skew-symmetric matrix , but conversely , not all skew-symmetric matrices correspond to a symplectic 2-form . In this paper , we propose an efficient model by learning the symplectic 1-form that derives the symplectic 2-form with neural networks . Learning the Hamilton equation from data in arbitrary coordinate systems . By using the coordinate-free representation of the Hamilton equation , it is possible to learn the Hamilton equation from data represented in a general coordinate system , not restricted to the Darboux coordinate system . In this way , the proposed method can determine whether the given data can be explained by the hidden theory of classical mechanics or not . A universal approximation theorem is also provided . 2 Related Work . Neural Networks for Hamiltonian Mechanics Neural ordinary differential equation ( NODE , [ 5 ] ) is a neural network that models the time-derivative of the states , thereby defining an ordinary differential equation ( ODE ) in a general way . Due to the generality , this model does not admit the energy conservation law . Hamiltonian neural network ( HNN ) is a neural network that models the HamiltonianH and defines the dynamics following the Hamiltonian mechanics , thereby ensuring the energy conservation law [ 11 ] . Although models of the form ( 1 ) are often used in the previous studies , the Hamilton equation has this form only in the Darboux coordinates [ 20 ] . The Darboux coordinate system , which is essentially the generalized momentum p , is defined using the Hamiltonian H , which is the target to be learned . Hence , training data in the coordinate is usually unavailable . In addition , Hamiltonian equations are defined on general symplectic manifolds ; however , in the existing studies , cotangent bundles are typically assumed as the symplectic manifold . A numerical integration of the Hamiltonian system is known to destroy the symplectic structure and does not conserve the Hamiltonian H , unless the integrator is carefully designed ( see , e.g. , [ 12 , 26 ] ) . Several studies focus on the numerical integration that conserves the Hamiltonian H [ 6 , 9 , 19 , 27 , 28 ] . The HNN was extended to energy-conserving partial differential equation ( PDE ) systems , such as the Korteweg–De Vries ( KdV ) equation [ 19 ] , to dissipative systems , such as a pendulum with friction [ 19 , 29 ] . In [ 15 ] , a discrete-time model is proposed for Poisson systems , which are extension of Hamiltonian systems , where the skew symmetric matrix can be degenerate . In particular , in [ 15 ] , dynamics with state-dependent skew symmetric matrices are leaned by introducing coordinate transformations for learning the dynamics in the latent space . The proposed method is different from this study in that our method does not use the coordinate transformations and hence has an advantage in interpretability . Another approach is employed in [ 4 , 16 ] , where the symplectic map is modeled . SympNets [ 16 ] are also shown to be universal approximators , and , in [ 4 ] , a bound on the prediction error is provided . The proposed method can be combined with these discrete-time approaches . Modeling using machine learning has also been performed in the field of quantum mechanics , for example , by Tkatchenko and coworkers ( e.g . [ 22 , 25 ] ) . Some breakthroughs have been reported that have not been possible with conventional computational chemistry methods . The relationship with these studies needs to be investigated in the future . Neural Networks for Lagrangian Mechanics Another branch of studies focus on Lagrangian mechanics . Lagrangian neural network ( LNN ) is a neural network that models the Lagrangian L in a general way [ 7 ] , and deep Lagrangian network explicitly defines the kinetics energy with a trainable mass matrix [ 17 ] . Lagrangian mechanics defines Lagrangian systems on tangent bundles [ 18 ] , where the state is the pair of the position q and velocity q̇ . The systems have specific symplectic structures , which are equivalent to Hamiltonian systems in general coordinate systems on cotangent bundles . Because the LNN does not assume equations of a specific form , it can learn a wider class of systems , including a double pendulum , in addition to the systems that the HNN can learn ( see Table 1 ) . Similarly to the HNN , numerical integrators that preserve the symplectic structure have been investigated [ 8 , 23 ] . Neural network architectures that ensure translational and rotational symmetries have also been investigated [ 10 , 24 ] . As mentioned before , not all Hamiltonian systems are defined on cotangent bundles . For examples , the some polynomial equations including the Lotka–Volterra equation have a different symplectic structure [ 13 ] . In fact , the Lotka–Volterra equation is actually a Hamiltonian system , even though its states are not position , velocity , nor generalized momentum . These equations are out of the scopes of the HNN and LNN ( see Table 1 ) . 3 Methods . Coordinate-Free Representation of Hamiltonian Equations In this paper , we propose a neural network model based on a coordinate-free representation of Hamiltonian equations . First , we describe this representation . Because detailed knowledge of geometry is required for a precise description of this representation , the details are omitted here . For more details , see Appendix A . The model in this paper can be used on a general symplectic manifold , but for simplicity , we describe the case where the phase space isM = R2N . A differential 2-form ω onM is a skew-symmetric bilinear function that maps given two vectors into a real number , depending on each point u onM . The skew-symmetric bilinear function defined by ω has the following matrix representation : ωu ( v1 , v2 ) = v > 1 Wuv2 , for all v1 , v2 ∈ R2N , where Wu is a skew-symmetric matrix , and the subscript u denotes that ω and hence its matrix representation Wu depend on u . A symplectic 2-form is a differential 2-form that is nondegenerate and closed . See Appendix A for the precise meanings of these terms . The following is the coordinate-free form of Hamiltonian equations [ 18 ] du dt = XH , ω ( XH , v ) = dH ( v ) for all v ∈ R2N . ( 2 ) Here , dH is the Fréchet derivative of the Hamiltonian H , and XH is a vector field depending on H . This equation is satisfied regardless of the coordinate system in which the state variable u is expressed . Therefore , by using this equation as a model , as long as the given data is described by the Hamilton equation , it is possible to learn both the symplectic 2-form and the Hamiltonian that define the Hamilton equation , no matter what coordinate system the data is given in . Naive method : the skew matrix learning By replacing the symplectic 2-form with the matrix Wu , ( 4 ) can be rewritten as du dt = XH , X > HWuv = v · ∇H for all v ∈ R2N ⇔ du dt = W− > u ∇H . A natural model based on this representation would be a model in which Wu and H are modeled by multilayer perceptrons : du dt = W− > u , NN∇HNN ( u ) , ( 3 ) where HNN is a function of u defined by a multilayer perceptron , and Wu , NN is a skew-symmetric matrix depending on u represented by another multilayer perceptron . We refer this model as the skew matrix learning . If u is represented in the Darboux coordinate system , ( 3 ) becomes ( 1 ) , and hence the model is the same as the Hamiltonian neural network . Similarly to Hamiltonian neural networks , this model has the energy conservation law . Theorem 1 . Solutions to ( 3 ) satisfy dHNN/dt = 0 . See Appendix C for the proof . Neural Symplectic Form Although the model in the previous section is natural , it is a redundant model and inefficient for learning as explained below . As explained in Introduction , although a differential 2-form corresponds to a skew-symmetric matrix , not all skew-symmetric matrices define a symplectic 2-form . Symplectic 2-forms have the characteristic feature of being closed . In this paper , we propose a model in which the learned 2-form is guaranteed to be closed . We refer the learned 2-form as the neural symplectic form . First , the necessary terminology is briefly explained . See Appendix A for details . A differential 0-form onM = R2N is a function fromM to R. A differential 1-form θ onM = R2N is a field of linear functions each of which maps a vector v ∈ R2N to θu ( v ) ∈ R , depending on each point u ∈ M. In general , a linear function from R2N to R can be expressed as an inner product with a vector , so a differential 1-form can be expressed as a vector field depending on u . A differential operation called the exterior derivative d is defined for differential forms . The exterior derivative is a graded linear map , i.e . a linear map depending on an integer k , which transfers a differential k-form to a differential ( k + 1 ) -form and has the property that dd = 0 . The differential form in Ker d is called a closed form . Since a symplectic 2-form is a closed form , in order to learn the symplectic 2-form by neural networks , neural networks should be designed so that differential 2-forms represented by the neural networks are contained in Ker d. Meanwhile , due to the property of the exterior derivative , dd = 0 , it holds that Im d ⊂ Ker d. Actually , according to the de Rham theorem , when the phase space is R2N , these two spaces coincide : Im d = Ker d. The difference between these two spaces Im d/Ker d is called the cohomology space . The de Rham theorem states that the cohomology space is isomorphic to the homology space , which is roughly a space of `` spatial holes . '' Because R2N contains no holes , the homology space must vanish , and hence Im d = Ker d holds ( see Figure 2 . ) Even when the phase space has a hole , in many cases the space can be embedded in a large Euclid space without holes , and the model should be defined on that space . See Appendix B . Therefore , in this paper , instead of learning the symplectic 2-form directly , we propose a method to learn the symplectic 2-form by learning the differential 1-form of which exterior derivative gives the symplectic 2-form . The following is the coordinate-free form of the proposed model : ω̃ = dθNN , du dt = X̃HNN , ω ( X̃HNN , v ) = dHNN ( v ) for all v ∈ R2N ( 4 ) Unlike ( 3 ) , in this model skew-symmetric matrices that do not correspond to symplectic forms are not explored . Therefore , this model can be trained much more efficiently than ( 3 ) . In fact , suppose that the dimension of the phase space is 2N . Then , the number of components of the skew-symmetric matrix is N ( 2N − 1 ) , while the differential 1-form is represented as a vector field , the number of which components is 2N . Consequently , a model using the neural symplectic form can reduce the order of the number of functions to be learned from O ( N2 ) to O ( N ) without sacrificing the expressive power . As per the above , the differential 1-form can be expressed as a vector field . Hence the neural network for modeling the 1-form in the proposed model essentially models a vector field YNN , which represents the differential 1-form θNN . As shown in Appendix A , the vector field YNN is transformed to the matrix W̃u representing the symplectic 2-form as follows : ( W̃u ) i , j = ∂ ( YNN ) i ∂uj − ∂ ( YNN ) j ∂ui . In the actual model , YNN given by the neural network is differentiated by the automatic differentiation , and Wu , NN is obtained by substituting the derivatives into the above equation . Thus we have the model expressed in terms of vectors and a matrix , without using the differential forms : du dt = W̃− > u ∇HNN ( u ) , ( W̃u ) i , j = ∂ ( YNN ) i ∂uj − ∂ ( YNN ) j ∂ui . ( 5 ) The proposed model has the energy conservation law and the universal approximation property . For the proofs , see Appendix C. Theorem 2 . Solutions to ( 4 ) , or equivalently , to ( 5 ) satisfy dHNN/dt = 0 . Theorem 3 . Suppose that the Hamilton equation to be learned can be represented in the form ( 2 ) using a symplectic 2-form ω derived from a 1-form θ ∈ W d , p and the Hamiltonian H ∈ W 1 , p . Suppose also that the phase space is compact . The model ( 4 ) with multilayer perceptrons with a sufficiently smooth activation function has the universal approximation property in the sense that θ and H can be approximated by the multilayer perceptrons with arbitrary accuracy . W 1 , p and W d , p are Sobolev spaces ; see [ 2 , 14 ] . Learning dynamics by using the neural symplectic form The proposed model learns the symplectic form and the Hamiltonian by minimizing the squared error between the left- and right-hand sides of ( 4 ) , assuming that time series data of state vectors { u ( n ) } and its time-derivatives { dudt ( n ) } are given ; minimize ∑ n ∣∣∣∣∣dudt ( n ) − W̃− > u ( n ) ∇HNN ( u ( n ) ) ∣∣∣∣∣ 2 , ( W̃u ( n ) ) i , j = ∂ ( YNN ) i ∂uj ( u ( n ) ) − ∂ ( YNN ) j ∂ui ( u ( n ) ) . If the time-derivatives are not available , interpolated data should be used . For each state vector u ( n ) , first YNN ( u ( n ) ) is computed . Then , the derivatives of YNN ( u ( n ) ) are calculated by the automatic differentiation to obtain ( W̃− > u ) i , j . Meanwhile , the gradient of the Hamiltonian is computed in the same way ; first HNN ( ( n ) ) is computed , and then ∇HNN ( ( n ) ) is obtained by the automatic differentiation . In the standard Hamiltonian neural networks only the computation of the gradient of the Hamiltonian is required . Hence the computational cost of the neural symplectic form is roughly 2N times the computational cost of Hamiltonian neural networks , where 2N is the number of the state variables , because the number of components of YNN ( u ( n ) ) is 2N . When N is large , such as in many-body problems , the most computationally expensive part may be computation of the inverse of the matrix W̃u ( n ) , which requires O ( N3 ) operations . To avoid the computation of the inverse , one may be tempted to minimize ∑ n ∣∣∣∣∣W̃ > u ( n ) dudt ( n ) −∇HNN ( u ( n ) ) ∣∣∣∣∣ 2 . However , this model does not work because this is trivially minimized by setting W̃ > u ( n ) = O and HNN = ( constant . ) Learning the inverse while maintaining the symplectic structure is future work .

Summary: This paper considers the learning of non-canonical Hamiltonian dynamics from time-series data. Canonical Hamiltonian systems in Euclidean spaces correspond to $\dot{x}=J \nabla H(x)$ where $J=[0,  I; -I, 0]$ is a constant matrix corresponding to the canonical symplectic form. Non-canonical Hamiltonian systems are in the same form, however $J$ can be $x$ dependent, and it has to be a skew-symmetric matrix satisfying Leibniz’s rule and Jacobi identity. This paper studies the Euclidean case, which has nice enough topology so that $J:\mathbb{R}^{d}\to \mathbb{R}^{d\times d}$ satisfying these conditions can be expressed using a vector field $Y: \mathbb{R}^{d}\to \mathbb{R}^d$ instead, which is no longer constrained. The proposed method represents $H$ and $Y$ by neural networks and trains them to match the data. Some numerical investigations are provided.