Symplectic ODE-Net: Learning Hamiltonian Dynamics with Control

1 INTRODUCTION . In recent years , deep neural networks ( Goodfellow et al. , 2016 ) have become very accurate and widely used in many application domains , such as image recognition ( He et al. , 2016 ) , language comprehension ( Devlin et al. , 2019 ) , and sequential decision making ( Silver et al. , 2017 ) . To learn underlying patterns from data and enable generalization beyond the training set , the learning approach incorporates appropriate inductive bias ( Haussler , 1988 ; Baxter , 2000 ) by promoting representations which are simple in some sense . It typically manifests itself via a set of assumptions , which in turn can guide a learning algorithm to pick one hypothesis over another . The success in predicting an outcome for previously unseen data then depends on how well the inductive bias captures the ground reality . Inductive bias can be introduced as the prior in a Bayesian model , or via the choice of computation graphs in a neural network . In a variety of settings , especially in physical systems , wherein laws of physics are primarily responsible for shaping the outcome , generalization in neural networks can be improved by leveraging underlying physics for designing the computation graphs . Here , by leveraging a generalization of the Hamiltonian dynamics , we develop a learning framework which exploits the underlying physics in the associated computation graph . Our results show that incorporation of such physics-based inductive bias offers insight about relevant physical properties of the system , such as inertia , potential energy , total conserved energy . These insights , in turn , enable a more accurate prediction of future behavior and improvement in out-of-sample behavior . Furthermore , learning a physically-consistent model of the underlying dynamics can subsequently enable usage of model-based controllers which can provide performance guarantees for complex , nonlinear systems . In particular , insight about ∗This research was carried out during Y. D. Zhong ’ s internship at Siemens Corporate Technology . 1We use the word Symplectic to emphasize that the learned dynamics endows a symplectic structure ( Arnold et al. , 2001 ) on the underlying space . kinetic and potential energy of a physical system can be leveraged to synthesize appropriate control strategies , such as the method of controlled Lagrangian ( Bloch et al. , 2001 ) and interconnection & damping assignment ( Ortega et al. , 2002 ) , which can reshape the closed-loop energy landscape to achieve a broad range of control objectives ( regulation , tracking , etc. ) . RELATED WORK Physics-based Priors for Learning in Dynamical Systems : The last few years have witnessed a significant interest in incorporating physics-based priors into deep learning frameworks . Such approaches , in contrast to more rigid parametric system identification techniques ( Söderström & Stoica , 1988 ) , use neural networks to approximate the state-transition dynamics and therefore are more expressive . Sanchez-Gonzalez et al . ( 2018 ) , by representing the causal relationships in a physical system as a directed graph , use a recurrent graph network to infer latent space dynamics of robotic systems . Lutter et al . ( 2019 ) and Gupta et al . ( 2019 ) leverage Lagrangian mechanics to learn the dynamics of kinematic structures from time-series data of position , velocity , and acceleration . A more recent ( concurrent ) work by Greydanus et al . ( 2019 ) uses Hamiltonian mechanics to learn the dynamics of autonomous , energy-conserved mechanical systems from time-series data of position , momentum , and their derivatives . A key difference between these approaches and the proposed one is that our framework does not require any information about higher-order derivatives ( e.g. , acceleration ) and can incorporate external control into the Hamiltonian formalism . Neural Networks for Dynamics and Control : Inferring underlying dynamics from time-series data plays a critical role in controlling closed-loop response of dynamical systems , such as robotic manipulators ( Lillicrap et al. , 2015 ) and building HVAC systems ( Wei et al. , 2017 ) . Although the use of neural networks towards identification and control of dynamical systems dates back to more than three decades ago ( Narendra & Parthasarathy , 1990 ) , recent advances in deep neural networks have led to renewed interest in this domain . Watter et al . ( 2015 ) learn dynamics with control from high-dimensional observations ( raw image sequences ) using a variational approach and synthesize an iterative LQR controller to control physical systems by imposing a locally linear constraint . Karl et al . ( 2016 ) and Krishnan et al . ( 2017 ) adopt a variational approach and use recurrent architectures to learn state-space models from noisy observation . SE3-Nets ( Byravan & Fox , 2017 ) learn SE ( 3 ) transformation of rigid bodies from point cloud data . Ayed et al . ( 2019 ) use partial information about the system state to learn a nonlinear state-space model . However , this body of work , while attempting to learn state-space models , does not take physics-based priors into consideration . CONTRIBUTION The main contribution of this work is two-fold . First , we introduce a learning framework called Symplectic ODE-Net ( SymODEN ) which encodes a generalization of the Hamiltonian dynamics . This generalization , by adding an external control term to the standard Hamiltonian dynamics , allows us to learn the system dynamics which conforms to Hamiltonian dynamics with control . With the learned structured dynamics , we are able to synthesize controllers to control the system to track a reference configuration . Moreover , by encoding the structure , we can achieve better predictions with smaller network sizes . Second , we take one step forward in combining the physics-based prior and the data-driven approach . Previous approaches ( Lutter et al. , 2019 ; Greydanus et al. , 2019 ) require data in the form of generalized coordinates and their derivatives up to the second order . However , a large number of physical systems accommodate generalized coordinates which are non-Euclidean ( e.g. , angles ) , and such angle data is often obtained in the embedded form , i.e. , ( cos q , sin q ) instead of the coordinate ( q ) itself . The underlying reason is that an angular coordinate lies on S1 instead of R1 . In contrast to previous approaches which do not address this aspect , SymODEN has been designed to work with angle data in the embedded form . Additionally , we leverage differentiable ODE solvers to avoid the need for estimating second-order derivatives of generalized coordinates . Code for the SymODEN framework and experiments is available at https : //github.com/d-biswa/Symplectic-ODENet . 2 PRELIMINARY CONCEPTS . 2.1 HAMILTONIAN DYNAMICS . Lagrangian dynamics and Hamiltonian dynamics are both reformulations of Newtonian dynamics . They provide novel insights into the laws of mechanics . In these formulations , the configuration of a system is described by its generalized coordinates . Over time , the configuration point of the system moves in the configuration space , tracing out a trajectory . Lagrangian dynamics describes the evolution of this trajectory , i.e. , the equations of motion , in the configuration space . Hamiltonian dynamics , however , tracks the change of system states in the phase space , i.e . the product space of generalized coordinates q = ( q1 , q2 , ... , qn ) and generalized momenta p = ( p1 , p2 , ... , pn ) . In other words , Hamiltonian dynamics treats q and p on an equal footing . This not only provides symmetric equations of motion but also leads to a whole new approach to classical mechanics ( Goldstein et al. , 2002 ) . Hamiltonian dynamics is also widely used in statistical and quantum mechanics . In Hamiltonian dynamics , the time-evolution of a system is described by the Hamiltonian H ( q , p ) , a scalar function of generalized coordinates and momenta . Moreover , in almost all physical systems , the Hamiltonian is the same as the total energy and hence can be expressed as H ( q , p ) = 1 2 pTM−1 ( q ) p+ V ( q ) , ( 1 ) where the mass matrix M ( q ) is symmetric positive definite and V ( q ) represents the potential energy of the system . Correspondingly , the time-evolution of the system is governed by q̇ = ∂H ∂p ṗ = −∂H ∂q , ( 2 ) where we have dropped explicit dependence on q and p for brevity of notation . Moreover , since Ḣ = ( ∂H ∂q ) T q̇+ ( ∂H ∂p ) T ṗ = 0 , ( 3 ) the total energy is conserved along a trajectory of the system . The RHS of Equation ( 2 ) is called the symplectic gradient ( Rowe et al. , 1980 ) of H , and Equation ( 3 ) shows that moving along the symplectic gradient keeps the Hamiltonian constant . In this work , we consider a generalization of the Hamiltonian dynamics which provides a means to incorporate external control ( u ) , such as force and torque . As external control is usually affine and only influences changes in the generalized momenta , we can express this generalization as [ q̇ ṗ ] = [ ∂H ∂p −∂H∂q ] + [ 0 g ( q ) ] u , ( 4 ) where the input matrix g ( q ) is typically assumed to have full column rank . For u = 0 , the generalized dynamics reduces to the classical Hamiltonian dynamics ( 2 ) and the total energy is conserved ; however , when u 6= 0 , the system has a dissipation-free energy exchange with the environment . 2.2 CONTROL VIA ENERGY SHAPING . Once we have learned the dynamics of a system , the learned model can be used to synthesize a controller for driving the system to a reference configuration q ? . As the proposed approach offers insight about the energy associated with a system , it is a natural choice to exploit this information for synthesizing controllers via energy shaping ( Ortega et al. , 2001 ) . As energy is a fundamental aspect of physical systems , reshaping the associated energy landscape enables us to specify a broad range of control objectives and synthesize nonlinear controllers with provable performance guarantees . If rank ( g ( q ) ) = rank ( q ) , the system is fully-actuated and we have control over any dimension of “ acceleration ” in ṗ . For such fully-actuated systems , a controller u ( q , p ) = β ( q ) + v ( p ) can be synthesized via potential energy shaping β ( q ) and damping injection v ( p ) . For completeness , we restate this procedure ( Ortega et al. , 2001 ) using our notation . As the name suggests , the goal of potential energy shaping is to synthesize β ( q ) such that the closed-loop system behaves as if its time-evolution is governed by a desired Hamiltonian Hd . With this , we have [ q̇ ṗ ] = [ ∂H ∂p −∂H∂q ] + [ 0 g ( q ) ] β ( q ) = [ ∂Hd ∂p −∂Hd∂q ] , ( 5 ) where the difference between the desired Hamiltonian and the original one lies in their potential energy term , i.e . Hd ( q , p ) = 1 2 pTM−1 ( q ) p+ Vd ( q ) . ( 6 ) In other words , β ( q ) shape the potential energy such that the desired Hamiltonian Hd ( q , p ) has a minimum at ( q ? ,0 ) . Then , by substituting Equation ( 1 ) and Equation ( 6 ) into Equation ( 5 ) , we get β ( q ) = gT ( ggT ) −1 ( ∂V ∂q − ∂Vd ∂q ) . ( 7 ) Thus , with potential energy shaping , we ensure that the system has the lowest energy at the desired reference configuration . Furthermore , to ensure that trajectories actually converge to this configuration , we add an additional damping term2 given by v ( p ) = −gT ( ggT ) −1 ( Kdp ) . ( 8 ) However , for underactuated systems , potential energy shaping alone cannot3 drive the system to a desired configuration . We also need kinetic energy shaping for this purpose ( Chang et al. , 2002 ) . Remark If the desired potential energy is chosen to be a quadratic of the form Vd ( q ) = 1 2 ( q− q ? ) TKp ( q− q ? ) , ( 9 ) the external forcing term can be expressed as u = gT ( ggT ) −1 ( ∂V ∂q −Kp ( q− q ? ) −Kdp ) . ( 10 ) This can be interpreted as a PD controller with an additional energy compensation term . 4

Summary: In this paper, the authors propose a framework for learning the dynamics of a system with underlying Hamiltonian structure subjected to external control. Based on the extended equations of motion, the authors suggest how to apply NeuralODE in a way that makes use of the prior information that the unconstrained system is Hamiltonian and subjected to a control term. For a range of tasks, the authors then demonstrate that the proposed SymODEN framework can learn the dynamics and recover the known analytical solution, and that they can derive a controller that allows them to drive the system to a target configuration.