Deep Lagrangian Networks: Using Physics as Model Prior for Deep Learning

1 Introduction . In the last five years , deep learning has propelled most areas of learning forward at an impressive pace ( Krizhevsky et al. , 2012 ; Mnih et al. , 2015 ; Silver et al. , 2017 ) – with the exception of physically embodied systems . This lag in comparison to other application areas is somewhat surprising as learning physical models is critical for applications that control embodied systems , reason about prior actions or plan future actions ( e.g. , service robotics , industrial automation ) . Instead , most engineers prefer classical off-the-shelf modeling as it ensures physical plausibility – at a high cost of precise measurements1 and engineering effort . These plausible representations are preferred as these models guarantee to extrapolate to new samples , while learned models only achieve good performance in the vicinity of the training data . To learn amodel that obtains physically plausible representations , we propose to use the insights from physics as amodel prior for deep learning . In particular , the combination of deep learning and physics seems natural as the compositional structure of deep networks enables the efficient computation of the derivatives at machine precision ( Raissi & Karniadakis , 2018 ) and , thus , can encode a differential equation describing physical processes . Therefore , we suggest to encode the physics prior in the form of a differential in the network topology . This adapted topology amplifies the information content of the training samples , regularizes the end-to-end training , and emphasizes robust models capable of extrapolating to new samples while simultaneously ensuring physical plausibility . Hereby , we concentrate on learning models of mechanical systems using the Euler-Lagrange-Equation , a second order ordinary differential equation ( ODE ) originating from Lagrangian Mechanics , as physics prior . ∗Max Planck Institute for Intelligent Systems , Spemannstr . 41 , 72076 Tübingen , Germany 1Highly precise models usually require taking the physical system apart and measuring the separated pieces ( Albu-Schäffer , 2002 ) . We focus on learning models of mechanical systems as this problem is one of the fundamental challenges of robotics ( de Wit et al. , 2012 ; Schaal et al. , 2002 ) . Contribution The contribution of this work is twofold . First , we derive a network topology called Deep Lagrangian Networks ( DeLaN ) encoding the Euler-Lagrange equation originating from Lagrangian Mechanics . This topology can be trained using standard end-to-end optimization techniques while maintaining physical plausibility . Therefore , the obtained model must comply with physics . Unlike previous approaches to learning physics ( Atkeson et al. , 1986 ; Ledezma & Haddadin , 2017 ) , which engineered fixed features from physical assumptions requiring knowledge of the specific physical embodiment , we are ‘ only ’ enforcing physics upon a generic deep network . For DeLaN only the system state and the control signal are specific to the physical system but neither the proposed network structure nor the training procedure . Second , we extensively evaluate the proposed approach by using the model to control a simulated 2 degrees of freedom ( dof ) robot and the physical 7-dof robot Barrett WAM in real time . We demonstrate DeLaN ’ s control performance where DeLaN learns the dynamics model online starting from random initialization . In comparison to analytic- and other learned models , DeLaN yields a better control performance while at the same time extrapolates to new desired trajectories . In the following we provide an overview about related work ( Section 2 ) and briefly summarize Lagrangian Mechanics ( Section 3 ) . Subsequently , we derive our proposed approach DeLaN and the necessary characteristics for end-to-end training are shown ( Section 4 ) . Finally , the experiments in Section 5 evaluate the model learning performance for both simulated and physical robots . Here , DeLaN outperforms existing approaches . 2 Related Work . Models describing system dynamics , i.e . the coupling of control input τ and system state q , are essential for model-based control approaches ( Ioannou & Sun , 1996 ) . Depending on the control approach , the control law relies either on the forward model f , mapping from control input to the change of system state , or on the inverse model f −1 , mapping from system change to control input , i.e. , f ( q , Ûq , τ ) = Üq , f −1 ( q , Ûq , Üq ) = τ . ( 1 ) Examples for application of these models are inverse dynamics control ( de Wit et al. , 2012 ) , which uses the inverse model to compensate system dynamics , while model-predictive control ( Camacho & Alba , 2013 ) and optimal control ( Zhou et al. , 1996 ) use the forward model to plan the control input . These models can be either derived from physics or learned from data . The physics models must be derived for the individual system embodiment and requires precise knowledge of the physical properties ( Albu-Schäffer , 2002 ) . When learning the model2 , mostly standard machine learning techniques are applied to fit either the forward- or inverse-model to the training data . E.g. , authors used Linear Regression ( Schaal et al. , 2002 ; Haruno et al. , 2001 ) , Gaussian Mixture Regression ( Calinon et al. , 2010 ; Khansari-Zadeh & Billard , 2011 ) , Gaussian Process Regression ( Kocijan et al. , 2004 ; Nguyen-Tuong et al. , 2009 ; Nguyen-Tuong & Peters , 2010 ) , Support Vector Regression ( Choi et al. , 2007 ; Ferreira et al. , 2007 ) , feedforward- ( Jansen , 1994 ; Lenz et al. , 2015 ; Ledezma & Haddadin , 2017 ; Sanchez-Gonzalez et al. , 2018 ) or recurrent neural networks ( Rueckert et al. , 2017 ) to fit the model to the observed measurements . Only few approaches incorporate prior knowledge into the learning problem . Sanchez-Gonzalez et al . ( 2018 ) use the graph representation of the kinematic structure as input . While the work of Atkeson et al . ( 1986 ) , commonly referenced as the standard system identification technique for robot manipulators ( Siciliano & Khatib , 2016 ) , uses the Newton-Euler formalism to derive physics features using the kinematic structure and the joint measurements such that the learning of the dynamics model simplifies to linear regression . Similarly , Ledezma & Haddadin ( 2017 ) hard-code these physics features within a neural network and learn the dynamics parameters using gradient descent rather than linear regression . Even though these physics features are derived from physics , the 2Further information can be found in the model learning survey by Nguyen-Tuong & Peters ( 2011 ) . learned parameters for mass , center of gravity and inertia must not necessarily comply with physics as the learned parameters may violate the positive definiteness of the inertia matrix or the parallel axis theorem ( Ting et al. , 2006 ) . Furthermore , the linear regression is commonly underdetermined and only allows to infer linear combinations of the dynamics parameters and can not be applied to close-loop kinematics ( Siciliano & Khatib , 2016 ) . DeLaN follows the line of structured learning problems but in contrast to previous approaches guarantees physical plausibility and provides a more general formulation . This general formulation enables DeLaN to learn the dynamics for any kinematic structure , including kinematic trees and closed-loop kinematics , and in addition does not require any knowledge about the kinematic structure . Therefore , DeLaN is identical for all mechanical systems , which is in strong contrast to the NewtonEuler approaches , where the features are specific to the kinematic structure . Only the system state and input is specific to the system but neither the network topology nor the optimization procedure . The combination of differential equations and Neural Networks has previously been investigated in literature . Early on Lagaris et al . ( 1998 ; 2000 ) proposed to learn the solution of partial differential equations ( PDE ) using neural networks and currently this topic is being rediscovered by Raissi & Karniadakis ( 2018 ) ; Sirignano & Spiliopoulos ( 2017 ) ; Long et al . ( 2017 ) . Most research focuses on using machine learning to overcome the limitations of PDE solvers . E.g. , Sirignano & Spiliopoulos ( 2017 ) proposed the Deep Galerkin method to solve a high-dimensional PDE from scattered data . Only thework ofRaissi et al . ( 2017 ) took the opposite standpoint of using the knowledge of the specific differential equation to structure the learning problem and achieve lower sample complexity . In this paper , we follow the same motivation as Raissi et al . ( 2017 ) but take a different approach . Rather than explicitly solving the differential equation , DeLaN only uses the structure of the differential equation to guide the learning problem of inferring the equations of motion . Thereby the differential equation is only implicitly solved . In addition , the proposed approach uses different encoding of the partial derivatives , which achieves the efficient computation within a single feed-forward pass , enabling the application within control loops . 3 Preliminaries : Lagrangian Mechanics . Describing the equations of motion for mechanical systems has been extensively studied and various formalisms to derive these equations exist . The most prominent are Newtonian- , Hamiltonianand Lagrangian-Mechanics . Within this work Lagrangian Mechanics is used , more specifically the Euler-Lagrange formulation with non-conservative forces and generalized coordinates.3 Generalized coordinates are coordinates that uniquely define the system configuration . This formalism defines the Lagrangian L as a function of generalized coordinates q describing the complete dynamics of a given system . The Lagrangian is not unique and every L which yields the correct equations of motion is valid . The Lagrangian is generally chosen to be L = T −V ( 2 ) where T is the kinetic energy and V is the potential energy . The kinetic energy T can be computed for all choices of generalized coordinates using T = 12 ÛqTH ( q ) Ûq , whereas H ( q ) is the symmetric and positive definite inertia matrix ( de Wit et al. , 2012 ) . The positive definiteness ensures that all non-zero velocities lead to positive kinetic energy . Applying the calculus of variations yields the Euler-Lagrange equation with non-conservative forces described by d dt ∂L ∂ Ûqi − ∂L ∂qi = τ i ( 3 ) where τ are generalized forces . Substituting L and dV/dq = g ( q ) into Equation 3 yields the second order ordinary differential equation ( ODE ) described by H ( q ) Üq+ ÛH ( q ) Ûq− 1 2 ( ∂ ∂q ( ÛqTH ( q ) Ûq ) ) T ︸ ︷︷ ︸ Bc ( q , Ûq ) +g ( q ) = τ ( 4 ) 3More information can be found in the textbooks ( Greenwood , 2006 ; de Wit et al. , 2012 ; Featherstone , 2007 ) where c describes the forces generated by the Centripetal and Coriolis forces ( Featherstone , 2007 ) . Using this ODE any multi-particle mechanical system with holonomic constraints can be described . For example various authors used this ODE to manually derived the equations of motion for coupled pendulums ( Greenwood , 2006 ) , robotic manipulators with flexible joints ( Book , 1984 ; Spong , 1987 ) , parallel robots ( Miller , 1992 ; Geng et al. , 1992 ; Liu et al. , 1993 ) or legged robots ( Hemami & Wyman , 1979 ; Golliday & Hemami , 1977 ) . 4 Incorporating Lagrangian Mechanics into Deep Learning . Starting from the Euler-Lagrange equation ( Equation 4 ) , traditional engineering approaches would estimate H ( q ) and g ( q ) from the approximated or measured masses , lengths and moments of inertia . On the contrary most traditional model learning approaches would ignore the structure and learn the inverse dynamics model directly from data . DeLaN bridges this gap by incorporating the structure introduced by the ODE into the learning problem and learns the parameters in an end-to-end fashion . More concretely , DeLaN approximates the inversemodel by representing the unknown functions g ( q ) and H ( q ) as a feed-forward networks . Rather than representing H ( q ) directly , the lower-triangular matrix L ( q ) is represented as deep network . Therefore , g ( q ) and H ( q ) are described by Ĥ ( q ) = L̂ ( q ; θ ) L̂ ( q ; θ ) T ĝ ( q ) = ĝ ( q ; ψ ) where .̂ refers to an approximation and θ andψ are the respective network parameters . The parameters θ and ψ can be obtained by minimizing the violation of the physical law described by Lagrangian Mechanics . Therefore , the optimization problem is described by ( θ∗ , ψ∗ ) = argmin θ , ψ ` ( f̂ −1 ( q , Ûq , Üq ; θ , ψ ) , τ ) ( 5 ) with f̂ −1 ( q , Ûq , Üq ; θ , ψ ) = L̂ L̂T Üq+ d dt ( L̂ L̂T ) Ûq− 1 2 ( ∂ ∂q ( ÛqT L̂ L̂T Ûq ) ) T + ĝ ( 6 ) s.t . 0 < xT L̂ L̂T x ∀ x ∈ Rn0 ( 7 ) where f̂ −1 is the inverse model and ` can be any differentiable loss function . The computational graph of f̂ −1 is shown in Figure 1 . Using this formulation one can conclude further properties of the learned model . Neither L̂ nor ĝ are functions of Ûq or Üq and , hence , the obtained parameters should , within limits , generalize to arbitrary velocities and accelerations . In addition , the obtained model can be reformulated and used as a forward model . Solving Equation 6 for Üq yields the forward model described by f̂ ( q , Ûq , τ ; θ , ψ ) = ( L̂ L̂T ) −1 ( τ − d dt ( L̂ L̂T ) Ûq+ 1 2 ( ∂ ∂q ( ÛqT L̂ L̂T Ûq ) ) T − ĝ ) ( 8 ) where L̂ L̂T is guaranteed to be invertible due to the positive definite constraint ( Equation 7 ) . However , solving the optimization problem of Equation 5 directly is not possible due to the ill-posedness of the Lagrangian L not being unique . The Euler-Lagrange equation is invariant to linear transformation and , hence , the Lagrangian L ′ = αL + β solves the Euler-Lagrange equation if α is non-zero and L is a valid Lagrangian . This problem can be mitigated by adding an additional penalty term to Equation 5 described by ( θ∗ , ψ∗ ) = argmin θ , ψ ` ( f̂ −1 ( q , Ûq , Üq ; θ , ψ ) , τ ) +λ Ω ( θ , ψ ) ( 9 ) where Ω is the L2-norm of the network weights . Solving the optimization problem of Equation 9 with a gradient based end-to-end learning approach is non-trivial due to the positive definite constraint ( Equation 7 ) and the derivatives contained in f̂ −1 . In particular , d ( LLT ) /dt and ∂ ( ÛqTLLT Ûq ) /∂qi can not be computed using automatic differentiation as t is not an input of the network and most implementations of automatic differentiation do not allow the backpropagation of the gradient through the computed derivatives . Therefore , the derivatives contained in f̂ −1 must be computed analytically to exploit the full gradient information for training of the parameters . In the following we introduce a network structure that fulfills the positive-definite constraint for all parameters ( Section 4.1 ) , prove that the derivatives d ( LLT ) /dt and ∂ ( ÛqTLLT Ûq ) /∂qi can be computed analytically ( Section 4.2 ) and show an efficient implementation for computing the derivatives using a single feed-forward pass ( Section 4.3 ) . Using these three properties the resulting network architecture can be usedwithin a real-time control loop and trained using standard end-to-end optimization techniques .

Summary: This paper looks at system identification for a multi-link robot based upon combining a neural network with the manipulator equations.  Specifically, the authors propose to model the robot dynamics using the typical manipulator equations, but have a deep neural network parameterize the H(q) and g(q) matrices.  They illustrate that the method can control the systems of a simualted 2-dof robot and real Barrett WAM arm, better than a pure neural network modeling approach, PID control, or an analytic model.