Original: Deep Lagrangian Networks: Using Physics as Model Prior for Deep Learning
Revised: Deep Lagrangian Networks: Using Physics as Model Prior for Deep Learning

Original: Anonymous authors Paper under double-blind review

Original: Abstract
Revised: Abstract

Original: Deep learning has achieved astonishing results on many tasks with large amounts of data and generalization within the proximity of training data.
Revised: Deep learning has achieved astonishing results on many tasks with large amounts of data and generalization within the proximity of training data.

Original: For many important real-world applications, these requirements are unfeasible and additional prior knowledge on the task domain is required to overcome the resulting problems.
Revised: For many important real-world applications, these requirements are unfeasible and additional prior knowledge on the task domain is required to overcome the resulting problems.

Original: In particular, learning physics models for model-based control requires robust extrapolation from fewer samples – often collected online in real-time – and model errors may lead to drastic damages of the system.
Revised: In particular, learning physics models for model-based control requires robust extrapolation from fewer samples – often collected online in real-time – and model errors may lead to drastic damages of the system.

Original: Directly incorporating physical insight has enabled us to obtain a novel deep model learning approach that extrapolates well while requiring fewer samples.
Revised: Directly incorporating physical insight has enabled us to obtain a novel deep model learning approach that extrapolates well while requiring fewer samples.

Original: As a ﬁrst example, we propose Deep Lagrangian Networks (DeLaN) as a deep network structure upon which Lagrangian Mechanics have been imposed.
Revised: As a ﬁrst example, we propose Deep Lagrangian Networks (DeLaN) as a deep network structure upon which Lagrangian Mechanics have been imposed.

Original: DeLaN can learn the equations of motion of a mechanical system (i.e., system dynamics) with a deep network eﬃciently while ensuring physical plausibility.
Revised: DeLaN can learn the equations of motion of a mechanical system (i.e., system dynamics) with a deep network eﬃciently while ensuring physical plausibility.

Original: The resulting DeLaN network performs very well at robot tracking control.
Revised: The resulting DeLaN network performs very well at robot tracking control.

Original: The proposed method did not only outperform previous model learning approaches at learning speed but exhibits substantially improved and more robust extrapolation to novel trajectories and learns online in real-time.
Revised: The proposed method did not only outperform previous model learning approaches at learning speed but exhibits substantially improved and more robust extrapolation to novel trajectories and learns online in real-time.

Original:  Introduction
Revised:  Introduction

Original: In the last ﬁve years, deep learning has propelled most areas of learning forward at an impressive pace (Krizhevsky et al., 2012;
Revised: In the last ﬁve years, deep learning has propelled most areas of learning forward at an impressive pace (Krizhevsky et al., 2012;

Original: Mnih et al., 2015; Silver et al., 2017) – with the exception of physically embodied systems.
Revised: Mnih et al., 2015; Silver et al., 2017) – with the exception of physically embodied systems.

Original: This lag in comparison to other application areas is somewhat surprising as learning physical models is critical for applications that control embodied systems, reason about prior actions or plan future actions (e g., service robotics, industrial automation).
Revised: This lag in comparison to other application areas is somewhat surprising as learning physical models is critical for applications that control embodied systems, reason about prior actions or plan future actions (e.g., service robotics, industrial automation).

Original: Instead, most engineers prefer classical oﬀ-the-shelf modeling as it ensures physical plausibility – at a high cost of precise measurements 1 and engineering eﬀort.
Revised: Instead, most engineers prefer classical oﬀ-the-shelf modeling as it ensures physical plausibility – at a high cost of precise measurements 1 and engineering eﬀort.

Original: These plausible representations are preferred as these models guarantee to extrapolate to new samples, while learning models  achieve good performanceonly in the vicinity of the training data.
Revised: These plausible representations are preferred as these models guarantee to extrapolate to new samples, while learned models only achieve good performance in the vicinity of the training data.

Original: To learn a model that obtains physically plausible representations, we propose to use the insights from physics as a model prior for deep learning.
Revised: To learn a model that obtains physically plausible representations, we propose to use the insights from physics as a model prior for deep learning.

Original: In particular, the combination of deep learning and physics seems natural as the compositional structure of deep networks enables the eﬃcient computation of the derivatives at machine precision (Raissi & Karniadakis, 2018) and, thus, can encode a diﬀerential equation describing physical processes.
Revised: In particular, the combination of deep learning and physics seems natural as the compositional structure of deep networks enables the eﬃcient computation of the derivatives at machine precision (Raissi & Karniadakis, 2018) and, thus, can encode a diﬀerential equation describing physical processes.

Original: Therefore, we suggest to encode the physics prior in the form of a diﬀerential in the network topology.
Revised: Therefore, we suggest to encode the physics prior in the form of a diﬀerential in the network topology.

Original: This adapted topology ampliﬁes the information content of the training samples, regularizes the end-to-end training, and emphasizes robust models capable of extrapolating to new samples while simultaneously ensuring physical plausibility.
Revised: This adapted topology ampliﬁes the information content of the training samples, regularizes the end-to-end training, and emphasizes robust models capable of extrapolating to new samples while simultaneously ensuring physical plausibility.

Original: Hereby, we concentrate on learning models of mechanical systems using the Euler-Lagrange-Equation, a second order ordinary diﬀerential equation (ODE) originating from Lagrangian Mechanics, as physics prior.
Revised: Hereby, we concentrate on learning models of mechanical systems using the Euler-Lagrange-Equation, a second order ordinary diﬀerential equation (ODE) originating from Lagrangian Mechanics, as physics prior.

Original: We focus on learning models of mechanical systems as this problem is one of the fundamental challenges of robotics (de Wit et al., 2012; Schaal et al., 2002).
Revised: We focus on learning models of mechanical systems as this problem is one of the fundamental challenges of robotics (de Wit et al., 2012; Schaal et al., 2002).

Original: Contribution
Revised: Contribution

Original: The contribution of this work is twofold.
Revised: The contribution of this work is twofold.

Original: First, we derive a network topology called Deep Lagrangian
Revised: First, we derive a network topology called Deep Lagrangian

Original: Networks (DeLaN) encoding the Euler-Lagrange equation originating from Lagrangian Mechanics.
Revised: Networks (DeLaN) encoding the Euler-Lagrange equation originating from Lagrangian Mechanics.

Original: This topology can be trained using standard end-to-end optimization techniques while maintaining physical plausibility.
Revised: This topology can be trained using standard end-to-end optimization techniques while maintaining physical plausibility.

Original: Therefore, the obtained model must comply with physics.
Revised: Therefore, the obtained model must comply with physics.

Original: Unlike previous approaches to learning physics (Atkeson et al., 1986; Ledezma & Haddadin, 2017), which engineered ﬁxed features from physical assumptions requiring knowledge of the speciﬁc physical embodiment, we are ‘only’ enforcing physics upon a generic deep network.
Revised: Unlike previous approaches to learning physics (Atkeson et al., 1986; Ledezma & Haddadin, 2017), which engineered ﬁxed features from physical assumptions requiring knowledge of the speciﬁc physical embodiment, we are ‘only’ enforcing physics upon a generic deep network.

Original: For DeLaN only the system state and the control signal are speciﬁc to the physical system but neither the proposed network structure nor the training procedure.
Revised: For DeLaN only the system state and the control signal are speciﬁc to the physical system but neither the proposed network structure nor the training procedure.

Original: Second, we extensively evaluate the proposed approach by using the model to control a simulated 2 degrees of freedom (dof) robot and the physical 7-dof robot Barrett WAM in real time.
Revised: Second, we extensively evaluate the proposed approach by using the model to control a simulated 2 degrees of freedom (dof) robot and the physical 7-dof robot Barrett WAM in real time.

Original: We demonstrate DeLaN’s control performance where DeLaN learns the dynamics model online starting from random initialization.
Revised: We demonstrate DeLaN’s control performance where DeLaN learns the dynamics model online starting from random initialization.

Original: In comparison to analytic- and other learned models,
Revised: In comparison to analytic- and other learned models,

Original: DeLaN yields a better control performance while at the same time extrapolates to new desired trajectories.
Revised: DeLaN yields a better control performance while at the same time extrapolates to new desired trajectories.

Original: In the following we provide an overview about related work (Section 2) and brieﬂy summarize
Revised: In the following we provide an overview about related work (Section 2) and brieﬂy summarize

Original: Lagrangian Mechanics (Section 3).
Revised: Lagrangian Mechanics (Section 3).

Original: Subsequently, we derive our proposed approach DeLaN and the necessary characteristics for end-to-end training are shown (Section 4).
Revised: Subsequently, we derive our proposed approach DeLaN and the necessary characteristics for end-to-end training are shown (Section 4).

Original: Finally, the experiments in
Revised: Finally, the experiments in

Original: Section 5 evaluate the model learning performance for both simulated and physical robots.
Revised: Section 5 evaluate the model learning performance for both simulated and physical robots.

Original: Here,
Revised: Here,

Original: DeLaN outperforms existing approaches.
Revised: DeLaN outperforms existing approaches.

Original:  Related Work
Revised:  Related Work

Original: Models describing system dynamics, i.e. the coupling of control input τττ and system state q , are essential for model-based control approaches (Ioannou & Sun, 1996).
Revised: Models describing system dynamics, i.e. the coupling of control input τττ and system state q , are essential for model-based control approaches (Ioannou & Sun, 1996).

Original: Depending on the control approach, the control law relies either on the forward model f , mapping from control input to the change of system state, or on the inverse model f − 1 , mapping from system change to control input, i.e.,
Revised: Depending on the control approach, the control law relies either on the forward model f , mapping from control input to the change of system state, or on the inverse model f − 1 , mapping from system change to control input, i.e.,

Original: [Equation]
Revised: [Equation]

Original: Examples for application of these models are inverse dynamics control (de Wit et al., 2012), which uses the inverse model to compensate system dynamics, while model-predictive control (Camacho &
Revised: Examples for application of these models are inverse dynamics control (de Wit et al., 2012), which uses the inverse model to compensate system dynamics, while model-predictive control (Camacho &

Original: Alba, 2013) and optimal control (Zhou et al., 1996) use the forward model to plan the control input.
Revised: Alba, 2013) and optimal control (Zhou et al., 1996) use the forward model to plan the control input.

Original: These models can be either derived from physics or learned from data.
Revised: These models can be either derived from physics or learned from data.

Original: The physics models must be derived for the individual system embodiment and requires precise knowledge of the physical properties (Albu-Schäﬀer, 2002).
Revised: The physics models must be derived for the individual system embodiment and requires precise knowledge of the physical properties (Albu-Schäﬀer, 2002).

Original: When learning the model 2 , mostly standard machine learning techniques are applied to ﬁt either the forward- or inverse-model to the training data.
Revised: When learning the model 2 , mostly standard machine learning techniques are applied to ﬁt either the forward- or inverse-model to the training data.

Original: E.g., authors used Linear Regression (Schaal et al., 2002; Haruno et al., 2001), Gaussian Mixture Regression (Calinon et al., 2010; Khansari-Zadeh & Billard, 2011), Gaussian Process Regression (Kocijan et al., 2004; Nguyen-Tuong et al., 2009; Nguyen-Tuong & Peters, 2010), Support Vector Regression (Choi et al., 2007; Ferreira et al., 2007), feedforward- (Jansen, 1994; Lenz et al., 2015; Ledezma &
Revised: E.g., authors used Linear Regression (Schaal et al., 2002; Haruno et al., 2001), Gaussian Mixture Regression (Calinon et al., 2010; Khansari-Zadeh & Billard, 2011), Gaussian Process Regression (Kocijan et al., 2004; Nguyen-Tuong et al., 2009; Nguyen-Tuong & Peters, 2010), Support Vector Regression (Choi et al., 2007; Ferreira et al., 2007), feedforward- (Jansen, 1994; Lenz et al., 2015; Ledezma &

Original: Haddadin, 2017; Sanchez-Gonzalez et al., 2018) or recurrent neural networks (Rueckert et al., 2017) to ﬁt the model to the observed measurements.
Revised: Haddadin, 2017; Sanchez-Gonzalez et al., 2018) or recurrent neural networks (Rueckert et al., 2017) to ﬁt the model to the observed measurements.

Original: Only few approaches incorporate prior knowledge into the learning problem.
Revised: Only few approaches incorporate prior knowledge into the learning problem.

Original: Sanchez-Gonzalez et al.
Revised: Sanchez-Gonzalez et al.

Original: (2018) use the graph representation of the kinematic structure as input.
Revised: (2018) use the graph representation of the kinematic structure as input.

Original: While the work of Atkeson et al.
Revised: While the work of Atkeson et al.

Original: (1986), commonly referenced as the standard system identiﬁcation technique for robot manipulators (Siciliano & Khatib, 2016), uses the Newton-Euler formalism to derive physics features using the kinematic structure and the joint measurements such that the learning of the dynamics model simpliﬁes to linear regression.
Revised: (1986), commonly referenced as the standard system identiﬁcation technique for robot manipulators (Siciliano & Khatib, 2016), uses the Newton-Euler formalism to derive physics features using the kinematic structure and the joint measurements such that the learning of the dynamics model simpliﬁes to linear regression.

Original: Similarly, Ledezma & Haddadin (2017) hard-code these physics features within a neural network and learn the dynamics parameters using gradient descent rather than linear regression.
Revised: Similarly, Ledezma & Haddadin (2017) hard-code these physics features within a neural network and learn the dynamics parameters using gradient descent rather than linear regression.

Original: Even though these physics features are derived from physics, the learned parameters for mass, center of gravity and inertia must not necessarily comply with physics as the learned parameters may violate the positive deﬁniteness of the inertia matrix or the parallel axis theorem (Ting et al., 2006).
Revised: Even though these physics features are derived from physics, the learned parameters for mass, center of gravity and inertia must not necessarily comply with physics as the learned parameters may violate the positive deﬁniteness of the inertia matrix or the parallel axis theorem (Ting et al., 2006).

Original: Furthermore, the linear regression is commonly underdetermined and only allows to infer linear combinations of the dynamics parameters and cannot be applied to close-loop kinematics (Siciliano & Khatib, 2016).
Revised: Furthermore, the linear regression is commonly underdetermined and only allows to infer linear combinations of the dynamics parameters and cannot be applied to close-loop kinematics (Siciliano & Khatib, 2016).

Original: DeLaN follows the line of structured learning problems but in contrast to previous approaches guarantees physical plausibility and provides a more general formulation.
Revised: DeLaN follows the line of structured learning problems but in contrast to previous approaches guarantees physical plausibility and provides a more general formulation.

Original: This general formulation enables DeLaN to learn the dynamics for any kinematic structure, including kinematic trees and closed-loop kinematics, and in addition does not require any knowledge about the kinematic structure.
Revised: This general formulation enables DeLaN to learn the dynamics for any kinematic structure, including kinematic trees and closed-loop kinematics, and in addition does not require any knowledge about the kinematic structure.

Original: Therefore, DeLaN is identical for all mechanical systems, which is in strong contrast to the NewtonEuler approaches, where the features are speciﬁc to the kinematic structure.
Revised: Therefore, DeLaN is identical for all mechanical systems, which is in strong contrast to the NewtonEuler approaches, where the features are speciﬁc to the kinematic structure.

Original: Only the system state and input is speciﬁc to the system but neither the network topology nor the optimization procedure.
Revised: Only the system state and input is speciﬁc to the system but neither the network topology nor the optimization procedure.

Original: The combination of diﬀerential equations and Neural Networks has previously been investigated in literature.
Revised: The combination of diﬀerential equations and Neural Networks has previously been investigated in literature.

Original: Early on Lagaris et al.
Revised: Early on Lagaris et al.

Original: (1998; 2000) proposed to learn the solution of partial diﬀerential equations (PDE) using neural networks and currently this topic is being rediscovered by Raissi &
Revised: (1998; 2000) proposed to learn the solution of partial diﬀerential equations (PDE) using neural networks and currently this topic is being rediscovered by Raissi &

Original: Karniadakis (2018); Sirignano & Spiliopoulos (2017); Long et al.
Revised: Karniadakis (2018); Sirignano & Spiliopoulos (2017); Long et al.

Original: Most research focuses on using machine learning to overcome the limitations of PDE solvers.
Revised: Most research focuses on using machine learning to overcome the limitations of PDE solvers.

Original: E.g., Sirignano & Spiliopoulos (2017) proposed the Deep Galerkin method to solve a high-dimensional PDE from scattered data.
Revised: E.g., Sirignano & Spiliopoulos (2017) proposed the Deep Galerkin method to solve a high-dimensional PDE from scattered data.

Original: Only the work of Raissi et al.
Revised: Only the work of Raissi et al.

Original: (2017) took the opposite standpoint of using the knowledge of the speciﬁc diﬀerential equation to structure the learning problem and achieve lower sample complexity.
Revised: (2017) took the opposite standpoint of using the knowledge of the speciﬁc diﬀerential equation to structure the learning problem and achieve lower sample complexity.

Original: In this paper, we follow the same motivation as Raissi et al.
Revised: In this paper, we follow the same motivation as Raissi et al.

Original: but take a diﬀerent approach.
Revised: but take a diﬀerent approach.

Original: Rather than explicitly solving the diﬀerential equation, DeLaN only uses the structure of the diﬀerential equation to guide the learning problem of inferring the equations of motion.
Revised: Rather than explicitly solving the diﬀerential equation, DeLaN only uses the structure of the diﬀerential equation to guide the learning problem of inferring the equations of motion.

Original: Thereby the diﬀerential equation is only implicitly solved.
Revised: Thereby the diﬀerential equation is only implicitly solved.

Original: In addition, the proposed approach uses diﬀerent encoding of the partial derivatives, which achieves the eﬃcient computation within a single feed-forward pass, enabling the application within control loops.
Revised: In addition, the proposed approach uses diﬀerent encoding of the partial derivatives, which achieves the eﬃcient computation within a single feed-forward pass, enabling the application within control loops.

Original:  Preliminaries: Lagrangian Mechanics
Revised:  Preliminaries: Lagrangian Mechanics

Original: Describing the equations of motion for mechanical systems has been extensively studied and various formalisms to derive these equations exist.
Revised: Describing the equations of motion for mechanical systems has been extensively studied and various formalisms to derive these equations exist.

Original: The most prominent are Newtonian-, Hamiltonianand Lagrangian-Mechanics.
Revised: The most prominent are Newtonian-, Hamiltonianand Lagrangian-Mechanics.

Original: Within this work Lagrangian Mechanics is used, more speciﬁcally the
Revised: Within this work Lagrangian Mechanics is used, more speciﬁcally the

Original: Euler-Lagrange formulation with non-conservative forces and generalized coordinates.
Revised: Euler-Lagrange formulation with non-conservative forces and generalized coordinates.

Original: 3 Generalized coordinates are coordinates that uniquely deﬁne the system conﬁguration.
Revised: 3 Generalized coordinates are coordinates that uniquely deﬁne the system conﬁguration.

Original: This formalism deﬁnes the Lagrangian L as a function of generalized coordinates q describing the complete dynamics of a given system.
Revised: This formalism deﬁnes the Lagrangian L as a function of generalized coordinates q describing the complete dynamics of a given system.

Original: The Lagrangian is not unique and every L which yields the correct equations of motion is valid.
Revised: The Lagrangian is not unique and every L which yields the correct equations of motion is valid.

Original: The Lagrangian is generally chosen to be
Revised: The Lagrangian is generally chosen to be

Original: [Equation]
Revised: [Equation]

Original: where T is the kinetic energy and V is the potential energy.
Revised: where T is the kinetic energy and V is the potential energy.

Original: The kinetic energy T can be computed for all choices of generalized coordinates using T = 12 (cid:219) q T H ( q )(cid:219) q , whereas H ( q ) is the symmetric and positive deﬁnite inertia matrix (de Wit et al., 2012).
Revised: The kinetic energy T can be computed for all choices of generalized coordinates using T = 1 2 (cid:219) q T H ( q )(cid:219) q , whereas H ( q ) is the symmetric and positive deﬁnite inertia matrix (de Wit et al., 2012).

Original: The positive deﬁniteness ensures that all non-zero velocities lead to positive kinetic energy.
Revised: The positive deﬁniteness ensures that all non-zero velocities lead to positive kinetic energy.

Original: Applying the calculus of variations yields the
Revised: Applying the calculus of variations yields the

Original: Euler-Lagrange equation with non-conservative forces described by
Revised: Euler-Lagrange equation with non-conservative forces described by

Original: [Equation]
Revised: [Equation]

Original: where τττ are generalized forces.
Revised: where τττ are generalized forces.

Original: Substituting L and dV / d q = g ( q ) into Equation 3 yields the second order ordinary diﬀerential equation (ODE) described by
Revised: Substituting L and dV / d q = g ( q ) into Equation 3 yields the second order ordinary diﬀerential equation (ODE) described by

Original: [Equation]
Revised: [Equation]

Original: [Table]
Revised: [Table]

Revised: c (

Original: [Equation]
Revised: [Equation]

Revised: , (cid:219)

Revised: [Equation]

Revised: [Equation]

Original: where c describes the forces generated by the Centripetal and Coriolis forces (Featherstone, 2007).
Revised: where c describes the forces generated by the Centripetal and Coriolis forces (Featherstone, 2007).

Original: Using this ODE any multi-particle mechanical system with holonomic constraints can be described.
Revised: Using this ODE any multi-particle mechanical system with holonomic constraints can be described.

Original: For example various authors  manually derived the equations of motion for coupled pendulums (Greenwood, 2006), robotic manipulators with ﬂexible joints (Book, 1984; Spong, 1987), parallel robots (Miller, 1992; Geng et al., 1992; Liu et al., 1993) or legged robots (Hemami & Wyman, 1979; Golliday & Hemami, 1977) using Equation 4.
Revised: For example various authors used this ODE to manually derived the equations of motion for coupled pendulums (Greenwood, 2006), robotic manipulators with ﬂexible joints (Book, 1984; Spong, 1987), parallel robots (Miller, 1992; Geng et al., 1992; Liu et al., 1993) or legged robots (Hemami & Wyman, 1979; Golliday & Hemami, 1977).

Original:  Incorporating Lagrangian Mechanics into Deep Learning
Revised:  Incorporating Lagrangian Mechanics into Deep Learning

Original: Starting from the Euler-Lagrange equation (Equation 4), traditional engineering approaches would estimate H ( q ) and g ( q ) from the approximated or measured masses, lengths and moments of inertia.
Revised: Starting from the Euler-Lagrange equation (Equation 4), traditional engineering approaches would estimate H ( q ) and g ( q ) from the approximated or measured masses, lengths and moments of inertia.

Original: On the contrary most traditional model learning approaches would ignore the structure and learn the inverse dynamics model directly from data.
Revised: On the contrary most traditional model learning approaches would ignore the structure and learn the inverse dynamics model directly from data.

Original: DeLaN bridges this gap by incorporating the structure introduced by the ODE into the learning problem and learns the parameters in an end-to-end fashion.
Revised: DeLaN bridges this gap by incorporating the structure introduced by the ODE into the learning problem and learns the parameters in an end-to-end fashion.

Original: More concretely, DeLaN approximates the inverse model by representing the unknown functions g ( q ) and H ( q ) as a feed-forward networks.
Revised: More concretely, DeLaN approximates the inverse model by representing the unknown functions g ( q ) and H ( q ) as a feed-forward networks.

Original: Rather than representing H ( q ) directly, the lower-triangular matrix L ( q ) is represented as deep network.
Revised: Rather than representing H ( q ) directly, the lower-triangular matrix L ( q ) is represented as deep network.

Original: Therefore, g ( q ) and H ( q ) are described by
Revised: Therefore, g ( q ) and H ( q ) are described by

Original: [Equation]
Revised: [Equation]

Original: whereˆ .
Revised: whereˆ .

Original: refers to an approximation and θ and ψ are the respective network parameters.
Revised: refers to an approximation and θ and ψ are the respective network parameters.

Original: The parameters θ and ψ can be obtained by minimizing the violation of the physical law described by Lagrangian
Revised: The parameters θ and ψ can be obtained by minimizing the violation of the physical law described by Lagrangian

Original: Mechanics.
Revised: Mechanics.

Original: Therefore, the optimization problem is described by
Revised: Therefore, the optimization problem is described by

Original: [Equation]
Revised: [Equation]

Original: ) with
Revised: with

Original: [Equation]
Revised: [Equation]

Original: where ˆ f − 1 is the inverse model and (cid:96) can be any diﬀerentiable loss function.
Revised: where ˆ f − 1 is the inverse model and (cid:96) can be any diﬀerentiable loss function.

Original: The computational graph of ˆ f − 1 is shown in Figure 1.
Revised: The computational graph of ˆ f − 1 is shown in Figure 1.

Original: Using this formulation one can conclude further properties of the learned model.
Revised: Using this formulation one can conclude further properties of the learned model.

Original: Neither ˆL nor ˆg are functions of (cid:219) q or (cid:220) q and, hence, the obtained parameters should, within limits, generalize to arbitrary velocities and accelerations.
Revised: Neither ˆL nor ˆg are functions of (cid:219) q or (cid:220) q and, hence, the obtained parameters should, within limits, generalize to arbitrary velocities and accelerations.

Original: In addition, the obtained model can be reformulated and used as a forward model.
Revised: In addition, the obtained model can be reformulated and used as a forward model.

Original: Solving Equation 6 for (cid:220) q yields the forward model described by
Revised: Solving Equation 6 for (cid:220) q yields the forward model described by

Original: [Equation]
Revised: [Equation]

Original: where ˆLˆL T is guaranteed to be invertible due to the positive deﬁnite constraint (Equation 7).
Revised: where ˆLˆL T is guaranteed to be invertible due to the positive deﬁnite constraint (Equation 7).

Original: However, solving the optimization problem of Equation 5 directly is not possible due to the ill-posedness of the
Revised: However, solving the optimization problem of Equation 5 directly is not possible due to the ill-posedness of the

Original: Lagrangian L not being unique.
Revised: Lagrangian L not being unique.

Original: The Euler-Lagrange equation is invariant to linear transformation and, hence, the Lagrangian L (cid:48) =
Revised: The Euler-Lagrange equation is invariant to linear transformation and, hence, the Lagrangian L (cid:48) =

Original: α L + β solves the Euler-Lagrange equation if α is non-zero and
Revised: α L + β solves the Euler-Lagrange equation if α is non-zero and

Original: L is a valid Lagrangian.
Revised: L is a valid Lagrangian.

Original: This problem can be mitigated by adding an additional penalty term to
Revised: This problem can be mitigated by adding an additional penalty term to

Original: Equation 5 described by
Revised: Equation 5 described by

Original: [Equation]
Revised: [Equation]

Original: (9) where Ω is the L 2 -norm of the network weights
Revised: where Ω is the L 2 -norm of the network weights.

Original: Solving the optimization problem of Equation 9 with a gradient based end-to-end learning approach is non-trivial due to the positive deﬁnite constraint (Equation 7) and the derivatives contained in ˆ f − 1 .
Revised: Solving the optimization problem of Equation 9 with a gradient based end-to-end learning approach is non-trivial due to the positive deﬁnite constraint (Equation 7) and the derivatives contained in ˆ f − 1 .

Original: In particular, d ( LL T )/ dt and ∂ (cid:0) (cid:219) q T LL T (cid:219) q (cid:1) /
Revised: In particular, d ( LL T )/ dt and ∂ (cid:0) (cid:219) q T LL T (cid:219) q (cid:1) /

Original: ∂
Revised: ∂

Original: q i cannot be computed using automatic diﬀerentiation as t is not an input of the network and most implementations of automatic diﬀerentiation do not allow the backpropagation of the gradient through the computed derivatives.
Revised: q i cannot be computed using automatic diﬀerentiation as t is not an input of the network and most implementations of automatic diﬀerentiation do not allow the backpropagation of the gradient through the computed derivatives.

Original: Therefore, the derivatives contained in ˆ f − 1 must be computed analytically to exploit the full gradient information for training of the parameters.
Revised: Therefore, the derivatives contained in ˆ f − 1 must be computed analytically to exploit the full gradient information for training of the parameters.

Original: In the following we introduce a network structure that fulﬁlls the positive-deﬁnite constraint for all parameters (Section 4.1), prove that the derivatives d ( LL T )/ dt and ∂ (cid:0) (cid:219) q T LL T (cid:219) q (cid:1) /
Revised: In the following we introduce a network structure that fulﬁlls the positive-deﬁnite constraint for all parameters (Section 4.1), prove that the derivatives d ( LL T )/ dt and ∂ (cid:0) (cid:219) q T LL T (cid:219) q (cid:1) /

Original: ∂
Revised: ∂

Original: q i can be computed analytically (Section 4.2) and show an eﬃcient implementation for computing the derivatives using a single feed-forward pass (Section 4.3).
Revised: q i can be computed analytically (Section 4.2) and show an eﬃcient implementation for computing the derivatives using a single feed-forward pass (Section 4.3).

Original: Using these three properties the resulting network architecture can be used within a real-time control loop and trained using standard end-to-end optimization techniques.# $ ! ! % & $ ' "$
Revised: Using these three properties the resulting network architecture can be used within a real-time control loop and trained using standard end-to-end optimization techniques.

Original: H $ % ̇% ̈% *

Original: [-ReLuNetwork Physics Transformations 0 0 0 ∗ 0∗ ∗- . - = " + + + ∗ 0 0 
 ∗0 0 ∗ + + b
 Linear Network-]
Revised: [-ReLuNetwork Physics Transformations 0 0 0 ∗ 0∗ ∗- . - = " + + + ∗ 0 0 
 ∗0 0 ∗ + + b
 Linear Network-][Figure]

Original: 4.1 Symmetry and Positive Definiteness of H
Revised: 4.1 Symmetry and Positive Definiteness of H

Original: Ensuring the symmetry and positive deﬁniteness of H is essential as this constraint enforces positive kinetic energy for all non-zero velocities.
Revised: Ensuring the symmetry and positive deﬁniteness of H is essential as this constraint enforces positive kinetic energy for all non-zero velocities.

Original: In addition, the positive deﬁniteness ensures that H is invertible and the obtained model can be used as forward model.
Revised: In addition, the positive deﬁniteness ensures that H is invertible and the obtained model can be used as forward model.

Original: By representing the matrix H as the product of a lower-triangular matrix the symmetry and the positive semi-deﬁniteness is ensured while simultaneously reducing the number of parameters.
Revised: By representing the matrix H as the product of a lower-triangular matrix the symmetry and the positive semi-deﬁniteness is ensured while simultaneously reducing the number of parameters.

Original: The positive deﬁniteness is obtained if the diagonal of L is positive.
Revised: The positive deﬁniteness is obtained if the diagonal of L is positive.

Original: This positive diagonal also guarantees that L is invertible.
Revised: This positive diagonal also guarantees that L is invertible.

Original: Using a deep network with diﬀerent heads and altering the activation of the output layer one can obtain a positive diagonal.
Revised: Using a deep network with diﬀerent heads and altering the activation of the output layer one can obtain a positive diagonal.

Original: The oﬀ-diagonal elements L o use a linear activation while the diagonal elements L d use a non-negative activation, e.g., ReLu or Softplus.
Revised: The oﬀ-diagonal elements L o use a linear activation while the diagonal elements L d use a non-negative activation, e.g., ReLu or Softplus.

Original: In addition, a positive scalar b is added to diagonal elements.
Revised: In addition, a positive scalar b is added to diagonal elements.

Original: Thereby, lower bounding the  eigenvalues of Hby b 2 .
Revised: Thereby, ensuring a positive diagonal of L and the positive eigenvalues of H .

Original: In addition, we chose to share parameters between L and g as both rely on the same physical embodiment.
Revised: In addition, we chose to share parameters between L and g as both rely on the same physical embodiment.

Original: The network architecture, with three-heads representing the diagonal l d and oﬀ-diagonal l o entries of L and g , is shown in Figure 1.
Revised: The network architecture, with three-heads representing the diagonal l d and oﬀ-diagonal l o entries of L and g , is shown in Figure 1.

Original: 4.2 Deriving the derivatives
Revised: 4.2 Deriving the derivatives

Original: The derivatives d (cid:0) LL T (cid:1) / dt and ∂ (cid:0) (cid:219) q T LL T (cid:219) q (cid:1) /
Revised: The derivatives d (cid:0) LL T (cid:1) / dt and ∂ (cid:0) (cid:219) q T LL T (cid:219) q (cid:1) /

Original: ∂
Revised: ∂

Original: q i are required for computing the control signal τττ using the inverse model and, hence, must be available within the forward pass.
Revised: q i are required for computing the control signal τττ using the inverse model and, hence, must be available within the forward pass.

Original: In addition, the second order derivatives, used within the backpropagation of the gradients, must exist to train the network using end-to-end training.
Revised: In addition, the second order derivatives, used within the backpropagation of the gradients, must exist to train the network using end-to-end training.

Original: To enable the computation of the second order derivatives using automatic diﬀerentiation the forward computation must be performed analytically.
Revised: To enable the computation of the second order derivatives using automatic diﬀerentiation the forward computation must be performed analytically.

Original: Both derivatives, d (cid:0) LL T (cid:1) / dt and ∂ (cid:0) (cid:219) q T LL T (cid:219) q (cid:1) /
Revised: Both derivatives, d (cid:0) LL T (cid:1) / dt and ∂ (cid:0) (cid:219) q T LL T (cid:219) q (cid:1) /

Original: ∂
Revised: ∂

Original: q i , have closed form solutions and can be derived by ﬁrst computing the respective derivative of L and second substituting the reshaped derivative of the vectorized form l .
Revised: q i , have closed form solutions and can be derived by ﬁrst computing the respective derivative of L and second substituting the reshaped derivative of the vectorized form l .

Original: For the temporal derivative d (cid:0) LL T (cid:1) / dt this yields
Revised: For the temporal derivative d (cid:0) LL T (cid:1) / dt this yields

Original: [Equation]
Revised: [Equation]

Original:  whereas d L / dt can be substituted with the reshaped form of
Revised: (10) whereas d L / dt can be substituted with the reshaped form of

Original: [Equation]
Revised: [Equation]

Original:  where i refers to the i -th network layer consisting of an aﬃne transformation and the non-linearity g 
Revised: (11) where i refers to the i -th network layer consisting of an aﬃne transformation and the non-linearity g , i.e., h i

Original: , i.e., h i =
Revised: =

Original: g i (cid:0) W Ti h
Revised: g i (cid:0) W T i h

Original: i − 

Revised: i −

Original: + b i (cid:1) .
Revised: + b i (cid:1) .

Original: Equation 11 can be simpliﬁed as the network weights W i and biases
Revised: Equation 11 can be simpliﬁed as the network weights W i and biases

Original: b iare time-invariant, i.e., d W i / dt = 0 and d b i / dt = 0. Therefore, d l / dt is described by [Equation] Due to the compositional structure of the network and the diﬀerentiability of the non-linearity, the derivative with respect to the network input d l / d q can be computed by recursively applying the chain rule, i.e.,
Revised: b i

Original: [Equation]

Original: (b) ∗ 0 0 ∗ ∗ 0 ∗ ∗ ∗ ∗ 0 0 ∗ ∗ 0 ∗ ∗ ∗ 0 ∗ 0 0 ∗ 0 0 ∗ 0 W 1 ba# $#$%
Revised: (b) ∗ 0 0 ∗ ∗ 0 ∗ ∗ ∗ ∗ 0 0 ∗ ∗ 0 ∗ ∗ ∗ 0 ∗ 0 0 ∗ 0 0 ∗ 0 W 1 ba# $#$%

Original: W 0 ba∗ 0 0 ∗ ∗ 0 ∗ ∗ ∗ $# $% ̇% % ̇% ' ('(% ) $'$* (a)
Revised: W 0 ba∗ 0 0 ∗ ∗ 0 ∗ ∗ ∗ $# $% ̇% % ̇% ' ('(% ) $'$* (a)

Original: W i b i a i MatMul
Revised: W i b i a i MatMul

Original: Add g i
Revised: Add g i

Original: MatMul diag( a i ) g
Revised: MatMul diag( a i ) g

Revised: ‘ i +, - +, -./ , - , -./ are time-invariant, i.e., d W i / dt = 0 and d b i / dt = 0.

Revised: Therefore, d l / dt is described by

Revised: [Equation]

Revised: Due to the compositional structure of the network and the diﬀerentiability of the non-linearity, the derivative with respect to the network input d l / d q can be computed by recursively applying the chain rule, i.e.,

Original: ‘ i +, - +, -./ , - , -./ where g (cid:48) is the derivative of the non-linearity.
Revised: [Equation] (13) where g (cid:48) is the derivative of the non-linearity.

Original: Similarly to the previous derivation, the partial derivative of the quadratic term can be computed using the chain rule, which yields∂
Revised: Similarly to the previous derivation, the partial derivative of the quadratic term can be computed using the chain rule, which yields

Original: [Equation]
Revised: [Equation]

Original: (14) whereas ∂ L / ∂ q i can be constructed using the columns of previously derived ∂ l / ∂ q .
Revised: whereas ∂ L / ∂ q i can be constructed using the columns of previously derived ∂ l / ∂ q .

Original: Therefore, all derivatives included within ˆ f can be computed in closed form.
Revised: Therefore, all derivatives included within ˆ f can be computed in closed form.

Original: 4.3 Computing the Derivatives
Revised: 4.3 Computing the Derivatives

Original: The derivatives of Section 4.2 must be computed within a real-time control loop and only add minimal computational complexity in order to not break the real-time constraint.
Revised: The derivatives of Section 4.2 must be computed within a real-time control loop and only add minimal computational complexity in order to not break the real-time constraint.

Original: l and ∂ l / ∂ q , required within Equation 10 and Equation 14, can be simultaneously computed using an extended standard layer.
Revised: l and ∂ l / ∂ q , required within Equation 10 and Equation 14, can be simultaneously computed using an extended standard layer.

Original: Extending the aﬃne transformation and non-linearity of the standard layer with an additional sub-graph for computing ∂ h
Revised: Extending the aﬃne transformation and non-linearity of the standard layer with an additional sub-graph for computing ∂ h

Original: i / ∂ h
Revised: i / ∂ h

Original: i − 1 yields the Lagrangian layer described by
Revised: i − 1 yields the Lagrangian layer described by

Original: [Equation]
Revised: [Equation]

Original: The computational graph of the Lagrangian layer is shown in Figure 2a.
Revised: The computational graph of the Lagrangian layer is shown in Figure 2a.

Original: Chaining the Lagrangian layer yields the compositional structure of ∂ l / ∂ q (Equation 13) and enables the eﬃcient computation of ∂ l / ∂ q .
Revised: Chaining the Lagrangian layer yields the compositional structure of ∂ l / ∂ q (Equation 13) and enables the eﬃcient computation of ∂ l / ∂ q .

Original: Additional reshaping operations compute d L / dt and ∂ L / ∂ q i .
Revised: Additional reshaping operations compute d L / dt and ∂ L / ∂ q i .

Original:  Experimental Evaluation: Learning an Inverse Dynamics Model for Robot Control
Revised:  Experimental Evaluation: Learning an Inverse Dynamics Model for Robot Control

Original: To demonstrate the applicability and extrapolation of DeLaN, the proposed network topology is applied to model-based control for a simulated 2-dof robot (Figure 3b) and the physical 7-dof robot 
Revised: To demonstrate the applicability and extrapolation of DeLaN, the proposed network topology is applied to model-based control for a simulated 2-dof robot (Figure 3b) and the physical 7-dof robot [Figure]

Original: Barrett WAM (Figure 3d).
Revised: Barrett WAM (Figure 3d).

Original: The performance of DeLaN is evaluated using the tracking error on train and test trajectories and compared to a learned and analytic model.
Revised: The performance of DeLaN is evaluated using the tracking error on train and test trajectories and compared to a learned and analytic model.

Original: This evaluation scheme follows existing work (Nguyen-Tuong et al., 2009; Sanchez-Gonzalez et al., 2018) as the tracking error is the relevant performance indicator while the mean squared error (MSE)  obtained using sample based optimization exaggerates model performance (Hobbs & Hepenstal, 1989).
Revised: This evaluation scheme follows existing work (Nguyen-Tuong et al., 2009; Sanchez-Gonzalez et al., 2018) as the tracking error is the relevant performance indicator while the mean squared error (MSE) 4 obtained using sample based optimization exaggerates model performance (Hobbs & Hepenstal, 1989).

Original: In addition to most previous work, we strictly limit all model predictions to real-time and perform the learning online, i.e., the models are randomly initialized and must learn the model during the experiment.
Revised: In addition to most previous work, we strictly limit all model predictions to real-time and perform the learning online, i.e., the models are randomly initialized and must learn the model during the experiment.

Original: Experimental Setup
Revised: Experimental Setup

Original: Within the experiment the robot executes multiple desired trajectories with speciﬁed joint positions, velocities and accelerations.
Revised: Within the experiment the robot executes multiple desired trajectories with speciﬁed joint positions, velocities and accelerations.

Original: The control signal, consisting of motor torques, is generated using[Figure] a non-linear feedforward controller, i.e., a low gain PD-Controller augmented with a feed-forward torque τττ f f to compensate system dynamics.
Revised: The control signal, consisting of motor torques, is generated using a non-linear feedforward controller, i.e., a low gain PD-Controller augmented with a feed-forward torque τττ f f to compensate system dynamics.

Original: The control law is described by
Revised: The control law is described by

Original: [Equation]
Revised: [Equation]

Original:  with 
Revised: f f with τττ f f

Original: [Equation] where K p , K d are the controller gains and q d , (cid:219) q d , (cid:220) 
Revised: = ˆ f − 1 ( q d , (cid:219) q d , (cid:220) q d ) where K p , K d are the controller gains and q d , (cid:219) q d , (cid:220) q

Original: q d the desired joint positions, velocities and accelerations.
Revised: d the desired joint positions, velocities and accelerations.

Original: The control-loop is shown in Figure 3a.
Revised: The control-loop is shown in Figure 3a.

Original: For all experiments the control frequency is set to 500Hz while the desired joint state and respectively τττ f f is updated with a frequency of f d = 200Hz.
Revised: For all experiments the control frequency is set to 500Hz while the desired joint state and respectively τττ f f is updated with a frequency of f d = 200Hz.

Original: All feed-forward torques are computed online and, hence, the computation time is strictly limited to T ≤ 1 / 200s.
Revised: All feed-forward torques are computed online and, hence, the computation time is strictly limited to T ≤ 1 / 200s.

Original: The tracking performance is deﬁned as the sum of the MSE evaluated at the sampling points of the reference trajectory.
Revised: The tracking performance is deﬁned as the sum of the MSE evaluated at the sampling points of the reference trajectory.

Original: For the desired trajectories two diﬀerent data sets are used.
Revised: For the desired trajectories two diﬀerent data sets are used.

Original: The ﬁrst data set contains all single stroke characters 4 while the second data set uses cosine curves in joint space (Figure 3c).
Revised: The ﬁrst data set contains all single stroke characters 5 while the second data set uses cosine curves in joint space (Figure 3c).

Original: The 20 characters are spatially and temporally re-scaled to comply with the robot kinematics.
Revised: The 20 characters are spatially and temporally re-scaled to comply with the robot kinematics.

Original: The joint references are computed using the inverse kinematics.
Revised: The joint references are computed using the inverse kinematics.

Original: Due to the diﬀerent characters, the desired trajectories contain smooth and sharp turns and cover a wide variety of diﬀerent shapes but are limited to a small task space region.
Revised: Due to the diﬀerent characters, the desired trajectories contain smooth and sharp turns and cover a wide variety of diﬀerent shapes but are limited to a small task space region.

Original: In contrast, the cosine trajectories are smooth but cover a large task space region.
Revised: In contrast, the cosine trajectories are smooth but cover a large task space region.

Original: Baselines
Revised: Baselines

Original: The performance of DeLaN is compared to an analytic inverse dynamics model, a standard feedforward neural network (FF-NN) and a PD-Controller.
Revised: The performance of DeLaN is compared to an analytic inverse dynamics model, a standard feedforward neural network (FF-NN) and a PD-Controller.

Original: For the analytic models the torque is computed using the Recursive Newton-Euler algorithm (RNE) (Luh et al., 1980), which computes the feedforward torque using estimated physical properties of the system, i.e. the link dimensions, masses and moments of inertia.
Revised: For the analytic models the torque is computed using the Recursive Newton-Euler algorithm (RNE) (Luh et al., 1980), which computes the feedforward torque using estimated physical properties of the system, i.e. the link dimensions, masses and moments of inertia.

Original: For implementations the open-source library PyBullet (Coumans & Bai, 2016–2018) is used.
Revised: For implementations the open-source library PyBullet (Coumans & Bai, 2016–2018) is used.

Original: Both  networks use the same dimensionality and ReLu nonlinearities. The FF-NN andDeLaN must learn the system dynamics online starting from random initialization.
Revised: Both deep networks use the same dimensionality, ReLu nonlinearities and must learn the system dynamics online starting from random initialization.

Original: The training samples containing joint states and applied torques ( q , (cid:219) q , (cid:220) q , τττ ) 0 ,... T are directly read from the control loop as shown in Figure 3a.
Revised: The training samples containing joint states and applied torques ( q , (cid:219) q , (cid:220) q , τττ ) 0 ,... T are directly read from the control loop as shown in Figure 3a.

Original: The training runs in a separate process on the same machine and solves the optimization problem online.

Original: Once the training process computed a new model, the inverse model ˆ f − 1 of the control loop is updated.

Original: 5.1 Simulated Robot Experiments

Original: The 2-dof robot shown in Figure 3b is simulated using PyBullet and executes the character and cosine trajectories. Figure 4 shows the ground truth torques of the characters ’a’, ’d’, ’e’, the torque ground truth components and the learned decomposition using DeLaN (Figure 4a-d). Even though DeLaN is trained on the super-imposed torques, DeLaN learns to disambiguate the inertial force
Revised: d a e

Original: H (cid:220) q , the Coriolis and Centrifugal force c ( q , (cid:219) q ) and the gravitational force g ( q ) as the respective curves overlap closely.

Original: Hence, DeLaN is capable of learning the underlying physical model using the proposed network topology trained with standard end-to-end optimization.

Original: Figure 4d shows d a e

Original: T o r qu e
Revised: T o r qu e

Original: [ N m ]
Revised: [ N m ]

Original: J o i n tτ
Revised: J o i n tτ

Original: DeLaNGroundTruth d a e − 1 .− 0 .0 .0 .
Revised: DeLaNGroundTruth d a e − 1 .− 0 .0 .0 .

Original: T o r qu e
Revised: T o r qu e

Original: [ N m ]
Revised: [ N m ]

Original: J o i n t(a)
Revised: J o i n t(a)

Original: d a e −
Revised: d a e −

Original: T o r qu e
Revised: T o r qu e

Original: [ N m ]
Revised: [ N m ]

Original: H ( q )¨ q d a e − 0 .
Revised: H ( q )¨ q d a e − 0 .

Original: 75 − 0 .− 0 .
Revised: 75 − 0 .− 0 .

Original: T o r qu e
Revised: T o r qu e

Original: [ N m ] (b) d a e − 0 .0 .0 .0 .0 .
Revised: [ N m ] (b) d a e − 0 .0 .0 .0 .0 .

Original: T o r qu e
Revised: T o r qu e

Original: [ N m ] c ( q , ˙ q ) d a e 0 .0 .0 .0 .
Revised: [ N m ] c ( q , ˙ q ) d a e 0 .0 .0 .0 .

Original: T o r qu e
Revised: T o r qu e

Original: [ N m ] (c) d a e 0 .0 .
Revised: [ N m ] (c) d a e 0 .0 .

Original: T o r qu e
Revised: T o r qu e

Original: [ N m ] g ( q ) d a e − 0 .
Revised: [ N m ] g ( q ) d a e − 0 .

Original: 4 − 0 .0 . 

Revised: 4 − 0 .0 .

Original: T o r qu e
Revised: T o r qu e

Original: [ N m ] (d) 12 4 6 8 10 12 14 16 18 20 10 −10 −10 −

Revised: [ N m ] (d) 12 4 6 8 10 12 14 16 18 20 10 −10 −10 −

Original: M e a n S qu a r e d
Revised: M e a n S qu a r e d

Original: E rrr o r
Revised: E rrr o r

Original: OﬄineTestingError
Revised: OﬄineTestingError

Original: DeLaNFF-NN 12 4 6 8 10 12 14 16 18 20 TrainCharacters 10 −10 −10 −

Revised: DeLaNFF-NN 12 4 6 8 10 12 14 16 18 20 TrainCharacters 10 −10 −10 −

Original: M e a n S qu a r e d
Revised: M e a n S qu a r e d

Revised: E rr o r (e) the inertial force H (cid:220) q (b), the Coriolis and Centrifugal forces c ( q , (cid:219) q ) (c) and the gravitational force g ( q ) (d).

Revised: All components match closely the ground truth data.

Original: E rr o r (e)  the oﬄine MSE on the test set averaged over multiple seeds for  the FF-NN and DeLaN w.r.t.
Revised: (e) shows the oﬄine MSE of the feed-forward neural network and DeLaN for each joint. The training runs in a separate process on the same machine and solves the optimization problem online. Once the training process computed a new model, the inverse model ˆ f − 1 of the control loop is updated. 5.1 Simulated Robot Experiments

Revised: The 2-dof robot shown in Figure 3b is simulated using PyBullet and executes the character and cosine trajectories.

Revised: Figure 4 shows the ground truth torques of the characters ’a’, ’d’, ’e’, the torque ground truth components and the learned decomposition using DeLaN (Figure 4a-d).

Revised: Even though

Revised: DeLaN is trained on the super-imposed torques, DeLaN learns to disambiguate the inertial force

Revised: H (cid:220) q , the Coriolis and Centrifugal force c ( q , (cid:219) q ) and the gravitational force g ( q ) as the respective curves overlap closely.

Revised: Hence, DeLaN is capable of learning the underlying physical model using the proposed network topology trained with standard end-to-end optimization.

Revised: Figure 4d shows the oﬄine MSE on the test set averaged over multiple seeds for the FF-NN and DeLaN w.r.t.

Original: to diﬀerent training set sizes.
Revised: to diﬀerent training set sizes.

Original: The diﬀerent training set sizes correspond to the combination of n random characters, i.e., a training set size of 1 corresponds to training the model on a single character and evaluating the performance on the remaining 19 characters.
Revised: The diﬀerent training set sizes correspond to the combination of n random characters, i.e., a training set size of 1 corresponds to training the model on a single character and evaluating the performance on the remaining 19 characters.

Original: DeLaN clearly obtains a lower test MSE compared to the FF-NN.
Revised: DeLaN clearly obtains a lower test MSE compared to the FF-NN.

Original: Especially the diﬀerence in performance increases when the training set is reduced.
Revised: Especially the diﬀerence in performance increases when the training set is reduced.

Original: This increasing diﬀerence on the test MSE highlights the reduced sample complexity and the good extrapolation to unseen samples.
Revised: This increasing diﬀerence on the test MSE highlights the reduced sample complexity and the good extrapolation to unseen samples.

Original: This diﬀerence in performance is ampliﬁed on the real-time control-task where the models are learned online starting from random initialization.
Revised: This diﬀerence in performance is ampliﬁed on the real-time control-task where the models are learned online starting from random initialization.

Original: Figure 5a and b shows the accumulated tracking error per testing character and the testing error averaged over all test characters while Figure 5c shows the qualitative comparison of the control performance 5 .
Revised: Figure 5a and b shows the accumulated tracking error per testing character and the testing error averaged over all test characters while Figure 5c shows the qualitative comparison of the control performance 6 .

Original: It is important to point out that all shown results are averaged over multiple seeds and only incorporate characters not used for training and, hence, focus the evaluation on the extrapolation to new trajectories.
Revised: It is important to point out that all shown results are averaged over multiple seeds and only incorporate characters not used for training and, hence, focus the evaluation on the extrapolation to new trajectories.

Original: The qualitative comparison shows that DeLaN is able to execute all 20 characters when trained on 8 random characters.
Revised: The qualitative comparison shows that DeLaN is able to execute all 20 characters when trained on 8 random characters.

Original: The obtained tracking error is comparable to the analytic model, which in this case contains the simulation parameters and is optimal.
Revised: The obtained tracking error is comparable to the analytic model, which in this case contains the simulation parameters and is optimal.

Original: In contrast, the FF-NN shows signiﬁcant deviation from the desired trajectories when trained on 8 random characters.
Revised: In contrast, the FF-NN shows signiﬁcant deviation from the desired trajectories when trained on 8 random characters.

Original: The quantitative comparison of the accumulated tracking error over seeds (Figure 5b) shows that DeLaN obtains lower tracking error on all training set sizes compared to the FF-NN.
Revised: The quantitative comparison of the accumulated tracking error over seeds (Figure 5b) shows that DeLaN obtains lower tracking error on all training set sizes compared to the FF-NN.

Original: This good performance using only few training characters shows that DeLaN has a lower sample complexity and better extrapolation to unseen trajectories compared to the FF-NN.
Revised: This good performance using only few training characters shows that DeLaN has a lower sample complexity and better extrapolation to unseen trajectories compared to the FF-NN.

Original: Figure 6a and b show the performance on the cosine trajectories.
Revised: Figure 6a and b show the performance on the cosine trajectories.

Original: For this experiment the models are only trained online on two trajectories with a velocity scale of 1x.
Revised: For this experiment the models are only trained online on two trajectories with a velocity scale of 1x.

Original: To assess the extrapolation w.r.t.
Revised: To assess the extrapolation w.r.t.

Original: velocities and accelerations the learned models are tested on the same trajectories with scaled velocities (gray area of Figure 6).
Revised: velocities and accelerations the learned models are tested on the same trajectories with scaled velocities (gray area of Figure 6).

Original: On the training trajectories DeLaN and the FF-NN performcomparable.
Revised: On the training trajectories DeLaN and the FF-NN perform

Original: When the velocities are increased the performance of FF-NN deteriorates because the new trajectories do not lie within the vicinity of the training distribution as the domain of the FF-NN is deﬁned as ( q , (cid:219) q , (cid:220) q ) .

Original: Therefore, FF-NN cannot extrapolate to the testing data.

Original: In contrast, the domain of the networks ˆL and ˆg composing DeLaN only consist of q , rather than ( q , (cid:219) q , (cid:220) q ) .

Original: This reduced domain enables DeLaN, within limit, to extrapolate to the test trajectories.

Original: The increase in tracking error is caused by the structure of ˆ f − 1 , where model errors to scale quadratic with velocities.

Original: However, the obtained tracking error on the testing trajectories is signiﬁcantly lower compared to

Original: FF-NN.

Original: 5.2 Physical Robot Experiments

Original: For physical experiments the desired trajectories are executed on the Barrett WAM, a robot with direct cable drives.

Original: The direct cable drives produce high torques generating fast and dexterous

Original: RNE
Revised: RNE

Original: PD-Controller FF-NNn=
Revised: PD-Controller FF-NNn=

Original: FF-NNn= FF-NNn=
Revised: FF-NNn= FF-NNn=

Original: FF-NNn= (c)
Revised: FF-NNn= (c)

Original: DeLaNn= DeLaNn=
Revised: DeLaNn= DeLaNn=

Original: DeLaNn=
Revised: DeLaNn=

Original: DeLaNn= 1  8 10 12 14 16 18 20 TrainCharacters 10 −

Revised: DeLaNn= 1  8 10 12 14 16 18 20 TrainCharacters 10 −

Original: A cc u m u l a t e d T r a c k
Revised: A cc u m u l a t e d T r a c k

Original: i n g E rr o r (b)
Revised: i n g E rr o r (b)

Original: TrackingError
Revised: TrackingError

Original: FF-NNDeLaNRNEPD -Controller
Revised: FF-NNDeLaNRNEPD -Controller

Original: a b c d e g h l m n o p q r s u v w y z 10 −10 −10 0 1010 2 

Revised: a b c d e g h l m n o p q r s u v w y z 10 −10 −10 0 1010 2

Original: T r a c k i n g E rr o r (a)
Revised: T r a c k i n g E rr o r (a)

Original: MLP
Revised: MLP

Original: Lagrangian
Revised: Lagrangian

Original: RNE
Revised: RNE

Revised: PD-Controller comparable.

Revised: When the velocities are increased the performance of FF-NN deteriorates because the new trajectories do not lie within the vicinity of the training distribution as the domain of the FF-NN is deﬁned as ( q , (cid:219) q , (cid:220) q ) .

Revised: Therefore, FF-NN cannot extrapolate to the testing data.

Revised: In contrast, the domain of the networks ˆL and ˆg composing DeLaN only consist of q , rather than ( q , (cid:219) q , (cid:220) q ) .

Revised: This reduced domain enables DeLaN, within limit, to extrapolate to the test trajectories.

Original: PD-Controller movements but yield complex dynamics, which cannot be modelled using rigid-body dynamics.
Revised: The increase in tracking error is caused by the structure of ˆ f − 1 , where model errors to scale quadratic with velocities. However, the obtained tracking error on the testing trajectories is signiﬁcantly lower compared to FF-NN. 5.2 Physical Robot Experiments

Revised: For physical experiments the desired trajectories are executed on the Barrett WAM, a robot with direct cable drives.

Revised: The direct cable drives produce high torques generating fast and dexterous movements but yield complex dynamics, which cannot be modelled using rigid-body dynamics due to the variable stiﬀness and lengths of the cables 7 .

Original: Therefore, the Barrett WAM is ideal for testing the applicability of model learning and analytic models 6 on complex dynamics.
Revised: Therefore, the Barrett WAM is ideal for testing the applicability of model learning and analytic models 8 on complex dynamics.

Original: For the physical experiments we focus on the cosine trajectories as these trajectories produce dynamic movements while character trajectories are mainly dominated by the gravitational forces.
Revised: For the physical experiments we focus on the cosine trajectories as these trajectories produce dynamic movements while character trajectories are mainly dominated by the gravitational forces.

Revised: In addition, only the dynamics of the four lower joints are learned because these joints dominate the dynamics and the upper joints cannot be suﬃciently excited to retrieve the dynamics parameters.

Original: Figure 6c and d show the tracking error on the cosine trajectories using thephysical and the simulated 
Revised: Figure 6c and d show the tracking error on the cosine trajectories using the the simulated Barrett

Original:  Barrett WAM.
Revised: WAM while Figure 6e and f show the tracking error of the physical Barrett WAM.

Original: It is important to note, that the simulation only simulates the rigid-body dynamics not including the direct cables drives and the simulation parameters are inconsistent with the parameters of the analytic model.
Revised: It is important to note, that the simulation only simulates the rigid-body dynamics not including the direct cables drives and the simulation parameters are inconsistent with the parameters of the analytic model.

Original: Therefore, the analytic model is not optimal.
Revised: Therefore, the analytic model is not optimal.

Original: On the training trajectories executed on the physical system the FF-NN performs better compared to DeLaN and the analytic model.
Revised: On the training trajectories executed on the physical system the FF-NN performs better compared to DeLaN and the analytic model.

Original: DeLaN achieves slightly better tracking error than the analytic model, which uses the same rigid-body assumptions as DeLaN. That shows DeLaN can learn a dynamics model of the WAM but is limited by the model assumptions of Lagrangian Mechanics.
Revised: DeLaN achieves slightly better tracking error than the analytic model, which uses the same rigid-body assumptions as DeLaN. That shows DeLaN can learn a dynamics model of the WAM but is limited by the model assumptions of Lagrangian Mechanics.

Original: These assumptions cannot represent the dynamics of thecable drives.
Revised: These assumptions cannot represent the dynamics of the

Original: When comparing to the simulated results, DeLaN and the FF-NN perform comparable but signiﬁcantly better than the analytic model.
Revised: [Figure]

Original: These simulation results show that DeLaN can learn an accurate model of the WAM, when the underlying assumptions of the physics prior hold.
Revised: accurate model of the WAM, when the underlying assumptions of the physics prior hold.

Original: The tracking performance on the physical system and the simulation indicate that DeLaN can learn a model within the model class of the physics prior but also inherits the limitations of the physics prior.
Revised: The tracking performance on the physical system and the simulation indicate that DeLaN can learn a model within the model class of the physics prior but also inherits the limitations of the physics prior.

Original: For this speciﬁc experiment the FF-NN can locally learn correlations of the torques w.r.t.
Revised: For this speciﬁc experiment the FF-NN can locally learn correlations of the torques w.r.t.

Original: q , (cid:219) q and (cid:220) q while such correlation cannot be represented by the network topology of DeLaN because such correlation should, by deﬁnition of the physics prior, not exist.
Revised: q , (cid:219) q and (cid:220) q while such correlation cannot be represented by the network topology of DeLaN because such correlation should, by deﬁnition of the physics prior, not exist.

Original: When extrapolating to the identical trajectories with higher velocities (gray area of Figure 6) the tracking error of the FF-NN deteriorates much faster compared to DeLaN, because the FF-NN overﬁts to the training data.
Revised: When extrapolating to the identical trajectories with higher velocities (gray area of Figure 6) the tracking error of the FF-NN deteriorates much faster compared to DeLaN, because the FF-NN overﬁts to the training data.

Original: The tracking error of the analytic model remains constant and demonstrates the guaranteed extrapolation of the analytic models.
Revised: The tracking error of the analytic model remains constant and demonstrates the guaranteed extrapolation of the analytic models.

Original: When comparing the simulated results, the
Revised: When comparing the simulated results, the

Original: FF-NN cannot extrapolate to the new velocities and the tracking error deteriorates similarly to the performance on the physical robot.
Revised: FF-NN cannot extrapolate to the new velocities and the tracking error deteriorates similarly to the performance on the physical robot.

Original: In contrast to the FF-NN, DeLaN can extrapolate to the higher velocities and maintains a good tracking error.
Revised: In contrast to the FF-NN, DeLaN can extrapolate to the higher velocities and maintains a good tracking error.

Original: Even further, DeLaN obtains a better tracking error compared the analytic model on all velocity scales.
Revised: Even further, DeLaN obtains a better tracking error compared the analytic model on all velocity scales.

Original: This low tracking error on all test trajectories highlights the improved extrapolation of DeLaN compared to other model learning approaches.[Figure]
Revised: This low tracking error on all test trajectories highlights the improved extrapolation of DeLaN compared to other model learning approaches.

Original:  Conclusion
Revised:  Conclusion

Original: We introduced the concept of incorporating a physics prior within the deep learning framework to achieve lower sample complexity and better extrapolation.
Revised: We introduced the concept of incorporating a physics prior within the deep learning framework to achieve lower sample complexity and better extrapolation.

Original: In particular, we proposed Deep
Revised: In particular, we proposed Deep

Original: Lagrangian Networks (DeLaN), a deep network on which Lagrangian Mechanics is imposed.
Revised: Lagrangian Networks (DeLaN), a deep network on which Lagrangian Mechanics is imposed.

Original: This speciﬁc network topology enabled us to learn the system dynamics using end-to-end training while maintaining physical plausibility.
Revised: This speciﬁc network topology enabled us to learn the system dynamics using end-to-end training while maintaining physical plausibility.

Original: We showed that DeLaN is able to learn the underlying physics from a super-imposed signal, as DeLaN can recover the contribution of the inertial-, gravitational and centripetal forces from sensor data.
Revised: We showed that DeLaN is able to learn the underlying physics from a super-imposed signal, as DeLaN can recover the contribution of the inertial-, gravitational and centripetal forces from sensor data.

Original: The quantitative evaluation within a real-time control loop assessing the tracking error showed that DeLaN can learn the system dynamics online, obtains lower sample complexity and better generalization compared to a feed-forward neural network.
Revised: The quantitative evaluation within a real-time control loop assessing the tracking error showed that DeLaN can learn the system dynamics online, obtains lower sample complexity and better generalization compared to a feed-forward neural network.

Original: DeLaN can extrapolate to new trajectories as well as to increased velocities, where the performance of the feedforward network deteriorates due to the overﬁtting to the training data.
Revised: DeLaN can extrapolate to new trajectories as well as to increased velocities, where the performance of the feedforward network deteriorates due to the overﬁtting to the training data.

Original: When applied to a physical systems with complex dynamics the bounded representational power of the physics prior can be limiting.
Revised: When applied to a physical systems with complex dynamics the bounded representational power of the physics prior can be limiting.

Original: However, this limited representational power enforces the physical plausibility and obtains the lower sample complexity and substantially better generalization.
Revised: However, this limited representational power enforces the physical plausibility and obtains the lower sample complexity and substantially better generalization.

Original: In future work the physics prior should be extended to represent a wider system class by introducing additional non-conservative forces within the Lagrangian.
Revised: In future work the physics prior should be extended to represent a wider system class by introducing additional non-conservative forces within the Lagrangian.

Original: Acknowledgments
Revised: Acknowledgments

Original: Will be added in the ﬁnal version.
Revised: This project has received funding from the European Union’s Horizon 2020 research and innovation program under grant agreement No #640554 (SKILLS4ROBOTS). Furthermore, this research was also supported by grants from NVIDIA and the NVIDIA DGX Station.

Original: Appendix A: Offline Benchmarks
Revised: Appendix A: Offline Benchmarks

Original: [Figure]
Revised: [Figure]

Original: To evaluate the performance of DeLaN without the control task, DeLaN was trained oﬄine on previously collected data and evaluated using the mean squared error (MSE) on the test and training set.
Revised: To evaluate the performance of DeLaN without the control task, DeLaN was trained oﬄine on previously collected data and evaluated using the mean squared error (MSE) on the test and training set.

Original: For comparison, DeLaN is compared to the system identiﬁcation approach (SI) described by Atkeson et al.
Revised: For comparison, DeLaN is compared to the system identiﬁcation approach (SI) described by Atkeson et al.

Original: (1986), a feed-forward neural network (FF-NN) and the Recursive Newton Euler algorithm (RNE) using an analytic model.
Revised: (1986), a feed-forward neural network (FF-NN) and the Recursive Newton Euler algorithm (RNE) using an analytic model.

Original: For this comparison, one must point out that the system identiﬁcation approach relies on the availability of the kinematics, as the Jacobians and transformations w.r.t.
Revised: For this comparison, one must point out that the system identiﬁcation approach relies on the availability of the kinematics, as the Jacobians and transformations w.r.t.

Original: to every link must be known to compute the necessary features.
Revised: to every link must be known to compute the necessary features.

Original: In contrast, neither DeLaN nor the
Revised: In contrast, neither DeLaN nor the

Original: FF-NN require this knowledge and must implicitly also learn the kinematics.
Revised: FF-NN require this knowledge and must implicitly also learn the kinematics.

Original: Figure 7 shows the MSE averaged over 20 seeds on the character data set executed on the two-joint robot.
Revised: Figure 7 shows the MSE averaged over 20 seeds on the character data set executed on the two-joint robot.

Original: For this data set, the models are trained using noisy samples and evaluated on the noise-free and previously unseen characters.
Revised: For this data set, the models are trained using noisy samples and evaluated on the noise-free and previously unseen characters.

Original: The FF-NN performs the best on the training set, but overﬁts to the training data.
Revised: The FF-NN performs the best on the training set, but overﬁts to the training data.

Original: Therefore, the FF-NN does not generalize to unseen characters.
Revised: Therefore, the FF-NN does not generalize to unseen characters.

Original: In contrast, the SI approach does not overﬁt to the noise and extrapolates to previously unseen characters.
Revised: In contrast, the SI approach does not overﬁt to the noise and extrapolates to previously unseen characters.

Original: In comparison, the structure of DeLaN regularizes the training and prevents the overﬁtting to the corrupted training data.
Revised: In comparison, the structure of DeLaN regularizes the training and prevents the overﬁtting to the corrupted training data.

Original: Therefore, DeLaN extrapolates better than the FF-NN but not as good as the
Revised: Therefore, DeLaN extrapolates better than the FF-NN but not as good as the

Original: SI approach.
Revised: SI approach.

Original: Similar results can be observed on the cosine data set using the Barrett WAM simulated in SL (Figure 8 a, b).
Revised: Similar results can be observed on the cosine data set using the Barrett WAM simulated in SL (Figure 8 a, b).

Original: The FF-NN performs best on the training trajectory but the performance deteriorates when this network extrapolates to higher velocities.
Revised: The FF-NN performs best on the training trajectory but the performance deteriorates when this network extrapolates to higher velocities.

Original: SI performs worse on the training trajectory but extrapolates to higher velocities.
Revised: SI performs worse on the training trajectory but extrapolates to higher velocities.

Original: In comparison, DeLaN performs comparable to the
Revised: In comparison, DeLaN performs comparable to the

Original: SI approach on the training trajectory, extrapolates signiﬁcantly better than the FF-NN but does not extrapolate as good as the SI approach.
Revised: SI approach on the training trajectory, extrapolates signiﬁcantly better than the FF-NN but does not extrapolate as good as the SI approach.

Original: For the physical system (Figure 8 c, d), the results diﬀer from the results in simulation.
Revised: For the physical system (Figure 8 c, d), the results diﬀer from the results in simulation.

Original: On the physical system the SI approach only achieves the same performance as RNE, which is signiﬁcantly worse compared to the performance of DeLaN and the FF-NN.
Revised: On the physical system the SI approach only achieves the same performance as RNE, which is signiﬁcantly worse compared to the performance of DeLaN and the FF-NN.

Original: When evaluating the extrapolation to higher velocities, the analytic model and the SI approach extrapolate to higher velocities, while the MSE for the FF-NN signiﬁcantly increases.
Revised: When evaluating the extrapolation to higher velocities, the analytic model and the SI approach extrapolate to higher velocities, while the MSE for the FF-NN signiﬁcantly increases.

Original: In comparison, DeLaN extrapolates better compared to the FF-NN but not as good as the analytic model or the SI approach.
Revised: In comparison, DeLaN extrapolates better compared to the FF-NN but not as good as the analytic model or the SI approach.

Original: This performance diﬀerence between the simulation and physical system can be explained by the underlying model assumptions and the robustness to noise.
Revised: This performance diﬀerence between the simulation and physical system can be explained by the underlying model assumptions and the robustness to noise.

Original: While DeLaN only assumes rigidbody dynamics, the SI approach also assumes the exact knowledge of the kinematic structure.
Revised: While DeLaN only assumes rigidbody dynamics, the SI approach also assumes the exact knowledge of the kinematic structure.

Original: For simulation both assumptions are valid.
Revised: For simulation both assumptions are valid.

Original: However, for the physical system, the exact kinematics are[Figure] unknown due to production imperfections and the direct cable drives applying torques to ﬂexible joints violate the rigid-body assumption.
Revised: However, for the physical system, the exact kinematics are unknown due to production imperfections and the direct cable drives applying torques to ﬂexible joints violate the rigid-body assumption.

Original: Therefore, the SI approach performs signiﬁcantly worse on the physical system.
Revised: Therefore, the SI approach performs signiﬁcantly worse on the physical system.

Original: Furthermore, the noise robustness becomes more important for the physical system due to the inherent sensor noise.
Revised: Furthermore, the noise robustness becomes more important for the physical system due to the inherent sensor noise.

Original: While the linear regression of the SI approach is easily corrupted by noise or outliers, the gradient based optimization of the networks is more robust to noise.
Revised: While the linear regression of the SI approach is easily corrupted by noise or outliers, the gradient based optimization of the networks is more robust to noise.

Original: This robustness can be observed in Figure 9, which shows the correlation between the variance of Gaussian noise corrupting the training data and the MSE of the simulated and noise-free cosine trajectories.
Revised: This robustness can be observed in Figure 9, which shows the correlation between the variance of Gaussian noise corrupting the training data and the MSE of the simulated and noise-free cosine [Figure]

Original:  With increasing noise levels, the MSE of the SI approach increases signiﬁcantly faster compared to the models learned using gradient descent.
Revised: trajectories. With increasing noise levels, the MSE of the SI approach increases signiﬁcantly faster compared to the models learned using gradient descent.

Original: Concluding, the extrapolation of DeLaN to unseen trajectories and higher velocities is not as good as the SI approach but signiﬁcantly better than the generic FF-NN.
Revised: Concluding, the extrapolation of DeLaN to unseen trajectories and higher velocities is not as good as the SI approach but signiﬁcantly better than the generic FF-NN.

Original: This increased extrapolation compared to the generic network is achieved by the Lagrangian Mechanics prior of DeLaN. Even though this prior promotes extrapolation, the prior also hinders the performance on the physical robot, because the prior cannot represent the dynamics of the direct cable drives.
Revised: This increased extrapolation compared to the generic network is achieved by the Lagrangian Mechanics prior of DeLaN. Even though this prior promotes extrapolation, the prior also hinders the performance on the physical robot, because the prior cannot represent the dynamics of the direct cable drives.

Original: Therefore, DeLaN performs worse than the FF-NN, which does not assume any model structure.
Revised: Therefore, DeLaN performs worse than the FF-NN, which does not assume any model structure.

Original: However, DeLaN outperforms the SI approach on the physical system, which also assumes rigid-body dynamics and requires the exact knowledge of the kinematics.
Revised: However, DeLaN outperforms the SI approach on the physical system, which also assumes rigid-body dynamics and requires the exact knowledge of the kinematics.

Original: [Figure]
Revised: [Figure]

Original: Appendix B: Complete  Results
Revised: Appendix B: Complete Online Results

Original: [Figure]
Revised: [Figure]

Original: a b c d e g h l m n o p q r s u v w y z
Revised: a b c d e g h l m n o p q r s u v w y z

Original: C h a r a c t e r 101010101010 3 Tracking Error
Revised: C h a r a c t e r 101010101010 3 Tracking Error

Original: D e L a N
Revised: D e L a N

Original: FF - NN
Revised: FF - NN

Original: P D - C o n t r o ll e r
Revised: P D - C o n t r o ll e r

Original: R N E
Revised: R N E