Using Physics Knowledge for Learning Rigid-body Forward Dynamics with Gaussian Process Force Priors
Authors: Lucas Rath
Andreas René Geist
Sebastian Trimpe
Venue: CoRL2021 Poster (2021)
Keywords: Machine Learning
Robotics
Analytical mechanics

Abstract: If a robot's dynamics are difficult to model solely through analytical mechanics, it is an attractive option to directly learn it from data. Yet, solely data-driven approaches require considerable amounts of data for training and do not extrapolate well to unseen regions of the system's state space. In this work, we emphasize that when a robot's links are sufficiently rigid, many analytical functions such as kinematics, inertia functions, and surface constraints encode informative prior knowledge on its dynamics. To this effect, we propose a framework for learning probabilistic forward dynamics that combines physics knowledge with Gaussian processes utilizing automatic differentiation with GPU acceleration. Compared to solely data-driven modeling, the model's data efficiency improves while the model also respects physical constraints. We illustrate the proposed structured model on a seven joint robot arm in PyBullet. Our implementation of the proposed framework can be found here: https://git.io/JP4Fs

Using Physics Knowledge for Learning Rigid-body Forward Dynamics with Gaussian Process Force Priors
Lucas Rath
A. René Geist".l
Sebastian Trimpe 1.2
Maa Planck Institute for Intelligent Systems, Stuttgart {rath, geist feis.mpe de
Institute for Data Science in Mechanical Engineering, RWTH Aachen University trimpe@deme..rth-aaccccci wwth-aachen. de
Abstract: If robot's dynamics are difficult to model solely through analytical mechanics, it is an attractive option to directly learn it from data. Yet, solely data- driven approaches require considerable amounts of data for training and do not extrapolate well to unseen regions of the system's 's state space. In this work, we em- phasize that when robot's links sufficiently rigid, many analytical functions such as kinematics, inertia functions, and surface constraints encode informative prior knowledge its dynamics. To this effect, we propose framework for learn- ing probabilistic forward dynamics that combines physics knowledge with Gaus- sian processes utilizing automatic differentiation with GPU acceleration. Com- pared to solely data-driven modeling, the model's s data efficiency improves while the model also respects physical constraints. We illustrate the proposed structured model on a seven joint robot arm in PyBullet. Our implementation of the proposed framework can be found here: https://gitio//PPFs
Keywords: Machine learning, Robotics, Analytical mechanics
Introduction
An accurate dynamics model is essential for computing a robot's actions via model-based control approaches such as model predictive control [1-4] model-based reinforcement learning [5-10]. Yet. in many robotic systems including legged robots and robot arms, physical phenomena such as friction and elasticities considerably affect the dynamics, are often environmental dependent, and unknown a-priori which in turn aggravates analytical physics modeling. As an alternative to analytical modeling, recent works resort to learning dynamics solely from input-output data using
Multiodyy Dynamics
Surface constraint Afce fxli (o:))))
11 ar AAYAAA "47)"
tangent plane, N(AA
Tneriis motrix M(xsda]
Gr A7(A 'ATy (b- AM 'G)
Kinemarics
Analytical furces Quixidal
Equations of motion I= 30-100+ Q.-+)
Regression
prior
Q Q, GPV dx) &ayles ('u)))
surface
Semi Surmi-prramttic model
SP (xr wwtaat-1 Hhul""
end-effector
Figure 1: Computational graph of the proposed framework, A robot arm's forward dynamics model augmented by a GP approximating unknown forces Qu- The semi-parametric model predicts acceleration while potentially including knowledge in form of analytical functions (M, A, b. Qx)
"cqual contribution.
5th Conference on Robot Learning (CoRI. 2021), London, UK.

---

regression models. Yet, data collection on robotic systems must be done in real-time using poten- tially sub-optimal control strategies that only operate in small regions of the system's state-space. Therefore, data sets that stem from physical systems are often limited in size and information value. This raises a considerable obstacle for successfully training data-driven models such as a neural net- work (NN) or a Gaussian process (GP), as these are data-hungry compared to physics models and usually do not extrapolate well. To overcome the limitations of data-driven modeling, recent works combine analytical mechanics with data-driven models resulting in so-called structured models or graybox models. In structured models, the target function, that is the parts of an analytical physics model that are difficult to model a-priori, is being approximated by a carefully designed data-driven model. By use of physical prior knowledge, the data efficiency of a structured model is potentially improved by: i) Reducing the complexity of the target function; ii) Reducing the dimensionality of the target function; or iii) Building prior knowledge on physical properties of the target function into the data-driven model. This target function ideally contains all erroneous parts of the physies model while the sample-complexity of a regression model is smaller when learning such a target function compared to simply learning the residual function in the physics model's output.
In general, finding suitable candidate for a target function inside an analytical physics model is not trivial. In the following, we suggest that for sufficiently rigid bodies the majority of the analytical model errors arise from an inaccurate depiction of forces. As many of these forces are potentially uncertain and stochastic in nature, we propose probabilistic structured dynamics model in which we approximate unknown forces inside an analytical dynamics model by a GP. In particular, the contribution of this work are follows:
We propose a modelling framework using PyTorch that combines a library for (implicitly constrained) rigid-body mechanies with a novel implementation of structured multi-task GP regression. We analyze the proposed framework by learning the dynamics of seven dimensional robot arm whose end-effector touches a surface in PyBullet.
We show that analytical parameters can be efficiently estimated alongside seven- dimensional structured multi-output GP with 154 hyperparameters.
We emphasize the new research opportunities that evolve from the proposed framework. Such the prediction of implicit contact forces by solely resorting to measurements of the robot's constrained acceleration as well as using Baumgarte stabilization to enforce that long-term trajectory predictions made using the structured GP respect implicit constraints.
1.1 Problem Formulation
We assume that the following assumptions apply to a robot whose forward dynamics (FD) we aim to identify:
The underlying kinematics function the the robot arm and kinematic surface equation, as well as their parameters, are known a-priori.
Assumption
Assumption The bodies (aka links) of the mechanical system are rigid.
Further, we assume that the state of a robot's forward dynamics description given by q E R"",, d. q. and u. E Th= being the joint angles, velocity, acceleration, and control input respectively. As discussed in Section 1.3, these assumptions are commonly found in literature on nonlinear robot dynamics identification [11-15]. Given input points I&- [qk. Gk. uz]", accelcration measurements are modeled as
(*x) ((*x) + Ex with ey ~ N (0,0 Eny,
(1)
where Ey denotes diagonal matrix containing the measurement variances. Further, an infor- mative data-set D (X,Y} is available consisting of the input data vector X [a] ... . M and the output data vector Y ly ... V. The main objective of this work is, for high- dimensional robot arm, to propose a modeling framework that enables the data-efficient prediction of - = E(i(..)|D) and K K D = cov(ū(æ.) ) (D) at a test input x. with a structured GP model & ~ GP(mq. Kg). We refer to å as structured as this model results from combining physical prior knowledge underlying the system's 's equations of motion (EOM) with GP prior placed on the system's 's unknown forces Qu ~ GP(O, Ku).
2

---

1.2 Forward Dynamics of Robot Arms
In what follows, we omit the arguments of functions to avoid cluttered notation these are clear from the context. In this work, limit the discussion to Lagrangian FD of robot arms as described by equations of motion (EOM) taking the form
q = fa
(Q +
(2)
with the analytical model parameters Ox symmetric and positive definite inertia matrix (g; 0x), torques acting on the unconstrained system Qa:;0x torques Q₂ (2; 0x) caused by Cartesian end- effector friction forces F2z, and torques Qr (x; 9A) that are caused by Cartesian surface normal forces Fy also acting on the end-effector. That is, if Q presses the robot's 's end-effector onto a surface, the reaction the force F'y that always remains normal to the surface. The transforma- tion of Fl into generalized coordinates yields the torques Q₁. The force F2 often causes dissipative forces between the end-effector and the surface. These dissipative forces Q₂ lie always tangential to the surface. The torques acting in the unconstrained system Q(z; 0x) = QD + Qu + QG + Qc consist of bias forces Qc gravitational forces Qg. dissipative forces inside the joints Qp. and ac- tuation forces Qu. Often in literature Qu + Qn is denoted by T being referred to as joint torques which are the torques that measurable in robot's joints. The analytical model parameters 0^ can be divided into kinematic parameters (e.g., the length between adjacent joints), and inertia pa- rameters (e.g., CoG positions, masses, and inertias). The priori estimated model parameters 0A often deviate from the real physical values of the system and need to be further identified using data. For kinematic trees, such as most robot arms, the FD (a; 0A) are straightforwardly obtained using the articulated body algorithm. Alternatively, inverse dynamics (ID) seek a mapping from {q, 4, i to u. However, unlike ID, FD allow the direct simulation of system's 's motion. In turn, one could use an accurate structured FD model to train an RL policy in simulation. Moreover, the largest noise usually lies on acceleration estimates denoting the output of FD models. In the case of GPs is simpler to model noise on the outputs than inputs [16].
surface applies
In the absence of end-effector forces, the coordinates q uniquely define every admissible state of the robot arm such that the robot's acceleration is given as q = M However, the presence of Qr enforces that the end-effector's motion during contact respects the holonomic surface constraints <(g) = O, with c(q) R + RT, effectively reducing the set of admissible states the robot arm may acquire. Instead of working directly with c(q) = 0, one can compute its second derivaaiie to obtain a constraint equation that is linear in q. writing
Aî = b,
with A(g, q: 0x): 2R + Rmxn, b(g, q; 0x) Rd + Rm, and m Rg. As detailed in [17], the part of all forces doing virtual work, Q' E Q, and the implicit constraint forces Q₁ doing no virtual work, must lie in Q2 E Q' N(A) as well Q1 E R(AT), with the null space of A E R being defined as N(A) = {ā e R'v Aq = 0} and its range space as R(A) = (b E R" : Eq e IT such that 6 = Aq} [18]. The fact that Q₁- E R(AT) motivates its parametrization in terms of Lagrange multipliers X, writing Q = AT X. As detailed in [19] one obtains an explicit formula for the Lagrange multipliers as x= = (AM " (b- - AM
AM- Q). where the A denotes the Moore- Penrose (MP) pseudo inverse of A. Subsequently. the system's EOM are obtained as
$ = M- Q HQQ
= - --------T(AA--- 300++AM-11,0 Q= Q Q2. and Q2 AT(AM-1AT)+b, Fur- ther details on implicitly constrained dynamics are given in Section 1 of the supplementary material.
1.3
Related Work
The identification of the errors in a robot's analytical EOM often forms a metaphorical Gordian knot consisting of errors that arise from the interplay of wrong analytical parameters (e.g., inertia parameters), wrong analytical functions (e.g., friction forces), and in the worst case the analytical model class itself being an inaccurate depiction of the real dynamics (e.g., describing flexible bodies with rigid-body dynamics). In what follows, we discuss recent works that combine data-driven models with rigid-body dynamics.
3

---

Parameter estimation of rigid-body dynamics models Pioneering works on robot dynamics identification focused on the estimation of analytical parameters. For example, the well-known approach of Atkeson et al. [11] uses the linearity of a rigid robot arm's 's ID with respect to its param- eters to estimate these via linear regression. With the increased popularity of libraries for automatic differentiation, recent works [12-15] use gradient-based optimization to estimate the physical pa- rameters inside the analytical model from data. Such approaches work well if the robot arm is rigid and the forces acting on it are sufficiently known. However, the parameter estimation of an insufficient analytical representation ter estimates as well as unsatisfactory model accuracy. While we assume that the analytical model differs from the real dynamics, we also implement a differentiable analytical model in Py PyTorch such that we can estimate analytical parameters alongside the parameters of a data-driven model.
robot's 's dynamies leads to physically inconsistent parame-
Building rigid-body dynamics into neural networks As an alternative approach to analytical modeling, [20-23] model dynamics via the Euler-Lagrange equations in which a NN approximates the system's 's Lagrangian or the entries of the inertia matrix M. In turn, such a Lagrangian NN increases the flexibility of the functional approximation of mass-related dynamic terms {M. QG. Qc} compared to analytical models while still respecting energy conservation. These works often assume =u and thereby potentially neglect the joint friction QD. If forces are the major sources of analytical errors and mass related quantities are not, then Lagrangian NNs add flexibility through the NN in the functional representation of a robot's dynamics where might not be required. So far this hypothesis has not been refuted, as the works on Lagrangian NNs test their algorithms solely on pendulums or low-dimensional robot arms without end-effector contacts.
A recent branch in structured modeling combines analytical models of mass-related quantities and presumably known forces with data-driven modeling. Lutter et al. [24] augments an analytical FD model with a NN approximating т. The parameters of the analytical model and NN are estimated jointly via automatic differentiation. Recently, [10] and subsequently [25] combined an analytical simulation with pre-trained NN predicting measured joint torques т. Notably, Qr and Q= modeled analytically as detailed in [9] while their physical parameters are varied to robustify the trained control policy towards parameter uncertainties. Significantly, NN control policies that were trained on such structured models achieved so far unseen robustness of a quadruped's 's locomotion policy. These seminal works indicate that the gap between modeled and real dynamics (aka sim2real gap) can be closed by consciously combining analytical mechanics with data-driven modeling.
Building rigid-body dynamics into Gaussian Processes On real robotic system's, noise and uncertainty are often significantly impacting the robot's dynamics. For example, friction force denotes a macroscopic abstraction of microscopic tribologic phenomena which aggravates deter- ministic modeling. In addition, depending on the robot's sensors, the changes in the joint torques due to elasticities and backlash are often not observed accurately and subsequently also introduce uncertainty into the dynamics. As shown in [6, 26]. the synthesis of a robot's control policy can significantly benefit from the availability of an uncertainty measure for the planned motion. Popular models for the identification of uncertain and noisy dynamics are either building on Bayesian linear regression as e.g. [27]. or often resort to GPs [8. 28-32]. GPs allow for the incorporation of var- ious model assumptions through the covariance function (kernel) and are often more -ata-efficieet compared to NNs. Yet, vanilla GP regression requires the computation of the inverse covariance matrix which demands a computational complexity of (((DN)*) and memory requirement of (((D)))) where D denotes the number of the GP's correlated outputs. In addition, the inversion of the covariance matrix is prone to numerical problems while the computational efficient implemen- tation of multi-output GPs forms considerable obstacle. Despite these challenges, several models have been proposed that combine GP regression with analytical dynamics.
Nguyen-Tuooe and Peters [33] used [11] as a linear parametric mean of a GP model (cf. [16, Chap- ter 2.7D. If Assumption 2 applies onto the robot arm, the GP in [33] effectively approximates the joint torques 7 (in the absence of end-effector forces). As outlined in Section 2, we also approximate unknown forces through a GP. Albeit, as a robot arm's 's FD are rarely linear with respect to analyti- cal parameters, structured GP regression is more challenging for FD compared to ID. Conceptually being similar to [33], Saveriano et al. [34] approximate the residuals of analytical transition dynam- ics, being the mapping from (gk, qx, ) to {qa+1, ( J. via one-dimensional GPs. Our approach does not directly approximate dynamics residuals with GPs, but instead approximates the residual function by placing a GP prior inside the analytical model.

---

Cheng et al. [35] placed GP on the Lagrangian inside a robot arm's Euler-Lagrangian ID. Yet, for GPs is not clear how formulate such model for FD and to which extent the end-effector forces can be incorporated. A Lagrangian GP leverages that the derivative of GP if it exists, is itself a GP [36]. Jidling et al. [37] emphasized that more generally GPs are closed under linear functionals. In turn, the authors proposed linearly transformed GP such that its predictions fulfil a linear operator equation egg., the mechanical stress field inside a linear elastic material. Our work also transforms GP by linear operator, that is, the matrix-values function Il and in turn enforce that predictions respect the linear-affine equation (3), Methodically, our work builds on Geist and Trimpe [38]. In addition to [38], suggest placing GP on the unknown forces of the system rather than its unconstrained acceleration. As we detail in Section 2.1, this alternative approach potentially improves the model's -ffple-efficiency well eases the inclusion of prior knowledge on force properties into the GP. In addition, [38] only tested their model on low-dimensional systems such as a point-mass sliding over surface. In contrast, we propose framework that allows for learning high-dimensional dynamics. Our framework is significantly faster compared to [38] by resorting to automatic differentiation with GPU accelerated computation of gradients, and shows how to transform regression models via recursively computed rigid-body dynamics.
2 Semi-parametric Regression of Dynamics with Gaussian Processes
In this section, we extend the findings of Geist and Trimpe [38] and use knowledge on the parametric functions {M, A, b) stated in (2) and (3) to linearly transform a GP. Here, we leverage that analytical mechanics defines the acceleration of system in terms of affine transformations of force functions which approximate by GP. An introduction to multi-output GP regression is provided in Section 2 of the supplementary material.
2.1 Combining Data-driven with Analytical Modeling
In this section, we propose framework that combines Gaussian process regression with parametric analytical modeling. To develop a thorough understanding of the underlying assumptions we first discuss the main building blocks of the analytical EOM (4) as also shown schematically in Figure 1. As the dynamics (4) are expressed in generalized coordinates, the entries of M, QG. Qc, A, and b depend kinematic functions. The kinematic functions themselves depend on the system's 's state {q, o) and kinematic parameters such as the length of the bodics. In practice, precise estimates for kinematic parameter
the kinematic parameters are obtained from computer-aided design (CAD) estimation techniques. In Cartesian space, an accurate model for (3) that is ( (A, b) can be straightfor- wardly obtained by e.g., using camera system. In turn, (3) is obtained in generalized coordinates using kinematic coordinate transformations. Besides kinematic expressions, the inertia matrix M contains inertia parameters, namely the CoG positions, inertias, and masses. We assume that a prior for the inertia parameters can be obtained using also CAD. Yet, for FD it is critical that we mitigate errors in the inertia parameters being part of 9 as all forces (4) are multiplied by M M-'(x: 0x).
As pointed out in Section 1, is often difficult to analytically describe force functions that arise from clasticities and friction phenomena. Therefore, we divide the analytical forces Q into known part Qx(x; A and an unknown part Qu(x). The term known is defined in the following as the existence of optimal A, f such that the error between physical function and its analytical description in terms of f approaches zero. Knowledge on 0% can be unavailable a-priori such that its entries need be estimated from data. Subsequently, we model the generalized forces as GP, writing
i >>pP(QQ(x; Ow), (L,,x';0))))
(5)
in which kqu denotes an appropriately chosen kernel function and 6M its haper-paramettrs. By inserting (5) into (4), we obtain structured model for the system's 's FD as
i ~ GP(M-'Q, + HQk, Hkqu HT).
(6)
The above model is itself a GP as q follows from an affine transformation of Q. To make predictions with (6), the GP conditional posterior formula used. Note that (6), we placed the GP prior on
the unknown forces instead on placing it on the unconstrained acceleration MM-QQu proposed by [38]. Since M known and its entries are usually nonlinear functions, learning function after has been transformed by M is likely to require more expressive data-driven model to achieve a
5

---

comparable prediction accuracy [17]. In addition, as elaborated in Section 3 of the supplementary material, GP force prior can be designed as a sum of GPs which each model specific force. In turn, our framework (6) enables a straightforward inclusion of prior knowledge on the properties of forces into the GP. Moreover, a trained GP force prior (5) can provide predictions on other analytical processes such as the generalized normal forces Q7.
2.2 Type-II MLE of Structured Gaussian Processes
While merit of the model presented in (6) forms its simplicity, the optimization of both analytical and kernel parameters poses a significant challenge. this work, we resort to Type-ll maximum likelihood estimation (MLB). MLE forms standard approach for estimating GP parameters. Given p( N E _x.x + Σy) resulting from (6), type-II MLE seeks a l {0) B mini- mizing the negative log likelihood, writing
( 8r arg min 2
( +Ex ( log
|
N log 2m.
D XX, + Σy
2
The left term inside the bracket in (7) assigns a cost to the data-fit while the log-determinant term penalises the function's 's complexity [16. p. 113]. As (6) denotes multi-output GP such that D 1, the computational cost increases significantly compared to standard GP regression. Moreover, as pointed out in [16], the objective function in (7) is highly nonlinear such that if local optimization techniques are considered we need to restart the optimization several times with random initial parameter values. To solve (7) using restarts for high- high-dimensional dynamics in reasonable amounts of time, we require a fast computation of analytical parameter gradients. However, such an approach to (7) requires the computation of partial gradients of the analytical parametric function p (1;; 0^) as well as >x being transformation of <z 0m) by H (xx 0x). Fortunately, recent ML libraries such JAX [39] and PyTorch [40] allow the optimization of functions via automatic differentiation (building on AutoGrad [41]) in combination with libraries for GPU accelerated computation such as XLA. In turn, JAX and PyTorch provide fast computation of analytical function gradients as long as the function is expressed in terms of an underlying numerical linear algebra library such as Numpy or PyTorch's torch functions. Therefore, we implemented a multi-body library using native PyTorch functions resulting in {A, b, M, Qk} as well as HH. H}. Then, while using the mean and kernel function from GPyTorch [42], we designed structured GP inference framework from scratch using Py PyTorhh to obtain and optimize the likelihood in (7). The optimization is done using automatic differentiation [43] and GPU acceleration resulting in significant speed-up compared to numerical gradient approximation techniques.
3 Simulation Results
In this section, the proposed structured GP (6) is analyzed and compared to different baseline mod- els. The chosen dynamical system is a KUKA arm with rotational joints simulated in PyBullet [44] as depicted in Figure 2a. A detailed description of the simulation setup is given in Section 5 of the supplementary material. Data is collected by controlling the robot's end-effector along lin- trajectories while pressing onto the surface. This task can be seen as an abstraction to a robot arm performing welding, cutting, or marking maneuvers on surfaces. The contact between the end- effector and the surface creates ideal-constraint forces, which avoids the end-effector penetrating the surface, as well as friction forces which are challenging to be modeled beforehand. While running the simulation, we measured the joint angles q and the control inputs u. The joint velocity q and ac- celeration q are obtained by using low-pass filter and appealing to numerical differentiation. After data collection, we applied Farthest Point Sampling [45] to remove adjacent data points as well as reduce the data set's 's size. The post-prout procespoe data is split into a training and test data set.
Figure 2b and 3 Prediction accuracy and data efficiency As baseline for comparison, we trained the parameters of (4) with Q = as well as a feed-forward NN on ten thousand data points. The trained analytical model and the NN achieved on the test data set mean absolute error (MAE) of 0.57 and 0.13, respectively. For all GP models, we choose a squared exponential (SE) kernel. The 154
of each GP were estimated according to (7) using Adam

---

10"
Nean absotute erter
GP
Moss errrr
0.25
5GP aubliu
mmm
-0.25
Mean absokete ertor CoO po wror
200
4.00
600
800
1000
Number training points
No ecod sedooyy the surruce
Mean abs. constraine: vidlation
0.00
10-1
-2.0
10
10"
10-1
200
6:00
800
1000
100
200
300
400
4.40-035 4.00005
Number training
Iteraaion
(a)
(b)
Figure 2: a) Robot arm in PyBullet (Top) and end-effector trajectory during collection of training data (Bottom). b) The hyper-parametee training points, then the mean absolute acceleration prediction error as well as acceleration- constraint eиrог on the test data set are compared. e) Parameter optimization results of analytical model (black) and the proposed structured GP model (blue).
of GP FD models are trained on an increasing number of
Guped 90.5e
2. -6
44 ce 41.00
Gepo0 G mm
1.2
1.0
1.0
0.0
-33
-25
-1.2
-20
-1.0
dms
dmme
(a) Analytical dynamics model
(b) foowwrd
(c) GP
(d) S-GP with
(c) S-GP with analytical mean
NN
zeTo mcan
Figure Acceleration predictions (eep (Top) and corresponding prediction error (Bottom) of different forward dynamics models. The black lines depicts the system's 's noisy acceleration over time, the purple line the predicted mean acceleration and light purple available the #2 std. deviation confidence region. Note that the y-axis scaling of Figure 3a deviates from the other figures.
[46] without parameter constraints. Figure 2b illustrates the GPs' test-data MAE of each joint- dimension over an increasing size of the training data set. In this figure, the term "S-GP" refers to the model in (6) with Qx while the term "S-GP + analytical mean" assumes an analytical model Qk QG + Qe Qu as the GP's prior mean function. For both S-GP models, accurate analytical parameters are given, that is 0^ = 0%. We assume QG + Qc as known as these analytical functions are being derived solely in terms of known kinematics and inertia functions, as well as the gravitational acceleration constant [17]. The structured GP models are compared to standard GP regression in which each acceleration function modelled by a single independent SE GP. As shown in Figure 2b (Top), analytical prior knowledge improves the data efficiency of the GP. The proposed S-GP models compares also favorably to placing GP prior on the system's unconstrained acceleration as initially proposed by [38] being denoted as "S-GP [38]". Moreover, the incorporation of implicit constraint knowledge (6) significantly reduces the constraint error Aû as illustrated in Figure 2b (Bottom). Figure 3 illustrates different acceleration predictions made with these models using 1000 training points. The analytical bascline model does not contain
assume that
7

---

151114--1
12"
0.0050
12
53
0.0025
13-9
10-3
64
111T net
10
-0.0035
uim
Figure 4: Long-term trajectory prediction and corresponding emor functions obtained by numeri- cal integration with integration time step of (1 (2/240) using the different GP models' acceleration predictions of the fourth output dimension. The true trajectory is depicted by the black dotted line.
function describing the surface friction which in turn causes the model to yield large prediction errors in the robot arm's 's joints that are close to the end-effector.
Figure 2c: Joint estimation of inertia and kernel parameters Another important aspect of the proposed structured modeling framework forms the simultaneous estimation of 0M and 0Ą. To illus- trate how 0M and A are estimated jointly, we train the same S-GP with QK = 0 as in the previous simulation on thousand data points, Yet, instead of assuming that we obtained good prior for PA, For
we now estimate the end-effector's inertia parameters alongside the GP's 154 example, one could imagine that tool such as a brush or milling machine is being fixed to the end- effector changing its inertia parameters. Figure 2c illustrates the optimization results. Compared to training the parameters of an analytical model without surface friction, the parameter estimates of proposed S-GP converge faster while the prediction's 's MAE for all joint-dimensions improves.
Figure 4: Long-term trajectory prediction Figure 4 illustrates trajectory prediction for one of the joint dimensions which computed by resorting to symplectic Euler numerical integration using the different GP models. The different GP models are the same in Figure 2b using 1000 training points. Due measurement noise in the initial state as well as numerical integration and prediction errors the GPs diverge from the true trajectory over time. As consequence of Assumption I and 2, one can guarantee that trajectory predictions computed with (6) converge onto the surface equation c(g) = 0 by adjusting b inside the S-GP model through Baumgarte stabilization [47] as detailed in Section 4 of the supplementary material. turn, for the GP model's using Baumgarte stabilization the error in the position-level surface constraints converges to the zero.
4 Conclusion
In this paper, we introduce modeling framework that combines differentiable Lagrangian dynam- ics with data-driven modeling. This framework incorporates structural prior knowledge in forn of kinematic, inertia and implicit constraint equations to improve the data efficiency of a GP model. Further, we illustrate that the dyper-paramede of the GP and parameters of analytical model can be estimated jointly using automatic differentiation. The framework requires that we have access to an accurate kinematics description of the system as well as that its bodies sufficiently rigid. No- tably, we illustrate that the dynamics of high- nigh-dimensional robot arms whose end-effector is subject to forces can be estimated using multi-output GPs.
Particularly the combination with GP regression, the proposed framework opens up exciting re- search directions in the field of robotics modeling. So far, by letting the data-driven model approx- imate forces inside an analytical model, the data efficiency of the model is increased as we do not need to learn inertia and constraint functions from scratch. Yet, we emphasized in the intro- duction, structured modeling can also improve a model's data efficiency by decreasing the target function's 's dimensionality. In particular, one can use the robot's differential kinematics to approxi- mate with a GP the end-effector forces {F., in the Cartesian space.
For the sake of illustration, we chose a SE kernel. In practice, such kernel might not suffice as it assumes that the process is stationary and smooth. Both of these assumptions will likely not be met in practice. Instead, one can parameterize kernels using NNs [48, 49]. Further, rather than assigning all states as inputs to the kernel force prior, one can use additional prior model knowledge and use only physically relevant states as kernel inputs.
8

---

Acknowledgments
We thank Vincent Berenz for his valuable advice and the many insightful discussions. This work has been supported in part by the Max Planck Society and part by the Cyber Valley initiative. The authors thank the International Max Planck Research School for Intelligent Systems (IMPRS-IS) for supporting A. René Geist.
References
[1] F. Allgöwer and A. Zheng. Nonlinear model predictive control, volume 26. Birkhäuser, 2012.
[2] M. V. Minniti, F. Farshidian, R. Grandia, and M. Hutter. Whole-body mpe for dynamically stable mobile manipulator. IEEE Robotics and Automation Letters, 4(4): 3694, 2019.
[3] J. Nubert, J. Köhler, V. Berenz, F. Allgöwer, and S. Trimpe. Safe and fast tracking a robot manipulator: Robust mpe and neural network control. IEEE Robotics and Automation Letters, 52):3050-3057, 2020.
[4] Kleff, Meduri, R. Budhiraja, N. Mansard, and L. Righetti. High-Frequency Nonlin- ear Model Predictive Control of Manipulator. In 2021 IEEE International Conference Robotics and Automation (ICRA), Xi an, China, May 2021.
[5] R. S. Sutton, A. G. Barto, et al. Introduction to reinforcement learning, volume 135, MIT press Cambridge, 1998.
[6] M. Deisenroth and C. E. Rasmussen. Pilco: A model-based and data-efficient approach to pol- icy search. In Proceedings of the 28th International Conference on machine learning (ICML- 11), pages 465-472, 2011.
[7] Parmas, C. E. Rasmussen, J. Peters, and K. Doya. PIPPS: Flexible model-basse policy search robust to the curse of chaos. In J. Dy and A. Krause, editors, Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learn- ing Research, pages 4065-4077 PMLR, 10-15 Jul 2018.
[8] A. Doerr. C. Daniel, D. Nguyen-Tuood Optimizing long-term predictions for modell-aseed policy search. In Conference on Robot Learning, pages 227-238. 2017.
Marco, S. Schaal, M. Toussaint, and S. Trimpe.
[9] J. Hwangbo, J. Lee, and M. Hutter. Per-contact iteration method for solving contact dynamics. IEEE Robotics and Automation Letters, 3(2):895 2018.
[10] J. Hwangbo, J. Lee, A. Dosovitskiy, D. Bellicoso, V. Tsounis, V. Koltun, and M. Hutter. Learn- ing agile and dynamic motor skills for legged robots. Science Robotics, 4(26), 2019.
[11] C. G. Atkeson, C. H. An, and J. M. Hollerbach. Estimation of inertial parameters manipu- lator loads and links. The International Journal of Robotics Research, 533): 1986.
[12] D. Ledezma and Haddadin. First-order- princcelppncciple---- baaeeepeee-baedd constructive network topologies: An application to robot inverse dynamics. In 2017 TEEE-RAS 17th International Conference on Humanoid Robotics (Humanoids), pages 438-445 IEEE, 2017.
[13] F. D. Ledezma and S. Haddadin. Fop networks for learning humanoid body schema and dy- namics. In 2018 HEEE-RAS 18th International Conference on Humanoid Robots (Humanoids), pages 1-9. JEEE, 2018.
[14] G. Sutanto, Wang, Y. Lin, M. Mukadam, G. Sukhatme, A. Rai, and Meier. Encod- ing physical constraints in differentiable newton-euler algorithm. In Proceedings of the 2nd Conference on Learning for Dynamics and Control, volume 120 of Proceedings of Machine Learning Research, pages 804-813, The Cloud, 10-11 Jun 2020. PMLR.
[15) M. Lutter, J. Silberbauer, J. Watson, and J. Peters. Differentiable physies models real-world offline model-basee reinforcement learning, arXiv preprint arXiv:2011.0173 2020.

---

[16] C. K. Williams and C. E. Rasmussen. Gaussian processes for machine learning, volume 2. MIT press Cambridge, MA, 2006.
[17] A. R. Geist and S. Trimpe. Structured learning rigid-body dynamics: survey and unified view from a robotics perspective. GAMM Mitteilungen, 44(2):34, 2021.
[18] R. W. Beard. Linear operator equations with applications in control and signal processing. IEEE Control Systems Magazine, 22(2):69-79, 2002.
[19] F. E. Udwadia and R. E. Kalaba. On the foundations of analytical dynamics. International Journal of non-linear mechanics, 1079-109,, 2002.
[20] M. Lutter, C. Ritter, and J. Peters. Deep lagrangian networks: Using physics as model prior for deep learning. May 2019.
[21] M. Lutter, K. Listmann, and J. Peters. Deep lagrangian networks for end-to-end learning of energy-based control for under-actuated systems. In 2019 JEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 7718-7725. IEEE, 2019.
[22] M. Cranmer, S. Greydanus. S. Hoyer, P. Battaglia, D. Spergel, and S. Ho. Lagrangian neural networks. ICLR 2020 Workshop on Integration of Deep Neural Models and Differential Equations, 2020.
[23] J. K. Gupta, K. Menda, Z. Manchester, and M. Kochenderfer. Structured mechanical models for robot learning and control. In Proceedings of the 2nd Conference on Learning for Dynamics and Control, volume 120 of Proceedings of Machine Learning Research, pages 328-337, The Cloud, 10-11 Jun 2020. PMLR.
[24] M. Lutter, J. Silberbauer, Watson, and J. Peters. A differentiable newton euler algorithm for multi-body model learning. arXiv preprint arXiv:2010.09800 2020.
[25] J. Lee, J. Hwangbo, L. Wellhausen, V. Koltun, and M. Hutter. Learning quadrupedal locomo- tion over challenging terrain. Science robotics, 5(47). 2020.
[26] K. Chua, Calandra, R. McAllister, and Levine. Deep reinforcement learning in hand- ful of trials using probabilistic dynamics models. In S. Bengio. H. Wallach, H. Larochelle, K. Grauman, N. Cesa- Dasn-Biachhi, and R. Garnett, editors, Advances in Neural Information Pro- cessing Systems, volume 31. Curran Associates, Inc., 2018.
[27] J. Ting, M. Mistry, J. Nakanishi, J. Peters, and S. Schaal. A bayesian approach to nonlinear parameter identification for rigid body dynamics. In Robotics: Science and Systems (RSS 2006). cambridge, ma: mit press, 2006.
[28] J. Koeijan, A. Girard, B. Banko, and R. Murray-Smith. Dynamic systems identification with Gaussian processes. Mathematical and Computer Modelling of Dynamical Systems, 11(4): 411 4-142, 2005.
[29] R. Frigola, F. Lindsten, T. B. Schön, and C. F. Rasmussen. Bayesian inference and learning in Gaussian process state-space models with particle MCMC. In Advances in Neural Information Processing Systems, pages 3156-316 3164, 2013.
[30] C. L. C. Mattos, A. Damianou, G. A. Barreto, and N. D. Lawrence. Latent autoregressive Gaussian processes models for robust system identification. PaAer-PapessOniimee 49(7):1121- 1 1126, 2016.
[31] S. Eleftheriadis, T. Nicholson, M. Deisenroth, and Hensman. Identification of Gaussian process state space models. In Advances in neural information processing systems, pages 3309-5331,2011
[32] A. Doerr, C. Daniel, M. Schiegg, N.-T. Duy, S. Schaal, M. Toussaint, and S. Trimpe. Prob- abilistic recurrent state-space models. volume 80 of Proceedings of Machine Learning Re- search, Stockholm Sweden, 10-15 Jul 2018, PMLR.
[33] D. Squyen-Tuong and J. Peters. Using model knowledge for learning inverse dynamics. In 2010 IEEE International Conference on Robotics and Automation, pages 2677-2682, 2010.
10

---

[34] M. Saveriano, Y. Yin, P. Falco, and D. Lee. Data-efficient control policy search using residual dynamics learning. In 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 4709-4715 2017. 200:10001000RRS5222770266666
[35] C.-A. Cheng, H.-P. Huang, H.-K Hsu, W.-z. Lai, and C.-C. Cheng. Learning the inverse dynamics of robotic manipulators in structured reproducing kernel hilbert space. IEEE trans- actions on cybernetics, 46(7):1691-11)) 2015.
[36] E. Solak, R. Murray-Smith, W. E. Leithead, D. J. Leith, and C. E. Rasmussen. Derivative observations in gaussian process models of dynamic systems, 2003.
[37] C. Jidling, N. Wahlström, A. Wills, and T. B. Schön. Linearly constrained gaussian processes.
In Advances in Neural Information Processing Systems (NeurIPS), pages 1215-1224, 2017.
[38] A. Geist and S. Trimpe. Learning constrained dynamics with gauss' principle adhering gaus- sian processes, In Proceedings of the 2nd Conference on Learning for Dynamics and Control, volume 120 of Proceedings of Machine Learning Research, pages 225-234--4422222222222222---2222 PMLR, 10-11 Jun 2020.
[39] J. Bradbury, R. Frostig, P. Hawkins, M. J. Johnson, C. Leary, D. Maclaurin, G. Necula, A. Paszke, J. VanderPlas, S. Wanderman-Milhe and Q. Zhang. JAX: composable transfor- mations of Python+NumPy programs, 2018. URL http: pttt:://githie //github. caaiib..oo/oooooo nom/google/jax
[40] A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Kopf, E. Yang. Z. Vito, M. Raison, A. Te- jani, Chilamkurthy, B. Steiner, L. Fang. Bai, and S. Chintala. Pytorch: An imperative style. high-performance deep learning library. In H. Wallach, H. Larochelle, A. Beygelzimer, d' Alché Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems 32, pages 8024 8035. Curran Associates, Inc., 2019.
[41] D. Maclaurin, D. Duvenaud, and R. P. Adams. Autograd: Reverse-mode differentiation of native python. In ICML. workshop on Automatic Machine Learning. 2015.
[42] J. R. Gardner, G. Pleiss, D. Bindel, K. Q. Weinberger, and A. G. Wilson. Gpytorch: Black- box matrix-matrix gaussian process inference with gpu acceleration. In Advances in Neural Information Processing Systems, 2018.
[43] R. E. Wengert. A simple automatic derivative evaluation program. Commun. ACM, 7(8): 463-464, Aug. 1964. ISSN 0001-0782.
[44] E. Coumans and Y. Bai. Pybullet, python module for physics simulation for games, robotics and machine learning. http://pybullet.co org, 2016-203 2022..
[45] C. R. Qi, I. Yi, H. Su, and L. J. Guibas. Pointnet++:: Deep hierarchical feature learning on point sets in a metric space. In Proceedings of the 31st International Conference on Neural In- formation Processing Systems, NIPS' 17, page 5105-5114 14, Red Hook, NY, USA, 2017. Curran Associates Inc. ISBN 9781510860964.
[46] D. P. Kingma and J. Ba. Adam: A Method for Stochastic Optimization. arXiv:/412.6600 [cs], Dec. 2014. URL http:// /aarivv//iiv.////////// www.aiwww.waaaaaaa.eaea.oo 1412. 6980. arXiv: 1412.6980.
[47] J. Baumgarte. Stabilization of constraints and integrals of motion in dynamical systems. Com- puter methods in applied mechanics and engineering, 1(1):1-16, 1972.
[48] A. G. Wilson, Z. Hu, R. Salakhutdinov, and E. P. Xing. Deep kernel learning. In Artificial intelligence and statisfics, pages 370-378. PMLR, 2016.
[49] S. W. Ober, C. E. Rasmussen, and M. van der Wilk. The promises and pitfalls of deep kernel learning, CoRR, 12100 2021. URL https: /
12108.


Reviews:
[{"review_id": "agvkr6r3KX", "reviewer": "Reviewer_2B5H", "summary": "This paper presents a method for embedding prior knowledge on the dynamics of a robot (e.g. in the form of physical constraints that the system has to satisfy, the inertia matrix, etc.) inside a dynamics-learning framework in order to help the model generalize better. This is done by modeling the robot dynamics as a Gaussian process, and incorporating the a-priori known dynamics information in the GP's mean function. The method is benchmarked on a 7-DOF arm following a planar end-effector trajectory and the dynamics models learned using the approach are shown to generalize better than baseline approaches.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n- The paper is very clearly written, providing an extensive motivation of the problem studied and an extensive overview of prior work.\n- The paper is technically solid, though it is admittedly easy to get lost in the notation at times (a table summarizing all the different forces would be helpful for the reader).\n\nWeaknesses\n- Overall, the contribution appears to be a somewhat minimal modification to reference [35] in the paper. From my understanding, the differences are twofold: 1) swapping out modeling the unknown accelerations with modeling the unknown forces and 2) implementing the GP training in GPyTorch. The paper claims on line 192 that the first change can improve sample efficiency, yet there is no baseline comparison in the results with a method that learns the accelerations directly (i.e. a variant of [35] which is trained using GPyTorch), which leaves me uncertain of the relative improvement of this paper's method over [35].\n- The use of \\theta_A is rather confusing, as it is used to refer to different subsets of the mass matrix/forces and to both known and unknown parameters. In the discussion near line 80, \\theta_A refers to any model parameters, including the mass matrix and all forces. In line 231, it is used to denote possibly erroneous parameters of the mass matrix M, but in line 243 it is claimed that M is known. Later in the results section (line 300), it seems to be instead used to denote components of Q_G, Q_C, and Q_u. It would help clarify things if \\theta_A were explicitly described for each example involving the 7-DOF arm in the results.\n- While the primary benefit of modeling the forces instead of the accelerations is to take advantage of prior knowledge in known components of the forces/the inertia matrix, the paper doesn't sufficiently motivate why we might have prior knowledge for some of the discussed components. The mass matrix M and A, b (arising from the constraint manifold) are motivated in the paragraphs, but it's not clear why we might have prior knowledge on Q_C or Q_D, for instance. More discussion on why it might be plausible to have prior knowledge on each analytical dynamics parameter would be useful.\n- The results are a bit confusing:\n---- In Figure 2b, why are there multiple curves with the same color? Are these just the loss curves for different training runs?\n---- There is a baseline comparison to a neural network discussed in the text, which based on those numbers seems to have similar performance as the proposed methods, but it does not appear in Figure 2; why is this? Also, from the supplementary, it seems like the NN is trained using more points than the GP-based methods; using the same number of points would help ease comparison. More discussion comparing the NN to the proposed approach would help.\n---- Q is modeled as Q_u + Q_G + Q_C in this example; why are dissipative forces Q_D not considered?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "While the paper is overall well-written and technically proficient, I am not fully convinced of the method's contribution compared to prior work [35] or the plausibility of having prior information on certain dynamics parameters. A revision along the lines of what follows would help improve my confidence in the paper.", "technical_quality": "Very Good:  Convincing.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed.", "reviewer_expertise": "Very good: Comprehensive knowledge of the area"}, "extra_text": {"issues": "- I would be more convinced of the utility of the method if there was a baseline comparison with [35], with the training procedure changed to use GPyTorch-based inference.\n- I would also be more convinced if the authors could provide justification on where the a-priori information on the unknown parameters might come from.\n- It would help to expand the discussion of the results and clarify the usage of \\theta_A, as described above."}}, {"review_id": "RGIDIJWRLhF", "reviewer": "Reviewer_Ahf1", "summary": "In this paper, the authors propose a structured learning approach to model the forward dynamics of a robot with implicitly constrained dynamics. For this purpose, they put a Gaussian Process prior on the unknown forces of the system. The basic structure of the equations of motion and the constrained dynamics are included in the Gaussian Process model by exploiting that GPs are closed under linear transformations. The hyperparameters are optimized by GPU accelerated computation. A simulation with a 7-dof robot shows the effectiveness of the proposed approach.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength\n-\tHigh quality simulation that is described in detail in the supplementary material\n\nWeakness\n-\tSloppy problem setting and motivation \n- Approach is very similar to existing work [35]\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "The problem presented in this paper is of interest and relevance. However, the solution seems to be very similar to [35] by simply exploiting the fact the GPs are closed under linear transformation. I do not see a contribution in the GPU acceleration as the MLE (14) is standard and GPU optimization packages are quite common nowadays. Thus, I see the main contribution in the thorough simulation. Therefore, it's mostly incremental theoretical work but with a nice simulation.\n\nUpdate: Score remains unchanged. See reviewer comment for more information.", "technical_quality": "Very Good:  Convincing.", "clarity_of_presentation": "Good: Minor revision needed to improve organization or clarity.", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed.", "reviewer_expertise": "Very good: Comprehensive knowledge of the area"}, "extra_text": {"issues": "Major:\n- The problem setting is very sloppy. What means “sufficiently rigid”? What is an “accurate representation? Why “NOISY input point”? $q_k$, etc. are not defined at this point. What is an “informative data set”? “EOM” not defined here….\nI would suggest to clarify in particular Assumption 2. The current form is not acceptable. Just consider rigid body motion.\n- The general motivation to put the prior on the forces and not on the acceleration as in [35] in not sufficiently discussed. Furthermore, in line 152 “for some applications it can be more practically […] as we propose in this work? Which applications? Why?\n\nMinor:\n- The motivation for the usage of data-driven model is given by (stochastic) friction, backlash, etc. Yet, the simulation contains a very simple friction model (viscous friction). The results would be more convincing if you include more “advanced” types of friction in the system.\n- Line 16/17: I would rather say any model-based control approach (includes standard control approaches such as computed torque for example)\n- Line 26 and more: The term “analytical model” is not well defined. Do you mean parametric models? I think a GP is a (probabilistic) analytical model as well.\n- Line 85: “$Q_u+Q_D$ […] being the joint torques that are measurable in the robot joints” Why is the gravitational force not measurable in the joints?\n-Line 277: It’s a simulation and not an experiment.\n"}}, {"review_id": "9dnOEXpyP5K", "reviewer": "Reviewer_HDXR", "summary": "The paper proposes a grey box model of rigid body dynamics (RBD) by combining a white box RBD model with a Gaussian process over external forces. \nThis model is evaluated in simulation of a contact-heavy manipulatior task.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "I thought this was a beautiful paper and was the best on my stack. \nThe idea of greybox modelling via the external forces is original and makes sense. \nThe inclusion of constrained dynamics is also a nice touch compared to previous data-driven RBD work.. \n\nThe literature review was nice and informative. The only missing work I could think of is [A].\n\nThe technical section was very readable. \n\nThe weakness is in the experiments. For model learning, the real world is often very different to simulators (especially w.r.t. contract forces) so its a shame that we cannot see how the model scales to the real world. Do the authors have access to a real manipulator?\n\nA second issue is with the scalability of GPs. I imagine sparse GPs slot into this framework quite easily, but would be good to see how the approximation effects the performance.  \n\nA neural network baseline of [20 30] is too small, especially has MBRL dynamics models are easily in the size of [200 200 200 200]. \n\nA final weakness is the title. The current title is far too general and doesn't distinguish this work from prior work. The title should include rigid body dynamics, Gaussian processes and their use as modelling external forces. \n\nLine 339: isn't a quality of the SE kernel that it perfoms ARD?\n\nMinor formatting suggestions:\n- consider using boldface for matrices and vectors  \n- JMLR capitalization rules hold only for names of people and places\n- it looks like you use \\intercal and \\top as your transpose symbol. \n\n\n[A] A Bayesian Approach to Nonlinear Parameter Identification for Rigid Body Dynamics RSS Ting et al. 2006", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "Well executed paper on gray box model learning with GPs via external forces. A real robot experiment would be really good to properly evaluate the model learning.  ", "technical_quality": "Very Good:  Convincing.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "4: The work contains interesting new ideas that will potentially have major impact in robotics or machine learning.", "issues": "See above.", "reviewer_expertise": "Very good: Comprehensive knowledge of the area"}, "extra_text": {}}, {"review_id": "4VURsprRryv", "reviewer": "Reviewer_ZmdF", "summary": "The paper proposes a method to learn forward dynamics while explicitly enforcing kinematic, inertia and implicit constraint equations. The constraint matrix, inertia matrix and known forces are explicitly estimated as the mean of the Gaussian Process (GP) Regression, while the variance of the GP captures uncertainties. The model parameters are trained using auto-differentiation. Experiment results in simulation show that, training models with structured knowledge converges faster and leads to smaller prediction errors.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- The paper proposes a novel method to combine analytical and data-driven dynamics modeling. Despite its complexity, the model can be trained using auto-differentiation with fast GPU acceleration, which is promising.\n\nWeaknesses:\n- Lack of sufficient comparisons with existing baseline:\nThe author only compared their approach with standard GP and a standard feed-forward neural network. It would be nice to see the comparison with PETS [1], which uses an ensemble of probabilistic models to capture uncertainty. Reading from Fig.2 and 3 in the appendix, it seems like a major benefit of GP is the modeling of uncertainty and lower prediction errors, which is achievable by PETS.\n\n- Organization:\nThe authors should better organize the main paper and appendices. Currently, the main paper focuses excessively heavy on problem setup and preliminary information, while a lot of interesting results and analyses remain in the appendix. My suggestions is as follows:\n* Trim down section 1.2 and focus more on intuitions. What does $Q$, $Q_z$, $Q_I$ mean? Some examples could be helpful here ($Q_I, Q_Z$ in Fig.1 isn't exactly clear).\n* Trim down the related works section. Right now it's overly detailed.\n* Trim down section 2.1, or move it to Appendix. \n* Move Fig.2/Fig.3 from the Appendix to the main paper, as well as related analysis. For example, why does the analytical model have significantly larger errors in dimension 4, 5, 6 only? Why does GP have significantly higher uncertainty in dimension 2 and 4 compared to other dimensions? It would be nice to investigate this in greater details.\n* Move Section 5 in Appendix to the main paper, and add more results on long-term prediction accuracy.\n\nReferences:\n[1] Chua, Kurtland, et al. \"Deep reinforcement learning in a handful of trials using probabilistic dynamics models.\" arXiv preprint arXiv:1805.12114 (2018).", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "[Update on Aug.30]\n\nUpdate: I have updated the recommendation based on the author's responses. The paper's clarity is also improved in the revision. I have NOT updated the \"clarity of representation\" score yet.\n\n[Original Comment]\n\nThe paper presents a novel way to model forward dynamics by combining both physics knowledge and data-driven regression. The main reason for the current recommendation is the organization: the main paper contains an excessive amount of details and preliminary information, while the results and analysis remain concise. I would love to re-evaluate the paper after revision.", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Fair: Major revision needed to improve organization or clarity.", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "4: The work contains interesting new ideas that will potentially have major impact in robotics or machine learning.", "issues": "See \"Strengths And Weaknesses\" above.", "reviewer_expertise": "Poor: Limited knowledge of the area"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes the use of additional structure/model knowledge to improve the learning of forward dynamics models.

Quality:

(-) Reviewers 2B5H, Ahf1, and ZmdF have concerns about the contribution of the manuscript compared to existing literature. Adding an experimental comparison to past literature would significantly strengthen the paper and better demonstrate the contributions of the proposed approach.

Clarity:

(+) The paper is generally clear

(-) Reviewers Ahf1 and ZmdF noted that the problem setting should be improved.

(-) All the reviewers suggested several improvements in notation, presentation of the results, and even title. Please address these comments.

Originality: 

(-) There have been several works in the literature exploring the use of structured priors and/or model priors to improve the learning of forward dynamics models.  
  
(+) To my knowledge, the use of this specific parametrization is novel.

Significance: 

(+) This topic is important and impactful 

(-) As highlighted by Reviewers 2B5H and ZmdF, the manuscript does not compare against any of the existing literature. As such, it is impossible to evaluate the significance of the contribution.
Decision: Accept (Poster)