Venue: CoRL 2021 Conference

Reviews:

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This paper presents a method for embedding prior knowledge on the dynamics of a robot (e.g. in the form of physical constraints that the system has to satisfy, the inertia matrix, etc.) inside a dynamics-learning framework in order to help the model generalize better. This is done by modeling the robot dynamics as a Gaussian process, and incorporating the a-priori known dynamics information in the GP's mean function. The method is benchmarked on a 7-DOF arm following a planar end-effector trajectory and the dynamics models learned using the approach are shown to generalize better than baseline approaches.
strengths_and_weaknesses: Strengths
- The paper is very clearly written, providing an extensive motivation of the problem studied and an extensive overview of prior work.
- The paper is technically solid, though it is admittedly easy to get lost in the notation at times (a table summarizing all the different forces would be helpful for the reader).

Weaknesses
- Overall, the contribution appears to be a somewhat minimal modification to reference [35] in the paper. From my understanding, the differences are twofold: 1) swapping out modeling the unknown accelerations with modeling the unknown forces and 2) implementing the GP training in GPyTorch. The paper claims on line 192 that the first change can improve sample efficiency, yet there is no baseline comparison in the results with a method that learns the accelerations directly (i.e. a variant of [35] which is trained using GPyTorch), which leaves me uncertain of the relative improvement of this paper's method over [35].
- The use of \theta_A is rather confusing, as it is used to refer to different subsets of the mass matrix/forces and to both known and unknown parameters. In the discussion near line 80, \theta_A refers to any model parameters, including the mass matrix and all forces. In line 231, it is used to denote possibly erroneous parameters of the mass matrix M, but in line 243 it is claimed that M is known. Later in the results section (line 300), it seems to be instead used to denote components of Q_G, Q_C, and Q_u. It would help clarify things if \theta_A were explicitly described for each example involving the 7-DOF arm in the results.
- While the primary benefit of modeling the forces instead of the accelerations is to take advantage of prior knowledge in known components of the forces/the inertia matrix, the paper doesn't sufficiently motivate why we might have prior knowledge for some of the discussed components. The mass matrix M and A, b (arising from the constraint manifold) are motivated in the paragraphs, but it's not clear why we might have prior knowledge on Q_C or Q_D, for instance. More discussion on why it might be plausible to have prior knowledge on each analytical dynamics parameter would be useful.
- The results are a bit confusing:
---- In Figure 2b, why are there multiple curves with the same color? Are these just the loss curves for different training runs?
---- There is a baseline comparison to a neural network discussed in the text, which based on those numbers seems to have similar performance as the proposed methods, but it does not appear in Figure 2; why is this? Also, from the supplementary, it seems like the NN is trained using more points than the GP-based methods; using the same number of points would help ease comparison. More discussion comparing the NN to the proposed approach would help.
---- Q is modeled as Q_u + Q_G + Q_C in this example; why are dissipative forces Q_D not considered?
summary_of_recommendation: While the paper is overall well-written and technically proficient, I am not fully convinced of the method's contribution compared to prior work [35] or the plausibility of having prior information on certain dynamics parameters. A revision along the lines of what follows would help improve my confidence in the paper.
issues: - I would be more convinced of the utility of the method if there was a baseline comparison with [35], with the training procedure changed to use GPyTorch-based inference.
- I would also be more convinced if the authors could provide justification on where the a-priori information on the unknown parameters might come from.
- It would help to expand the discussion of the results and clarify the usage of \theta_A, as described above.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: In this paper, the authors propose a structured learning approach to model the forward dynamics of a robot with implicitly constrained dynamics. For this purpose, they put a Gaussian Process prior on the unknown forces of the system. The basic structure of the equations of motion and the constrained dynamics are included in the Gaussian Process model by exploiting that GPs are closed under linear transformations. The hyperparameters are optimized by GPU accelerated computation. A simulation with a 7-dof robot shows the effectiveness of the proposed approach.
strengths_and_weaknesses: Strength
-	High quality simulation that is described in detail in the supplementary material

Weakness
-	Sloppy problem setting and motivation 
- Approach is very similar to existing work [35]

summary_of_recommendation: The problem presented in this paper is of interest and relevance. However, the solution seems to be very similar to [35] by simply exploiting the fact the GPs are closed under linear transformation. I do not see a contribution in the GPU acceleration as the MLE (14) is standard and GPU optimization packages are quite common nowadays. Thus, I see the main contribution in the thorough simulation. Therefore, it's mostly incremental theoretical work but with a nice simulation.

Update: Score remains unchanged. See reviewer comment for more information.
issues: Major:
- The problem setting is very sloppy. What means “sufficiently rigid”? What is an “accurate representation? Why “NOISY input point”? $q_k$, etc. are not defined at this point. What is an “informative data set”? “EOM” not defined here….
I would suggest to clarify in particular Assumption 2. The current form is not acceptable. Just consider rigid body motion.
- The general motivation to put the prior on the forces and not on the acceleration as in [35] in not sufficiently discussed. Furthermore, in line 152 “for some applications it can be more practically […] as we propose in this work? Which applications? Why?

Minor:
- The motivation for the usage of data-driven model is given by (stochastic) friction, backlash, etc. Yet, the simulation contains a very simple friction model (viscous friction). The results would be more convincing if you include more “advanced” types of friction in the system.
- Line 16/17: I would rather say any model-based control approach (includes standard control approaches such as computed torque for example)
- Line 26 and more: The term “analytical model” is not well defined. Do you mean parametric models? I think a GP is a (probabilistic) analytical model as well.
- Line 85: “$Q_u+Q_D$ […] being the joint torques that are measurable in the robot joints” Why is the gravitational force not measurable in the joints?
-Line 277: It’s a simulation and not an experiment.
review_title: null

final_score: rating: Strong Accept: I recommend accepting the paper and will argue for my recommendation even if other reviewers hold a different opinion.
review_content: summary: The paper proposes a grey box model of rigid body dynamics (RBD) by combining a white box RBD model with a Gaussian process over external forces. 
This model is evaluated in simulation of a contact-heavy manipulatior task.
strengths_and_weaknesses: I thought this was a beautiful paper and was the best on my stack. 
The idea of greybox modelling via the external forces is original and makes sense. 
The inclusion of constrained dynamics is also a nice touch compared to previous data-driven RBD work.. 

The literature review was nice and informative. The only missing work I could think of is [A].

The technical section was very readable. 

The weakness is in the experiments. For model learning, the real world is often very different to simulators (especially w.r.t. contract forces) so its a shame that we cannot see how the model scales to the real world. Do the authors have access to a real manipulator?

A second issue is with the scalability of GPs. I imagine sparse GPs slot into this framework quite easily, but would be good to see how the approximation effects the performance.  

A neural network baseline of [20 30] is too small, especially has MBRL dynamics models are easily in the size of [200 200 200 200]. 

A final weakness is the title. The current title is far too general and doesn't distinguish this work from prior work. The title should include rigid body dynamics, Gaussian processes and their use as modelling external forces. 

Line 339: isn't a quality of the SE kernel that it perfoms ARD?

Minor formatting suggestions:
- consider using boldface for matrices and vectors  
- JMLR capitalization rules hold only for names of people and places
- it looks like you use \intercal and \top as your transpose symbol. 


[A] A Bayesian Approach to Nonlinear Parameter Identification for Rigid Body Dynamics RSS Ting et al. 2006
summary_of_recommendation: Well executed paper on gray box model learning with GPs via external forces. A real robot experiment would be really good to properly evaluate the model learning.  
issues: See above.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: The paper proposes a method to learn forward dynamics while explicitly enforcing kinematic, inertia and implicit constraint equations. The constraint matrix, inertia matrix and known forces are explicitly estimated as the mean of the Gaussian Process (GP) Regression, while the variance of the GP captures uncertainties. The model parameters are trained using auto-differentiation. Experiment results in simulation show that, training models with structured knowledge converges faster and leads to smaller prediction errors.
strengths_and_weaknesses: Strengths:
- The paper proposes a novel method to combine analytical and data-driven dynamics modeling. Despite its complexity, the model can be trained using auto-differentiation with fast GPU acceleration, which is promising.

Weaknesses:
- Lack of sufficient comparisons with existing baseline:
The author only compared their approach with standard GP and a standard feed-forward neural network. It would be nice to see the comparison with PETS [1], which uses an ensemble of probabilistic models to capture uncertainty. Reading from Fig.2 and 3 in the appendix, it seems like a major benefit of GP is the modeling of uncertainty and lower prediction errors, which is achievable by PETS.

- Organization:
The authors should better organize the main paper and appendices. Currently, the main paper focuses excessively heavy on problem setup and preliminary information, while a lot of interesting results and analyses remain in the appendix. My suggestions is as follows:
* Trim down section 1.2 and focus more on intuitions. What does $Q$, $Q_z$, $Q_I$ mean? Some examples could be helpful here ($Q_I, Q_Z$ in Fig.1 isn't exactly clear).
* Trim down the related works section. Right now it's overly detailed.
* Trim down section 2.1, or move it to Appendix. 
* Move Fig.2/Fig.3 from the Appendix to the main paper, as well as related analysis. For example, why does the analytical model have significantly larger errors in dimension 4, 5, 6 only? Why does GP have significantly higher uncertainty in dimension 2 and 4 compared to other dimensions? It would be nice to investigate this in greater details.
* Move Section 5 in Appendix to the main paper, and add more results on long-term prediction accuracy.

References:
[1] Chua, Kurtland, et al. "Deep reinforcement learning in a handful of trials using probabilistic dynamics models." arXiv preprint arXiv:1805.12114 (2018).
summary_of_recommendation: [Update on Aug.30]

Update: I have updated the recommendation based on the author's responses. The paper's clarity is also improved in the revision. I have NOT updated the "clarity of representation" score yet.

[Original Comment]

The paper presents a novel way to model forward dynamics by combining both physics knowledge and data-driven regression. The main reason for the current recommendation is the organization: the main paper contains an excessive amount of details and preliminary information, while the results and analysis remain concise. I would love to re-evaluate the paper after revision.
issues: See "Strengths And Weaknesses" above.
review_title: null

Meta-review: This paper proposes the use of additional structure/model knowledge to improve the learning of forward dynamics models.

Quality:

(-) Reviewers 2B5H, Ahf1, and ZmdF have concerns about the contribution of the manuscript compared to existing literature. Adding an experimental comparison to past literature would significantly strengthen the paper and better demonstrate the contributions of the proposed approach.

Clarity:

(+) The paper is generally clear

(-) Reviewers Ahf1 and ZmdF noted that the problem setting should be improved.

(-) All the reviewers suggested several improvements in notation, presentation of the results, and even title. Please address these comments.

Originality: 

(-) There have been several works in the literature exploring the use of structured priors and/or model priors to improve the learning of forward dynamics models.  
  
(+) To my knowledge, the use of this specific parametrization is novel.

Significance: 

(+) This topic is important and impactful 

(-) As highlighted by Reviewers 2B5H and ZmdF, the manuscript does not compare against any of the existing literature. As such, it is impossible to evaluate the significance of the contribution.
Decision: Accept (Poster)