Venue: CoRL 2023 Conference

Reviews:

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary_of_paper: This paper proposes a framework for simultaneously learning smooth motion and contacts. This is done by starting with an implicit method for resolving contacts by solving appropriate convex optimization problems, and parameterizing it using a dynamics model. Rather than differentiating through the contact events directly, the authors use a violation-based implicit loss to balance contact events with parameter learning through a bilevel optimization problem. The authors create and release a dataset of tosses of a two-link rigid object, and evaluate their ideas using this dataset.
review: Strengths
* **Important topic.** Figuring out how to simultaneously model systems with smooth behavior and contacts is a critically important direction for robotics, and I entirely agree with the authors desire to study this.
* **Non-trivial problem.** Simultaneous modeling of smooth dynamics and contacts is known to be hard, so the authors approach - and indeed any approach which performs well even on simple problems - is very respectable.
* **Two-link articulated object experiment** is particularly non-trivial compared to a pure rigid object compared to a plastic cube, because one needs to simultaneously understand how the object rotates and how it moves due to contacts.
* **Dataset contribution.** The authors make available a dataset of simulated trajectories, which is very good because it allows others who may not have the same hardware access to also work on similar problems. From the pictures, I have reasonable confidence that the dataset collected is of reasonable quality and does not have so much noise, etc in, and is a good contribution.

Weaknesses
* **Misleading, overly general title.** Simultaneous learning of contact and continuous dynamics has already been studied in many prior works. This paper is proposing a very specific way to learn both components together. Even for a review paper, the present title would have been inappropriate without the addition of the word "review" 
* **In too many places, the writing is vague and not specific enough.** See "questions regarding vagueness" in the questions section.
* **Not clear what is from prior work and what are the authors' original ideas**. Prediction-based losses are definitely not new, but the authors cite papers for their violation-based loss too. Is this new? If this is a paper that brings together a bunch of existing ideas to do a fancy experiment, that's totally fine, but it needs to be made clear that this is what is happening.
* **No explicit consideration of noise - a key issue - in the formalism** in simultaneous modeling of smooth and contact dynamics. One of the key difficulties working in this area in my experience is that simultaneous noise and contacts introduce a fundamental ambiguity: one can explain discontinuous motion due to contacts, or due to noise, and in the absence of some kind of signal it need not be possible, even for a human, to distinguish one from the other. This is written about in some detail by Hochlehnert et al (2021), "Learning Contact Dynamics using Physically Structured Neural Networks".
* **Blurry figures, hard to view.** Figures 1, 2, and others are not vector graphics and should be converted to those to ensure the text is large and sharp enough to be accessible.
robotics_focus: Highly relevant to robotics but no hardware experiments
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: My view is that this paper is borderline, and contains both good reasons to accept such as the dataset made available, and good reasons to reject such as confusing presentation. In general, I think this paper would be significantly improved if the authors edit it and submit a revised version to a future conference on basis of feedback obtained via reviewing. However, I am not very confident because it is not completely clear to me what is new and what is not, and I am very curious to see other reviewers' opinions, and could therefore probably be persuaded by other reviewers to change my mind if there are good reasons for this.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary_of_paper: The main idea and contributions of the paper are the following points: 
* The authors developed a system identification method using a violation-based implicit loss and residual physics to learn continuous and contact dynamics simultaneously.
* The authors demonstrated that the proposed method could learn the continuous and contact dynamics more accurately than differentiable simulation and end-to-end learning methods.

review: Strengths:
* The paper is well organized and seems to provide sufficient information to the reader. Therefore, the clarity of the paper is high.

Weaknesses:
* The experimental settings for the real experiments are not appropriate since the effectiveness of the residual physics cannot be not demonstrated. Moreover, it is unclear why the differentiable simulation using Anitescu [34] was chosen for comparison. Thus, the quality of the paper is not high.
* Since the validity of the residual physics is not fully demonstrated, the paper seems to be a minor update of ContactNets [7]. Thus, the originality of the paper seems week.
* Although the proposed method outperformed the differentiable simulation and end-to-end learning methods, from the accompanying media, there still seems to have a large prediction error. Therefore, it is difficult to assess the significance of the paper.

robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: The contribution of the paper is not sufficient: weakness in both quality and originality of the paper. The effectiveness of a part of the proposed method has not been adequately demonstrated.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary_of_paper: The authors introduce a method for learning the continuous dynamics of articulated rigid body dynamics in conjunction with the contact dynamics from real-world object motions. The system identification method extends the ContactNets method to introduce a violation-based loss function that constraints the violation of physical constraints and laws of motion for multi-body systems. In addition, this work introduces a new dataset of real-world tosses of an articulated object onto a table. In addition to the real-world cube tossing dataset and simulated scenarios, the proposed method is compared against explicit simulation-based and end-to-end dynamics learning approaches.
review: ## Strengths

The approach to formulate the dynamics violation as a soft constraint inside the system identification loss is promising and the results clearly show the benefit of this technique.

The proposed violation-based loss can be easily adapted by other system identification works to improve upon the commonly used prediction loss.

The authors promise to release the dataset of tossing articulated objects which would be very valuable to this area of research.

## Weaknesses

The technical contributions appear quite incremental where the ContactNets approach is extended via a new violation-based implicit loss function to learn continuous dynamics parameters for articulated rigid-body systems.

The system ID approach assumed that the articulation is known upfront, i.e. the joint connectivity and number of rigid bodies.
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: Although the technical contributions are quite incremental, the execution of the paper is well done with a detailed analysis of the proposed system ID method against common baselines. The release of the multi-body tossing dataset is a welcome addition to the field of learning continuous and contact dynamics, hence I recommend acceptance of this work.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary_of_paper: The paper proposes an extension to "ContactNets" proposed in [1]. 

ContactNets assume that the smooth "non-contact" impulses and the system's inertia matrix governing the system's dynamics are known. [1] proposed that one can directly learn the parameters of the signed distance function and its partial derivative using a "violation-based loss function". In this loss an implicit function is included explicitly imposing structure onto the contact forces. As pointed out in [2], such a loss compares favorable to the loss of a "naive implicit model".

The authors contributions are:

1) While ContactNets assume that the "non-contact" dynamics are known, the authors learn these dynamics using residual physics. That is, the "non-contact" dynamics are modeled analytically where in particular the learnable parameters of the inertia matrix are constrained as done in [3] and the analytical model's residuals are learned with an MLP.
2) A data set of a two-link articulated object is provided.
review: The paper proposes an interesting extension to ContactNets. The work appears to be novel and is clearly relevant to the field.
Below I list of major comments and minor comments to the proposed work.

C1: Quite simple dynamical systems and limited experiment scope.
The chosen dynamical systems in the absence of contacts are really simple. The first system are two rigid-bodies that are connected by a revolute joint and the second system is a single rigid body that is pulled by a force field. In the absence of contacts it would be trivial to learn these dynamics with a purely data-driven or physics-informed model. As the violation-based loss as in [1] and [2] handles the discontiniuous nature of contacts, it is not really surprising that learning some residuals of very simple dynamics is possible. Moreover, most of the proposed paper focuses on the questions "Does it work?". However, it would be more insightful to look at when such a residual physics approach does not work. For example... How large can the error in the inertia parameters or initial signed distance functions be until the parameters do not converge? Can CCN-R learn the dynamics of a soft contact surface? What are the limitations of this approach?

C2: Model baselines
It is not stated why DiffSim has been chosen as a baseline of comparison. I would be interested to know the reasons for choosing DiffSim being an approach from 2006 while a lot has happened in contact dynamics identification in the last 17 years. Moreover, that the end-to-end model is chosen as a four layer MLP is a bit surprising. I found that ResNets and RNNs significantly outperform MLPs when learning dynamics. When benchmarking CCN to the state-of-the-art, the authors should consider adding a sign activation to the network's first layer to give it a realistic chance of learning discontinuous contact dynamics. After all, a network using smooth activation functions such as Tanh and Softplus might require substantial amounts of data to learn discontinuities.

C3: Figure 2
It took me some time to decipher what the authors want to convey with Figure 2. I guess that the figure shall visualize the model structure proposed in [2] using a "violation-based loss" in comparison to an explicit model / "naive implicit model". However, the terms "contact solver", "learning", and "optimization" depending on the context may be used interchangeably while the term "realism measure" is not introduced in the text. In addition, even if the figure would be clear it only shows the model structure as proposed in [1] and discussed in detail in [2]. I would  prefer that the authors use the page space to illustrate their own contributions.

C4: The published workshop paper "Simultaneously Learning Contact and Continuous Dynamics" 
https://neural-implicit-workshop.stanford.edu/assets/pdf/bianchini.pdf
has not been cited even though it contains the same figure. I kindly ask the authors to cite this work and emphasize in the related work section how the proposed work forms a significant extension to this prior publication.

MC1: Writing style

Paper structure:
- The model baselines are introduced in the description of Table 1. I would prefer a brief paragraph detailing the models instead. 

Verbose writing:
- Table 1: "End-to-end baseline ABLATES the physical structure imposed by the rigid body model-based parameterization" - While you indeed do an ablation study, I would simply say that you use an MLP for the end-to-end approach.
- Why mention "Newton’s third law" in line 106? Instead you could just say that the function g is obtained by "forward integration of the systems dynamics and adding the contact impulse transfer". Ideally, after pointing to the "system dynamics" you also write the number of the equation that you are referring to.

Imprecise wording:
- Page 6 - "Featuring actual dynamics" - what does "actual" mean in this context?
- Line 162 to Line 164: "... penalizes errors in velocity space, since configurations are an affine function of velocity predictions (6b)." - The loss is on $x$ being the concatenation of position and velocity variables. So of course it penalizes velocity errors. How does this relate to (6b) describing position through a forward euler discretization aka "affine function of velocity predictions"?

MC2: Some paragraphs of the related work section to not detail how the proposed work relates to the mentioned prior work. For example, it would help if the authors add a sentence on how previous developed mappings to the inertia parameters relate to the particular mapping used in this work.

MC3: The title of citation number 7 in the proposed work includes an "of" that is not part of the paper's actual title.

MC4: Some Figures depict confidence intervals, but I could not find a statement detailing the number of training runs and what exactly has been random in these runs.

MC5: The code was not accessible during review time. I would have liked to check the code and data set. Especially, as the data set forms a contribution of this work.

Remark 1: Equation 11 seems related to "Gauss Principle of least constraint". Gauss postulated that nature chooses the implicit constraint forces $Q=J^T \lambda'$ such that a functional $G(q) = Q(q, \dot q)^T M(q)^{-1}Q(q, \dot q)$ is minimized. If one minimizes this functional by introducing a Lagrange multiplier vector (the implicit constraint forces aka contact normal force) then one arrives at the standard EOM of an implicitly-constrained system.

[1] "Contactnets: Learning discontinuous contact dynamics with smooth, implicit representations", Pfrommer et al., CORL 2020
[2] "Generalization Bounded Implicit Learning of Nearly Discontinuous Functions", Bianchini et al., L4DC 2022
[3] "Smooth parameterization of rigid-body inertia", Rucker and Wensing, RA-L 2022
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: While the paper has some notable strengths, I lean toward a marginal accept. The research topic is relevant and the methodology is generally sound, although the experimental result section requires further refinement. The findings offer modest contributions to the field but may lack strong impact considering the already extensive literature on learning "non-contact" dynamics with residual physics. The writing is clear, but the writing style uses at times ambiguous wording and is not on point. The weakest point of this paper are the chosen model baselines which have been insufficiently been discussed and do not form the state-of-the-art. Overall, this submission has potential and warrants further discussion, but would clearly benefit from a minor revision to strengthen its impact.
review_title: null

Meta-review: All reviewers agreed that the paper containes a interesting contribution for a very challenging problem, learning contact and continuous dynamics. Moreover, the paper offers an interesting dataset that will be useful for the community. Initial  concerns regarding clarity of the paper and novelty have been addressed well by the authors. All reviewers vote for acceptance of the paper. I agree with this judgement.
Decision: Accept (Poster)