Extending Lagrangian and Hamiltonian Neural Networks with Differentiable Contact Models

Abstract: The incorporation of appropriate inductive bias plays a critical role in learning dynamics from data. A growing body of work has been exploring ways to enforce energy conservation in the learned dynamics by encoding Lagrangian or Hamiltonian dynamics into the neural network architecture. These existing approaches are based on differential equations, which do not allow discontinuity in the states and thereby limit the class of systems one can learn. However, in reality, most physical systems, such as legged robots and robotic manipulators, involve contacts and collisions, which introduce discontinuities in the states. In this paper, we introduce a differentiable contact model, which can capture contact mechanics: frictionless/frictional, as well as elastic/inelastic. This model can also accommodate inequality constraints, such as limits on the joint angles. The proposed contact model extends the scope of Lagrangian and Hamiltonian neural networks by allowing simultaneous learning of contact and system properties. We demonstrate this framework on a series of challenging 2D and 3D physical systems with different coefficients of restitution and friction. The learned dynamics can be used as a differentiable physics simulator for downstream gradient-based optimization tasks, such as planning and control.


Reviews and author responses:

 Thank you very much for your update. After reading your rebuttal, I maintain my score to 7, conditioned on that the following requests can be fulfilled:

- I am not very familiar with NeuralSim codebase. The pseudo-code you showed seems to be an inefficient implementation of backpropagation. If so, please state this issue and explain how this affects your comparisons with NeuralSim in your revised manuscript.

- Thank you for the comments on the pros and cons of your method and the other methods. Please add them to your manuscript, and mention the pros and cons of your method explicitly in your introduction if possible.

This paper propose a novel method for learning simultaneously the properties of dynamics and contacts.
This method is implemented by a combination between C[HL]NN, Differentiable Convex Optimization Layers
and neural ODE. The main contribution is the ability to model frictional contacts both with or without elasticity.

The experiments demonstrate the superiority of the proposed method over the relevant baselines.


  originality:

the proposed combination between the existing frameworks (C[HL]NN, Differentiable Convex Optimization Layers
and neural ODE) for handling contacts with friction is novel 

quality:
the main claims of the paper are demonstrated to be true, and the method/model is reasonable.


clarity:
I strongly recommend to add a description of the entire method by a block diagram/picture/pseudo code.
Also, frequent jumping between the paper and the appendices affects the readability. 

significance:
even though the method is original, it is limited to particular classes of contact dynamics, and/or dimensionality of dynamics.
Specifically, there is no improvement in "Rope" environment, which indicates that the proposed solution is not general and/or scalable.  Consequently,  it is unclear when this method improves the baselines. Does it work only with a small number of contacts ( <= 8)? What is the main limitation? Does it perform better with more training data and/or larger networks? E.g., Table 3 might shed light on this issue, if the corresponding errors (as in Fig2+Fig3) would be added. Addressing this concern and providing a conclusion will improve the score. 


post rebuttal: following the clarifications and improvements by the authors I change my score to 7.

 the authors adequately addressed the limitations. 

 Thank you for your suggestion! We'll incorporate relevant discussion in camera-ready. 

From our side, we still see the score 5 instead of 7. Is it a technical glitch? If you haven't, could you kindly update the score in the system? Thank you!

 > Could you clarify, since you are saying: "train the model on CM generated data and LCP generated data". Are you also evaluating the LCP data trained model on LCP data? 

We've misunderstood your question and the errors in the previous table are all evaluated on CM validation data (100 trajectories.) We've created an update table below. The numbers here are trajectory relative error (mean with its 95% confidence interval). The first row is the same as in the previous response. The second row is based on LCP generated validation data (100 trajectories).

|                          | BP5-e            | CP3-e            | CP3              | Gyro             |
|--------------------------|------------------|------------------|------------------|------------------|
| CM data   | 2.34e-5(1.29e-5) | 3.85e-3(9.51e-4) | 2.83e-3(3.64e-4) | 2.39e-3(1.36e-3) |
| LCP data  | 7.35e-4(3.32e-4) | 1.31e-3(3.26e-4) | 5.46e-3(2.30e-3) | 4.51e-4(2.65e-4) |

Here we can see our learning framework performs well on CP3-e, CP3 and Gyro tasks. The performance on BP5-e drops by an order of magnitude though. 

> Thanks for running this experiment. However, I don't share your interpretation of the results here. 

We agree that it is strange that in some cases that training loss is higher than the validation loss. In addition to issues with optimization, this could also be attributed to other factors (e.g., limitations with the baselines). However, we are not able to narrow it down to a specific explanation. 

> I acknowledge the limitations of compliant contact models but feel that these could be a simpler to "swap-in" baseline for comparison to other differentiable contact models (at least compared to LCP).

We agree that compliant contact models are very relevant here. The main reason that we didn't include a comparison here is the time constraint. We are in the middle of understanding the potential challenges of implicit integration schemes (usually used with compliant contact models) in the learning setting. We are looking forward to explore compliant contact models from a learning perspective.

Please let us know what you think!



 I think it is worth to examine further whether there is a convergency-related issue, or there is something incomplete in the current model which affects the scalability. Please add your conclusions regarding the scalability to."Conclusion" and propose future directions to resolve the scalability issue. 

Overall, this work shows a progress in the field and a novel approach. It has limitations, which are clarified and might be addressed in the future work. I increase my score to 7. 


 **Performance - different number of training trajectories**: In the following table we show trajectory relative error (the same metric used in Figure 2) of the rope (D=400) trained with 
different numbers of trajectories (N). (The mean and 95% confidence interval are calculated based on the same 100 validation trajectories. )

|         | CM-CD-CLNN       | MLP-CD-CLNN      | IN-CP-CLNN       | IN-SP-CP         |
|---------|------------------|------------------|------------------|------------------|
| N=25    | 1.91e-3(3.06e-4) | 2.96e-2(8.04e-3) | 3.73e-2(3.41e-3) | 5.96e-2(7.10e-3) |
| N=50    | 1.87e-3(9.60e-4) | 1.25e-2(2.32e-3) | 3.06e-2(2.35e-3) | 6.49e-2(7.71e-3) |
| N=100   | 2.04e-3(2.91e-4) | 7.36e-3(9.63-e4) | 2.10e-2(1.13e-3) | 2.82e-2(1.92e-3) |
| N=200   | 1.92e-3(3.12e-4) | 8.66e-3(1.04e-3) | 2.68e-2(1.61e-3) | 3.59e-2(2.39e-3) |
| N=400   | 1.90e-3(3.40e-4) | 7.02e-3(7.76e-4) | 1.44e-2(1.01e-3) | 2.71e-2(1.61e-3) |
| N=800   | 3.20e-3(5.69e-4) | 5.98e-3(5.21e-4) | 6.87e-3(5.11e-4) | 8.94e-3(5.30e-4) |
| N=1600  | 2.27e-3(3.51e-4) | 5.94e-3(5.26e-4) | 6.86e-3(4.88e-4) | 9.19e-3(5.27e-4) |
| N=3200  | 1.64e-3(2.73e-4) | 5.98e-3(5.27e-4) | 6.21e-3(5.14e-4) | 7.33e-3(5.26e-4) |
| N=6400  | 1.57e-3(2.27e-4) | 5.92e-3(5.26e-4) | 5.90e-3(5.27e-4) | 7.72e-3(5.85e-4) |
| N=12800 | 1.20e-3(2.88e-4) | 5.93e-3(5.29e-4) | 5.90e-3(5.27e-4) | 7.40e-3(5.31e-4) |

As expected, for all of the four models, the error decreases with increasing number of training trajectories. For **CM-CD-CLNN**, it seems the decreasing trend hasn't converge yet. For **MLP-CD-CLNN**, the errors doesn't change much from N=800 to N=12800. For **IN-CP-CLNN** and **IN-SP-CP**, it is hard to tell if the decrease has converged or not, but it is clear that they have the highest errors across four models (each row). 

If we compare CM-CD-CLNN and MLP-CD-CLNN, we can clearly see that the gap between our model and the baseline decreases from N=25 to N=800 and increases from N=800 to N=12800. If we compare our model and the other two baselines, the gap decreases from N=25 to N=800, but the trend from N=800 to N=12800 is unclear. 

The decreasing trend might be unexpected for the reviewer. The underlying reason is that our model performs well with a small amount data. For our model, the difference between N=25 and N=12800 is just 2.9e-4. For baseline models, this difference (between N=25 and N=12800) is at least one order of magnitude higher. Since our model has strong physics priors, this result is expected and shows that our model is data efficient. 

As we've said in our previous response, our model might not be as scalable as we thought. We will discuss this in the limitations. To tackle contact-rich scenarios, one promising model is the Incremental Potential Contact (IPC) mentioned by Reviewer snrk. IPC can scale up to contact-rich scenarios. IPC is also a differentiable model, but the differentiability is not the focus on the IPC paper and the performance of IPC on learning system dynamics and contact properties is not evaluated. It is not clear a) if IPC is able to learn system dynamics and contact properties b) is the learning process scalable. We think it is a promising direction to leverage ideas from IPC and build deep learning models for dynamics learning and system identification. 

Please let us know what you think!

 Thanks for the update. See my comments below:
1. Could you clarify, since you are saying: "train the model on CM generated data and LCP generated data". Are you also evaluating the LCP data trained model on LCP data? From your description of the comparison you seem to be evaluating the LCP trained data on trajectories from your model.
What I'm trying to get at is how well your proposed model of contact is able to model real contact. In the absence of IRL data, the next best proxy would be to see if your model has the flexibility to model trajectories from other contact models (as compared to baselines). Sorry if I didn't make this clear in my original response as I was assuming that this is what was actually being done.

2. Thanks for running this experiment. However, I don't share your interpretation of the results here. It seems that there is an optimization problem rather than an over-fitting problem. Most of the difference in performance can be accounted for in the training loss. In fact, the training loss is in some cases higher than validation and seems to oscillate, whereas CM-CD-CLNN reaches 0 training error immediately. It may be prudent to investigate whether this is a problem with the optimization settings/capacity of the baselines. In the limit of infinite data, the generalization gap should close.

5. I acknowledge the limitations of compliant contact models but feel that these could be a simpler to "swap-in" baseline for comparison to other differentiable contact models (at least compared to LCP).

 Thank you for the comments. The previous update are based on 800 training trajectories. We'll report the results on different numbers of trajectories in the next update. We are also curious to see what happens. 

 how many training samples do you use for D=100, 200, 400 respectively?

does the gap between CM-CD-CLNN and the baselines increase in contact-reach scenarios with more training data?

I suggest to perform this experiment. (As I see in your another response you did a similar experiment but not for the contact-rich case "# training traj.	25	50	100	200	400	800	1600	3200	6400	12800"..)

if the gap is still negligible (with significantly more training data), I recommend to mention that explicitly in the limitations, and to propose reasonable future research directions (i) how to tackle the scalability issue in contact-rich scenarios. if the gap grows, then the work would be stronger (ii).

in either case (i) or (ii) I promise to increase my score. 

 It seems some of your questions are not properly addressed in our initial response. We elaborate on our thoughts below.

> If gradSim and ADD haven't released their code, I am OK with not adding experiments to compare your method with theirs. Are there any updates on NeuralSim that you would like to share?

We have limited progress on NeuralSim. NeuralSim has its own automatic differentiation mechanism. In current (C++ and python) examples, we need to get the gradient of a scalar loss w.r.t. each scalar variable one at a time. Here is a snippet of their billiard optimization python script (with our annotation). One can see that in order to get `d_force_x` and `d_force_y`, we need to rollout the simulation twice. This is inefficient to incorporate into deep learning tasks.  We think if we can only get gradients in this way, it is almost impossible to incorporate the simulator to tackle modern machine learning tasks. We appreciate advice for how to proceed.
```python
def grad_dual(force_x, force_y, steps = 300, eps = dp.fraction(1,10000)):
  ##### getting the gradient of force_x
  fx = dp.TinyDualDouble(force_x.real(), 1.) # setting the "dual" attribute of fx to 1
  fy = dp.TinyDualDouble(force_y.real(), 0.) # setting the "dual" attribute of all other inputs to 0

  c = rollout(fx, fy, steps, False) # perform simulation, return the cost object
  cost = c.real() # this is the scalar cost, obtained by getting the "real" attribute of c
  # next line set up a "TinyDualDouble" object for the gradient of fx
  # the "real" attribute of d_force_x is set to be the "dual" attribute of c
  # the "real" attribute of c represents a scalar cost,
  # the "dual" attribute of c is the gradient of fx since we set the "dual" of fx to 1 here. 
  d_force_x = dp.TinyDualDouble(c.dual(), 0.)
  #####
  ##### getting the gradient of force_y
  fx = dp.TinyDualDouble(force_x.real(), 0.)
  fy = dp.TinyDualDouble(force_y.real(), 1.0)

  c = rollout(fx, fy, steps, False)
  d_force_y = dp.TinyDualDouble(c.dual(), 0.)
  #####
  return cost, d_force_x, d_force_y
```

> The added sentence is not exactly what I expect to read in a related work section. Could you tell us more about what you consider are the pros and cons of your differentiable contact model over the model proposed in Macklin et al [35]?

Macklin et al. [35] uses a *compliant contact model* (usually referred to as soft contact models). Compliant contact models assume contact surfaces are deformable (e.g. Fig. 7 (a) in [35]) and Coulomb friction law is relaxed (e.g. Fig. 7(c) in [35]). By using a compliant contact model, the “instantaneous velocity change” we refer to in our paper is treated as “a continuous (large) velocity change over a short amount of time.”  This treatment allows continuous simulation of dynamics. However, it poses challenges for integration since the dynamics (ODE) is stiff. Macklin et al. [35] formulated an implicit time-stepping scheme for the dynamics and presented a compliant contact model that is suitable for implicit integration. From the result, their proposed method can handle this stiffness challenge. The differentiability is also achieved in part due to the relaxation of contact models (Fig 7. (b) (d) in [35]).  Our contact model is not a compliant contact model in the sense that we assume the contact surfaces are rigid and we didn’t relax Coulomb friction law. As compared to [35], we don’t have stiffness issues in solving the ODEs, which allows us to use Neural ODE to learn the mass and potential energy. (The vanilla Neural ODE has been shown to perform bad on stiff dynamics and later works such as RNODE [Finley et. al., ICML 2020] aims to improve it.) The differentiability of our contact model is based on differentiable optimization. 

The **pros** of our model is that we show the learning of physical properties and contact properties, while [35] only show a trajectory optimization task (similar to the downstream tasks in our paper.) It is not clear if unknown dynamics and contact properties can be learned using the implicit time-stepping scheme in [35]. 

The **cons** of our model is that our framework might fail to correctly simulate systems which have extremely high mass ratios or stiffness ratios (see Fig. 4,5,8,9 in [35]). The primal method and dual method in [35] are shown to perform well on high mass ratios and high stiffness ratios, respectively. This is a limitation of our method and we’d like to add it to the limitation discussion. 

> Building a Python wrapper for a C++ codebase like IPC would not be technically challenging, so I wouldn't consider your simulator written in Python a big advantage. What I would like to ask is similar to my question about [35]: Could you elaborate on the difference between your contact model and IPC and comment on the pros and cons?

We agree that a Python simulator is not the major contribution of our work. Similar to [35], IPC also proposes its own compliant contact model (see Fig. 3,6 in IPC). As the dynamics are stiff, IPC proposes its own implicit time-stepping method. In this sense, the high-level methodologies of [35] and IPC are similar. 

The **pros** of our framework is still the learning of system and contact properties. Since the focus of IPC is an intersection and inversion-free model, there is no learning results presented in the paper. IPC is differentiable, but how unknown system dynamics and contact properties can be learned from the proposed implicit time-stepping method is unclear. 

The **cons** of our framework is scalability and robustness. IPC can handle contact-rich systems with large deformation (soft materials) and systems with high mass ratios and high stiffness ratios (Fig. 12 in IPC), while we focus on rigid body systems and our model might not be scalable and robust under these extreme scenarios. 

---------------------
We also have two remarks.

**Remark 1**: On Mujoco website, the Mujoco contact model is referred to as a 
“soft” contact model. As our CMr closely resembles the Mujoco model, one can view CMr as a “soft” version of CM. However, this is still not the same as compliant contact models.

**Remark 2**: gradSim also adopts a compliant contact model, but they use a semi-implicit Euler integration scheme. This might not help it simulate the stiff ODE in complex scenarios. Probably this is why gradSim has limited capability to handle contact-rich motion that introduces a large number of discontinuities (Appendix K in gradSim paper)

-------------------

Please let us know if you have any additional questions/concerns.

 We are glad we've properly addressed your concerns. Thank you for raising the score. 

 Thanks for your comments. We respond in details below.

> 1. I noted the "Robustness" experiment and look forward to the results on trajectory error experiment using the LCP data.

We added the trajectory relative error with 95% confidence interval below (the same metric as shown in Fig. 2). The model we used here is CM-CD-CLNN. We train the model on CM generated data and LCP generated data. For BP5-e, CP3, and Gyro, the model trained on LCP generated data results in higher prediction error and variance, as expected. Model mismatch does not result in unrealistic predictions and unexpectedly high prediction errors. This is mainly because the contact properties (Table S.4 in SM) are learned with high accuracy. What is a little bit surprising is that for CP3-e, model mismatch result in slightly lower error. One might think that probably this is because in the validation dataset in this example, there might not exist many collisions. In this example collision does not happen as frequently as in, say, BP5-e. However, this is not the case, since we make sure both the training dataset and the validation dataset contains enough collisions so that we can learn the contact properties efficiently and test the model performance properly. Since the difference in this example is not large, we guess the stochasticity in gradient descent plays a role. In other words, maybe the “LCP data” model happens to find a very good solution and the “CM data” models somehow only find a slightly suboptimal solution.  

|          | BP5-e            | CP3-e            | CP3              | Gyro             |
|----------|------------------|------------------|------------------|------------------|
| CM data  | 2.34e-5(1.29e-5) | 3.85e-3(9.51e-4) | 2.83e-3(3.64e-4) | 2.39e-3(1.36e-3) |
| LCP data | 2.54e-3(3.73e-3) | 3.73e-3(6.92e-4) | 1.57e-2(5.00e-3) | 5.50e-3(1.67e-3) |

> 2. Thanks for the additional results. This is very interesting to me. Is the performance gap due to overfitting of MLP and IN or is the training error also much worse? Would you expect these models to catch up eventually?

We added the training loss along with the validation loss. For MLP-CD-CLNN and IN-SP-CP, we observed overfitting of the training data. This is probably why the validation loss doesn’t decrease. Because of the overfitting, we don’t expect these models would be able to catch up eventually.

| # training traj.         | 1600     | 3200     | 6400     | 128000   |
|--------------------------|----------|----------|----------|----------|
| MLP-CD-CLNN (train loss) | 0.0091   | 0.0132   | 0.0091   | 0.0087   |
| MLP-CD-CLNN (val loss)   | 0.0107   | 0.0107   | 0.0107   | 0.0108   |
| IN-SP-CP (train loss)    | 0.0143   | 0.0092   | 0.0054   | 0.0161   |
| IN-SP-CP (val loss)      | 0.0110   | 0.0109   | 0.0109   | 0.0110   |
| CM-CD-CLNN (train loss)  | 0.000000 | 0.000000 | 0.000000 | 0.000000 |
| CM-CD-CLNN (val loss)    | 0.000579 | 0.000298 | 0.000138 | 0.000297 |

> 4.	I'm curious about this. Do you know how CxpyLayers deals with singular case, since it presumably must pick between multiple optima and Jacobian of the objective at optimum would also be singular?

We try to answer your question here but might not answer it to your satisfaction. We guess that in our particular optimization problem (5), a global minimum exists. We agree that for a general singular case, e.g., $x^TAx$ where $A$ is singular, there exist multiple optima and CvxpyLayer simply pick one of them as the solution. However, since our problem comes from a physical setting, for any physically plausible pair of $A$ and $V_C^{c-}$, subject to the constraints, there should exist a unique contact force that minimize the energy. (We haven’t rigorously proved this but this is our hypothesis.) If this is true, then it makes sense for CvxpyLayer to return a correct (global minimum) solution and gradients. 

> 5.	By soft contact, I meant a relaxed contact model (such as the one presented in gradSim[38] see section C.2).

GradSim Sec C.2 describes the compliant contact model it uses. A disadvantage of the compliant contact model is that for relatively stiff contact surfaces, it will produce stiff ODEs. Since the compliant contact models use a quite different setting as compared to our model, we didn’t investigate those models in this work, but it would be an interesting future work to investigate the pros and cons of compliant contact models and rigid contact models in the learning setting. We also noticed that when gradSim introduces the relaxed Coulomb model in section A.1, it cited Todorov (2014). We believe this is a typo since Todorov (2014) doesn’t adopt a relaxed Coulomb model. We’d also like to point out that the Mujoco model is also said to be “soft”, in the sense that it is analytically invertible and the constraints are “softened.” As our CMr closely resembles the Mujoco model, one can view CMr as a “soft” version of CM. However, this is still not the same as compliant contact models.

Please let us know if you have any additional questions/concerns.

 We updated results on the rope example to address the scalability question. Here are the details.

- **Performance - different number of contacts**: Here we report the trajectory relative error (the same metric used in Figure 2) of ropes discretized in different number of segments (The configurations in Table 3). We find that the performance gap between our model and baselines is the smallest in D=400 scenario. This indicates that our model might not have a clear advantage over baselines in contact-rich scenarios.  
|       | CM-CD-CLNN       | MLP-CD-CLNN      | IN-CP-CLNN       | IN-SP-CP         | 
|-------|------------------|------------------|------------------|------------------|
| D=100 | 1.97e-3(4.72e-4) | 1.85e-2(2.53e-3) | 2.31e-2(2.12e-3) | 2.93e-2(2.05e-3) |  
| D=200 | 9.39e-4(3.26e-4) | 1.08e-2(8.91e-4) | 1.27e-2(7.76e-4) | 1.37e-2(8.14e-4) | 
| D=400 | 3.20e-3(5.69e-4) | 5.98e-3(5.21e-4) | 6.87e-3(5.11e-4) | 8.94e-3(5.30e-4) |  

(Number in parenthesis indicates the 95% confidence level.)
- **Performance – large network**: Our default network used in CLNN (to approximate potential energy) is an MLP with 3 layers with hidden sizes of 256. We enlarge the network with 6 layers with hidden sizes of 512. We find that large networks doesn’t improve our models performance. This is likely because with the default size, the potential energy is already estimated well enough. 
|       | CM-CD-CLNN (defualt size) | CM-CD-CLNN (large size) |
|-------|---------------------------|-------------------------|
| D=400 | 3.20e-3(5.69e-4)          | 6.89e-3(5.07e-4)        |

**Remark**: Thanks for the reviewer for suggesting these experiments. We didn’t realize that the small gap between model performances is due to the large contact numbers before doing these experiments. These results show that our method might not be as scalable as we think. We also want to point out a previous inaccurate statement we made. In D=400 case, the ratio between our model and baselines is around 2-fold instead of an order of magnitude. Still, we think this shows certain advantage of our model over the baselines. In D=200 and D=100 case, we do see an order of magnitude difference. The fact that large network doesn’t improve performance rules out the possibility that the potential energy is learned inaccurately in D=400 case. It seems that the default network already approximate the potential energy well and increasing the network sizes doesn’t help. 

We’d also like to point out that even if our model does not have a clear advantage in contact-rich scenarios, our main contribution is to demonstrate the framework’s ability in simultaneously learning of unknown system dynamics and contact properties from trajectory data. Other differentiable simulators such as ADD[34] and NeuralSim[36] demonstrate their differentiability by optimizing initial conditions with known dynamics and contact properties (similar to the downstream tasks in our paper.) Their ability to learn unknown dynamics has not been evaluated. Incremental potential contact (IPC) (which is pointed out by Reviewer snrk) is differentiable and can scale up to contact-rich scenarios, but their performance on dynamics learning or trajectory optimization (downstream tasks) has not been evaluated. Differentiable simulator in the deep learning era is an exciting direction. We will pay extra attention to scalability in our future research. 

Please let us know if you have any additional questions/concerns. 


The paper proposes a differentiable contact model, whose unknown parameters (restitution and friction coefficients) can be learned together with other system parameters using Hamiltonian and/or Lagrangian neural networks. The proposed contact model consists of two constrained convex optimization problems, which are made differentiable by the recently proposed differentiable convex optimization layers. Variants of the proposed methods outperform baselines on multiple 2D and 3D physical systems. The learned dynamics model can also be used for downstream planning and control tasks.  Originality: The proposed method is a novel combination of the two-phase contact model, the differentiable convex optimization layers and the Hamiltonian & Lagrangian neural networks. Related works are sufficiently discussed.

Quality:
The paper is technically sound, and the empirical evaluation is thorough. Some weaknesses are listed below:
- The choice of the target velocity $v_C^*$ to fix penetration seems to require a lot of engineering and not very principled. It is also unclear how sensitive the performance is to the choice of the target velocities, as it is not discussed in the experiments.
- One of the improvements made from the previous submission is that the authors fixed an error in the code base. However, the code is still not included in this submission. 

Clarity: The paper is generally well-written. Some improvements can be made on the presentation of the proposed method. For example, an algorithm box outlining the training procedure would help. Also, the current introduction repeats part of the related work section, and can be made cleaner.

Significance: The contributions of this paper are incremental and moderately significant. The proposed method outperforms the baselines on prediction error and energy conservation, but also requires a known collision detection module and the nontrivial choice of the target velocity. In general, it doesn’t seem to have a clear advantage over existing methods, but can still be a meaningful contribution.

==== Post rebuttal ====

My concerns are well addressed by the author response. I have increased the score to 7.
 I think the authors have adequately addressed the limitations and potential negative societal impact of the work.

 I would like to thank the authors for the detailed response. I believe my concerns are well addressed, and I'm raising my score to 7.

 Thank you for your detailed responses!

- If gradSim and ADD haven't released their code, I am OK with not adding experiments to compare your method with theirs. Are there any updates on NeuralSim that you would like to share?

- The added sentence is not exactly what I expect to read in a related work section. Could you tell us more about what you consider are the pros and cons of your differentiable contact model over the model proposed in Macklin et al [35]?

- OK. I will let you know if I have follow-up comments on that.

- Building a Python wrapper for a C++ codebase like IPC would not be technically challenging, so I wouldn't consider your simulator written in Python a big advantage. What I would like to ask is similar to my question about [35]: Could you elaborate on the difference between your contact model and IPC and comment on the pros and cons?

 Thanks for the detailed reply. I read through it carefully and address the necessary ones below:

1. I noted the "Robustness" experiment and look forward to the results on trajectory error experiment using the LCP data.
2. Thanks for the additional results. This is very interesting to me. Is the performance gap due to overfitting of MLP and IN or is the training error also much worse? Would you expect these models to catch up eventually?
4. Thanks for the detailed explanation. I understand it now.
6. I'm curious about this. Do you know how CxpyLayers deals with singular case, since it presumably must pick between multiple optima and Jacobian of the objective at optimum would also be singular?
7. By soft contact, I meant a relaxed contact model (such as the one presented in gradSim[38] see section C.2).



 i do not have other questions. looking forward to seeing your updated response. 

 We’d like to thank the anonymous reviewers. We greatly appreciate your feedback and comments! We’ve responded to your comments/questions as replies to the individual comments. Also, responses to some of your questions require additional experiments; we are currently running those additional experiments and expect to report the extra results as soon as possible. 

 Thank you for your feedback! We address your questions below:
1. The only test data for comparing trajectory error is generated using the proposed model itself. In this case there is zero model mismatch. An additional prediction task using real world data (ideally) or trajectories generated from a separate physics simulator should be used to compare the method against baseline.
    - We have a “Robustness” paragraph in Sec. 4 where we use LCP generated data to train our proposed model. We’ve only reported on the accuracy of learned contact parameters. We plan to update the trajectory error by the end of the rebuttal period. We agree it would be ideal to use real world data, this is what we would like to test in the future. In this work, we mainly focused on simulation data.
2. It would be interesting to extend Figure 5 to see when/if MLP and IN baseline could catch up to the proposed method (if the CM method is too expensive at least the baselines could be extended).
    - We run extra experiments and conclude that MLP and IN couldn't catch up to our proposed method. Please see the figures below. 

|# training traj.             | 25      | 50      | 100     | 200     | 400     | 800     | 1600     | 3200     | 6400     | 12800    |
|-------------|---------|---------|---------|---------|---------|---------|----------|----------|----------|----------|
| MLP-CD-CLNN | 0.0138  | 0.0138  | 0.0128  | 0.0117  | 0.0108  | 0.0107  | 0.0107   | 0.0107   | 0.0107   | 0.0108   |
| IN-SP-CP    | 0.0144  | 0.0128  | 0.0122  | 0.0130  | 0.0122  | 0.0131  | 0.0110   | 0.0109   | 0.0109   | 0.0110   |
| CM-CD-CLNN  | 0.00741 | 0.00827 | 0.00839 | 0.00839 | 0.00762 | 0.00774 | 0.000579 | 0.000298 | 0.000138 | 0.000297 |

3. This is cleared up in the appendix but it would be clearer if the notation used in Algorithm1 ($\Delta v$) were explicitly related in the section 3 description.
    - We have updated the ($\Delta v$) description in main text of Sec 2.2 (side by side with Algorithm 1.) This will be incorporated in camera-ready. 
4. In the appendix S.10 shows $f_E$ can be expressed in terms of $f_C$ but doesn’t $f_E$ depend also on impulse from unconstrained dynamics f.e. acceleration due to gravity (the Mujoco paper includes this as “s” term)?
    - Unlike the formulation in Mujoco, in our formulation, $f_E$ doesn’t depend on the impulse from unconstrained dynamics. The difference can be clearly seen by comparing Eqn. S.9 in our SM and Eqn. 1 in [23]. In our formulation, we treat “continuous dynamics” and “instantaneous velocity change” in different steps, and $f_E$ only contributes to the instantaneous velocity change, so it doesn’t depend on any terms from the dynamics. In Mujoco (Eqn. 1 in [23]), the integration of dynamics and the velocity change due to contact are treated in one equation, so the $f_E$ there, by definition, is different from the $f_E$ in our formulation. We choose this formulation since it allows us to use more sophisticated ODE solvers (e.g. RK4) to integrate the dynamics, which is independent of the contact solver. The formulation of Mujoco is more suitable for Euler method. For Mujoco to use RK4, it needs to “repeat the sequence (of computations) three more times” [mujoco website -> computation tab. We found this hard to fit in the Neural ODE training scheme. As RK4 can better show the conservation of energy of Hamiltonian and Lagrangian neural networks [4][11], we choose the current formulation. 
5. Some details on the IN and MLP baselines are missing. What features are being passed to them?
    - For our proposed models and baseline models, the goal is to predict future states based on the initial condition. Hence, the initial condition (initial position and velocity) are passed to all these models. The MLP-CD-CLNN baseline use an MLP to replace our contact model so there is no additional input. Interaction network (IN) assumes the mass, external forces and contact properties are known, so the IN-CP-SP model have true mass and external forces as additional inputs passed to it. Actually the “CP (contact properties)” and “SP (system properties)” indicates true contact and system properties are passed. For IN-CP-CLNN, the dynamic is learned by a CLNN, so we assume the true system properties (mass and external forces) are not known and are estimated from CLNN. These estimated system properties along with true contact properties are extra inputs to IN-CP-CLNN. The “CP” in the name indicates true contact properties are passed. We briefly explain this extra inputs and naming conventions in Sec. 4.2. We will expand this section to include more detailed discussion in camera-ready.
6. The regularized version CMr is briefly introduced and not discussed too much. In which scenarios would the regularization be important?
    - Our method is inspired by the Mujoco contact models. In Mujoco, the regularization is added “because matrix A can be singular and also because it introduces smoothing that is necessary to define the inverse later” [23]. In our study, it is true that matrix A can be singular, but we can still formulate the convex optimization problem in Disciplined-Parametrized-Programming-compliant form [17] and leverage CvxpyLayers [17] to not only solve the problem, but also to get the gradients. Our results show that the contact model without regularization is more accurate and should be preferred over the regularized version. We think the regularization is important to formulate an inverse dynamic, as pointed out by [23]. However, in this study, our problem and method don’t require inverse dynamics, so regularization does not play a critical role. 
7. I understand that the LCP baseline was infeasible due to nan issues. Another possible baseline which should be comparatively stable is soft contact model.
    - For baselines, we try to use other differentiable contact models. Could the reviewer point us to some works that make soft contact model differentiable?
8. I believe [34] do test energy conservation for their contact model somewhat.
    - Thanks for pointing out that [34] tested energy conservation for their contact model. Fig. 6 in [34] shows total energy over time of a bouncing rigid cubes with different damping coefficients. We’d like to point out that we show energy conservation on predicted trajectories of learned models (Fig. 4), where both dynamics (mass and potential energy) as well as contact properties (coefficient of friction and restitution) are learned from data, while Fig. 6 in [34] only shows energy conservation of simulation of given damping coefficients. In [34], we couldn’t find report on the accuracy of learning damping coefficients. We can only find the learning of coefficient of friction (Table 1 in [34]). However, the ground truth value used in these parameter estimation examples are not given, which makes us hard to evaluate how well [34] can learn contact properties. 

Please let us know if you have any other questions!

 Thank you for your feedback! We address your questions below:
- I strongly recommend to add a description of the entire method by a block diagram/picture/pseudo code. Also, frequent jumping between the paper and the appendices affects the readability. 
    - We have made a diagram but we find that this rebuttal period is not like the ICLR one where the authors can update manuscripts and supplementary materials. Thus, we cannot update now but we will incorporate it into camera-ready. We draw the diagram from the perspective of training and hopefully it complements Algorithm 1 to give a high-level picture of our method.
- even though the method is original, it is limited to particular classes of contact dynamics, and/or dimensionality of dynamics. Specifically, there is no improvement in "Rope" environment, which indicates that the proposed solution is not general and/or scalable. Consequently, it is unclear when this method improves the baselines. Does it work only with a small number of contacts ( <= 8)? What is the main limitation? 
    - We didn’t think that our method “is limited to particular classes of contact dynamics, and/or dimensionality of dynamics”. Our method can model both frictional and frictionless as well as elastic and inelastic contacts. It can also model the “limits” in joint angles as shown in the rope example.  We think this includes a broad class of contacts. Could the reviewer let us know what classes of contact you have in mind that our method cannot model? It might motivate us to further extend this work in the future! Our method is able to work with a large number of contacts. The purpose of the rope example is to demonstrate this. One disadvantage of our method is that our method currently is not optimized for speed as compared to other C++ simulators, so it is currently not efficient to simulate a large number of contacts. The purpose of Table 3 is to quantify scalability. However, our work is written entirely in Python so it might be easier for people to play with and easier to integrate with machine learning frameworks. As for the rope example, from Figure 3 we can still see that the proposed methods outperform the baselines. The gap is not as large as some other examples but it is similar to the BD5 example. We’d like to emphasize that this is in log scale and our methods outperform the baselines by around an order of magnitude. Thus, we couldn’t agree that “there is no improvement in the Rope environment.” 
- Does it perform better with more training data and/or larger networks? E.g., Table 3 might shed light on this issue, if the corresponding errors (as in Fig2+Fig3) would be added. 
    - We are working on the suggested experiments to answer this question, we plan to update our response on this question by the end of the rebuttal period. 

Please let us know if you have any other questions!

 Thank you for your feedback! We address your questions below:
- The choice of the target velocity $v_C^*$ to fix penetration seems to require a lot of engineering and not very principled. 
    - The choice of the target velocity does not require a lot of engineering. The formula is presented in Appendix G and can be directly implemented. The only “hyperparameter” is the choice of the desired velocity $v_C^d$. As a rule of thumb, one can choose $v_C^d$ by “fixing penetration in 1 time step” (as described in Appendix G). The only downside of this choice is that for totally inelastic collision, one may still observe some restitution, as shown in Figure S.2. If this behavior is not desired, other choices can be used. In fact, Mujoco has a similar penetration compensation term (see [22] Eqn. 11 and the paragraph below it), which corresponds to our “fixing penetration in 1 time step” option. They didn’t explicitly write out the formula of this $v_{min}$, but if they’ve written it out, it would have looked similar to what we have in Appendix G. The purpose of Appendix G is to: a) give an explicit expression of how to compensate penetration, and b) to point out a simulation artefact of the option used in Mujoco and provide alternative options to fix the undesired behavior. 
- It is also unclear how sensitive the performance is to the choice of the target velocities, as it is not discussed in the experiments.
    - From our experiments, the performance is not sensitive to the choice of the target velocities. One reason is that the target velocities doesn't play a role in simulation where the coefficient of restitution is away from zero. In these scenarios, the penetration can be naturally fixed by restitution. 
- One of the improvements made from the previous submission is that the authors fixed an error in the code base. However, the code is still not included in this submission.
    - We didn’t submit our code for two reasons. 1. Our code is not well refactored. 2. The model and task names we used in our code are a little bit different from what we decided to use later in our paper. We think that these might confuse the reviewers so we didn’t submit the code. We are planning to open source our code on Github after the review period. We are planning to have an archive branch hosting the current status of our code for people who want to reproduce our results exactly in the paper, and a main branch hosting refactored code with all the names corrected to those we used in the paper. We’ll also upload our training dataset on Zenodo and pretrained models on github releases. 
- The paper is generally well-written. Some improvements can be made on the presentation of the proposed method. For example, an algorithm box outlining the training procedure would help. Also, the current introduction repeats part of the related work section, and can be made cleaner.
    - We’ve streamlined our introduction section to avoid repetition. This will be incorporated into camera-ready.

Please let us know if you have any other questions!

 Thank you for your feedback! We address your questions below:
- A consequence of the use of CHNN is the need for using Cartesian coordinates instead of generalized ones. This results in soft joint constraints when considering multi-body systems in robotics and also requires solving artificially higher dimension problems. 
    - The mathematical formulation of the contact model works for both Cartesian coordinates and generalized coordinates. From a simulation perspective, using generalized coordinates is better since the dimension of the dynamics is lower and the equality constraints are naturally preserved. From a learning perspective, the CHNN paper shows using data in Cartesian coordinates facilitates the learning of Hamiltonian and Lagrangian dynamics because the potential energy expressed in Cartesian coordinates are usually simpler and thus easier for neural network to learn. This is the main reason we use Cartesian coordinate data. This work demonstrates that the simultaneous learning of dynamics and contact properties is possible. An interesting future work would be exploring a method that enjoys the best of both worlds, i.e., using generalized coordinates so we have a low dimensional system and we can also learn the dynamics well.
- Moreover, the proposed approach does not account for external or control forces and such efforts could make it more challenging to enforce constraints. In order to consider control applications, would it still be possible for CHNN to learn the dynamics of a system under these forces? 
    - Symplectic ODE-Net [4] has shown how to incorporate control inputs into the Hamiltonian formulation with generalized coordinates. It is straightforward to implement the same techniques to CHNN, i.e., simultaneously learn an extra “input matrix” (which captures how control inputs interact with the system). The external forces or control inputs shouldn’t make it more challenging for CHNN to enforce equality constraints, since you can view gravity (or any forces caused by potential energy function) as an “external force” and CHNN can still enforce equality constraints with these forces. As the focus of this work is to extend CHNN/CLNN in order to model contacts, we didn’t include control inputs, but we believe incorporating control input should be straightforward as done in [4]. 
- The Signorini condition which enforces the complementarity between normal contact forces and velocities accounts for the non-inter penetrability between objects [The non-smooth contact dynamics method, Michel Jean 1999]. This condition is necessary to simulate rigid bodies and is ignored in this paper. The same approximation is done in the simulator Mujoco [Mujoco: a physics engine for model-based control, Todorov et al. 2012] and leads to simulation artefacts such as objects interacting even when they are not in contact. Did you observe similar effects? Could the Authors report on it?
    - We indeed has done approximation to formulate the problem as a convex optimization problem, as done in Mujoco. In this formulation, the Signorini condition is not explicitly expressed. We will add the following sentence to Sec. 3.2.: “This formulation is an approximation to the Signorini condition, please see [Jean 1999] [Todorov, 2012] for more details.” We didn’t observe simulation artefacts in our experiments. We also didn’t find the description to these artefacts in [Todorov 2012], could the reviewer please point us to a reference that describes this artefact?
- Does the SCS solver have convergence guarantees for positive semi-definite problems (case when the A matrix has a null eigenvalue)? This is a common case in robotics that corresponds to over-constrained problems and one should avoid using a regularization matrix R as it leads to less precise solutions.
    - In the SCS paper [24], the authors mention that they can handle semi-definite cones, but it is unclear if there exists convergence guarantees for positive semi-definite problems. In our examples and experiments, an A matrix with a null eigenvalue is common and we didn’t observe any issue with SCS solver during simulation (data generation). During the initial phase of learning, in rare cases, SCS solver would return a “solved/inaccurate” status. Even in these rare cases, the SCS solver wouldn’t complain after a few iterations. As our results show models without the regularization matrix R lead to better results, we agree that the version without matrix R should be preferred. 
- Experiments on dynamics and parameters learning demonstrate the superiority of the method when compared to a simple MLP or a more sophisticated Interaction-Networks. However, a comparison with a differentiable simulator such as gradSim [gradSim: differentiable simulation for system identification and visuomotor control, Murthy et al. 2021] would be interesting because the approach should be more competitive than other baselines (as shown by experiments from 5.). 
    - We agree that recent differentiable simulator such as gradSim [38], ADD[34] and NeuralSim[36] should give more competitive results than other baselines and we would really love to compare our model against them to understand the pros and cons of these models. However, both gradSim and ADD haven’t released their source code yet, which makes it challenging for us to compare. NeuralSim has made their C++ and python binding code open source. The API makes it easy to get the gradient of loss to each variable one at a time but incorporating it into the pytorch ecosystem is not that straightforward. We are still working to embed NeuralSim in the pytorch training pipeline. 
- Durations considered during the evaluation of energy conservation are short when compared to the ones involved in robotic applications. Is this property still valid for longer time horizons? 
    - We are working on the experiments to address this comment. We plan to update it by the end of the rebuttal period. 
- Concerning the experiment on robustness, why were the 3D examples generated from an LCP simulator and not with one accounting for the cone constraint? From my understanding, the learnt friction parameter does not have the same meaning as the original one because it corresponds to different friction models which makes it difficult to compare them [Differentiable simulation for physical system identification, Le Lidec et al. 2021]. 
    - Thanks for pointing us to this paper! We are not aware of this paper. Previously we’re only aware of the LCP formulation and the Mujoco formulation of contact models. This is why we use LCP simulator for robustness tests. If time is allowed, we’d like to add robustness test using data generated by Le Lidec et al.
- A billiard task is considered to evaluate the approach and compare it against the DiffTaichi differentiable simulator. This simulator includes gradients from the collision detection routine while it is not clear if this information is available in the collision detection module used in the approach proposed in the paper. I would expect this information to be important to solve a problem where one needs to reason about contact points as is the case in the billiard task. Could it be the reason behind the better performance of Difftaichi on this task? Did you consider different initial guesses or targets to assert the robustness of the approach on this task?
    - We think the “gradients from the collision detection routine” mentioned above is the “time of impact(TOI)” used in DiffTaichi. We didn’t implement TOI in our method. The main reason is that it is not clear how to calculate TOI with RK4 integrator. DiffTaichi uses the Euler integrator where calculating TOI is just a simple interpolation. As you pointed out, this could be the reason behind the better performance of DiffTaichi on this task. In fact, this might be able to explain why the optimized positions of the white balls in DiffTaichi and our method are on the right and left of the initial guess, respectively. As you can see from Figure 4 in DiffTaichi [33], the gradient w.r.t the initial position of the ball using naive integrator and TOI have different signs. Thanks for this insightful question. We will add this discussion to the camera-ready version.
- Overall, the limitations as presented by the Authors seems a bit limited. Indeed, the Authors did not discuss for instance the fact they did consider complementarity between motion and forces at the contact level. Additionally, they clearly did not discuss the fact they did a mix between acceleration-based simulation and time-stepping methods which will compute a corrective impulse at velocity level. What is the physical and mathematical meaning of such a choice?
    - We will add a statement that the convex optimization formulation is an approximation to the Signorini condition. The main purpose of a mix between acceleration-based simulation and time-stepping methods is that we’d like to treat “continuous dynamics” and “instantaneous velocity change” in different steps and use RK4 integrator for the dynamics, since RK4 can better enforce the conservation of energy in Hamiltonian and Lagrangian neural networks [4][11]. Mujoco didn’t use this kind of mix and treated dynamics and instantaneous velocity change in one step (Eqn. 1 in [23]). Instead, the formulation of Mujoco (Eqn. 1 in [23]) is more suitable for Euler method. For Mujoco to use RK4, it needs to “repeat the sequence (of computation) three more times” [mujoco website -> computation tab]. We found this hard to fit in the Neural ODE training scheme.

Please let us know if you have any other questions!

 Thank you for your feedback! We address your questions below:
- Differentiable contact models are not new to the ML/robotics/graphics communities, and previous works have proposed quite a few candidates. One of my biggest concerns for this paper is that it focuses on showing the performance of the proposed contact model but without comparing it to other differentiable contact models from previous papers. 
    - We’d like to compare our method with recent differentiable simulator such as gradSim [38], ADD[34] and NeuralSim[36] to understand the pros and cons of these models. However, both gradSim and ADD haven’t released their source code yet, which makes it challenging for us to compare. NeuralSim has made their C++ and python binding code open source. The API makes it easy to get the gradient of loss to each variable one at a time but incorporating it into the pytorch ecosystem is not that straightforward. We are still working to embed NeuralSim in the pytorch training pipeline.
- The authors are aware of the differentiable contact model presented in [35] but did not seem to comment [35] in their related work section.
    - We plan to add the following sentence to the camera-ready in the related work section: “Macklin et al. [35] develop a contact model using a variational formulation based on generalized velocities and soft constraints, and analyzes the impact of various system parameters on the accuracy of contact simulation.” We can expand the discussion if space is allowed in camera-ready.
- The original “hit” example in [35], which this paper got inspiration from (line 326), is a 3D deformable ball bounces between walls and grounds more than once, which is much more sophisticated than the two examples this paper presented in Fig. 7. 
    - The “hit” and “vertical” tasks are indeed simplified versions of those studied in [34] and [35]. However, unlike [34] and [35], the initial conditions in our tasks are optimized using the learned dynamics and contact properties instead of true ones. We’d like to use these two tasks as examples of downstream tasks.
- Another more recent, state-of-the-art differentiable contact model “Incremental Potential Contact: Intersection- and Inversion-free Large Deformation Dynamics” (IPC), is missing in this paper’s related work but I believe IPC deserves a careful discussion. I do agree this submission has some unique contributions in its machine learning experiments which are not covered by [35] or IPC, which is also why I choose to give a positive score. However, I think it should also pay more attention to other existing differentiable contact models and position its contributions more properly.
    - Thank you for letting us know about this paper! We are not aware of this paper and we’ll discuss it in the related work section in camera-ready. One advantage of our paper is that our simulator is written in Python (not focused on speed as those written in C++), which makes it easy for people to play with and integrate with machine learning frameworks. As you mentioned, we should discuss this aspect along with other differentiable contact models and position our contribution more properly. 

Please let us know if you have any other questions!

This paper adds explicit contact modelling to lagrangian/hamiltonian neural networks.
Specifically, they add a contact resolution step to CHNN using an impulse-based micro collision maximum dissipation contact model inspired by the contact model used in MuJoCo.
They also handle elastic collisions with a 2 phase collision based on poisson restitution.
Each phase of contact resolution are convex programs and CvxpyLayers is leveraged to compute analytic gradients through them.
Experimental evaluations are performed to test the proposed models ability to infer physical parameters from trajectories and predict future trajectories.
Two small experiments are also included to show the capability of the model to be used for control.
  Strengths:
1. The choices of the contact model are inspired by MuJoCo and novel for differentiable rigid body physics models and most other methods typically don’t handle elastic collisions.
2. Method is thoroughly detailed in the appendix.
3. Video shows qualitative results on the various toy tasks.

Issues:
1. The only test data for comparing trajectory error is generated using the proposed model itself. In this case there is zero model mismatch. An additional prediction task using real world data (ideally) or trajectories generated from a separate physics simulator should be used to compare the method against baseline.
2. It would be interesting to extend Figure 5 to see when/if MLP and IN baseline could catch up to the proposed method (if the CM method is too expensive at least the baselines could be extended).
3. This is cleared up in the appendix but it would be clearer if the notation used in Algorithm1 ($\Delta v$) were explicitly related in the section 3 description.
4. In the appendix S.10 shows $f_E$ can be expressed in terms of $f_C$ but doesn’t $f_E$ depend also on impulse from unconstrained dynamics f.e. acceleration due to gravity (the Mujoco paper includes this as “s” term)?
5. Some details on the IN and MLP baselines are missing. What features are being passed to them?
6. The regularized version CMr is briefly introduced and not discussed too much. In which scenarios would the regularization be important?
7. I understand that the LCP baseline was infeasible due to nan issues. Another possible baseline which should be comparatively stable is soft contact model.
8. I believe [34] do test energy conservation for their contact model somewhat.

Overall:
I think the paper provides a useful contribution to differentiable rigid body simulation which has seen a lot of interest. A new differentiable contact model is proposed and evaluated experimentally. My main issue is that comparisons and prediction errors are only evaluated on data generated by the proposed model itself. Perhaps the authors could elaborate on what cases the contact model could be inaccurate/unstable for. For example, would you expect any issues around resting contacts, large numbers of simultaneous contacts etc.

This paper presents a differentiable contact model and demonstrates its compatibility with deep learning frameworks and its usefulness in downstream applications. The proposed contact model is similar to that in MuJoCo but with several enhancements. The final contact model formulates the task of solving contact forces as a convex optimization problem and is differentiable. The authors demonstrate the usefulness of their proposed contact model in several standard 2D and 3D examples.  Differentiable contact models are not new to the ML/robotics/graphics communities, and previous works have proposed quite a few candidates. One of my biggest concerns for this paper is that it focuses on showing the performance of the proposed contact model but without comparing it to other differentiable contact models from previous papers. The authors are aware of the differentiable contact model presented in [35] but did not seem to comment [35] in their related work section. The original “hit” example in [35], which this paper got inspiration from (line 326), is a 3D deformable ball bounces between walls and grounds more than once, which is much more sophisticated than the two examples this paper presented in Fig. 7. Another more recent, state-of-the-art differentiable contact model “Incremental Potential Contact: Intersection- and Inversion-free Large Deformation Dynamics” (IPC), is missing in this paper’s related work but I believe IPC deserves a careful discussion. I do agree this submission has some unique contributions in its machine learning experiments which are not covered by [35] or IPC, which is also why I choose to give a positive score. However, I think it should also pay more attention to other existing differentiable contact models and position its contributions more properly. From what I understand, developing contact models typically focuses on one of the two goals: it either aims at physical accuracy and targets at real-world applications, or focuses on speed/robustness/simplicity and considers animation/learning problems as its main application. The position of the contact model presented in this work is not very clear to me: it is definitely more physically plausible than some of the simplest contact models in robotics/graphics/ML, but not accurate enough for a real-world application. I look forward to seeing a clarification and, even better, a real-world example in their future work.
 
Another limitation in this work is that a comprehensive discussion about the role of collision detection is missing. However, I notice that the authors are aware of this and consider it as future work, which I think is fair, so I won’t hold it against them.


This paper introduces Lagrangian/Hamiltonian neural networks for simulating systems with frictional contacts. Previous Constrained Hamiltonian Neural Networks (CHNN) are able to learn constrained motion while this paper extends to the case of constrained motions with frictional contacts. A first phase leverages the NeuralODE framework to simulate in a fully differentiable way, the constrained motion associated with the learnt Lagrangian/Hamiltonian. Contacts are handled by incorporating an additional contact-solving phase solved using differentiable optimization layers (cvxpylayers).  The approach is benchmarked against an MLP baseline and an Interaction-Network on the task of learning a differentiable simulator from data. The learnt differentiable simulator is then applied to control tasks and compared to the DIffTaichi simulator.   A consequence of the use of CHNN is the need for using Cartesian coordinates instead of generalized ones. This results in soft joint constraints when considering multi-body systems in robotics and also requires solving artificially higher dimension problems. Moreover, the proposed approach does not account for external or control forces  and such efforts could make it more challenging to enforce constraints. In order to consider control applications, would it still be possible for CHNN to learn the dynamics of a system under these forces? 

The Signorini condition which enforces the complementarity between normal contact forces and velocities accounts for the non-inter penetrability between objects [The non-smooth contact dynamics method, Michel Jean 1999]. This condition is necessary to simulate rigid bodies and is ignored in this paper. The same approximation is done in the simulator Mujoco [Mujoco: a physics engine for model-based control, Todorov et al. 2012] and leads to simulation artefacts such as objects interacting even when they are not in contact. Did you observe similar effects? Could the Authors report on it?

Does the SCS solver have convergence guarantees for positive semi-definite problems (case when the A matrix has a null eigenvalue)? This is a common case in robotics that corresponds to over-constrained problems and one should avoid using a regularization matrix R as it leads to less precise solutions.

Experiments on dynamics and parameters learning demonstrate the superiority of the method when compared to a simple MLP or a more sophisticated Interaction-Networks. However, a comparison with a differentiable simulator such as gradSim [gradSim: differentiable simulation for system identification and visuomotor control, Murthy et al. 2021] would be interesting because the approach should be more competitive than other baselines (as shown by experiments from 5.). Durations considered during the evaluation of energy conservation are short when compared to the ones involved in robotic applications. Is this property still valid for longer time horizons? 

Concerning the experiment on robustness, why were the 3D examples generated from an LCP simulator and not with one accounting for the cone constraint? From my understanding, the learnt friction parameter does not have the same meaning as the original one because it corresponds to different friction models which makes it difficult to compare them [Differentiable simulation for physical system identification, Le Lidec et al. 2021]. 

A billiard task is considered to evaluate the approach and compare it against the DiffTaichi differentiable simulator. This simulator includes gradients from the collision detection routine while it is not clear if this information is available in the collision detection module used in the approach proposed in the paper. I would expect this information to be important to solve a problem where one needs to reason about contact points as is the case in the billiard task. Could it be the reason behind the better performance of Difftaichi on this task? Did you consider different initial guesses or targets to assert the robustness of the approach on this task?
 Overall, the limitations as presented by the Authors seems a bit limited.
Indeed, the Authors did not discuss for instance the fact they did consider complementarity between motion and forces at the contact level.
Additionally, they clearly did not discuss the fact they did a mix between acceleration-based simulation and time-stepping methods which will compute a corrective impulse at velocity level. What is the physical and mathematical meaning of such a choice?

Meta-review: This paper proposes a differentiable contact model that can be used together with previously proposed Lagrangian and Hamiltonian neural networks, enabling gradient-based system identification and model-based control for hybrid dynamical systems. All reviewers recommend acceptance and by the end of the rebuttal process became satisfied with the additional experiments and clarifications by the authors. Some limitations on the scalability of the method in contact-rich (and high-dimensional state) scenarios were also illustrated through additional experiments, and reviewers were satisfied. I am recommending this paper for acceptance.    

Decision: accept