Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: summary: This paper takes on the timely problem of devising differentiable simulators for soft bodies, with a focus on articulated bodies where the soft nature of the material comes with structure in the way different parts are connected together - hence requiring formulation of equations of motion in the sense of multi-body dynamics.

The core contribution is to formulate the simulation problem in a projective dynamics framework, and to exploit the implicit matrix structure in the equations of motion to achieve computational speedups. 

This has been implemented in C++ libraries and demonstrated in a few different simulation settings, including a Baymax soft articulated body, a deformable ball falling on cloth and a bridge -  each representing a different form of structure. The main outcome is that this form of differentiable simulation enables faster system identification and control.
main_review: I think this is a timely piece of work, and it is well motivated. The authors have situated their work in context by referencing the relevant prior works - this is a large area (i.e., there are lots of papers on multi-body sim, FEM and so on), so I am content that they have referenced significant representative works to discuss different options. This includes recent interest in neural approximations, which do not really guarantee physical realism always. Also, I find the derivations to be sound.

That said, I find that the following could be better developed and explained in the paper:

(1) The experiment section is interesting in that the authors seem to have made a good attempt at comprehensively selecting different scenarios and teasing apart the reasons why their approach is better. However, some areas are lacking in detail. For instance, a crucial claim is that "this approach" outperforms RL and derivative free optimization. However, what is "this approach" in specific detail? The paper is almost entirely about the equations of motion and ways of simulating multi-body dynamics up to sec 4/5. Then, we do not have an optimization problem written down to say what the approach is for using the sim to synthesise motion. I can imagine doing this by a one-shot trajectory optimization over a horizon, I can do this by MPC, and so on - what is actually done (I suspect the former but is this stated?). In this sense, the point is not that this method outperforms RL but that a policy synthesis method with this sim as model performs better than policy synthesis without. Is this a fair characterisation of the claims (which I believe, but it isn't clearly stated)?

(2) Likewise, I find the system identification experiment interesting, because this is not just identification of a bulk modulus for the whole object but element by element Young's modulus and Poisson's ratio estimation - which is nice. It is good to see that the proposed method does a good job of quickly identifying this. However, here again I am a little bit confused - "our method" is shown to do better than derivative free estimation, but what is the framing of the optimization or error using the projected dynamics and frictions models developed in sec 4?
In this sense, the writing of sec 6 seems incomplete, and could benefit from revisiting.

(3) Without rederiving everything from first principles, I have studied the mathematical formulation - which seems sound. However, a few steps could be expanded on. For instance, after eq 7, we are told that non-rigid part of the vertices can be integrated by setting \frac{\partial{q^i}}{\partial{z}} to identity. However, non-rigid dynamics will require additional consistency conditions, e.g., rod or rope dynamics - how does that factor into all this?

(4) Lastly, it isn't entirely clear from the writing which parts of the paper are entirely original and which parts are a competent reimplementation of well understood ideas in mechanics. For instance, ideas like splitting schemes have a long tradition in applied mathematics, and the authors have drawn on this - which is great. However, it would be helpful to flag up in the prose where the novel departures are (beyond the list at the end of sec 1).

limitations_and_societal_impact: This is mainly a technical contribution to solver methodology. It addresses specific weaknesses in earlier solvers and proposes new computational methods.

This is of course relevant to practice - indeed, the entire rationale is to enable better simulation in practical domains. However, taken by itself, this contribution does not have any negative impacts beyond the toolchains for graphics and simulation as currently used.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper proposes a soft multi-body differentiable physics simulation framework based on projective dynamics. The authors propose a top-down matrix assembly algorithm for rigid body simulation algorithms and a new matrix splitting strategy for a generalized dry friction model for soft continuum. The experiments demonstrate that their designs make soft body simulation more stable and realistic compared to other frameworks, and the gradients help accelerate system identification and motion control of soft robots.
main_review: Differentiation of projective dynamics and simulation of an articulated soft body system are all noticeable features of the proposed framework and would benefit the machine learning and robotics community. The authors also provide code supplementarily. I would be happy to see it at the Nuerips conference. I am only a little bit worry if this paper fits Neurips's general audience. The paper focuses more on the simulation side, which might be better evaluated in other relevant conferences such as Siggraph or other soft robot conferences. Here are some comments:

- Algorithm 2 computes the Jacobian of each link. I wonder what is the differences between the proposed algorithm and the way of computing jacobian in rigid body simulation?
- Section 5 mentions a novel matrix splitting method and shows that in theory it convergences much faster. I wonder how this will affect the experimental/visual effects. I hope the authors could include experiments to justify their contributions.
- What's the efficiency of the proposed algorithm? How is it compared with MPM implemented with Taichi? Is it fast enough for RL research?
limitations_and_societal_impact: Yes
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: The paper presents a new soft-body physics simulator that is end-to-end differentiable, and shows how it supports efficient system identification and control.  Experiments show the value of the simulator for a range of complex realistic situations -- although only in simulation.  The experiments also report ablations of key model components (skeletons, collision contacts). 
main_review: I am not an expert in the technical aspect of differentiable simulation or soft-body physics, so my comments should be taken with an appropriate grain of salt.  But I have collaborated as a senior co-author on several projects using related simulators for AI and machine learning problem.  I liked this paper a lot: The topic is important, the writing is clear, and the contribution meaningful.  The experiments are well-done and well-described.  I can't judge originality like an expert in simulation could, but my impression is that the method is novel enough for a top conference contribution. I expect this work will be of interest to a number of different sub-communities at NeurIPS: RL, robotics, and maybe computer vision researchers interested in intuitive physics, and possibly some neuroscientists as well. 

The main thing I felt was missing is an evaluation of the system on scenarios that don't match the physics of the simulator: this could be either control of a simple real-world robot, or control or system identification in a simulator that has different physics than the differentiable simulator (i.e., simulating deployment of the model-based methods in the paper to real-world systems where there is some model mismatch).  I don't see this as a fatal flaw but it would have been very nice to see.  Anything like this that could be added in revision might lead me to raise my score. 
limitations_and_societal_impact: Limitations are described briefly at the end.  That's okay but I would encourage the authors to illustrate concretely (perhaps in the supplementary material) some failure cases and limitations of the current method, which point the way to future work.  I would be especially interested to see how the system breaks down when the contact model isn't completely right.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: The proposed paper introduces a novel method for solving the simulation of soft articulated bodies based on a differentiable solver. The key contribution of the introduced method are a top-down matrix assembly approach for projective dynamics, a generalized friction model, analytic models for muscles, and a unified framework that enables computing gradients in a differentiable manner. The method is evaluated based on a number of meaningful experiments and ablation studies. 
main_review: Overall, this is an interesting paper that has all things going for it. The method is technically sound, the paper is well-written, and the experiment setup along with the presented results are convincing. As also stated in the paper, the research direction the paper addresses is important for robotics and research in the direction of differentiable physics is potentially of high impact. W.r.t. the paper I specifically like the experiments towards system identification and motion planning. So, in summary, I am in support of accepting this work to NeurIPS. 


Specific comments: 

- At the beginning of Section 3 it would be helpful to first formally define the geometric representation (tetrahedral meshes). Also, at the beginning of Section 3 it is not clear why vertices are defined per frame. Some introducing/clarifying sentences would be required here.  

- It would also be helpful to discuss what implications exists due to relying on tetrahedral meshes. Can the method also be used for other (less constrained) geometric representations?

- Section 3: I would also suggest to move the introduction to Projective Dynamics to the Appendix. Equations (1)-(3) are either direct copies or reformulation and given that reference [5] is already provided, restating them here does not seem necessary. Similar for Algorithm 1. For Equation (4)-(5) it would be helpful to more clearly describe what the differences to [5] are. 

- Section 4: Here it would also help to first motivate why it is necessary to add articulated skeletons prior to stating that it is challenging to add them. 

- L148,L149: It is not clear what is meant by ' ... if the rigid body vertices are ordered
149 by their kinematic tree depth.'.

- L157: Please clearly state what 'the tree' is. Currently it is not defined. 

- L180: Missing ' ' between 'friction.The idea'

- I appreciated the discussion in L306-311. It would even be interesting to further expand on the reasons as to why the algorithms perform as observed. 

- Given the results shown, the statement that the 'method is able to simulate a variety of soft robots' seems somewhat unsubstantiated and should be toned-down. 

The authors may want to add the following papers to their discussion of related work on deformable body simulation and differentiable physics: 

- T. Hädrich, B. Benes, O. Deussen, S. Pirk, Interactive Modeling and Authoring of Climbing Plants, Computer Graphics Forum (Proceedings of Eurographics), 2017 

- H. Shao, T. Kugelstadt, W. Palubicki, J. Bender, S. Pirk, D. L. Michels, Accurately Solving Physical Systems with Graph Learning, 2020
limitations_and_societal_impact: Limitations of the method are appropriately discussed. I do not have concerns that the method may have potential negative societal impact.
review_title: null

Meta-review: Reviewers unanimously enjoyed this paper and thought it was a timely and impactful contribution to NeurIPS, especially in reinforcement learning and robotics. Reviewers especially highlighted the soft-body model and the motion planning experiments. There were some questions about the clarity of the exposition, and so the authors are encouraged to address the issues raised by the referees before the camera ready version of the paper is due.
Decision: Accept (Poster)