Differentiable Simulation of Soft Multi-body Systems
Authors: Yi-Ling Qiao
Junbang Liang
Vladlen Koltun
Ming Lin
Venue: NeurIPS 2021 Poster (2021)
Keywords: differentiable physics
physics simulation

Abstract: We present a method for differentiable simulation of soft articulated bodies. Our work enables the integration of differentiable physical dynamics into gradient-based pipelines. We develop a top-down matrix assembly algorithm within Projective Dynamics and derive a generalized dry friction model for soft continuum using a new matrix splitting strategy. We derive a differentiable control framework for soft articulated bodies driven by muscles, joint torques, or pneumatic tubes. The experiments demonstrate that our designs make soft body simulation more stable and realistic compared to other frameworks. Our method accelerates the solution of system identification problems by more than an order of magnitude, and enables efficient gradient-based learning of motion control with soft robots.

Differentiable Simulation of Soft Multi-body Systems
Yi-Ling Qiao
Junbang Liang
University of Maryland, College Park
University of Maryland, College Park
Vladlen Koltun
Ming C. Lin University of Maryland, College Park
Intel Labs
Abstract
We present method for differentiable simulation of soft articulated bodies. Our work enables the integration of differentiable physical dynamics into gradient-based pipelines. We develop top-down matrix assembly algorithm within Projective Dynamics and derive generalized dry friction model for soft continuum using new matrix splitting strategy. We derive a differentiable control framework for soft articulated bodies driven by muscles, joint torques, or pneumatic tubes. The experiments demonstrate that our designs make soft body simulation more stable and realistic compared to other frameworks. Our method accelerates the solution of system identification problems by more than an order of magnitude, and enables efficient gradient- based learning of motion control with soft robots.
1 Introduction
Soft articulated bodies have been studied and utilized a number of important applications, such as microsurgery [32], underwater robots (37), and adaptive soft grippers (20). Since the compliance of deformable materials can enable robots to operate more robustly and adaptively, soft biomimetic robots are drawing lot of attention and have made considerable progress. A snailfish robot dives at depth of 10,900 meters in the Mariana Trench (43) Drones equipped with soft manipulators grasp and transmit objects with 91.7% success rate [17]. Soft hands with pneumatic actuators able to grasp objects of different shapes, including water bottles, eyeglasses, and sheets of cloth [12] soft robots and efficient design control algorithms through virtual
To enable rapid prototyping experiments, we aim to create realistic deformable multi-body dynamics framework, in which soft articulated robots can be simulated learn powerful control policies.
Design and control of soft robots are challenging because of their nonlinear dynamics and many degrees of freedom. Differentiable physics has shown great promise deal with such complex problems 14 31 68). One possibility is to treat soft bodies as volumes that are modeled as sets of
particles or finite elements 29 [6]. These methods have made great progress, but the volumetric representations are difficult to scale to large multi-body systems and are poorly suited to modeling internal skeletons. Moreover, contact handling in recent differentiable physics frameworks [51] 30 often does not comply with Coulomb's 's Law, which is central to plausible visual realism and correct physical behavior.
In this paper, we design powerful and accurate differentiable simulator for soft multi-body dynamics. Since entire framework is differentiable, method can be embedded with gradient-based opti- mization and learning algorithms, supporting gradient-based system identification, motion planning, and motor control. Within the simulator, we first tetrahedral meshes to enable adaptive resolution and more accurate modeling. Next, to couple soft materials with articulated skeletons, we design
Equal contribution.
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

top-down matrix assembly algoritim within the local steps of Projective Dynamics [5]. For accurate contact handling, we extend and generalize dry friction model previously developed for cloth simulation [54] to soft solids and introduce new matrix splitting stralegy to stabilize the solver. In addition, our simulation framework incorporates actuator models widely used in robotics, including muscles [40,, joint torques (75], and pneumatic actuators [2].. With the support of the articulated skeleton constraints, dry frictional contact, and versatile actuators, our novel differentiable algorithm can simulate soft articulated robots and compute gradients for wide range of applications.
The key contributions of this work are as follows.
A top-down matrix assembly algorithm within Projective Dynamics to make soft-body dynamics compatible with coordiinae articulated systems (Sec. 4
An extended and generalized dry friction model for soft solids with a new matrix splitting strategy to stabilize the solver (Sec. 5).
Analytical models of muscles, joint torques, and pneumatic actuators to enable more realistic and stable simulation results (Sec. 4.3 and Appendix C
A unified differentiable framework that incorporates skeletons, contact, and actuators to enable gradient computation for learning and optimization (6.c.
Experimental validation demonstrating that differentiable physies accelerates system identi-
fication and motion control with soft articulated bodies up to orders of magnitude (Sec. (6)
In the following paper, Section 2 will discuss related papers in deformable body simulation and differentiable physics. Section 3 is basically a preliminary of projective dynamics explain the high-level simulation framework and define notations that will be used later. Section Saaooon/aam 5nd are about how to deal with articulated skeleton and contact in our method. Section will show the ablation studies and comparisons results with other learning methods. Code is available on our project page: /utps://githine com/
2 Related Work
Deformable body simulation using Finite Element Method (FEM) plays important role in many scientific and engineering problems [32 20 57.. Previous works model soft bodics using different representations and methods for specific tasks. There are several kinds of approaches for modeling body actuation. Pneumatic- based methods 99 change the rest shape to produce reaction forces. Rigid bones attached within soft materials are also used to control the motion of deformable To further simulate biologically realistic motion, it is common to apply joint use articulated body dynamics govern the motion while handling collisions using soft contact. Inspired by animals, different designs of muscle-like actuators for soft-body simulations were also proposed [41 42
bodies 522 38
18 45
torques in articulated skeletons (36.. For example, 333 76
Regarding contact modeling, spring-based penalty forces are widely used [58 67 27 for their simplicity. More advanced algorithms include inclastic projection 6 24 and barrier-base repul- sion 47 48). However, these methods do not always conform to Coulomb's 's frictional law. We opt for more realistic dry frictional model 44 to better handle collisions.
Projective Dynamics [5] is widely used for its robustness and efficiency for implicit time integration. It has been extended to model muscles [59], rigid skeletons [46), realistic materials (53), and accurate contact forces [54). Our method also adopts this framework for faster and more stable time integration. In contrast to the aforementioned methods using Projective Dynamics, our algorithm is the first to enable joint actuation in articulated skeletons together with a generalized dry frictional contact for soft body dynamics.
Differentiable physies has recently been successfully applied to solve control and optimization problems. There are several types of physically-ber based simulations that are differentiable, including rigid bodies I0 soft bodies [30 29 19 [6], cloth [2]]]] articulated bodies 21 63) and
fluids [T] 28 70.. Differentiable physics simulation can be used for system identification 68 26).
control (69] and design [4 For differentiable soft-body dynamics, Du et al. [16] propose a
system for FEM simulation represented by volume mesh This system has been applied to robot
design (56) and control [15]. Different from this work [16), our approach uses tetrahedral meshes
with adaptive resolution to model finer detail and scale better to complex articulated bodies.

---

Hu et al. [30], Krishna Murthy et al. [39 use source code transformation to differentiate the dynamics, but their contact model does not follow Coulomb's law. Geilinger et [19] simulate soft materials attached to rigid parts with penalty-based contact force, but their use of maximal coordinates makes difficult to incorporate joint torques. In comparison, our model has realistic contact handling, versatile actuators, and skeletons with joint constraints, thereby enabling our method to simulate much wider range of soft, multi-body systems not possible before.
There are other works that approximate physical dynamics using neural networks 50 These methods are inherently differentiable but cannot guarantee physical correctness outside the training distribution.
651
3 Soft Body Simulation Using Projective Dynamics
We use Projective Dynamics [5] to model the physics of soft, multi-body systems because its efficient implicit time integration make the simulation more stable. We briefly introduce Projective Dynamics below. The dynamics model can be written as
M(q - qn -n hv,) (²((E((n+1)) fezt).
with M being the mass matrix, qa the vertex locations frame n, h the time step, v.. the velocity, E the potential energy due to deformation, and fezt the external forces. We choose implict Fuler for stable time integration, then the state qn+1 can be solved by
1 (q sq)) M(q s.) + E(g).
4n+1 = arg min
242
where 8, qa + lv,, + M-M--fe Projective Dynamies reduces the computational cost by introducing an auxiliary variable represent the internal energy the Euclidean distance between p and q after a projection G:
F((g) Σ Wi |. 2
-p
where w is scalar weight, and E contains internal energy from different sources, such as deforma- tions, actuators, and constraints. The computation of w, G, and is dependent on the form of the energy. Combining all energy components into Eq.
[ we have
M h22
M Sn Jp) Jp).
9u+1 = arg min
29
+I)
+ +
9+
where L = W. and J J=- G S: and S; is the selector matrix. Since this is a quadratic optimization without constraints, its optimal point given by the solution of the following linear system:
M 1
+
9o+1
h2 Sn + Jp
(5)
Note that the estimation p is based on the current values of q. Therefore we need to alternate between computing p and solving until convergence. Luckily, both steps are fast and easy to solve: solving q Eq. 5 is easy because it a simple linear system, and computing p can be fast because local and can be parallelized. We show the generic Projective Dynamics method in Alg. T
Algorithm 1 Soft body simulation using Projective Dynamics
1: x f- initial condition
2: for = 1 to n do
3: while not converged do
4:
Compute p, for all energy components i according to q (Local step)
5: Solve in Eq. according to p, (Global step)
6: end while
7: end for
3

---

4 Articulated Skeletons
Skeletons indispensable for vertebrate animals and nowadays articulated robots. However, adding skeletons into the soft body simulation is challenging. First, the rigid bones cannot be simply replaced by soft materials with large stiffness, since this make the system unstable and unrealistic. Second, joint connections between bones must be physically valid at all times, and thus also cannot be modeled as soft constraints. Moreover, the formulation should support joint actuation as torques to drive the multi-body system like an articulated robot. Li et al. [45] proposed a method for passive articulated soft-body simulation with ball joint constraints. We extend this method enable rotational/prismatic joints, torque actuation, and precise joint connections without introducing extra constraints.
4.1 Rigid Body System
When integrated with hard skeletons, vertices the rigid parts can be expressed
q V
(6)
where Q (I 0) is the projection from homogeneous coordinates to 3D coordinates, TV g4x4 the rigid transformation matrix, and V E I 4xmx the rest-pose homogeneous coordinates of the hthh rigid body.
During the global step in Projective Dynamics, we do not directly solve for Té but for the increment Azи in its Pngree-of-freedont (DoF), avoid nonlinearity. This formulation restricts the changes of the rigid vertices to the tangent space yielded by the current T
++1 q + Agi ql + dqi. Azi q
where q are the vertex locations of the hthh body in the 7th iteration step, and Z₁. is the variable defining the DoF of the hthh rigid body, including the rotation and the translation. The nonrigid part of the vertices can also be integrated with this formulation simply with 0q' I. du
0q" be the Jacobian of the concatenated variables. Eq. can be rewritten Oh
Let B
1
M 1
((M) M
(
M Bn
Az arg min Az 2 Az
A
+
(21))BZ+++
BAz + B
+D))
(8)
After solving Az', the new vertex states q are derived by the new rigid transformation matrix TV using Eq. [6] which is subsequently computed in the local step, discussed below.
Local step. The variable Azi is composed of the increment of rotation wi and translation 1/. based on the current transformation T
Aqi. [-1aflx
I
Here [qL]x x is defined as the vertical stack of the cross product matrices of all vertices in the 1thh rigid body. During the local step, compute the SVD of the new transformation matrix after integrating wt and Ig.
I + win
0
1 0
T
T
=
U.VV,
(10)
and restrict it SO(3) obtain the new rigid transformation, T UVT
The local step of Projective Dynamics for a single rigid body the same as in [46]. However, we propose Eq. to generalize the coupling to kinematic trees [55] with precise and actuated articulation.
4.2 Top-down Matrix Assembly for Articulated Body Systems
The articulated body system formulation is similar to the rigid body one, except that the transformation matrix is now chained:
T
II A
(1I)
DEU

---

where Uj contains all ancestor of the hth link (inclusive), and A. is the local transformation matrix defined by joint u.
For rigid bodies, the vertex locations a rigid body only depend on the body's 's own DoF variables. In the articulated system, however, they also affected by the body's ancestors. Therefore, B changes from a block diagonal matrix to block lower triangular matrix if the rigid body vertices are ordered by their kinematic tree depth.
To compute the matrix B, we consider a link u with one its non-root ancestor v. By the definitions in Sec. 4.1 the corresponding block in matrix B is
OT, Vv. oz,
dA. Se,u Vus
Ba
QP
(12)
v Oz.
where P., the prefix product of the local transformation matrices of the link chain from root to (exclusive), and Sa,u is the suffix product from to u. In boundary cases where v, the formulation becomes
B = QPw dA Va-
(13)
Ōz.
When the root and thus has the same DoFs rigid body, using the results from Sec. 4.1 the formulation can be simplified to
B Burroot = Q[-[qu]x I].
(14)
Computing Eq. 12 requires the matrix products P.. and Sa,u of a link chain (v, u) in the tree. Straightforward approaches here could result in O(N3) complexity, where N is the number of links. However, by utilizing the kinematic tree and conducting the computation in top-down order, the complexity can be reduced O which optimal. The key observation is that the prefix and suffix products can be computed recursively:
P. = P.r Sp', S,,.,,
(15)
(16)
assuming v' is the parent link of v. When we traverse the kinematic tree in a depth-first order, the prefix product can be computed O(1). The suffix product also obtained as we iterate along the path back to the root. Algorithm shows the matrix assembly method starting from the root node:
Algorithm 2 Matrix Assembly for the Articulated System
1: Input: tree link
2: Compute P., using Eq.
15
3: +-
4: while u is not root do
5: Compute S,,. using Eq. 16
6: Compute Ba, using Eq. T2
7: t parent(v)
8: end while
9: Compute Bu,root using Eq.
14
10: for in descendants(u) do
Solve link recursively
12: end for
The transformation matrix A and the Jacobian of a joint depend on the joint type. This is further derived in Appendix(C
4.3 Articulated Joint Actuation
Eq. g is quadratic optimization, the optimal Az' is given by the linear system
HAz k
(17)
5

---

where H = B" (M +L) B and k = BB ((
+ +L) q' ( +p)) Reordering the vertices
into sets of deformable and rigid ones yields the following partitioning of the matrix:
H H
H
Az Azi
kd k
(18)
where * and K,- represents the deformable and the rigid parts, respectively. The joint actuation be directly added k, since the linear system is analogous to the basic formulation Ma = f where the right hand side represents the sum of forces and/or torques. The formulation of pneumatic and muscle actuators can be found in Appendix D
Contact Modeling
We handle the contact using Coulomb's frictional law via Jacobian. To compute the velocities after collisions, we split the left-hand side of Equation into the diagonal mass matrix M and the constraint matrix 12 and move the latter to the right-hand side:
Mv =f h'vv' L''',
(19)
where f Ms, - (M + q² + J'Jp and the contact force e is determined according to f Lvv'' (the current momentum) to enforce non-penctration and static/sliding friction. The idea here is to enforce Coulomb's ' law at every iteration, which ensured by solving v'+1 using the inverse of diagonal matrix M. As long the solver converges the end, the final and will conform to the frictional law.
This method works well for cloth contacts 154] but cannot be directly applied to soft bodies, because solid continuum much stiffer than thin sheets, much larger than those in M on the left-hand side, resulting in severe oscillation or even divergence during the iterative solve.
elements in IL on the right-hand side are
We show that in order to guarantee the convergence of Equation 19 the time step h has to satisfy a certain condition:
Proposition 1. Assuming f and & are fixed, Equation 19
converges if the time step h satisfies
122
(20)
24/3T D |1 goll
where p is the density, μ is the stiffness, T is the number of tetrahedra, and qi are the vertex positions.
Details of the proof can be found in Appendix A Using the setting in our experiments, where ss 1000, y Rs 3 x 103, ||q qo R 10 10-2, and Re 1, we would need to set h < 1/1934 in order to ensure the convergence, which is too strict for the simulation to be useful in general applications.
Splitting scheme. We improve this method be compatible with soft body dynamics by introducing a new splitting scheme. Eq. 19 is reformulated as
(M + D'D)v'++ =f f 122 - DD-D)v' +
(21)
where D are the diagonals of L. Our key observation is that (a) the diagonals of L are necessary and sufficient to stabilize the Jacobian iteration, and (b) adding extra diagonal elements to the left-hand side will not break the Coulomb friction law. We show in Appendix that under the same assumption as Proposition]] I our method is guaranteed to converge no matter how big h is. This improvement accelerates the simulation since larger time steps mean faster computation.
We also note that the new splitting scheme will not modify the behavior of the collision response because the convergence point of Eq. 19 is the same as that of Eq. 21 and thus Coulomb's 's Law is still satisfied at convergence.
Experiments
In this section, we first introduce our implementation and then report ablation studies that demonstrate the importance of skeletons and collision contacts in soft-body dynamics. Subsequently, the

---

gradients computed by our method to perform system identification; specifically, we estimate the physical parameters of bridges. Finally, we perform gradient- based learning of grasping and motion planning on robots with various actuators, including pneumatic gripper, an octopus with muscles, and a skeleton-driven fish. Our method can converge more than an order of magnitude faster than reinforcement learning and derivative-fre baselines.
6.1 Implementation
Our simulator is written in C++, the learning algorithms are implemented in PyTorch 60) and Pybind (34] is used as the interface. We run our experiments on two desktops, one with an Intel Xeon W-2123 CPU @ 3.6GHz and the other with Intel 19-10980XF @ 3.0GHz, respectively. For differentiation, the numerical data structure in our simulator is templatized and integrated into the C++ Eigen library, such that our method can conveniently interoperate with autodiff' tools to differentiate the dynamics. Our method can also run in pure C++ to perform forward simulation.
We refer the open-source code from [59] 2.0., [54] (GNU GPL v3.0), and [45] (MPL2)
during our implementation. More details can be found in our code in the supplement.
To further improve the memory efficiency, we introduce check- pointing scheme 7 into our pipeline. Instead of storing the entire simulation history, we only store the system's 's state in each step. During the backward pass, we reload the saved state vector and
Table I: Memory footprint (GB).
steps
w/o ckpt
w/ ckpt
10
0.9
0.1
resume all the intermediate variables before the backpropagation. This strategy can save a major part of the memory, compared to
20
1.4
0.1
100
6.9
0.1
the brute-force implementation. We conduct an experiment compare the memory consumption with and without this check- pointing scheme. The results are reported in Table CppAD [3 is used to differentiate the simulation here. In this experiment, we simulate bridge and estimate its material properties as shown in Fig- aa) The results show that the memory footprint of the baseline scales linearly with simulation length, while our checkpointing scheme keeps memory consumption nearly constant.
200
15.7
0.1
Bone length 0.13 + 0.00
Body deformation error
Joint angle error 2.45 + 0.87
Runtime (ms)
No skeleton
0.20 = 0.14
132 = 2
Rigid 62
0.07 :
1.81 +
727 258
Passive 45
0.04 - 0.03
2.34 + 0.78
163
MPM 30
0.48 + 0.32
0.12 ::: 0.03
NaN
11 +
Ours
0.05 - 0.03
0.41 = 0.30
191
Ablation study of skeleton realization. 'Bone length error' measures the length change of rigid bone, 'body deformation error' measures the deviation from the desired body length, and 'joint angle error' is the deviation from the target joint configuration. For all metries, lower is better and 0 in error indicates the highest accuracy possible. Our model is the most physically realistic.
Figure I:
6.2 Ablation Study
Controlling soft characters via skeletons is natural and convenient: vertebrate animals are soft, but are driven by piccewise-rigid skeletons. Our simulator supports skeletons and joint torques within soft bodies. This ablation study compares other designs with ours. In this experiment, a Baymax model 13 in its T-pose is released from above the ground, as shown in Figure We embed 5 bones inside Baymax (4 in arms and legs, and in the torso). When Baymax falls to the ground, we also add torques on its shoulders so it can lift its arms to a target Y-pose. More details of the setting and qualitative results Three metrics, summarized in Figure [] are used to measure realism. The metrics are averaged over repetitions with different initial positions and velocities. For comparison, Baymax without the support of rigid bones. Its bone error is non-zero because of the deformation. The Baymax a differentiable rigid body simulator (62) is rigid, so the body length error is non-zero. Li et al. [45] simulate the 'Passive' skeleton case where there is no joint actuation and joint angles cannot be adjusted the desired configuration. We also run DifftaichiMMPM [30] by converting the mesh model to the point-based MPM representation. 'MPM' does not have skeletons so the errors are high. The arms also detach from the body so the joint error is NaN. Our method attains the highest degree of physical realism and correctness overall.
Skeleton constraints.
be found in Appendix B and the supplementary video.
simulate a 'No skeleton'
7

---

Penetration error
Compression
Stretching
Runtime (ms)
Cloth
14
0.041 = 0.004
No
No
930 = 52
Rigid
62
0
No
No 520 135
MPM
30
NaN
Yes
Yes
14 + 0
Ours
0
Yes
Yes
51 = 3
Figure 2: Ablation study of collision bandling scheme. 'Penetration error' is greater than 0 if objects interpenetrate as a result the simulation. 'Compression" and 'stretching' indicate whether the simulation allows the ball to compress and stretch in the vertical and horizontal directions, respectively. Our method can model a soft ball correctly while preventing interpenetration.
2.00
0.00
Objectie famplio
oxjectee tundon
0.00
284
VODDYCA
9.00
0.02
Nelder-Mas Dua
0.02
180
(a) Suspension
(h) Loss
(c) Arch
(d) Loss
Figure 3: System identification. Given the desired displacement under gravity, we estimate Poisson's ratio and Young's 's modulus of each element of the two bridges with different structures. Our method converges much faster than gradient-free methods.
Contact handling. Good contact handling critical for simulating multi-body systems that interact with their environment. In this experiment, we throw 3D soft ball against 2D thin sheet. Metries in this experiment are penetration error and indicators of vertical compression and horizontal stretching. Zero penetration error is ideal. 'Yes' for compression/stretchin model the deformation of the soft ball correctly. The metrics are averaged over experiments with different initial positions and velocities. The dry frictional contact model of Ly et al. [54] does not model the deformation of soft solids, and there could be penetration when the resolutions of the ball and cloth differ a lot due to the nodal collision handling scheme. The rigid differentiable simulator of Qiao et al. 622 can prevent interpenetration. but the ball remains rigid. MPM [30] can model the deformation of both the ball and the cloth, but the cloth is torn apart by the ball and penetration cannot be quantified. In contrast, our method accurately handles collision to avoid interpenctration and correctly simulates the deformation of the ball.
indicates that the simulator can
6.3 Applications
System identification. Determining the material parameters of deformable objects can be chal- lenging given their high dimensionality and complex dynamics. In this experiment, we use our differentiable simulator identify the material property of each finite element cell within the soft body. As shown in Figure 3 there are two bridges with unknown materials: a suspension bridge with both ends fixed and the entire bridge being soft, and arch bridge that has three piers attached to the ground. Given that the movement of the barycenter under gravity, compared to its rest pose, Ax Scm, we estimate Young's modulus and Poisson's ratio of each finite element cell in the bridge. The loss function is the distance from the actual barycenter to the target. The suspension bridge has 668 cells and the arch one has 2911 cells. The number of unknowns is 2n. We compare our method with four derivative-Fo free methods (CMA-ES (25), LEAP 88, BOB YQA [61), and Nelder-Meat (66)) Each experiment is repeated 5 times with different random seeds. As shown in the figure, our method converges in ~0 iterations while others fail to converge even after 100 iterations, indicating that derivative-free methods in this high-dimensional setting become too inefficient to converge to a reasonable solution. By making use of the gradients provided by our method, common gradient-hasee based algorithms can quickly reach the target configuration.
Motion planning. Controlling the motion of deformable bodies is challenging due to their flexible shapes. In this experiment, summarized in Figure 4 the task is to control robots with different actuator types. In general, given an initial state Xo, control policy de, (-), and material parameters 02, the simulator can generate trajectory of the states at all time steps i, Lsim,(Xo de, (-), 02)) = (X1, X2...., ).). If we want the system to reach target state X Xtarget at the end of the simulation, we can define an objective function (X(x,, 01, 02) = [sim (No (Xo. do, (+), 02) the optimization variable are 0 and/or 0z.
Xtarget) ||2, where
8

---

objective function
objective function
objective function
SAC EQL CMA-ES Dur
CMA-ES
0.02
SAC
CHA-ES
0.01
0.5
Ours
Dum
0
200
400
100
200
300
400
500
200
400
episode
Episode
episode
(a) Gripper
(b) Octopus
(c) Fish
Figure 4: Motion control experiments. We simulate (a) a gripper with pneumatic cells in four arms, (b) an octopus with 16 muscles, and (c) a fish with bones inside the body. The goal is to optimize actuation schedules get the robots to the target location. Our method is up orders of magnitude faster than reinforcement learning and derivative-free baselines.
We compare method with Reinforcement Learning algorithms (SAC 23 SQL 22 and
PPO (64). and the best derivative-free optimization method from the last experiment, CMA-ES. We also tried MBPO [35], but found that this method takes too much memory and could not finish any test. All RL methods use the negative of the loss as the reward.
The pneumatic gripper in Figure 4 a) has 56 pneumatic cells in four arms and is attached to an (invisible) drone as in [17]. The pncumatic activation can control the volume of a tetrahedron. When the cells inflate, the arms will move inwards and hold the ball tighter. We control the pneumatic activation as well as the movement of the drone to move the ball from the start (0, O, O) to our target (0, 0.3, 0) in 50 steps. The loss is the distance from the actual position the target position. Our method converges in 10 episodes while CMA-ES and PPO gradually converge in 200 and 500 episodes, respectively.
The muscle-driven octopus in Figure 4() has 8 legs, each with muscles inside. moves forward by actuating the muscles, being pushed by drag and thrust forces induced by the water on the octopus's surface [59]. The octopus starts at (0, 0, 0) and our target location is i (-0.4, 0.8, -0.4). We set the objective be the distance between the current location and the desired location. The length of the simulation is 400 steps, and the control input in each step is 64-dimensional. In total, there are 64 400 25600 variables to optimize. Our method converges in 50 episodes while other methods fail to converge in 500 episodes.
The fish with embedded skeleton in Figure 4 () has 6 bones: 3 in its body, 2 in the fins, and in the tail. The hydrodynamics in this environment is the same as the octopus experiment. The fish starts at (0, 0) in step I and the target location in step 100 is (0, O, 0.15). The objective function the distance from the actual location to the target location. For each step, there will be torque vector of size 5 that represents the joint actuation level. In total, the optimization variable has 500 dimensions. Our method with gradient- based optimization can converge in roughly 50 episodes, while others cannot converge even after 500 episodes.
In summary, gradient-fres optimization methods and RL. algorithms meet substantial difficulties when tackling problems with high dimensionality, such soft, multi-body systems. Even when the action space as small as the one in the gripper case, RL methods still fail to rapidly optimize the policy. By introducing the gradients of the simulation, simple gradient-based optimization outperforms other algorithms. This work hopefully may inspire improvements in RL algorithms that tackle such high-dimensional problems.
7 Conclusion
Our paper has developed a differentiable physics framework for soft, articulated bodies with dry frictional contact. To make the simulation realistic and casy-to-use, we designed a recursive matrix

---

assembly algorithm and generalized dry frictional model for soft continuum with a new matrix splitting strategy. Integrated with joint, muscle, and pneumatic actuators, our method can simulate variety of soft robots. Using our differentiable physics to enable gradient-baset optimization, our method converges more than an order of magnitude faster than the baselines and other existing alternatives.
There are some limitations in our contact handling and soft body dynamics. Currently, though our algorithm is more extensive and generalized than existing differentiable physics algorithms and our implementation handles the most commonly found contact configuration, vertex-face collisions, there could still be edge-edge penetration missed in some corner cases. Moreover, the Projective Dynamics pipeline limits the energy have the form E Gq - p Some nonlinear material models (e.g.. neo-Hooke Hookean) are not captured in this framework and new models for differentiable physics will be required for handling nonlinear and heterogeneous materials. For future work, we aim to add edge-edge collision handling in the Projective Dynamics pipeline. The techniques in (3 can be used
to incorporate addition material types. GPU or other parallel computing implementation can be used to boost the performance of gradient computation.
Acknowledgements. This research is supported in part by Army Research Office, National Science Foundation, Dr. Barry Mersky and Capital One Endowed Professorship.
References
[1] Baptiste Angles, Daniel Rebain, Miles Macklin, Brian Wyvill, Loic Barthe, JP Lewis, Javier Von Der Pahlen, Shahram Izadi, Julien Valentin, Sofien Bouazzz, al. Viper: Volume invariant position-based elastic rods. Proceedings of the ACM on Computer Graphics and Interactive Techniques, 2(2):1-26, 2019.
[2] Filipe de Avila Belbute-Peres, Thomas D. Economon. and J. Zico Kolter. Combining differentiable PDE solvers and graph neural networks for fluid flow prediction. In JCML, 2020.
[3] Bradley M. Bell et al. CppAD: C++ algorithmic differentiation. bstps::/projjctss.col org/ CppAD 2018.
[4] James M. Bern, Pol Banzet, Roi Poranne, and Stelian Coros. Trajectory optimization for cable-driven soft robot locomotion. In Robotics: Science and Systems (RSS), 2019.
[5] Sofien Bouaziz, Sebastian Martin, Tiantian Liu, Ladislav Kavan, and Mark Pauly. Projective dynamics: Fusing constraint projections for fast simulation. ACM Transuctions on Gruphics (TOG), 33(4). 2014.
[6] Robert Bridson, Ronald Fedkiw, and John Anderson. Robust treatment of collisions, contact and friction for cloth animation. In SIGGRAPH, 2002.
[7] Tianqi Chen, Bing Xu, Chiyuan Zhang, and Carlos Guestrin. Training deep nets with sublinear memory cost. arXiv: 1604.06674 74, 2016.
[8] Mark Coleti, Eric O Scott, and Jeffrey K Bassett. Library evolutionary algorithms Python (LEAP). In Genetic and Evolutionary Computation Conference, 2020.
[9] Stelian Coros, Sebastian Martin, Bernhard Thomaszewski. Christian Schumacher, Robert Sumner, and Markus Gross. Deformable objects alive! ACM Transactions on Graphics (TOG), 31(4), 2012.
[10] Filipe de Avila Belbute-Peres, Kevin A. Smith, Kelsey Allen, Josh Tenenbaum, and J. Zico Kolter. End-to-end differentiable physies for learning and control. In Neural Information Processing Systems, 2018.
[11] Jonas Degrave, Michiel Hermans. Joni Dambre, and Francis wyffels. A differentiable physics engine for deep learning in robotics. Frontiers in Neurorobotics, 13, 2019.
[12] Raphael Deimel and Oliver Brock. compliant hand based a novel pneumatic actuator. In ICRA, 2013.
[13] DoodleNotes. Baymax walk cycle animation. htttss//bbberdsssppcc/////eel//d
[14] Tao Du, Kui Wu, Andrew Spielberg. Wojciech Matusik, Bo Zhu, and Eftychios Sifakis. Functional optimization of fluidic devices with differentiable stokes flow. ACM Transactions on Graphics (TOG). 2020.
[15] Tao Du Josie Hughes, Sebastien Wah, Wojciech Matusik, and Daniela Rus. Underwater soft robot modeling and control with differentiable simulation. IEEE Robotics and Automation Letters, 2021.
10

---

[16] Tao Du, Kui Wu, Pingchuan Ma, Sebastien Wah, Andrew Spielberg. Daniela Rus, and Wojeiech Matusik. DifIPD: Differentiable projective dynamics with contact. arXiv:2101.05917, 2021.
[17] Joshua Fishman, Samuel Ubellacker. Nathan Hughes, and Luca Carlone. Dynamic grasping with "soft" drone: From theory practice. arXiv:2103.0 06466, 2021.
[18] Nico Galoppo, Miguel A Otaduy, Serhat Tekin, Markus Gross, and Ming C Lin. Soft articulated characters with fast contact handling. In Computer Graphics Forum, volume 26, 2007.
[19] Moritz Geilinger, David Hahn, Jonas Zehnder, Moritz Bächer, Bernhard Thomaszewski, and Stelian Coros. ADD: Analytically differentiable dynamics for multi-body systems with frictional contact. ACM Transactions on Graphics (TOG), (9))). 2020.
[20] Thomas George Thuruthel, Yasmin Ansari, Egidio Falotico, and Cecilia Laschi. Control strategies for soft robotic manipulators: A survey. Soft Robotics, 5(2):149- 2018
[21] Schoon Ha, Stelian Coros, Alexander Alspach, Joohyung Kim, and Katsu Yamane. Joint optimization of robot design and motion parameters using the implicit function theorem. In Robotics: Science and Systems (RSS), 2017.
[22] Tuomas Haarnoja, Haoran Tang, Pieter Abbeel, and Sergey Levine. Reinforcement learning with deep energy-based policies. In ICML, 2017.
[23] Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, and Sergey Levine. Soft actor-critic: Off Off-policy maximum entropy deep reinforcement learning with stochastic actor. In ICML, 2018
[24] Torsten Hädrich, Bedrich Benes, Oliver Deussen, and Sören Pirk. Interactive modeling and authoring of climbing plants. In Computer Graphics Forum, volume 36, pages 49-61. Wiley Online Library, 2017.
[25] Nikolaus Hansen. The CMA evolution strategy: A tutorial. arXiv: 1604.0077, 2016.
[26] Eric Heiden, David Millard, Erwin Coumans, Yizhou Sheng, and Gaurav S. Sukhatme. NeuralSim: Augmenting differentiable simulators with neural networks. arXiv:201.002117, 2020.
[27] Jonathan Hiller and Hod Lipson. Dynamic simulation of soft multimaterial 3D-printed objects. Seft Robotics, 1(1):88-100 2014.
[28] Philipp Holl, Vladlen Koltun, and Nils Thuerey. Learning control PDEs with differentiable physics. In ICLR, 2020.
[29] Yuanming Hu, Jiancheng Liu, Andrew Spielherg, Joshua B. Tenenhaum, William T. Freeman, Jiajun Wu, Daniela Rus, and Wojeiech Matusik. ChainQueen: A real-time differentiable physical simulator for soft robotics. In JCRA, 2019.
[30] Yuanming Hu, Luke Anderson, Tzu-Mao Li, Qi Sun, Nathan Carr, Jonathan Ragan- KelleK, and Frédo Durand. DiffTaichi: Differentiable programming for physical simulation. In ICLR, 2020.
[31] Zhiao Huang, Yuanming Hu, Tao Du, Siyuan Zhou, Hao Su, Joshua Tenenbaum, and Chuang Gan. PlasticineLab: A soft-body manipulation benchmark with diffcrentiable physics. In ICLR, 2021.
[32] Filip Ilievski, Aaron D Mazzeo, Rohert F Shepherd, Xin Chen, and George M Whitesides. Soft roboties for chemists. Angewandte Chemie, 123(8), 2011.
[33] Sumit Jain and Karen Liu. Controlling physics-based characters using soft contacts. In SIGGRAPH Asia, 2011.
[34] Wenzel Jakob, Jason Rhinelander, and Dean Moldovan. pybindl I seamless operability between c++1l and python, 2016. https://iihub..co//yyy hom.b///oooooooobbb/nnnnn.......
[35] Michael Janner, Justin Fu, Marvin Zhang, and Sergey Levine. When to trust your model: Model-based policy optimization. In Advances Neural Information Processing Systems, 2019.
[36] Yifeng Jiang, Tom Van Wouwe, Friedl De Groote, and C Karen Liu. Synthesis of biologically realistic human motion using joint torque actuation. ACM Transactions On Graphics (TOG), 38(4), 2019.
[37] Robert K Katzschmann. Joseph DelPreto, Robert MacCurdy, and Daniela Rus. Exploration of underwater life with acoustically controlled soft robotic fish. Science Robotics. 3(16), 2018
[38] Junggon Kim and Nancy S Pollard. Direct control of simulated nonhuman characters. IEEE Computer Graphics and Applications, 31(4))56-66 06.556666,220 2011.

---

[39] Jatavallabhula Krishna Murthy, Miles Macklin, Florian Golemo. Vikram Voleti, Linda Petrini, Martin Weiss, Breandan Considine, Jérôme Parent-Lévesque, Kevin Xie, Kenny Erleben. et al. gradSim: Differentiable simulation for system identification and visuomotor control. ICLR, 2021.
[40] Shunichi Kurumaya, Koichi Suzumori, Hiroyuki Nabae, and Shuichi Wakimoto, Musculoskeletal lower- limb robot driven by multifilament muscles. Robomech Journal, 301) 2016.
[41] Scunghwan Lee, Ri Yu, Jungnam Park, Mridul Aanjaneya, Eftychios Sifakis, and Jehee Lec. Dexterous manipulation and control with volumetric muscles. ACM Transactions on Graphics (TOG), 37(4), 2018.
[42] Seunghwan Lee, Moonseok Park, Kyoungmin Lee, and Jehee Lee. Scalable muscle-actuated human simulation and control. ACM Transactions On Gruphics (TOG), 38(4), 2019.
[43] Guorui Li. Xiangping Chen, Fanghao Zhou, Yiming Liang. Youhua Xiao, Xunuo Cao. Zhen Zhang, Minggi Zhang, Baosheng Wu, Shunyu Yin, al. seff-powered soft robot the mariana trench. Nature, 591(7848); 66 71, 2021.
[44] Jie Li, Gilles Daviet, Rahul Narain, Florence Pertails-Descouher Matthew Overby, George Brown, and Laurence Boissieux. An implicit frictional contact solver for adaptive cloth simulation, ACM Transactions on Graphics (TOG), 37(4), 2018.
[45] Jing Li, Tiantian Liu, and Ladislav Kavan. Fast simulation of deformable characters with articulated skeletons in projective dynamics. In ACM SIGGRAAPH/Enngrrpphis Symposium on Computer Animation, 2019.
[46] Jing Li, Tiantian Liu, and Ladislav Kavan. Soft articulated characters in projective dynamies. IEEE Transactions on Visualization and Computer Graphics, 2020.
[47] Minchen Li, Zachary Ferguson, Teseo Schneider, Timothy Langlois, Denis Zorin, Daniele Panozzo, Chenfanfu Jiang, and Danny M Kaufman. Incremental potential contact: Intersection-and inversion-free, large-deformation dynamics. ACM Transactions on Graphics (TOG), 2020.
[48] Minchen Li, Danny M Kaufman, and Chenfanfu Jiang. Codimensional incremental potential contact. arXiv:2012.04457. 2020.
[49] Xuan Li, Jessica McWilliams, Minchen Li, Cynthia Sung, and Chenfanfu Jiang. Soft hybrid aerial vehicle via bistable mechanism. arXiv:2011.0044 2020.
[50] Yunzhu Li, Jiajun Wu, Yan Zhu, Joshua B Tenenbaum, Antonio Torralba, and Russ Tedrake. Propaga- tion networks for model-based control under partial observation. In ICRA, 2019.
[51] Junbang Liang, Ming C. Lin, and Vladlen Koltun. Differentiable cloth simulation for inverse problems. In Neural Information Processing Systems, 2019.
[52] Libin Liu, KangKang Yin, Bin Wang, and Baining Guo. Simulation and control of skeleton-driven soft body characters. ACM Transactions on Graphics (TOG), 32(6), 2013.
[53] Tiantian Liu, Sofien Bouaziz, and Ladislav Kavan. Quasi-Newtt methods for sal-time simulation of hyperelastic materials. ACM Transactions on Graphics (TOG), 36(3), 2017.
[54] Mickaöl Ly, Jean Jouve, Laurence Boissicux, and Florence Bertails-Descoubbl Projective dynamics with dry frictional contact. ACM Transactions on Gruphics (TOG), 39(4), 2020.
[55] Kevin M Lynch and Frank C Park. Modern robotics. Cambridge University Press, 2017.
[56] Pingchuan Ma, Tao Du John Z Zhang, Kui Wu, Andrew Spielberg. Robert K Katzschmann, and Wojeiech Matusik. DiffAqua: A differentiable computational design pipeline for soft underwater swimmers with shape interpolation. In SIGGRAPH, 2021.
[57] Miles Macklin. Simulation for learning and rohotics, numerical methods for contact, deformation, and identification. In PhD Thesis, 2020.
[58] Aleka McAdams. Yongning Zhu, Andrew Selle, Mark Empey, Rasmus Tamstorf, Joseph Teran, and Eftychios Sifakis. Efficient elasticity for character skinning with contact and collisions. In SIGGRAPH. 2011.
[59] Sehee Min. Jungdam Won, Seunghwan Lee, Jungnam Park, and Jehee Lee. Softcon: simulation and control of soft-bodied animals with biomimetic actuators. ACM Transactions Graphics (TOG), 38(6). 2019.
12

---

[60] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chaman, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, al. PyTorch: An imperative style, high-performance deep learning library. Neural Information Processing Systems, 2019.
[61] Michael JD Powell. The BOBYQA algorithm for bound constrained optimization without derivatives. Cambridge NA Report NA2009/06, University of Cambridge, Cambridge, pages 26-46, 2009.
[62] Yi-Ling Qiao, Junbang Liang, Vladlen Koltun, and Ming C. Lin. Sealable differentiable physics for learning and control. In /CML, 2020.
[63] Yi-Ling Qiao, Junbang Liang Vladlen Koltun, and Ming C. Lin. Efficient differentiable simulation of articulated bodies. JCML, 2021.
[64] John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. (XXiv17707.064777 2017.
[65] Han Shao, Tassilo Kugelstadt, Torsten Hädrich, Wojciech Palubicki, Jan Bender, Sören Pirk, and Dominik L Michels. Accurately solving physical systems with graph learning. arXiv preprint arXiv:2006.0889 2020.
[66] Saša Singer and John Nelder. Nelder-Mead algorithm. Scholarpedia, 4(7):2928, 2009.
[67] Breannan Smith, Fernando De Goes, and Theodore Kim. Stable neo-hookean flesh simulation. ACM
Transactions on Graphics (TOG), 37(2). 2018.
[68] Changkyu Song and Abdeslam Boularias. Leamingg to slide unknown objects with diffcrentiable physics simulations. In Roboticx: Science and Systems (RSS), 2020.
[69] Andrew Spielberg, Allan Zhao, Yuamming Hu, Tao Du, Wojeiech Matusik, and Daniela Rus. Learning- in-the-loop optimization: End-to-end control and co-design of soft robots through learned deep latent representations. In Neural Information Processing Systems, 2019.
[70] Tetsuya Takahashi, Junbang Liang, Yi-Ling Qiao, and Ming C Lin. Differentiable fluids with solid coupling for learning and control. In AAAI, 2021.
[71] Kiwon Um, Robert Brand, Yun (Raymond) Fei, Philipp Holl, and Nils Thuerey. Salver-in-the-lopp Learn- ing from differentiable physics to interact with iterative PDE-solvers. In Neural Information Processing Systems, 2020.
[72] Benjamin Ummenhofer, Lukas Prantl, Nils Thuerey, and Vladlen Koltun. Lagrangian fluid simulation with continuous convolutions. In ICLR, 2020.
[73] Nils Wandel, Michael Weinmann, and Reinhard Klein. Learning incompressible fluid dynamics from scratch - towards fast, differentiable fluid models that generalize. In ICLR, 2021.
[74] Keenon Werling, Dalton Omens, Jeongseok Lee, Tonnis Exarchos, and C Karen Liu. Fast and feature- complete differentiable physics for articulated rigid bodies with contact. In Robotics: Science and Systems (RSS), 2021.
[75] Andy Zeng, Shuran Song, Kuan- Yu, Elliott Donlon, Francois R Hogan, Maria Bauza, Daolin Ma, Orion laylor, Melody Liu, Eudald Romo, et al. Robotic pick-and-place of novel objects in clutter with multi-affordance grasping and cross-domain image matching. In ICRA, 2018
[76] Yunbo Zhang, Wenhao Yu, C Karen Liu, Charlie Kemp, and Greg Turk. Learning manipulate amorphous materials. ACM Transuctions on Graphics (TOG), 39(6), 2020.
13


Reviews:
[{"review_id": "zOEz2MZHaGX", "reviewer": "Reviewer_PGkZ", "summary": "This paper proposes a soft multi-body differentiable physics simulation framework based on projective dynamics. The authors propose a top-down matrix assembly algorithm for rigid body simulation algorithms and a new matrix splitting strategy for a generalized dry friction model for soft continuum. The experiments demonstrate that their designs make soft body simulation more stable and realistic compared to other frameworks, and the gradients help accelerate system identification and motion control of soft robots.", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Differentiation of projective dynamics and simulation of an articulated soft body system are all noticeable features of the proposed framework and would benefit the machine learning and robotics community. The authors also provide code supplementarily. I would be happy to see it at the Nuerips conference. I am only a little bit worry if this paper fits Neurips's general audience. The paper focuses more on the simulation side, which might be better evaluated in other relevant conferences such as Siggraph or other soft robot conferences. Here are some comments:\n\n- Algorithm 2 computes the Jacobian of each link. I wonder what is the differences between the proposed algorithm and the way of computing jacobian in rigid body simulation?\n- Section 5 mentions a novel matrix splitting method and shows that in theory it convergences much faster. I wonder how this will affect the experimental/visual effects. I hope the authors could include experiments to justify their contributions.\n- What's the efficiency of the proposed algorithm? How is it compared with MPM implemented with Taichi? Is it fast enough for RL research?", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "tOtRD4MbxkX", "reviewer": "Reviewer_JAVC", "summary": "The proposed paper introduces a novel method for solving the simulation of soft articulated bodies based on a differentiable solver. The key contribution of the introduced method are a top-down matrix assembly approach for projective dynamics, a generalized friction model, analytic models for muscles, and a unified framework that enables computing gradients in a differentiable manner. The method is evaluated based on a number of meaningful experiments and ablation studies. ", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Overall, this is an interesting paper that has all things going for it. The method is technically sound, the paper is well-written, and the experiment setup along with the presented results are convincing. As also stated in the paper, the research direction the paper addresses is important for robotics and research in the direction of differentiable physics is potentially of high impact. W.r.t. the paper I specifically like the experiments towards system identification and motion planning. So, in summary, I am in support of accepting this work to NeurIPS. \n\n\nSpecific comments: \n\n- At the beginning of Section 3 it would be helpful to first formally define the geometric representation (tetrahedral meshes). Also, at the beginning of Section 3 it is not clear why vertices are defined per frame. Some introducing/clarifying sentences would be required here.  \n\n- It would also be helpful to discuss what implications exists due to relying on tetrahedral meshes. Can the method also be used for other (less constrained) geometric representations?\n\n- Section 3: I would also suggest to move the introduction to Projective Dynamics to the Appendix. Equations (1)-(3) are either direct copies or reformulation and given that reference [5] is already provided, restating them here does not seem necessary. Similar for Algorithm 1. For Equation (4)-(5) it would be helpful to more clearly describe what the differences to [5] are. \n\n- Section 4: Here it would also help to first motivate why it is necessary to add articulated skeletons prior to stating that it is challenging to add them. \n\n- L148,L149: It is not clear what is meant by ' ... if the rigid body vertices are ordered\n149 by their kinematic tree depth.'.\n\n- L157: Please clearly state what 'the tree' is. Currently it is not defined. \n\n- L180: Missing ' ' between 'friction.The idea'\n\n- I appreciated the discussion in L306-311. It would even be interesting to further expand on the reasons as to why the algorithms perform as observed. \n\n- Given the results shown, the statement that the 'method is able to simulate a variety of soft robots' seems somewhat unsubstantiated and should be toned-down. \n\nThe authors may want to add the following papers to their discussion of related work on deformable body simulation and differentiable physics: \n\n- T. Hädrich, B. Benes, O. Deussen, S. Pirk, Interactive Modeling and Authoring of Climbing Plants, Computer Graphics Forum (Proceedings of Eurographics), 2017 \n\n- H. Shao, T. Kugelstadt, W. Palubicki, J. Bender, S. Pirk, D. L. Michels, Accurately Solving Physical Systems with Graph Learning, 2020", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Limitations of the method are appropriately discussed. I do not have concerns that the method may have potential negative societal impact.", "review_text": "", "extra_scores": {"ethical_concerns": "I do not have any ethical or diversity concerns toward the topic this paper addresses or the paper in general. ", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3 hours. "}, "extra_text": {}}, {"review_id": "sVlvgHPG3n", "reviewer": "Reviewer_yjag", "summary": "The paper presents a new soft-body physics simulator that is end-to-end differentiable, and shows how it supports efficient system identification and control.  Experiments show the value of the simulator for a range of complex realistic situations -- although only in simulation.  The experiments also report ablations of key model components (skeletons, collision contacts). ", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "I am not an expert in the technical aspect of differentiable simulation or soft-body physics, so my comments should be taken with an appropriate grain of salt.  But I have collaborated as a senior co-author on several projects using related simulators for AI and machine learning problem.  I liked this paper a lot: The topic is important, the writing is clear, and the contribution meaningful.  The experiments are well-done and well-described.  I can't judge originality like an expert in simulation could, but my impression is that the method is novel enough for a top conference contribution. I expect this work will be of interest to a number of different sub-communities at NeurIPS: RL, robotics, and maybe computer vision researchers interested in intuitive physics, and possibly some neuroscientists as well. \n\nThe main thing I felt was missing is an evaluation of the system on scenarios that don't match the physics of the simulator: this could be either control of a simple real-world robot, or control or system identification in a simulator that has different physics than the differentiable simulator (i.e., simulating deployment of the model-based methods in the paper to real-world systems where there is some model mismatch).  I don't see this as a fatal flaw but it would have been very nice to see.  Anything like this that could be added in revision might lead me to raise my score. ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Limitations are described briefly at the end.  That's okay but I would encourage the authors to illustrate concretely (perhaps in the supplementary material) some failure cases and limitations of the current method, which point the way to future work.  I would be especially interested to see how the system breaks down when the contact model isn't completely right. ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "1"}, "extra_text": {}}, {"review_id": "DPx9WdE9Ge", "reviewer": "Reviewer_yZqv", "summary": "This paper takes on the timely problem of devising differentiable simulators for soft bodies, with a focus on articulated bodies where the soft nature of the material comes with structure in the way different parts are connected together - hence requiring formulation of equations of motion in the sense of multi-body dynamics.\n\nThe core contribution is to formulate the simulation problem in a projective dynamics framework, and to exploit the implicit matrix structure in the equations of motion to achieve computational speedups. \n\nThis has been implemented in C++ libraries and demonstrated in a few different simulation settings, including a Baymax soft articulated body, a deformable ball falling on cloth and a bridge -  each representing a different form of structure. The main outcome is that this form of differentiable simulation enables faster system identification and control.", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "I think this is a timely piece of work, and it is well motivated. The authors have situated their work in context by referencing the relevant prior works - this is a large area (i.e., there are lots of papers on multi-body sim, FEM and so on), so I am content that they have referenced significant representative works to discuss different options. This includes recent interest in neural approximations, which do not really guarantee physical realism always. Also, I find the derivations to be sound.\n\nThat said, I find that the following could be better developed and explained in the paper:\n\n(1) The experiment section is interesting in that the authors seem to have made a good attempt at comprehensively selecting different scenarios and teasing apart the reasons why their approach is better. However, some areas are lacking in detail. For instance, a crucial claim is that \"this approach\" outperforms RL and derivative free optimization. However, what is \"this approach\" in specific detail? The paper is almost entirely about the equations of motion and ways of simulating multi-body dynamics up to sec 4/5. Then, we do not have an optimization problem written down to say what the approach is for using the sim to synthesise motion. I can imagine doing this by a one-shot trajectory optimization over a horizon, I can do this by MPC, and so on - what is actually done (I suspect the former but is this stated?). In this sense, the point is not that this method outperforms RL but that a policy synthesis method with this sim as model performs better than policy synthesis without. Is this a fair characterisation of the claims (which I believe, but it isn't clearly stated)?\n\n(2) Likewise, I find the system identification experiment interesting, because this is not just identification of a bulk modulus for the whole object but element by element Young's modulus and Poisson's ratio estimation - which is nice. It is good to see that the proposed method does a good job of quickly identifying this. However, here again I am a little bit confused - \"our method\" is shown to do better than derivative free estimation, but what is the framing of the optimization or error using the projected dynamics and frictions models developed in sec 4?\nIn this sense, the writing of sec 6 seems incomplete, and could benefit from revisiting.\n\n(3) Without rederiving everything from first principles, I have studied the mathematical formulation - which seems sound. However, a few steps could be expanded on. For instance, after eq 7, we are told that non-rigid part of the vertices can be integrated by setting \\frac{\\partial{q^i}}{\\partial{z}} to identity. However, non-rigid dynamics will require additional consistency conditions, e.g., rod or rope dynamics - how does that factor into all this?\n\n(4) Lastly, it isn't entirely clear from the writing which parts of the paper are entirely original and which parts are a competent reimplementation of well understood ideas in mechanics. For instance, ideas like splitting schemes have a long tradition in applied mathematics, and the authors have drawn on this - which is great. However, it would be helpful to flag up in the prose where the novel departures are (beyond the list at the end of sec 1).\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "This is mainly a technical contribution to solver methodology. It addresses specific weaknesses in earlier solvers and proposes new computational methods.\n\nThis is of course relevant to practice - indeed, the entire rationale is to enable better simulation in practical domains. However, taken by itself, this contribution does not have any negative impacts beyond the toolchains for graphics and simulation as currently used. ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2"}, "extra_text": {"ethical_concerns": "This paper is mainly a technical contribution to a type of solver. So, there are no noteworthy ethical implications, and the authors have diligently completed the relevant sections - including code disclosure."}}]

Author rebuttal:

Meta-review: Reviewers unanimously enjoyed this paper and thought it was a timely and impactful contribution to NeurIPS, especially in reinforcement learning and robotics. Reviewers especially highlighted the soft-body model and the motion planning experiments. There were some questions about the clarity of the exposition, and so the authors are encouraged to address the issues raised by the referees before the camera ready version of the paper is due.
Decision: Accept (Poster)