Differentiable Simulation of Soft Multi-body Systems

1 Introduction . Soft articulated bodies have been studied and utilized in a number of important applications , such as microsurgery [ 32 ] , underwater robots [ 37 ] , and adaptive soft grippers [ 20 ] . Since the compliance of deformable materials can enable robots to operate more robustly and adaptively , soft biomimetic robots are drawing a lot of attention and have made considerable progress . A snailfish robot dives at a depth of 10,900 meters in the Mariana Trench [ 43 ] . Drones equipped with soft manipulators grasp and transmit objects with a 91.7 % success rate [ 17 ] . Soft hands with pneumatic actuators are able to grasp objects of different shapes , including water bottles , eyeglasses , and sheets of cloth [ 12 ] . To enable rapid prototyping of soft robots and efficient design of control algorithms through virtual experiments , we aim to create a realistic deformable multi-body dynamics framework , in which soft articulated robots can be simulated to learn powerful control policies . Design and control of soft robots are challenging because of their nonlinear dynamics and many degrees of freedom . Differentiable physics has shown great promise to deal with such complex problems [ 4 , 14 , 31 , 68 ] . One possibility is to treat soft bodies as volumes that are modeled as sets of particles or finite elements [ 29 , 16 ] . These methods have made great progress , but the volumetric representations are difficult to scale to large multi-body systems and are poorly suited to modeling internal skeletons . Moreover , contact handling in recent differentiable physics frameworks [ 51 , 30 ] often does not comply with Coulomb ’ s Law , which is central to plausible visual realism and correct physical behavior . In this paper , we design a powerful and accurate differentiable simulator for soft multi-body dynamics . Since our entire framework is differentiable , our method can be embedded with gradient-based optimization and learning algorithms , supporting gradient-based system identification , motion planning , and motor control . Within the simulator , we first use tetrahedral meshes to enable adaptive resolution and more accurate modeling . Next , to couple soft materials with articulated skeletons , we design a ∗Equal contribution . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . top-down matrix assembly algorithm within the local steps of Projective Dynamics [ 5 ] . For accurate contact handling , we extend and generalize a dry friction model previously developed for cloth simulation [ 54 ] to soft solids and introduce a new matrix splitting strategy to stabilize the solver . In addition , our simulation framework incorporates actuator models widely used in robotics , including muscles [ 40 ] , joint torques [ 75 ] , and pneumatic actuators [ 12 ] . With the support of the articulated skeleton constraints , dry frictional contact , and versatile actuators , our novel differentiable algorithm can simulate soft articulated robots and compute gradients for a wide range of applications . The key contributions of this work are as follows . • A top-down matrix assembly algorithm within Projective Dynamics to make soft-body dynamics compatible with reduced-coordinate articulated systems ( Sec . 4 ) . • An extended and generalized dry friction model for soft solids with a new matrix splitting strategy to stabilize the solver ( Sec . 5 ) . • Analytical models of muscles , joint torques , and pneumatic actuators to enable more realistic and stable simulation results ( Sec . 4.3 and Appendix C & D ) . • A unified differentiable framework that incorporates skeletons , contact , and actuators to enable gradient computation for learning and optimization ( Sec . 6 ) . • Experimental validation demonstrating that differentiable physics accelerates system identification and motion control with soft articulated bodies up to orders of magnitude ( Sec . 6 ) . In the following paper , Section 2 will discuss related papers in deformable body simulation and differentiable physics . Section 3 is basically a preliminary of projective dynamics to explain the high-level simulation framework and define notations that will be used later . Section 4 and 5 are about how to deal with articulated skeleton and contact in our method . Section 6 will show the ablation studies and comparisons results with other learning methods . Code is available on our project page : https : //github.com/YilingQiao/diff_fem 2 Related Work . Deformable body simulation using Finite Element Method ( FEM ) plays an important role in many scientific and engineering problems [ 32 , 20 , 57 ] . Previous works model soft bodies using different representations and methods for specific tasks . There are several kinds of approaches for modeling body actuation . Pneumatic-based methods [ 9 ] change the rest shape to produce reaction forces . Rigid bones attached within soft materials are also used to control the motion of deformable bodies [ 52 , 38 , 18 , 45 ] . To further simulate biologically realistic motion , it is common to apply joint torques in articulated skeletons [ 36 ] . For example , [ 33 , 76 ] use articulated body dynamics to govern the motion while handling collisions using soft contact . Inspired by animals , different designs of muscle-like actuators for soft-body simulations were also proposed [ 41 , 1 , 42 ] . Regarding contact modeling , spring-based penalty forces are widely used [ 58 , 67 , 27 ] for their simplicity . More advanced algorithms include inelastic projection [ 6 , 24 ] and barrier-based repulsion [ 47 , 48 ] . However , these methods do not always conform to Coulomb ’ s frictional law . We opt for a more realistic dry frictional model [ 44 ] to better handle collisions . Projective Dynamics [ 5 ] is widely used for its robustness and efficiency for implicit time integration . It has been extended to model muscles [ 59 ] , rigid skeletons [ 46 ] , realistic materials [ 53 ] , and accurate contact forces [ 54 ] . Our method also adopts this framework for faster and more stable time integration . In contrast to the aforementioned methods using Projective Dynamics , our algorithm is the first to enable joint actuation in articulated skeletons together with a generalized dry frictional contact for soft body dynamics . Differentiable physics has recently been successfully applied to solve control and optimization problems . There are several types of physically-based simulations that are differentiable , including rigid bodies [ 10 , 11 ] , soft bodies [ 30 , 29 , 39 , 19 , 16 ] , cloth [ 51 , 62 ] , articulated bodies [ 21 , 74 , 63 ] , and fluids [ 71 , 73 , 28 , 70 ] . Differentiable physics simulation can be used for system identification [ 68 , 26 ] , control [ 69 ] , and design [ 14 , 49 ] . For differentiable soft-body dynamics , Du et al . [ 16 ] propose a system for FEM simulation represented by volume mesh . This system has been applied to robot design [ 56 ] and control [ 15 ] . Different from this work [ 16 ] , our approach uses tetrahedral meshes with adaptive resolution to model finer detail and scale better to complex articulated bodies . Hu et al . [ 30 ] , Krishna Murthy et al . [ 39 ] use source code transformation to differentiate the dynamics , but their contact model does not follow Coulomb ’ s law . Geilinger et al . [ 19 ] simulate soft materials attached to rigid parts with penalty-based contact force , but their use of maximal coordinates makes it difficult to incorporate joint torques . In comparison , our model has realistic contact handling , versatile actuators , and skeletons with joint constraints , thereby enabling our method to simulate a much wider range of soft , multi-body systems not possible before . There are other works that approximate physical dynamics using neural networks [ 50 , 2 , 72 , 65 ] . These methods are inherently differentiable but can not guarantee physical correctness outside the training distribution . 3 Soft Body Simulation Using Projective Dynamics . We use Projective Dynamics [ 5 ] to model the physics of soft , multi-body systems because its efficient implicit time integration can make the simulation more stable . We briefly introduce Projective Dynamics below . The dynamics model can be written as M ( qn+1 − qn − hvn ) = h2 ( ∇E ( qn+1 ) + fext ) , ( 1 ) with M being the mass matrix , qn the vertex locations at frame n , h the time step , vn the velocity , E the potential energy due to deformation , and fext the external forces . We choose implict Euler for a stable time integration , then the state qn+1 can be solved by qn+1 = arg min q 1 2h2 ( q− sn ) > M ( q− sn ) + E ( q ) , ( 2 ) where sn = qn + hvn + h2M−1fext . Projective Dynamics reduces the computational cost by introducing an auxiliary variable p to represent the internal energy as the Euclidean distance between p and q after a projection G : E ( q ) = ∑ i ωi 2 ‖Giq− pi‖2F , ( 3 ) where ω is a scalar weight , and E contains internal energy from different sources , such as deformations , actuators , and constraints . The computation of ω , G , and p is dependent on the form of the energy . Combining all energy components into Eq . 2 , we have qn+1 = arg min q 1 2 q > ( M h2 + L ) q + q > ( M h2 sn + Jp ) , ( 4 ) where L = ∑ ωiG > i Gi and J = ∑ ωiG > i Si , and Si is the selector matrix . Since this is a quadratic optimization without constraints , its optimal point is given by the solution of the following linear system : ( M h2 + L ) qn+1 = M h2 sn + Jp ( 5 ) Note that the estimation of p is based on the current values of q . Therefore we need to alternate between computing p and solving q until convergence . Luckily , both steps are fast and easy to solve : solving q in Eq . 5 is easy because it is a simple linear system , and computing p can be fast because it is local and can be parallelized . We show the generic Projective Dynamics method in Alg . 1 . Algorithm 1 Soft body simulation using Projective Dynamics 1 : x1 ← initial condition 2 : for t = 1 to n− 1 do 3 : while not converged do 4 : Compute pi for all energy components i according to q ( Local step ) 5 : Solve q in Eq . 5 according to pi ( Global step ) 6 : end while 7 : end for 4 Articulated Skeletons . Skeletons are indispensable for vertebrate animals and nowadays articulated robots . However , adding skeletons into the soft body simulation is challenging . First , the rigid bones can not be simply replaced by soft materials with large stiffness , since this can make the system unstable and unrealistic . Second , joint connections between bones must be physically valid at all times , and thus also can not be modeled as soft constraints . Moreover , the formulation should support joint actuation as torques to drive the multi-body system like an articulated robot . Li et al . [ 45 ] proposed a method for passive articulated soft-body simulation with ball joint constraints . We extend this method to enable rotational/prismatic joints , torque actuation , and precise joint connections without introducing extra constraints .

Summary: This paper takes on the timely problem of devising differentiable simulators for soft bodies, with a focus on articulated bodies where the soft nature of the material comes with structure in the way different parts are connected together - hence requiring formulation of equations of motion in the sense of multi-body dynamics.  The core contribution is to formulate the simulation problem in a projective dynamics framework, and to exploit the implicit matrix structure in the equations of motion to achieve computational speedups.   This has been implemented in C++ libraries and demonstrated in a few different simulation settings, including a Baymax soft articulated body, a deformable ball falling on cloth and a bridge -  each representing a different form of structure. The main outcome is that this form of differentiable simulation enables faster system identification and control.