Learning Particle Dynamics for Manipulating Rigid Bodies, Deformable Objects, and Fluids

1 INTRODUCTION . Objects have distinct dynamics . Under the same push , a rigid box will slide , modeling clay will deform , and a cup full of water will fall with water spilling out . The diverse behavior of different objects poses challenges to traditional rigid-body simulators used in robotics ( Todorov et al. , 2012 ; Tedrake & the Drake Development Team , 2019 ) . Particle-based simulators aim to model the dynamics of these complex scenes ( Macklin et al. , 2014 ) ; however , relying on approximation techniques for the sake of perceptual realism , their simulation often deviates from real world physics , especially in the long term . Developing generalizable and accurate forward dynamics models is of critical importance for robot manipulation of distinct real-life objects . We propose to learn a differentiable , particle-based simulator for complex control tasks , drawing inspiration from recent development in differentiable physical engines ( Battaglia et al. , 2016 ; Chang et al. , 2017 ) . In robotics , the use of differentiable simulators , together with continuous and symbolic optimization algorithms , has enabled planning for increasingly complex whole body motions with multi-contact and multi-object interactions ( Toussaint et al. , 2018 ) . Yet these approaches have only tackled local interactions of rigid bodies . We develop dynamic particle interaction networks ( DPINets ) for learning particle dynamics , focusing on capturing the dynamic , hierarchical , and long-range interactions of particles ( Figure 1a-c ) . DPI-Nets can then be combined with classic perception and gradient-based control algorithms for robot manipulation of deformable objects ( Figure 1d ) . Learning a particle-based simulator brings in two major benefits . First , the learned simulator , just like other particle-based systems , acts widely on objects of different states . DPI-Nets have successfully captured the complex behaviors of deformable objects , fluids , and rigid-bodies . With learned DPINets , our robots have achieved success in manipulation tasks that involve deformable objects of complex physical properties , such as molding plasticine to a target shape . Our project page : http : //dpi.csail.mit.edu Second , the particle-based representation poses strong inductive bias for learning : particles of the same type have the same dynamics within . This enables the model to quickly adapt to new environments of unknown dynamics within a few observations . Experiments suggest that DPI-Nets quickly learn to adapt to characterize a novel object of unknown physical parameters by doing online system identification . The adapted model also helps the robot to successfully manipulate object in the real world . DPI-Nets combine three key features for effective particle-based simulation and control : multi-step spatial propagation , hierarchical particle structure , and dynamic interaction graphs . In particular , it employs dynamic interaction graphs , built on the fly throughout manipulation , to capture the meaningful interactions among particles of deformable objects and fluids . The use of dynamic graphs allows neural models to focus on learning meaningful interactions among particles , and is crucial for obtaining good simulation accuracy and high success rates in manipulation . As objects deform when robots interact with them , a fixed interaction graph over particles is insufficient for robot manipulating non-rigid objects . Experiments demonstrate that DPI-Nets significantly outperform interaction networks ( Battaglia et al. , 2016 ) , HRN ( Mrowca et al. , 2018 ) , and a few other baselines . More importantly , unlike previous paper that focused purely on forward simulation , we have applied our model to downstream control tasks . Our DPI-Nets enable complex manipulation tasks for deformable objects and fluids , and adapts to scenarios with unknown physical parameters that need to be identified online . We have also performed real-world experiments to demonstrate our model ’ s generalization ability . 2 RELATED WORK . Differentiable physics simulators . Researchers have developed many differentiable physical simulators ( Ehrhardt et al. , 2017 ; Degrave et al. , 2016 ; Todorov et al. , 2012 ) , among which some can also provide analytical gradients ( Tedrake & the Drake Development Team , 2019 ) . In particular , Battaglia et al . ( 2016 ) and Chang et al . ( 2017 ) have explored learning a simulator from data by approximating object interactions with neural networks . Li et al . ( 2019 ) proposed learning to propagate signals along the interaction graph and extended to partially observable scenarios . These methods mostly focus on modeling rigid body dynamics . Differentiable simulators for deformable objects have been less studied . Recently , Schenck & Fox ( 2018 ) proposed SPNets for differentiable simulation of position-based fluids ( Macklin & Müller , 2013 ) . An inspiring concurrent work from Mrowca et al . ( 2018 ) explored learning to approximate particle dynamics of deformable shapes with the Hierarchical Relation Network ( HRN ) . Compared with these papers , we introduce state-specific modeling and dynamic graphs for accurate forward prediction for different states of matter ( rigid bodies , deformable shapes , fluids ) . We also demonstrate how the learned dynamics model can be used for control in both simulation and real world . Our approach is also complementary to some recent work on learning to discover the interaction graphs ( van Steenkiste et al. , 2018 ; Kipf et al. , 2018 ) . Our model can also be naturally augmented with a perception module to handle raw visual input , as suggested by Watters et al . ( 2017 ) ; Wu et al . ( 2017 ) ; Fragkiadaki et al . ( 2016 ) . Model-predictive control with a differentiable simulator . Many recent papers have studied model-predictive control with deep networks ( Lenz et al. , 2015 ; Gu et al. , 2016 ; Nagabandi et al. , 2018 ; Farquhar et al. , 2018 ; Srinivas et al. , 2018 ) . They often learn an abstract state transition function , instead of an explicit account of the environment ( Silver et al. , 2017 ; Oh et al. , 2017 ) , and then use the learned function to facilitate training of a policy network . A few recent papers have employed analytical , differentiable simulators ( de Avila Belbute-Peres et al. , 2018 ; Schenck & Fox , 2018 ) for control problems , such as tool manipulation and tool-use planning ( Toussaint et al. , 2018 ) . Our model builds on and extends these approaches by learning a general physics simulator that takes raw object observations ( e.g. , positions , velocities ) of each particle as input . We then integrate it into classic trajectory optimization algorithms for control . Compared with pure analytical simulators , our learned simulator can better generalize to novel testing scenarios where object and environment parameters are unknown . A few papers have explored using interaction networks for planning and control . They often learn a policy based on interaction networks ’ rollouts ( Racanière et al. , 2017 ; Hamrick et al. , 2017 ; Pascanu et al. , 2017 ) . In contrast , our model learns a dynamics simulator and directly optimizes trajectories for continuous control . Recently , Sanchez-Gonzalez et al . ( 2018 ) have applied interaction networks for control , and Li et al . ( 2019 ) have further extended interaction nets to handle instance signal propagation for controlling multiple rigid bodies under partial observations . Compared with them , our dynamic particle interaction network simulate and control deformable , particle-based objects , using dynamic graphs to tackle scenes with complex object interactions . 3 APPROACH . 3.1 PRELIMINARIES . We first describe how interaction networks ( Battaglia et al. , 2016 ) represent the physical system ; we then extend them for particle-based dynamics . The interactions within a physical system are represented as a directed graph , G = 〈O , R〉 , where vertices O = { oi } represent objects and edges R = { rk } represent relations . Specifically , oi = 〈xi , aoi 〉 , where xi = 〈qi , q̇i〉 is the state of object i , containing its position qi and velocity q̇i . aoi denotes its attributes ( e.g. , mass , radius ) . For relation , we have rk = 〈uk , vk , ark〉 , 1 ≤ uk , vk ≤ |O| , where uk is the receiver , vk is the sender , and both are integers . ark is the type and attributes of relation k ( e.g. , collision , spring connection ) . The goal is to build a learnable physical engine to capture the underlying physical interactions using function approximators φ . The learned model can then be used to infer the system dynamics and predict the future from the current interaction graph as Gt+1 = φ ( Gt ) , where Gt denotes the scene state at time t. Interaction networks . Battaglia et al . ( 2016 ) proposed interaction networks ( IN ) , a generalpurpose , learnable physics engine that performs object- and relation-centric reasoning about physics . INs define an object function fO and a relation function fR to model objects and their relations in a compositional way . The future state at time t + 1 is predicted as ek , t = fR ( ouk , t , ovk , t , a r k ) k=1 ... |R| , ôi , t+1 = fO ( oi , t , ∑ k∈Ni ek , t ) i=1 ... |O| , where oi , t = 〈xi , t , a o i 〉 denotes object i at time t , uk and vk are the receiver and sender of relation rk respectively , and Ni denotes the relations where object i is the receiver . Propagation networks . A limitation of INs is that at every time step t , it only considers local information in the graph G and can not handle instantaneous propagation of forces , which however is a common phenomenon in rigid-body dynamics . Li et al . ( 2019 ) proposed propagation networks to handle the instantaneous propagation of forces by doing multi-step message passing . Specifically , they first employed the ideas on fast training of RNNs ( Lei & Zhang , 2017 ; Bradbury et al. , 2017 ) to encode the shared information beforehand and reuse them along the propagation steps . The encoders for objects are denoted as f encO and the encoder for relations as f encR , where we denote c o i , t = f enc O ( oi , t ) , c r k , t = f enc R ( ouk , t , ovk , t , a r k ) . At time t , denote the propagating influence from relation k at propagation step l as elk , t , and the propagating influence from object i as hli , t . For step 1 ≤ l ≤ L , propagation can be described as Step 0 : h0i , t = 0 , i = 1 . . . |O| , ( 1 ) Step l = 1 , . . . , L : elk , t = fR ( c r k , t , h l−1 uk , t , hl−1vk , t ) , k = 1 . . . |R| , ( 2 ) hli , t = fO ( c o i , t , ∑ k∈Ni elk , t , h l−1 i , t ) , i = 1 . . . |O| , ( 3 ) Output : ôi , t+1 = f output O ( h L i , t ) , i = 1 . . . |O| , ( 4 ) where fO denotes the object propagatorand fR denotes the relation propagator .

Summary: This paper tackles the problem of learning dynamics of non-rigid objects in a physics simulator. This learned dynamics can then be used for planning later. The non-rigid objects are represented via a particle-based system. The dynamics model is learned using NVIDIA's particle-based simulator "Flex". The main idea is to adapt Interaction Networks [Battaglia, 2016] which was earlier proposed for rigid-body simulators to particle-based simulators. Instead of maintaining interactions at the level of objects as in [Battaglia, 2016], the proposed approach models interaction at the level of particles.