Contact Points Discovery for Soft-Body Manipulations with Differentiable Physics

1 INTRODUCTION . Soft body manipulation has a wide application in cooking ( Bollini et al. , 2013 ) , fabric manipulation ( Wu et al. , 2020 ) , healthcare ( Mayer et al. , 2008 ) and manufacturing of deformable objects ( Sanchez et al. , 2018 ) . Differentiable physics has recently been shown as a powerful and effective tool for solving control problems for soft-body manipulation tasks . As demonstrated in Huang et al . ( 2021 ) , given a parameterized manipulation policy , the differentiable physics solver computes the gradients of the policy parameters , enabling gradient-based optimization much more efficiently than reinforcement learning algorithms at finding optimal solutions for soft-body manipulation tasks on a diverse collection of environments . However , the performance of the stand-alone gradient-based solver can be heavily influenced by the policy initialization . Especially , the end effectors ’ initial contact points with objects play critical roles in the optimization . Different contact points may lead to vast differences in manipulation performance due to local optima . Besides , some tasks require agents to switch contact points during the manipulation , where the local optima issue becomes a serious bottleneck for completing these multi-stage tasks . For example , as shown in Figure 2 , an agent needs to control the capsule “ pen ” to sculpt two scribbles on the surface of a yellow plasticine cube . In order to complete the second line , the agent needs to switch contact points after drawing the first one . While the stand-alone differentiable physics solver could possibly draw the first line , it often gets stuck and struggles to draw the second one , due to the lack of gradients that push the pen to a new contact point to begin the second line . How to automatically find proper contact points for soft body manipulation tasks remains a challenge in differentiable physics . 1https : //sites.google.com/view/cpdeformiclr2022 In this paper , we propose a principled framework that integrates an optimal transport-based contact discovery method into differentiable physics ( CPDeform ) to address this important challenge . CPDeform heuristically find contact points for end effectors by using transport priorities computed from optimal transport to compare the current shape with the target shape , where soft-body manipulation is treated as a particle transportation problem . After finding contact points , CPDeform can combine the differentiable physics solver to solve soft body manipulation tasks . On single-stage tasks that do not require contact point switching , CPDeform can find suitable initial contact points to finish the task . On multi-stage tasks , using an example shown in Figure 1 ( right ) where the goal is to reshape a plasticine cube into an airplane , CPDeform can iteratively switch the contact points of end effectors based on transport priorities . This iterative deformation process is motivated by how humans manipulate plasticine . As shown in Figure 1 ( left ) , when humans manipulate a plasticine dough , they tend to repeatedly focus on the point of interest and modify it towards the target shape . CPDeform can mimic this process by iteratively switching contact points of interests based on transport priorities and deforming the soft bodies into the target shape with the help of the differentiable solver . By integrating contact points discovery into the differentiable physics solver , CPDeform can skip over the local minima caused by contact switching and improve the performance of the stand-alone solver . To evaluate the effectiveness of CPDeform , we introduce PlasticineLab-M that extends the existing differentiable physics benchmark PlasticineLab ( Huang et al. , 2021 ) to seven new challenging multistage soft body tasks . Extensive experimental results suggest that : on single-stage tasks where the vanilla differentiable physics solver performs sub-optimally or near-optimally in PlasticineLab , we find that the backbone of CPDeform , a contact point discovery method based on optimal transport , single-handedly performs better than or on par with the manipulation performance obtained with random-selected or human-defined contact points . On multi-stage tasks that are infeasible for the vanilla gradient-based solver , we find that CPDeform performs reasonably well in practice and the iterative deformation method equipped with contact point discovery could serve as an alternative to the expensive long-horizon searching algorithm . In summary , our work makes the following contributions : • We perform an in-depth study of local optimum issues of differentiable physics solver for initial contact points and switching contact points . • We propose a principled framework , CPDeform , that integrates optimal transport-based contact discovery into differentiable physics . • We find that the backbone of CPDeform , the contact point discovery method , can be directly employed by the stand-alone solver to find better initial contact points for single-stage tasks . • On multi-stage tasks , which are infeasible for the vanilla solver , CPDeform employs a heuristic searching approach to iteratively complete the tasks . 2 MOTIVATION . In this section , we provide an intuitive analysis of the drawback of the differentiable physics solver through motivating toy examples . We start with a brief review of how the differentiable physics solver could be employed to optimize manipulation policies . We then demonstrate how initial contact points affect the optimization performance . Finally , we take a simple but representative multi-stage task as an example and discuss why contact switching would often lead to local minima . We study a Writer task as shown in Figure 2 ( a ) . In this task , an agent needs to manipulate the capsule “ pen ” to initiate contact with the yellow plasticine cube , and sculpt a line scribble on the plasticine surface . The agent can move the tip of the pen along three dimensions . To solve this task with differentiable physics , we manually initialize the end-effector “ pen ” near the suitable contact point that allows the `` pen '' to initiate contact with the plasticine . We then parameterize the desired motion trajectory of the pen as a sequence of three-dimensional actions { a1 , . . . , aT } where T is the number of simulation steps . Let { st } 0≤t≤T be simulation states at different time steps which include the state of the plasticine and manipulator . The differentiable simulator starts from the initial state s0 and completes the action sequences by repeatedly executing the forward function st+1 = φ ( st , at+1 ) until the ending state sT has been reached . The objective of this optimizer is to minimize the distance between the current shape and the target shape . We represent the objective as a loss function L ( sT , g ) where g is the target shape . Since the simulation forward function φ is fully-differentiable , we can compute gradients ∂L/∂at of the loss L with respect to action at , and run multiple gradient descent steps to optimize the action sequences by at = at − α∂L/∂at where α is the learning rate . As shown in Figure 2 ( b ) , we can see that the agent succeeds at sculpting the target scribble by moving the “ pen ” downwards . We refer the readers to Algorithm 2 in Appendix D for more details on differentiable physics for controller optimization . However , if we do not initialize the positions of the end effectors well , the solver would get stuck in the local minima . For example , in the task as shown in Figure 3 , we illustrate the optimization outcomes for end effectors with different contact points by showing their corresponding resulting shapes . Even with an arbitrarily large number of steps T given , the gradient-based solver is unable to discover a policy that moves away from the local optimum to the a new contact point that allows for task completion . Such phenomena are commonly observed across soft body manipulation tasks using the differentiable physics solver ( Huang et al. , 2021 ) . When end effectors are far away from the region of interest , it is unlikely that the gradient could push the end-effectors towards the desired region . This observation poses the question of how to efficiently find optimal contact points to place the end effectors . The local minima problem caused by inappropriate contact points becomes a more serious issue in multi-stage tasks . Taking the multi-stage writer task illustrated in Figure 2 ( c ) as an example , the agent now needs to write an additional straight line on the plasticine surface by switching its contact point . Even with a well-initialized contact point for the first line , the solver is unable to relocate to the new region of interest for the upcoming line . We observe that , differing from the vanilla differentiable physics solver , humans tend to employ an explicit “ iterative deformation ” schema to complete such a task . Humans would decompose this task into two stages . In each stage , we tend to iteratively derive the correspondence between the current shape and target shape to arrive at useful contact points from observations and then subsequently move the “ pen ” and write the lines . This motivates us to combine contact point discovery with iterative deformation . 3 METHOD . In this section , we introduce CPDeform , a principled framework that integrates optimal transportbased contact discovery into differentiable physics for solving challenging soft-body manipulation tasks . We first describe our contact point discovery method in relation to the transport priorities found by optimal transport in Section 3.1 . In Section 3.2 , we describe how transport priorities can be used to place end-effectors . Finally , in Section 3.3 , we show how CPDeform integrates contact points discovery with differentiable physics and iteratively deforms the soft bodies for multi-stage tasks . 3.1 OPTIMAL TRANSPORT AND CONTACT POINT DISCOVERY . One way to consider soft-body manipulation is by treating it as a particle transportation problem . By evaluating the cost of transporting the current state particles µ to the target state particles ν , optimal transport provides a useful framework for comparing differences between any given pair of shapes , which can guide us to discover contact points . Let all particles be weighted equally in our simulator . Given a cost matrix M , optimal transport finds a transportation plan P in transportation polytope U by minimizing the transportation cost minP∈U 〈P , M〉 . Casting the problem into dual form , we have OT ( α , β ) : = Eµ [ f ] +Eν [ g ] such that ∀i , j , Lagrange multipliers fi , gj satisfy f i+gj ≤Mij , where α , β are the mass vectors that for the particles in µ , ν respectively . We refer the reader to Appendix B for more details on optimal transport . We focus on the Lagrange multipliers f of the source particles , which we refer to as the dual potentials . Since it represents the supports of the source measure , we interpret f as the transport priorities for the source particles µ . Transport priorities are helpful for selecting contact points . Given a pair of current and target softbody shapes , we intuitively would place the end-effectors around the region of the largest difference between the two shapes , in order to substantially modify the shapes . This observation leads us to place the end-effectors at contact points whose corresponding optimal manipulation policies can minimize the difference between the current and target shapes . However , given the computation budget , it is not easy to directly evaluate the optimality of the contact point by enumerating through a set of contact points . Thus , we propose to use optimal transport priorities to heuristically identify contact points , based on a simple rule of selecting contact points with high transport priorities . We observe that contact points with high transport priorities mostly correspond with superior optimization performances .

Summary: The paper proposed an algorithm to discover appropriate contact points for deformable object manipulation. A key component of the proposed algorithm is to use an optimal-transport approach that computes a transport priority score for each particle in the deformable body. This score is then used to guide a grid search procedure to determine the best initial contact point for the manipulation. For multiple manipulators, a pre-defined set of poses are enumerated to find the best pose to use. The proposed algorithm is evaluated on tasks that requires a single manipulation motion or a sequence of motions to shape a deformable object into desired shapes. The result shows improved performance compared to prior methods and enables completion of novel tasks that prior methods failed.