Contact Points Discovery for Soft-Body Manipulations with Differentiable Physics
Authors: Sizhe Li
Zhiao Huang
Tao Du
Hao Su
Joshua B. Tenenbaum
Chuang Gan
Venue: ICLR 2022 Spotlight (2022)
Keywords: differentiable physics
soft body manipulation

Abstract: Differentiable physics has recently been shown as a powerful tool for solving soft-body manipulation tasks. However, the differentiable physics solver often gets stuck when the initial contact points of the end effectors are sub-optimal or when performing multi-stage tasks that require contact point switching, which often leads to local minima.
To address this challenge, we propose a  contact point discovery approach (CPDeform) that guides the stand-alone differentiable physics solver to deform various soft-body plasticines. The key idea of our approach is to integrate optimal transport-based contact points discovery into the differentiable physics solver to overcome the local minima from initial contact points or contact switching.
On single-stage tasks, our method can automatically find suitable initial contact points based on transport priorities. On complex multi-stage tasks, we can iteratively switch the contact points of end-effectors based on transport priorities. To evaluate the effectiveness of our method, we introduce PlasticineLab-M that extends the existing differentiable physics benchmark PlasticineLab to seven new challenging multi-stage soft-body manipulation tasks. Extensive experimental results suggest that: 1) on multi-stage tasks that are infeasible for the vanilla differentiable physics solver, our approach discovers contact points that efficiently guide the solver to completion; 2) on tasks where the vanilla solver performs sub-optimally or near-optimally, our contact point discovery method performs better than or on par with the manipulation performance obtained with handcrafted contact points.


CONTACT POINTS DISCOVERY FOR SOFT-BODY MA- NIPULATIONS WITH DIFFERENTIABLE PHYSICS
Sizhe Li
Zhiao Huang
Tao Du MIT
University of Rochester sl 11966uu.... rochester.eeeeoo edd
UC San Diego
z2huang@eng ucsd. edu
taoduêcsail www.ii.........mmm.. edu
Hao Su
Joshua Tenenbaum
Chuang Gan MIT-IBM Watson Lab
UC San Diego
MIT BCS, CBMM, CSAIL ib: @bt@mit.edu
haosußeng, ucsd. edu
ganchuang@csail mit edu
ABSTRACT
Differentiable physics has recently been shown as a powerful tool for solving soft-body manipulation tasks. However, the differentiable physics solver often gets stuck when the initial contact points of the end effectors are sub-optimal or when performing multi-stage tasks that require contact point switching, which often leads to local minima. To address this challenge, we propose a contact point discovery approach (CPDeform) that guides the stand-alone differentiable physics solver to deform various soft-body plasticines. The key idea of our approach is to integrate optimal transport-ban based contact points discovery into the differentiable physics solver to overcome the local minima from initial contact points contact switching. On single-stage tasks, our method can automatically find suitable initial contact points based transport priorities. On complex multi-stage tasks, iteratively switch the contact points of end-effectors based on transport priorities. To evaluate the effectiveness of our method, we introduce Plasticinel Lab-M that extends the existing differentiable physics benchmark Plasticinel .ab to seven new challenging multi-stage soft-body manipulation tasks. Extensive experimental results suggest that: 1) on multi-stage tasks that are infeasible for the vanilla differentiable physics solver, our approach discovers contact points that efficiently guide the solver to completion; 2) on tasks where the vanilla solver performs sub-optimally or optimally,, our contact point discovery method performs better than on par with the manipulation performance obtained with handcrafted contact points. Demos are available on our project page'.
INTRODUCTION
Soft body manipulation has a wide application in cooking (Bollini et al., 2013), fabric manip- ulation (Wu et al., 2020), healtheare (Mayer et al., 2008) and manufacturing of deformable ob- jects (Sanchez et al., 2018). Differentiable physics has recently been shown a powerful and effective tool for solving control problems for soft-body manipulation tasks. As demonstrated in Huang al. (2021), given parameterized manipulation policy, the differrnniablee physics solver computes the gradients of the policy parameters, enabling gradient-based optimization much more effi- ciently than reinforcement learning algorithms finding optimal solutions for soft-body manipulation tasks on diverse collection of environments.
However, the performance of the stand-alone gradient- based solver can be heavily influenced by the policy initialization. Especially, the end effectors' initial contact points with objects play critical roles in the optimization. Different contact points may lead to vast differences in manipulation performance due to local optima. Besides, some tasks require agents to switch contact points during the manipulation, where the local optima issue becomes serious bottleneck for completing these
"Equal Contribution
Project Page: http::///ddeeeeee csaf 1.mm e.ddu

---

multi-stage tasks. For example, as shown in Figure 2, agent needs to control the capsule "pen" to sculpt two scribbles on the surface of a yellow plasticine cube. In order to complete the second line, the agent needs to switch contact points after drawing the first one. While the stand-alone differentiable physics solver could possibly draw the first line, it often gets stuck and struggles to draw the second one, due the lack of gradients that push the pen new contact point to begin the second line. How to automatically find proper contact points for soft body manipulation tasks remains a challenge in differentiable physics.
In this paper, we propose principled framework that integrates an optimal transport-based contact discovery method into differentiable physics (CPDeform) address this important challenge. CPDe- form heuristically find contact points for end effectors by using transport priorities computed from optimal transport to compare the current shape with the target shape, where soft-body manipulation treated as a particle transportation problem. After finding contact points, CPDeform can combine the differentiable physies solver to solve soft body manipulation tasks. On single-stage tasks that do not require contact point switching, CPDeform can find suitable initial contact points to finish the task.
On multi-stage tasks, using an example shown in Figure (right) where the goal is to reshape a plasticine cube into an airplane, CPDeform can iteratively switch the contact points of end effectors based on transport priorities. This iterative deformation process is motivated by how humans manipulate plasticine. As shown in Figure 1 (left), when humans manipulate a plasticine dough, they tend to repeatedly focus on the point of interest and modify it towards the target shape. CPDeform mimic this process by iteratively switching contact points of interests based transport priorities and deforming the soft bodies into the target shape with the help of the differentiable solver. By integrating contact points discovery into the differentiable physics solver, CPDeform can skip over the local minima caused by contact switching and improve the performance of the stand-alone solver.
To evaluate the effectiveness of CPDeform, we introduce PlasticineLab-M that extends the existing differentiable physics benchmark PlasticineLab (Huang et al., 2021) to seven new challenging multi- stage soft body tasks. Extensive experimental results suggest that: on single-stage tasks where the vanilla differentiable physics solver performs sub-optimally or near-optimally in PlasticinelLab, we find that the backbone of CPDeform, a contact point discovery method based on optimal transport, single-handedi performs better than cundom-selectoo or human-defined contact points. On multi-stage tasks that are infeasible for the vanilla gradient- pradient-based solver, find that CPDeform performs reasonably well in practice and the iterative deformation method equipped with contact point discovery could serve as an alternative to the expensive long-horizon searching algorithm. In summary, our work makes the following contributions:
par with the manipulation performance obtained with
We perform in-depth study of local optimum issues of differentiable physics solver for initial contact points and switching contact points.
We propose a principled framework, CPDeform, that integrates optimal transport-based contact discovery into differentiable physics.
We find that the backbone of CPDeform, the contact point discovery method, can be directly employed by the hand-alone solver to find better initial contact points for single-stage tasks. On multi-stage tasks, which are infeasible for the vanilla solver, CPDeform employs heuristic searching approach to iteratively complete the tasks.
Stage1
Stage
Stage
Figure 1: Left: the process that a human applies to transform a plasticine dough into a bowl. Right: use images captured in time sequence to demonstrate how framework reshapes plasticine cube into a target airplane. The bottom row indicates the transport priorities found by our framework by computing the optimal transport between the current and target shapes. The top row illustrates the pose of the end-effectors selected by our framework based on the transport priorities.
2

---

viuck local optmumm
(a) Initial Shape
(b) physics optimization
(c) reaching local optimum
(d) Target Shape
Figure 2: (a) Writer manipulate "pen"; (b) given an initial handcrafted position of the "pen", the differentiable physics solver can control the "pen" to minimize the loss; (c) the solver gets stuck and fails to finish the second curve, due the lack of gradient to push the "pen" towards the second line. (d) shows the target shape.
MOTIVATION
In this section, we provide an intuitive analysis the drawback of the differentiable physics solver through motivating toy examples. We start with brief review of how the differentiable physics solver could be employed to optimize manipulation policies. We then demonstrate how initial contact points affect the optimization performance. Finally, take simple but representative multi-stage task as an example and discuss why contact switching would often lead to local minima.
We study Writer task as shown in Figure 2(a). In this task, agent needs to manipulate the capsule "pen" to initiate contact with the yellow plasticine cube, and sculpt a line scribble on the plasticine surface. The agent move the tip of the pen along three dimensions.
To solve this task with differentiable physics, we manually initialize the end-effector "pen" near the suitable contact point that allow the "pen" to initiate contact with the plasticine. We then parameterize the desired motion trajectory of the "pen" as sequence of three-dimensional actions {a)...., ar} where T is the number of simulation steps. Let (8e)OS be simulation states different time steps which include the state of the plasticine and manipulator. The differentiable simulator starts from the initial state se and completes the action sequences by repeatedly executing the forward function 86+1 (f8t, 0t+11 1) until the ending state 8r has been reached. The objective of the optimizer to minimize the distance between the current and target shapes. We represent the objective as loss function C(sT,g) gg) where is the target shape. Since the simulation forward function é is fully-differentiable, we can compute gradients ÔC/0a, the loss with respect to action a,, and gradient descent steps to optimize the action sequences by a, o₁ a&L/ /d,, where is the learning rate. As shown in Figure 2(b), we can see that the agent succeeds at sculpting the target scribble by moving the "pen" downwards. We refer the readers to Algorithm in Appendix D for more details on differentiable physics for controller optimization.
However, if positions of the end-effectors solver would get stuck in the local minima. Taking the task shown Figure as example, we illustrate the optimization outcomes with different contact points by showing their corresponding resulting shapes. Even with arbitrarily large number of steps T' given, the gradient- based solver is unable to discover policy that moves away from the local optimum new contact point that allows for task completion. Such phenomena are commonly observed across soft body manipulation tasks using the differentiable physics solver (Huang et al., 2021). When end-effectors are far away from the region of interest, it often unlikely that the gradient could push the end-effectors towards the desired region. This observation poses the question of how to efficiently find optimal contact points to place the end-effectors.
well-initialized, the
Figure 3: Writer-vl in Plas- ticineLab. The bottom-right figure shows the target shape.
The local minima problem caused by inappropriate contact points becomes stati-stage tasks. Taking the multi-stage writer task Figure 2(c) as an example, the agent needs write additional line the plasticine surface by switching its contact point. Even with
serious issue in

---

well- cel-iiniiillized contact point for the first line, the solver is unable to relocate to the new region of interest for the upcoming line. We observe that, differing from the vanilla differentiable physics solver, humans tend to employ an explicit "iterative deformation" schema to complete such task. Humans would decompose this task into two stages. In each stage, we tend to iteratively derive the correspondence between the current and target shapes arrive useful contact points from observations and then subsequently move the "pen" and write the lines. This motivates us to combine contact point discovery with iterative deformation.
3 METHOD
In this section, we introduce CPDeform, a principled framework that integrates optimal transport- based contact discovery into differentiable physics for solving challenging soft-body manipulation tasks. We first describe contact point discovery method in relation to the transport priorities found by optimal transport in Section 3.1. In Section 3.2, we describe how transport priorities can be used to place end-effectors. Finally, in Section 3.3, we show how CPDeform integrates contact points discovery with differentiable physics and iteratively deforms the soft bodies for multi-stage tasks.
3.1 OPTIMAL TRANSPORT AND CONTACT POINT DISCOVERY
One way to consider soft-body manipulation is by treating it as particle transportation problem. By evaluating the cost of transporting the current state particles μ to the target state particles v. optimal transport provides a useful framework for comparing differences between any given pair of shapes, which can guide us to discover contact points. Let all particles be weighted equally in our simulator. Given cost matrix M, optimal transport finds transportation plan P in transportation polytope U by minimizing the transportation cost min PEu (P, M). Casting the problem into dual form, we have OT B) := maxf.g En [f] + [o]g] such that Ji Lagrange multipliers fi,9j 9j satisfy fi + 9j S Mij. where at, 3 are the mass vectors for the particles in μu, v respectively. We refer the reader Appendix B for more details on optimal transport. We focus on the Lagrange multipliers of the source particles, which we refer to as the dual potentials. Since it represents the supports of the source measure, we interpret f as the transport priorities for the source particles jr.
Transport priorities are helpful for selecting contact points. Given pair of current and target shapes, we intuitively would place the end-effectors around the region of the largest difference between the two shapes, in order to substantially modify the shapes. This observation leads us to place the end-effectors at contact points whose corresponding optimal manipulation policies can minimize the shape difference. However, it is computationally prohibitive to directly evaluate the optimality of the contact point by exhaustively searching through a set of contact points. Thus, we propose to heuristically identify contact points, based on simple rule of selecting contact points with high transport priorities. We observe that contact points with high transport priorities mostly correspond with superior optimization performances.
3.2
END EFFECTOR PLACEMENT WITH TRANSPORT PRIORITIES
In this section, we describe how manipulators are placed at advantagcous locations based on optimal transport priorities. We first describe the single manipulator case, which can be extended to multiple manipulator environments by adding a heuristic.
Direct Single Manipulator Placement. The transport priorities can be directly employed place the single manipulator. We identify the source particle μ corresponding to the largest potential f". We then find suitable contact point around jr through grid search, where we create 3D) grid centered at 1r" with each dimension evenly spaced into N; intervals. The criterion of the grid search is as follows: We reject points whose placements of the manipulator lead to collisions with the particles. And we iterate over each grid point x E R2, and place the manipulator at the point with the maximal score according the following criterion: score(z)
ft N₂ ΣiAr, M(ca))+1 where d(p4, 2)) computes the signed distance between the particle μ4; to the closest point on the manipulator placed at grid point x. As shown in Figure (top), the criterion leads us to discover contact points that allow the manipulator to cover high potential particles while reducing subsequent changes to the low potential particles.
4

---

sapendui op4s
Shape
Transpori Priorties
Placement
Target Shape
Figure 4: Visualizations of the placement strategies. We visualize the current and target shapes, transport priorities for contact discovery, and the resulting placement of the manipulators. We show single manipulator case using Torus (top), and multiple-manipulttor case using Airplane (bottom)
Multiple Manipulator Placement With Heuristic. For multiple manipulator environments, we need to consider heuristically defined candidate pose set T, where each pose corresponds with example, it is more advantageous
different manipulation strategy. Taking Figure 4 (bottom) to use the highlighted pose than the ones drawn with low opacity. Each pose is specified by the orientations of the manipulators and the direction vectors for placing other manipulators in relation to placement strategy for the first
the first manipulator. For each pose, we employ the manipulator. Using the first placement position as a starting point, for the remaining manipulators, we search along their pose-specific directions to find placement positions with minimal distances from the first manipulator that do not result in collisions with soft bodies. Since the differentiable physics solver is able to effectively adjust manipulator orientations during optimization, in practice, the candidate pose does not need be exhaustively large. In practice, we use three poses cover left-right, top-bottom, and front-back grasping poses.
3.3 ITERATIVE DEFORMATION FOR MULTI-STAGE SOFT BODY MANIPULATION
Continuing with the focus on multi-stage soft-body manipulation tasks, we now describe how CPDeform combines the contact point discovery method with the of differentiable physics for these tasks. For every stage, we iteratively specify the contact points and perform stage-level manipulation with the differentiable physies solver. For the multiple manipulator case, search over contact plans corresponding to different poses, and choose the plan that achieves the lowest loss.
We present the detailed Algorithm in Appendix C. We start with the initial shape Sg. Our goal is to reach the target shape G within given number stage Nstage Each stage contains sstep simulation steps. As described in Section 3.2, consider heuristically defined candidate pose set T. In each stage, we first compute the optimal transport between the current shape S; and the target shape G to derive the transport priorities. Next, we select the particle p with the largest transport priority. For each pose, we search for the placement in relation to that maximizes the aforementioned criterion.
EXPERIMENTS
We conduct multiple experiments test the efficacy of CPDeform on soft-body manipulation tasks to address the following questions:
On multi-stage tasks that involve multiple contact switches, can CPDeform complete these tasks by iteratively manipulating the soft-bodies?
How robust is the backbone CPDeform, the contact point discovery method, when tested on single-stage tasks where it is only allowed to specify a single contact point in one shot?
How is transport priority compared with other contact discovery methods?
5

---

Airplane
Chair
Bottle
Star
Writer++
Move++
Rope++
Pinch
HLL
Figure 5: Task illustrations of PlasticineLab-M with target and initial shapes
4.1 DATASETS
To extensively evaluate our method, we introduce PlasticineLab-Me a dataset that extends the existing differentiable physics benchmark PlasticineLab with seven new challenging multi-stage soft-body manipulation tasks, and contains the multi-stage environment Pinch in Plasticinel.ab.
We show these eight multi-stage tasks in Figure 5, whose task descriptions are detailed in Appendix A. We also use the remaining single-stage tasks in PlasticineLab evaluate our contact point discovery method. For multi-stage environments, we use the Wasserstein-1 distance (Solomon et al., 2015) approximated by Sinkhorn iteration (Cuturi, 2013) between the source and target particles to quantify the fine-gre enn-grained difference between state and the goal. For single-stage environments, we evaluate our approach using the IoU metric for a fair and consistent comparison with PlasticineLab.
4.2 EVALUATION OF CPDEFORM ON MULTI- STAGE TASKS
We compare our approach with the following basclines: PlasticineLab: The vanilla gradient-based solver does not come with any contact point discovery features. It corresponds with the stand-alone differentiable physics solver described in Huang et al. (2021). Reinforcement Learning (RL): We evaluate the performance of the existing RL algorithms on our tasks. We use three model-free reinforcement learning algorithms: Soft Actor-Critic (SAC) (Haarnoja et al., 2017), Policy Proximal Optimization (PPO) (Schulman et al., 2017). and TD3 (Fujimoto al., 2018). For each stage, we optimize for 200 episodes for differentiable thysics-based approaches with learning rate 0.1. For each environment, we modestly choose horizon of 10 or 20. We restrict the number environment steps used for optimization under million. We train each RI. algorithm on each environment for 1000 episodes, with 1000 environment steps per episode, which accounts for the million environment-step limit. Our reward function is of the form is the Wasserstein-1 distance between the source and target particles for measuring shape differences. And R.grasp encourages the manipulators be closer the soft bodies. For fair comparison with Pinch from PlasticineLab, we use the reward formulation described in Huang et al. (2021).
RRhhpee Ryrasps where Ryhape
We show the quantitative results Table and the qualitative results Figure 6. We find that our approach is capable of finishing these complex tasks, and significantly outperforms the baselines. We find that with the discovered contact points, our approach able to iteratively build and refine the nose, tail, and wings of the Airplane. In Chair, we find that our approach guides the solver to first create the general seat, then refine the arm rest and back of the chair. In Bottle, our approach first pushes down the top of the plasticine cube to create the neck, before refining the sides of the bottle. For Move+t, our approach able to complete the transportation tasks the three cubes by selecting the most advantageous object to transfer each stage. In Rope++, approach first moves the rope to form the general shape, before refining the ends of the rope. In Writer++. our approach is capable to iteratively guide the solver print the "ICLR" letters the plasticine cube. Comparatively.

---

Airplane
Chair
Write+++
Initial Shape
CPDeform
PlasticineLab
Target Shape
Figure 6: Qualitative results of CPDeform and Plasticinel.an .ab
multi- stage task environments.
Env
Airplane
Chair
Bottle
Star
SAC
0.0437 0.0006
0.0319 0.0003
0.0383 0.0020
0.0382 0.0012
PPO
0.0507 0.0082
0.0329 0.0004
0.0394 0.0002
0.0352
0.0003
TD3
0.0555 + 0.0043
0.0364 + 0.0033
0.0443 0.0001
0.0403
0.0047
PlasticineLab
0.0324 + 0.0023
0.0218 0.0002
0.0283 0.0007
0.0274
0.0003
CPDeform
0.0073 1 0.0001
0. 0.0072 1 0.0001
0.0093 1 0.0001
0.0198
O. 0.0004
Env
Move++
Ropc++
Writer++
Pinch
0.1660 0.0118
0.0333 0.0038
0.0149 0.0002
0.0204
0.0002
PPO
0.1936 0.0508
0.0387 0.0082
0.0164 0.0001
0.0189 0.0002
TD3
0.2164 0.0057
0.0396 0.0023
0.0153 0.0083
0.0202 0.0003
PlasticineLab
0.1895 0.0001
0.0075 0.0004
0.0122 0.0001
0.0094 0.0016
CPDeform
0.0127 0.0030
0.0052 0.0004
0.0067 0.0001
0.0081 0.0001
Table The averaged Wasserstein-1 distance and the standard deviations of each method.
find that the stand-alone differentiable physics solver fails completely the multi-stage tasks, lacks the necessary exploration power to overcome the local minima. Take Move++ as an example, the vanilla solver is unable to away from the cube it first initiates contact with after stage. Similar to PlasticineLab (Huang et al., 2),1), we also observe that the RL approaches general perform worse than the vanilla gradient-based method.
4.3 EVALUATION OF CONTACT POINT DISCOVERY ON SINGLE STAGE TASKS
To further demonstrate the effectiveness of our approach, we compare the one-shot contact points discovered by the backbone CPDeform with the human-defined contact points on single-stage tasks from PlasticineLab. Table lists the normalized incremental loU scores, together with the standard deviations all approaches.
From Table 2 manipulation performance obtained from the human-defined initial end-effector positions from Plasticinel.ab.
that most tasks, CPDeform performs better than par with the
In Table, the agent needs to push one of the four legs of the plasticine. The initial position of the agent in PlasticineLab does not directly establish contact between the end effectors and the plasticine. The stand-alon differentiable physics solver from PlasticineLab with contact loss is inadequate for guiding the agent to find the correct "leg" to push. Whereas in CPDeform, the optimal priorities is

---

Env
Move
Tri. Move
Torus
Rope
PlasticineLab CPDeform
0.90 + 0.12
0.35 0.20
0.77 0.39 0.99 1 0.01
0.59 0.13
0.82 1 0.12
0.63 0.15
0.73 0.12
Env
RollingPin
Chopsticks
Assembly
Table
Plasticinel.ab
0.93 + 0.04
0.88
0.08
0.90 + 0.10
0.01 + 0.01
CPDeform
0.89 + 0.06
0.34 0.19
0.82 + 0.13
0.74 + 0.23
Table 2: We compare our contact point discovery backbone with human-definet contact points from PlasticineLab on single-stage tasks. We show the averaged normalized incremental IoU scores and the standard deviations of each method.
capable of identifying the "leg" of interest, resulting in suitable placement of the manipulator that significantly improves task completion. However, we recognize that CPDeform is not a panacea. We notice that CPDeform struggles Chopsticks due to the limitation of transport priorities for capturing the dramatic topological change. Specifically, we find that the transport priorities cannot discover the transport plan that preserves the continuity of the topology, as discussed in Feydy (2020). How to find a continuous correspondence to map a source shape to target remains to be an interesting future direction to explore.
4.4 COMPARISON WITH OTHER CONTACT DISCOVERY METHOD
Airplane
Bottle
Writer++ +
Rope++
+ove++
g8 0M
0.000
0.00
0.000
0.00
0.16
0.00
20099 Steps
20003
39000
20303
Steps
Steps
Steps
Steps
Ours
Surface
Figure 7: Ablation Learning Curves
In this section, perform an ablation study to verify if transport priorities help find accurate solutions efficiently, We compare CPDeform with random sampling contact points on the surface of the soft bodies, while maintaining all other settings and hyperparameters, including the number of stages and candidate pose sets. Figure 7 shows the comparison of randomly sampled contact points with CPDeform. We observe that our method in CPDeform outperforms the surface-poin sampling strategy by a large margin in terms of both accuracy and efficiency, proving the effectiveness of the cunsport-priority-aassee contact point discovery backbone employed by CPDeform.
RELATED WORK
Soft Body Manipulation. Soft body manipulation has a long history in robotics across multiple fields, such as fabric manipulation (Liang et al., 2019; Wu et al., 2020; Ha & Song, 2021), rope controlling (Yan al., 2020; Wu et al., 2020; Gan et al., 2021), and food preparation (Bollini et al., 2013; Heiden et al., 2019). In this work, we study the elastoplastic materials in PlasticineLab (Huang al., 2021) due to their wider applications than the pure elastic material model, leading more general and realistic modeling of real-world soft bodies. The high degrees of freedom of soft bodies (Hu et al., 2018; Essahbi et al., 2012) limit the application of motion planning methods (Kuffner & Valle, 2000; Kavraki et al., 1996) and most works focus on linear models like rope (Saha & Isto, 2006; 2007; Wakamatsu et al., 2006) or planar models like cloth (McConachie et al., 2020). Control methods such as Hirai & Wada (2000); Wada et al. (2001); Smolen & Patriciu (2009) bypass the expensive global planning by approximating local models and manipulating soft bodies with local controllers, which sacrifices the ability to plan for the multi-stage tasks that we study in this work.
Recent development of Deep RL (Mnih et al., 2013; Schulman et al., 2017; Haarnoja et al., 2017; Fujimoto et al., 2018) methods has enabled unified approach to learn both perception module and manipulation policy in an end-to-end way (Lin et al., 2020; Wu et al., 2020; Nair et al., 2017; Wang et al., 2019). However, policy learning suffers from exploration issue and usually require huge
8

---

amount of data additional expert demonstrations support imitation learning (Wu al., 2020; Lee et al., 2015; Matas et al., 2018; Seita et al., 2019). Another popular approach is to use neural networks to approximate soft body dynamics (Li et al., 2018; Lin et al., 2021; Hoque et al., 2021) and solve control problems with model predictive control (Rubinstein, 1999) or gradient- -hased optimization. While these methods gain the generalizability by employing shared local dynamic model, they need a way to use the model for planning. Similar to our iterative contact-and-deform approach, methods like Lin et al. (2020); Seita et al. (2019); Li et al. (2018) apply a pick-and-place action space to manipulate soft bodies like cloth and rope. While this action space enables efficient exploration and simplifies learning tasks, it poses restriction for the type of end effectors, making it unsuitable for the fine-grained manipulation of soft bodies such as plasticine. Comparatively, CPDeform leverages the differentiable physies solver to complete more sophisticated manipulation tasks.
Differentiable Physics for Trajectory Optimization. Our method uses the differentiable simulator developed in Plasticinel (Huang et al., 2021) for trajectory optimization. Encouraged by the success of gradient descent in neural network learning, differentiable physics with analytic physics models (Geilinger et al., 2020; Degrave et al., 2016; de Avila Belbute-Peree et al., 2018; Carpentier & Mansard, 2018; Giftthaler et al., 2017; Heiden et al., 2019; 2020; 2021; Toussaint et al., 2018; Hu et al., 2019; 2020; Qiao et al., 2020; Murthy et al., 2020; Millard et al., 2020; Werling et al., 2021; Du et al., 2021; Ma et al., 2021) has gained increasing popularity. As mentioned in Sec. 2, applying gradient- optimization with an inappropriate contact point would get stuck in the local minima, especially in tasks where multiple contact switches are needed. Previous research has explored various ways to handle contacts for rigid bodies. Contact-invariant optimization (Mordatch et al., 2012) imposes variables as soft representation of objects' contact relationship, and in (Posa et al., 2014; Sleiman et al., 2019), contacts are handled implicitly within analytical models. However, due to the nonlinear and contact-rich nature of manipulation tasks, we often have to combine search and optimization to solve mixed-integer program (Han & Tedrake, 2020) or logic-geometrie program (Toussaint et al., 2018) 18) problems, which are inefficient given the complexity of 3D soft bodies. We explore a complementary direction of previous approaches. By integrating visual cues into differentiable physics, are able to skip many local minima and boost the performance in various soft body manipulation tasks.
Grasping and Rigid Body Manipulation. Our contact point discovery approach shares similar spirit with grasp pose detection in rigid body manipulation. Determining contact point or grasp pose for different end effectors is one of the everlasting topics in rigid body manipulation (Bohg et al., 2014; Miller & Allen, 2004; Dang & Allen, 2012; Qin et al., 2020; Mahler et al., 2017). By analyzing 3D geometry (Sahbani et al., 2012; Hong al., 2021), one can find antipodal grasps that satisfy force closure (Nguyen, 1988; Chen & Burdick, 1993), and grasp objects without running simulation. Similarly, our contact point discovery method tries to find contact points through geometrical analysis. in an effort to extend geometric analysis for more general soft body manipulation tasks.
6 CONCLUSIONS AND FUTURE WORK
In this paper, we propose novel framework, CPDeform, that integrates optimal transport- -ased contact discovery into differentiable physics. Extensive experiments suggest that our proposed contact point discovery method, when directly employed by the differentiable solver, performs on par with or better than nuuman-define initial contact points on single-stage tasks. On multi-stage tasks that are infeasible for the vanilla solver, CPDeform employs heuristic searching approach iteratively solve the tasks. Our work demonstrates the importance of contact points in policy learning with differentiable physics and the advantage of geometric-analyste methods as heuristic.
Our framework requires a properly defined Wasserstein distance on the object's 's representation. The choice of material types does not affect our optimal transport heuristic because it relies on shape information only. We assume uniform density of the object template and moderately similar topology across initial and target shapes. Interesting avenues for future work include generalizing the discovery of useful contact points through learning methods for diverse set of shapes, and applying a similar contact point discovery principle to dexterous rigid body manipulation, or combining it with other planning approaches. We refer the readers to Appendix G for more discussions the limitation and future works.
Acknowledgement. We thank Hannah Skye Dunnigan for her help graphic design. This work was supported by MIT-IBM Watson AI Lab and its member company Nexplore, ONR MURI (N00014- 13-1-0333), DARPA Machine Common Sense program, ONR 1000001-18---200 and MERI.

---

REFERENCES
Jeannette Bohg, Antonio Morales, Tamim Asfour, and Danica Kragic. Data-driven grasp synthesis survey. IEEE Transactions on Robotics, 30(2):289--3) 2014, doi: 10.11897RR.2013:2229011
Mario Bollini, Stefanie Tellex, Tyler Thompson, Nicholas Roy, and Danicla Rus. Interpreting and executing recipes with cooking robot. In Experimental Robotics, pp. 481-495, Springer, 2013. 1,
Justin Carpentier and Nicolas Mansard. Analytical derivatives of rigid body dynamics algorithms. In Robotics: Science and systems (RSS 2018), 2018. 9
I-Ming Chen and J.W. Burdick. Finding antipodal point grasps irregularly shaped objects. IEEE Transactions on Robotics and Automation, 94):5007 512- 1993. doi: 10. 10.1109/70.246663
Marco Cuturi. Sinkhorn distances: Lightspeed computation of optimal transport. In
C. J. C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K. Q. Weinberger (eds.), Advances in Neural Information Processing Systems, volume 26. Curran Asso- ciates, Inc., 2013. URL https::/ppocceeeiiiiicce
neurips. oc httpc//ss..pppppp/////////////// /ttppppp///////le/////// af21d0c9 7tt.7///222777777777111141411110 Papez pdf.
Hao Dang and Peter K. Allen. Semantic grasping: Planning robotic grasps functionally suitable for an object manipulation task. In 2012 IEEE/RSJ International Conference on Intelligent Robots 9
and Systems, pp. 1311-1317, 2012. doi:
Filipe de Avila PaabuttePPres, Kevin Smith, Kelsey Allen, Josh Tenenbaum, and J 7ico Kolter. End-to-end differentiable physics for learning and control. In Advances in Neural Information Processing Systems, pp. 7178-7189, 2018.
Jonas Degrave, Michiel Hermans, Joni Dambre, et al. A differentiable physics engine for deep learning in robotics. arXiv preprint
2016. 9
Tao Du, Kui Wu, Pingchuan Ma, Sebastien Wah, Andrew Spielberg, Daniela Rus, and Wojeiech Ma- tusik. Diffpd: Diffcrentiable projective dynamics with contact. arXiv preprint arXiv:2101.05917, 2021. 9
Nabil Essahbi, Belhassen Chedli Bouzgarrou, and Grigore Gogu. Soft material modeling for robotic manipulation. In Applied Mechanics and Materials, volume 162, pp. 184-193. Trans Tech Publ, 2012. 8
J Feydy. Geometric data analysis, beyond convolutions. PhD thesis, PhD thesis, Université Paris- Saclay, 2020. 8
Scott Fujimoto, Herke Hoof, and David Meger. Addressing function approximation error in actor- critic methods. In International Conference on Machine Learning, pp. 1591, 2018. 6,
Chuang Gan, Jeremy Schwartz, Seth Alter, Martin Schrimpf, James Traer, Julian De Freitas, Jonas Kubilius, Abhishek Bhandwaldar, Nick Haber, Megumi Sano, et Threedworld: A platform for interactive multi-modal physical simulation. NeurIPS, 2021. 8
Moritz Geilinger, David Hahn, Jonas Zehnder, Moritz Bächer, Bernhard Thomaszewski, and Stelian Coros. Add: Analytically differentiable dynamics for multi-body systems with frictional contact. arXiv preprint arXiv:2007.00987 2020. 9
Markus Giftthaler. Michael Neunert, Markus Stäuble, Marco Frigerio, Claudio Semini, and Jonas Buchli. Automatic differentiation of rigid body dynamics for optimal control and estimation. Advanced Robotics, 1222)): 2017.
Huy Ha and Shuran Song. Flingbot: The unrcasonable effectiveness of dynamic manipulation for cloth unfolding. arXiv preprint 1rX:v:2105.03655. 2021. 8
10

---

Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, and Sergey Levine. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with stochastic actor. 2017. 6, 8
Weiqiao Han and Russ Tedrake. Local trajectory stabilization for dexterous manipulation via piece- wise affine approximations. 2020 IEEE International Conference on Robotics and Automation (ICRA), pp. 1884-8891. IEEE, 2020. 9
Eric Heiden, David Millard, Hejia Zhang, and Gaurav S Sukhatme. Interactive differentiable simulation. arXiv preprint 1905. 10706, 2019. 8,9
Eric Heiden, David Millard, Erwin Coumans, and Gaurav S Sukhatme. Augmenting differentiable simulators with neural networks to close the sim2real gap. arXiv preprint arXhv:2007.06045, 2020.
Eric Heiden, Miles Macklin, Yashraj Narang, Dieter Fox, Animesh Garg, and Fabio Ramos. Disect: A differentiable simulation engine for autonomous robotic cutting. arXiv preprint vrXiv:2105.12244 2021. 9
Shinichi Hirai and Takahiro Wada. Indirect simultaneous positioning of deformable objects with multi-pinching fingers based on uncertain model. Robotica, 18(1):3-11, 2000. 8
Yining Hong, Yi, Joshua Tenenbaum, Antonio Torralba, and Chuang Gan. Ptr: A benchmark for part-based conceptual, relational, and physical reasoning. In Advances In Neural Information Processing Systems, 2021.
Ryan Hoque, Daniel Seita, Ashwin Balakrishna, Aditya Ganapathi, Ajay Kumar Tanwani, Nawid Jamali, Katsu Yamane, Soshi Iba, and Ken Goldberg. Visuospatial foresight for physical sequential fabric manipulation. arXiv preprint wrxv:2202.00955. 2021.
Yuanming Hu, Yu Fang, Ziheng Ge, Ziyin Qu, Yixin Zhu, Andre Pradhana, and Chenfanfu Jiang. moving least squares material point method with displacement discontinuity and two-way rigid body coupling. ACM Transactions on Graphics (TOG), 37(4):1-14, 2018. 8
Yuanming Hu, Jiancheng Liu, Andrew Spielberg, Joshua B Tenenbaum, William T Freeman, Jiajun Wu, Daniela Rus, and Wojciech Matusik. Chainqueen: A real-time differentiable physical simulator for soft robotics. In 2019 International Conference Robotics and Automation (ICRA), pp. 6265- 6271. IEEE, 2019.
Yuanming Hu, Luke Anderson, Tzu-Mao Li, Qi Sun, Nathan Carr, Jonathan Ragan-Kellew and Frédo Durand. Difftaichi: Differentiable programming for physical simulation. ICLR, 2020. 9
Zhiao Huang, Yuanming Hu, Tao Du Siyuan Zhou, Hao Su, Joshua B Tenenbaum, and Chuang Gan. Plasticinelab; soft-body manipulation benchmark with diffcrentiable physics. arXiv preprint arXiv:2104.033/1 004.1011 2021. 2, 3, 7, 8,
Lydia E Kavraki, Petr Svestka, J-C Latombe, and Mark H Overmars. Probabilistic roadmaps for path planning in high-dimensional configuration spaces. IEEE transactions on Robotics and Automation, 12(():566-555 580, 1996. 8
James J Kuffner and Steven M Valal.. Rit-connect: An efficient approach to single-query path planning. In Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No. 00CH37065), volume 2, pp. 995-1001.
IEEE, 2000.
Alex X Lee, Abhishek Gupta, Henry Lu, Sergey Levine, and Pieter Abbeel. Learning from multiple demonstrations using trajectory-aware non-rigid registration with applications to deformable object manipulation. In 2015 IEEE/RSJ International Conference Intelligent Robots and Systems (IROS), pp. 5265-5272. IEEE, 2015. 9
Yunzhu Li, Jiajun Wu, Russ Tedrake, Joshua B Tenenbaum, and Antonio Torralba. Learning particle dynamics for manipulating rigid bodies, deformable objects, and fluids. arXiv preprint arXiv: 1.X.:1810.00560 2018. 9
11

---

Junbang Liang, Ming C Lin, and Vladlen Koltun. Differentiable cloth simulation for inverse problems. Advances in Neural Information Processing Systems, 2019. 8
Xingyu Lin, Yufei Wang. Jake Olkin, and David Held. Softgym: Benchmarking deep reinforcement learning for deformable object manipulation. arXiv preprint arXiv:2011.07215, 2020. 8,
Xingyu Lin, Yufei Wang, and David Held. Learning visible connectivity dynamics for cloth smoothing. arXiv preprint arXiv:2105. 10389, 2021. 9
Pingchuan Ma, Tao Du, John Z Zhang, Kui Wu, Andrew Spielberg, Robert K Katzschmann, and Wojciech Matusik. Diffaqua: differentiable computational design pipeline for soft underwater swimmers with shape interpolation. ACM Transactions on Graphics (TOG), 40(4):132, 2021. 9
Jeffrey Mahler, Jacky Liang, Sherdil Niyaz, Michael Laskey, Richard Doan, Xinyu Liu, Juan Aparicio Ojea, and Ken Goldberg. Dex-net 2.0: Deep learning plan robust grasps with synthetic point clouds and analytic grasp metrics. CoRR, abs//103.00912. 2017. URL http: /ttp://axxvvoooo////o
abs/ 170.. 09312.
Jan Matas, Stephen James, and Andrew Davison. Sim-to-real reinforcement learning for deformable object manipulation. arXiv preprint 1rX1v:1806.07857 2018.
Hermann Mayer, Faustino Gomez, Daan Wierstra, Istvan Nagy, Alois Knoll, and Jürgen Schmidhuber. A system for robotic heart surgery that learns tie knots using recurrent neural networks. Advanced Robotics, 22(13-14) 2008.
Dale McConachie, Andrew Dobson, Mengyao Ruan, and Dmitry Berenson. Manipulating deformable objects by interlcaving prediction, planning, and control. The International Journal of Robotics Research, 3957)9957-98,, 2020. 8
David Millard, Eric Heiden, Shubham Agrawal, and Gaurav S Sukhatme. Automatic differentiation and continuous sensitivity analysis of rigid body dynamics. arXiv preprint arXiv:2001.0855 2020. 9
A.T. Miller and P.K. Allen. Graspit! a versatile simulator for robotic grasping. IEEE Robotics Automation Magazine, 11(4):110-12) 2004. doi: 00.1109//R..2004.11377
Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller. Playing atari with deep reinforcement learning. arXiv preprint 1rXi5v1312.5602, 2013.
Igor Mordatch, Emanuel Todorov, and Zoran Popović. Discovery of complex behaviors through contact-invariant optimization. ACM Transactions on Graphics (TOG), 31())1188 2012.
J Krishna Murthy, Miles Macklin, Florian Golemo, Vikram Voleti, Linda Petrini, Martin Weiss, Breandan Considine, Jérôme Parent-Lévesquee, Kevin Xie, Kenny Erleben, et al. gradsim: Differ- entiable simulation for system identification and visuomotor control. In International Conference on Learning Representations, 2020.
Ashvin Nair, Dian Chen, Pulkit Agrawal, Phillip Isola, Pieter Abbeel, Jitendra Malik, and Sergey Levine. Combining self-supervised learning and imitation for vision-based rope manipulation. In 2017 IEEE international conference robotics and automation (ICRA), pp. 2114-2152 IEEE, 2017. 8
Van-Due Nguyen. Constructing force- closure grasps. The International Journal of Robotics Research, 7(3))3-16, 1988. doi: 10. 00.1177007736649800000033 URL /ttp::::///oio.o///....o/o 0278 027836498800700300
Michael Posa, Cecilia Cantu, and Russ Tedrake. A direct method for trajectory optimization of rigid bodies through contact. The International Journal of Robotics Research, 33(1):69-81, 2014.
Yi-Ling Qiao, Junbang Liang, Vladlen Koltun, and Ming C Lin. Scalable differentiable physics for learning and control. arXiv preprint arXiv:2007.022 020000..226666 2020. 9
12

---

Yuzhe Qin, Rui Chen, Hao Zhu, Meng Song, Jing Xu, and Hao Su. S4g: Amodal single-view single-shot se(3) grasp detection in cluttered scenes. In Leslie Pack Kaelbling, Danica Kragic, Robot Learning. volume 100 of
and Komei Sugiura (eds.), Proceedings of the Conference Proceedings of Machine Learning Research, pp. 53-65. PMLR, 30 Oct-01 Nov 2020. URL https:/ httts::/pproeeddii ngs ngs ml press.peesssee... / 00/ qi gin20a...... html. 9
Reuven Rubinstein. The cross-entropy method for combinatorial and continuous optimization.
Methodology and computing in applied probability, 1(2):127-190, 1999. 9
Mitul Saha and Pekka Isto. Motion planning for robotic manipulation of deformable linear objects. In Proceedings 2006 IEEE International Conference Robotics and Automation, 2006. ICRA 2006., pp. 2478-24888 2484. IEEE, 2006. 8
Mitul Saha and Pekka Isto. Manipulation planning for deformable linear objects. IEEE Transactions on Robotics, 23()) 2007. 8
A. Sahbani, S. El-Khoury, and P. Bidaud. An overview of 3d object grasp synthesis algorithms. Robotics and Autonomous Systems, 60(3):326-336, 2012. ISSN 0921-8890. doi: https://doi. org/10. 1tw.001166..mmbo22111.00000 articcle/p1/$30221888888
URL https: / www. sci enced rect. cre senn//ciencee 001405..0... Autonomous Grasping.
Jose Sanchez, Juan-Antonio Corrales, Belhassen-Chedl Bouzgarrou, and Youcef Mezouar. Robotic manipulation and sensing of deformable objects in domestic and industrial applications: a survey. The International Journal of Robotics Research, 37(7):688-716, 2018.
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint 1707.06777 2017, 8
Daniel Scita, Aditya Ganapathi, Ryan Hoque, Minho Hwang. Edward Cen, Ajay Kumar Tanwani, Ashwin Balakrishna, Brijen Thananjeyan, Jeffrey Ichnowski, Nawid Jamali, et al. Deep imitation 2, 2019. 9
learning of sequential fabric smoothing policies. arXiv preprint
Jean-Pierre Sleiman, Jan Carius, Ruben Grandia, Martin Wermelinger, and Marco Hutter. Contact- implicit trajectory optimization for dynamic object manipulation. In 2019 JEEE/RSJ International Conference Intelligent Robots and Systems (IROS), pp. 6814 6821.. IEEE, 2019. 9
Jerzy Smolen and Alexandru Patriciu. Deformation planning for robotic soft tissue manipulation. In 2009 Second International Conferences Advances in Computer-Human Interactions, pp- 199-204. IEEE, 2009. 8
Justin Solomon, Fernando de Goes, Gabriel Peyré, Marco Cuturi, Adrian Butscher, Andy Nguyen, Tao Du, and Leonidas Guibas. Convolutional wasserstein distances: Efficient optimal transportation on geometric domains. ACM Trans. Graph., 34(4), jul 2015. ISSN 0730-0301. doi: 10. 10.1145/2766963.
URL https::t//// //opp::://doi. /doi www.orog./..... 1444/15//2 2 2766663. 6
Mare A Toussaint, Kelsey Rebecca Allen, Kevin A Smith, and Joshua B Tenenbaum. Differentiable physics and stable modes for tool-use and manipulation planning. 2018.
Takahiro Wada, Shinichi Hirai, Sadao Kawamura, and Norimasa Kamiji. Robust manipulation of deformable objects by a simple pid feedback. In Proceedings 2001 ICRA. IEEE International Conference Robotics and Automation (Cat. No. 01CH37164), volume pp. 85-90. IEEE, 2001.
Hidefumi Wakamatsu, Eiji Arai, and Shinichi Hirai. Knotting/unknotting manipulation of deformable linear objects. The International Journal of Robotics Research, 25(4):371-395, 2006. 8
Angelina Wang, Thanard Kurutach, Kara Liu, Pieter Abbeel, and Aviv Tamar. Learning robotic manipulation through visual planning and acting. arXiv preprint arXivv/905.00 1905.04411,,, 2019. 8
Keenon Werling, Dalton Omens, Jeongseok Lee, Ioannis Exarchos, and C Karen Liu. Fast and feature-complete differentiable physics for articulated rigid bodies with contact. arXiv preprint 100i::2003.16021, 2021. 9
13

---

Yilin Wu, Wilson Yan, Thanard Kurutach, Letrel Pinto, and Pieter Abbeel. Learning to manipulate deformable objects without demonstrations, 2020. 1, 8.9
Mengyuan Yan, Yilin Zhu, Ning Jin, and Jeannette Bohg. Self-supervised learning of state estimation for manipulating deformable linear objects. IEEE Robotics and Automation Letters, 5(2)23372 2379, 2020. 8
14

---

A ENVIRONMENT DETAILS
Airplane The agent needs manipulate a pair of "spatulas" (represented as round boxes) to sculpt the nose, wing, and tail on the plasticine cube to match the target airplane shape.
Chair With pair of "spatulas," the agent needs to build the seat, back, and armrest from the plasticine cube to reach the target chair shape.
Bottle With pair of "spatulas." the agent needs to build the neck and body from the plasticine cube to reach the target bottle shape.
Star The agent needs to manipulate pair of "spatulas" to build the tips the star from the plasticine cube reach the target bottle shape.
Move++ The agent needs to use a single pair of sphere manipulators to separately transport three plasticine cubes fulfiil the three destinations of demand.
Rope++ The agent needs to use single pair of sphere manipulators to reshape the plasticine rope reach the target polyline shape.
Writer++ The agent manipulates "pen" (represented using vertical capsule) to print the letters
"ICLR" on the plasticine cube.
Pinch In this task, the agent manipulates one rigid sphere to create dents specified by the target shape on the plasticine box.
B MORE DETAILS ABOUT OPTIMAL TRANSPORT
Optimal transport is useful for comparing measures μu and v in a Lagrangian framework, by accounting for the cost of transporting one measure to another. Here, we consider the first measure μ as the soft-body particles in the source state and v as the soft-body particles in the target state that we wish to achieve through manipulation, where both input measures are on the space IR x3,, Let a, B E N be the mass vectors that for the particles μ., V respectively, Since all particles weighted equally in our simulator, & = 3 1. Let U(a,, 3) be the polytope that contains all transport non-negative matrices whose U(a, B) = {P& E R N₂ XN,
to a, respectively. We have P1= PT B}. Given cost function C RR3 x TR: - [0, oo). we create the cost matrix E R,,, where each element M,,- = C(u,, V2) measures the cost of transporting the particle μ4 to Vy. Now, the cost of mapping μ to be quantified (P, P)) We define the primal form the optimal transport (OT) problem as OT(o, B) := min PP.
plans, which
and columns
PeUla..)
C PSEUDOCODE OF CPDEFORM
Algorithm 1 CPDeform
Input: Current shape Sn with N particles, goal shape G, number of stages Dsaage number of steps
per stage Ratep, candidate pose set T
1: for < < Rstage do
2: Compute optimal transport priorities (asligı between S; and goal shape G
3: Find point with largest priorities ap
4: for pose / T do
5: Place manipulators around point p with largest heuristic value without collisions
6: Solve trajectories with differentiable physics generate plan p: and compute its shape
matching loss
7: end for
8: Execute the plan pt with the minimal loss
9: end for
D CONTROLLER OPTIMIZATION WITH DIFFERENTIABLE PHYSICS
Consider that the motion trajectory is parameterized as a sequence of ftree-dimensional actions (01...., aT) where 7 is the number of simulation steps. Let (stoseer be simulation states at
15

---

different time steps, which include the state of the plasticine and manipulator. The differentiable simulator starts from the initial state So and completes the action sequences by repeatedly executing the forward function S ((s, 0++1) until the ending state Sy has been reached. The objective of the optimizer is to minimize the distance between the final shape and the target shape. We represent loss function L(ST, g) where 9 is the target shape. Since the simulation forward
the objective function & is tully-iiffereniibbl, we can compute gradients / 0 of the loss L with respect to each action ar, and run gradient descent steps optimize the action sequences by a, = ar a&C / where a is the learning rate. We repeat this optimization loop for N iterations. The overall process is reflected by Algorithm 2.
Algorithm 2 Differentiable Physies Solver for Controller Optimization
Input: Target shape g. action sequence {a!...., (r), initial state sp, differentiable forward simula- tion function d, number of iterations N, learning rate
1: for <
2: for 0 $ t T do
3:
St-1 = Ø(s,, a(+1)
4: end for
5: Compute shape difference loss using [(ST.9) g)
6:
Update the action sequence by a; = at adC/ da:
7:
end for
E ADDITIONAL BASELINES
We compare CPDeform with two additional baselines, named Multiple-Restarts (Multi-Re) and Mayesian-Optimizatti (Bayes-Op)
Investigation multiple restarts For each stage, randomly sample collection of 15 contact plans. We then use the differentiable physic solver to optimize the action sequence. The execution of the solver for each plan corresponds with single restart an "initial guess." " We use the contact plan that achieves the lowest loss for manipulation.
Investigation on Bayesian optimization For each stage, use Bayesian optimization with Gaussian process to optimize the contact plan. We employ a black box function that takes a contact plan as its input and outputs the loss achieved by the differentiable physics solver with that contact plan. We then perform Bayesian optimization for 15 iterations to optimize the contact plan for the given stage.
Env
Airplane
Chair
Bottle
Star
Move++
Rope++
Writer++
Pinch
Multi-Re 0.0122
0.0098
0.0112
0.0221
0.1209
0.0055
0.0076
0.0094
Bayes-Op
0.0369
0.0103
0.0136
0.0212
0.1234
0.0067
0.0113
0.0092
PlasticineLab
0.0324
0.0218
0.0283
0.0274
0.1895
0.0075
0.0122
0.0094
CPDeform
0.0073
0.0072
0.0093
0.0198
0.0127
0.0052
0.0067
0.0081
Table 3: The averaged Wasserstein-1 distance of each method.
In general, we observe that Multiple-Restarts and Bayesian-Optimizate perform better than Plas- ticineLab, or the standalone differentiable physics solver. CPDeform outperforms these two additional baselines. Additionally, because both Multiple-Restaie trials and errors to find the suitable contact points, as illustrated in Figure 8, they are much more computationally expensive and less efficient than CPDeform.
and Sayesian-Optimizzute rely heavily
F RUNTIME
We show the runtime CPDeform, PlasticineLab, Peyesian-Optimiize
and Multiple-Resteee Restarts in Figure 8. RL approaches not drawn because they cannot complete the manipulation tasks, and their loss curves are close to the starting loss. We draw the wall-time in seconds on the x-axis and the corresponding Wasserstein-1 distance loss values the y-axis. Each data point represents the lowest
16

---

loss achieved for that method at the end of stage. Bayesian- Optimization and Multiple-Restate spend more time than the CPDeform and PlasticineLab on each stage due to the number of Bayesian optimization iterations and the number of restarts performed, respectively. We observe that CPDeform more efficient than other approaches.
Airplane
Chair
Bottle
Star
142
224
130
Ne 301
CDDeform
PlasticineLbb
Bayesian-Opiiiaation
Multiple-Restarts
Figure 8: Runtime of each method. We draw the wall-time in seconds on the x-axis and the corresponding Wasserstein- I distance loss values on the y-axis. Each data point represents the lowest loss achieved for that method at the end of stage.
G LIMITATION AND FUTURE WORKS
In this section, we discuss several implicit assumptions we made when constructing tasks. These limitations not common in tasks and could be addressed by including additional techniques; we did not take those factors into account in this work. We believe they constitute several interesting future directions explore.
Density Assumption Low or non-uniform density could potentially impact the optimization steps due varying optimal transport mapping. For instance, the optimal transport loss might be able to capture areas of the target shape whose particle distributions are extremely sparse (e.g., a few particles) and likely unable to guide the differentiable physics solver task completion. In such case, having low or non-uniform density might pose limit on our method. As solution, we have uniformly sampled particles inside the target shapes. By uniformly sampling particles inside a given 3D shape of mesh representation, we create its corresponding target template that the differentiable physics solver can use for the manipulation tasks. We think that considcrations of target shapes (e.g.. cuboid) with varying internal configurations (e.g., non-uniform density) is an interesting future direction.
Volume Assumption We assume that the volumes of the initial and target shapes similar. Since our tasks use manipulators that are tailored to compressing instead of expanding the soft materials, the initial shape cannot be significantly smaller than the target shape. Conversely, if the initial shape significantly larger than the target shape, it will require a lot of effort to compress the initial shape into the target shape. As solution, can meet this condition by cutting and assembling the initial plasticines.
Topology Assumption We assume that the initial and target shapes do have tremendous topologi- cal differences while requiring shape transformation that involves complex motion planning. When this condition false, the greedy algorithm that finds the best contact points at the current stage could be limited. For example, consider a task where the initial shape is straight rope, and the target shape is rope with knots. order to complete the task, the rope needs to be folded and wrapped, which might lead to temporary increment of shape- matchiig loss for few stages. The greedy approach is limited because might be unwilling to accept the temporary increment the loss. would potentially arrive at a solution whose general shape is matched with the target rope by omitting the detailed knots. As a possible solution, if we incorporate RL-based algorithms into our approach, the ability plan globally could allow the differentiable physics solver to overcome the temporary increment the loss at an intermediate stage and act for the best completion of the task at the final stage.
17

---

H INVESTIGATION ON THE OPTIMALITY OF CPDEFORM
We use Writer to study the optimality of our of the manipulator grid that covers the top surface of the plasticine cube. The grid is of size x0x 20, and its points are equidistant from each other on each row or column. We then record the loss achieved by executing the differentiable physics solver at each grid point (i.e., contact point). Having collected grid of loss values, we use interpolation to draw the 2D loss landscape, as illustrated in Figure 9. By exhaustively searching through each grid point, we observe that the higher transport priorities indicated by our heuristic tend to correlate with lower loss values, and vice versa.
based heuristic. Holding the height fixed value, illustrated in the leftmost figure in Figure we create 2D
Current Shape
Transport Priorities
Loss Landscape
Target Shape
Figure 9: demonstrate the visualizations of the transport priorities and the loss landscape using Writer. For the loss landscape, the color of each pixel corresponds with the loss achieved by applying the differentiable physics solver near the corresponding simulator coordinate, where the height the manipulator is set fixed value (illustrated in the current shape). Here, warmer colors indicate smaller loss values and colder colors indicate larger loss values.
18


Reviews:
[{"review_id": "zwJn5bmEq7B", "reviewer": "Reviewer_Ept9", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This method design an automatic strategy to stage long manipulation tasks and discover the contact points for each task. The use of optimal transport priority as a heuristic for contact areas is reasonable and intuitive. The same heuristic is also used in DiffAqua, the authors might want to cite and discuss their similarities.  As far as I can tell from the demo video, this method can successfully perform long manipulation tasks that I have never seen in other differentiable physics papers, which is impressive. However, the optimal transport heuristic might be only suitable for these MPM-based objects, it remains interesting how to apply this method to rigid or elastic soft materials.\n\nSome concerns that I have,\n1. What are the definitions of $\\alpha, \\beta$ in Sec. 3.1?\n2. This is actually a greedy algorithm, finding the best contact areas at the current stage and deforming the object to the target shape as close as possible. Without global planning, is it possible that the schedule is suboptimal or can not even converge？\n3. How long it will take to find an optimal motion path? \n\n", "summary_of_the_paper": "This paper proposes a method to solve the multistage manipulation tasks for soft materials. Differentiable physics has been used in controlling and manipulating soft materials since DiffTaichi. However, optimization methods based on local gradient information can be easily trapped in local minima. This paper divides a long task into small stages, within which the contact area will remain unchanged. The contact points are selected by choosing the regions with the largest transport priorities. The end effectors are then placed with poses from the candidate set. After running trajectory optimization using physics gradients, the final pose is finalized as the pose with minimum loss.", "summary_of_the_review": "This paper proposes an intuitive and effective way to utilize differentiable physics in long-horizon motion planning for soft materials. Compared to previous papers like DiffTaichi and PlasticineLab, this method can better handle multistage tasks. The demo videos show its capacity to plan very long tasks (up to 140s).", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "ym8ee3gKNSK", "reviewer": "Reviewer_rUod", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "### Strengths\n\n* **S1** This paper addresses an important gap in differentiable physics engines. Existing differentiable physics engines (Degrave et al., Qiao et al., and others) provide gradients through the physical simulation steps (i.e., evolution of state variables like velocity/acceleration/displacement) with respect to chosen initial physical parameters (forces/torques etc.). However, they provide accurate gradients for a task only if the contacts required to accomplish the task already are known a priori. To the best of my knowledge, there isn’t an approach that also differentiates through the ‘contact generation’ step, which involves reasoning over the space of all possible contacts.\n* **S2** This paper takes an optimal transport based approach, which is theoretically (physically) motivated and has advantages of simplicity.\n* **S3** The proposed approach significantly outperforms a gradient descent baseline (differentiable simulation without differentiating through contact generation) and select reinforcement learning baselines.\n* **S4** The paper is well-written and easy to follow\n\n\n### Weaknesses\n\n* **W1** This paper seems to make a few implicit assumptions that potentially have ramifications on whether this will generalize to noisy or simply incorrect object templates. I believe this requires some discussions, likely in a limitations (sub-)section. For instance, if the target trajectory is obtained from an object with similar external appearance (e.g., a cuboid) but with very different internal configuration (low or non-uniform density opposed to a uniform density structure), one would expect optimal transport mappings to vary significantly. This may in-turn impact subsequent optimization steps\n* **W2** In terms of baselines, the current manuscript compares with a gradient-based optimization method (PlasticineLab) and three RL approaches (SAC, PPO, TD3). However, as the primary aim of this paper is to design a gradient-based solution to multi-step contact-rich manipulation tasks, I believe there are other gradient-based variants that can be evaluated in this setup. One immediate approach that comes to mind is gradient descent with multiple restarts (i.e., differentiable simulation based optimization with multiple initial guesses opposed to a single guess). Another approach might also be to leverage Bayesian optimization for this task.\n* **W3** From the paper, it is unclear as to how the runtime of the approaches vary — reporting runtime overhead for each baseline would help better contextualize this paper\n\n\n### Minor remarks\n\nThese are nitpicks; I believe these are easily addressed in a minor revision. The authors need not respond to these in the rebuttal phase.\n\n* At multiple places in the manuscript, “differential” has been the word used to describe what I believe should be “differentiable” (e.g., differential simulator -> differentiable simulator)\n* Penultimate paragraph of page 3: “Such phenomena are commonly observed across soft body manipulation tasks using the differentiable physics solver.” Perhaps reference a concrete published work to corroborate this claim?\n* The best-performing approaches may be highlighted (boldface-d) in table 2\n\n", "summary_of_the_paper": "This paper presents an optimal transport based solution to differentiate through the process of contact generation in differentiable physics engines. This enables solving contact-rich multi-step manipulation tasks for deformable bodies.\n", "summary_of_the_review": "Overall, I believe this paper tackles an important and interesting problem in the differentiable physics community. While I believe the proposed approach has merit, I would like to see some of the aforementioned issues (particularly discussion of failure modes and choice of baselines) before I can recommend acceptance.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "bV3GYS9wSFH", "reviewer": "Reviewer_P7JN", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper is in general well written and the proposed method produces quite interesting results. However, I do have some questions that I hope the authors could help address:\n\n1) In the single manipulator placement algorithm: is the d() function the distance between the particle to the closest point on the manipulator, or the center of the manipulator? Also, is orientation considered during the placement optimization?\n\n2) How does the algorithm determine when to switch to the next stage during a multi-stage task? Is it time-based, state-based, or reward-based?\n\n3) For the task where the deformable object is translated/transported to a different location, does all particles receive the the same priority score? If so, how does the algorithm figure out that the best contact points? For example, if the deformable object is very flexible and the best contact point is to hold it from bottom. More generally, it’s not clear whether the current approach can take physical information into account.\n\n4) The heuristics approach of contact selection seems reasonable in general. It would be helpful if some ground-truth results could be shown to further support the optimality of the result, e.g. by comparing to an exhaustive grid search approach.\n\n5) The paper currently did not talk about how the controller optimization with differentiable physics is done. Although it’s mostly prior work, it would be nice to include some descriptions, maybe in appendix, to make the paper more self-contained.\n", "summary_of_the_paper": "The paper proposed an algorithm to discover appropriate contact points for deformable object manipulation. A key component of the proposed algorithm is to use an optimal-transport approach that computes a transport priority score for each particle in the deformable body. This score is then used to guide a grid search procedure to determine the best initial contact point for the manipulation. For multiple manipulators, a pre-defined set of poses are enumerated to find the best pose to use. The proposed algorithm is evaluated on tasks that requires a single manipulation motion or a sequence of motions to shape a deformable object into desired shapes. The result shows improved performance compared to prior methods and enables completion of novel tasks that prior methods failed.", "summary_of_the_review": "I think the paper presents interesting work that achieves non-trivial results in deformable object manipulation. Some of the details are missing from the text as discussed above, which would help better gauge the impact of the work and help reproduce the work by other researchers. Overall, I think the work makes a good contribution to the conference.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The reviewers are unanimous that this is a strong submission that deserves to be accepted.
Decision: Accept (Spotlight)