C ONTACT P OINTS D ISCOVERY FOR S OFT -B ODY M ANIPULATIONS WITH D IFFERENTIABLE P HYSICS
Anonymous authors
Paper under double-blind review
A BSTRACT
Differentiable physics has recently been shown as a powerful tool for solving soft-body manipulation tasks.
However, the differentiable physics solver often gets stuck when the initial contact points of the end effectors are sub-optimal or when performing multi-stage tasks that require contact point switching, which often leads to local minima.
To address this challenge, we propose a contact point discovery approach (CPDeform) that guides the stand-alone differentiable physics solver to deform various soft-body plasticines.
The key idea of our approach is to integrate optimal transport-based contact points discovery into the differentiable physics solver to overcome the local minima from initial contact points or contact switching.
On single-stage tasks, our method can automatically ﬁnd suitable initial contact points based on transport priorities.
On complex multi-stage tasks, we can iteratively switch the contact points of end-effectors based on transport priorities.
To evaluate the effectiveness of our method, we introduce PlasticineLab-M that extends the existing differentiable physics benchmark PlasticineLab to seven new challenging multi-stage soft-body manipulation tasks.
Extensive experimental results suggest that: 1) on multi-stage tasks that are infeasible for the vanilla differentiable physics solver, our approach discovers contact points that efﬁciently guide the solver to completion; 2) on tasks where the vanilla solver performs sub-optimally or near-optimally, our contact point discovery method performs better than or on par with the manipulation performance obtained with handcrafted contact points.
Demos are available at our project page 1 .
I NTRODUCTION
Soft body manipulation has a wide application in cooking (Bollini et al., 2013), fabric manipulation (Wu et al., 2020), healthcare (Mayer et al., 2008) and manufacturing of deformable objects (Sanchez et al., 2018).
Differentiable physics has recently been shown as a powerful and effective tool for solving control problems for soft-body manipulation tasks.
As demonstrated in Huang et al.
(2021), given a parameterized manipulation policy, the differentiable physics solver computes the gradients of the policy parameters, enabling gradient-based optimization much more efﬁciently than reinforcement learning algorithms at ﬁnding optimal solutions for soft-body manipulation tasks on a diverse collection of environments.
However, the performance of the stand-alone gradient-based solver can be heavily inﬂuenced by the policy initialization.
Especially, the end effectors’ initial contact points with objects play critical roles in the optimization.
Different contact points may lead to vast differences in manipulation performance due to local optima.
Besides, some tasks require agents to switch contact points during the manipulation, where the local optima issue becomes a serious bottleneck for completing these multi-stage tasks.
For example, as shown in Figure 2, an agent needs to control the capsule “pen” to sculpt two scribbles on the surface of a yellow plasticine cube.
In order to complete the second line, the agent needs to switch contact points after drawing the ﬁrst one.
While the stand-alone differentiable physics solver could possibly draw the ﬁrst line, it often gets stuck and struggles to draw the second one, due to the lack of gradients that push the pen to a new contact point to begin the second line.
How to automatically ﬁnd proper contact points for soft body manipulation tasks remains a challenge in differentiable physics.
[Figure]
In this paper, we propose a principled framework that integrates an optimal transport-based contact discovery method into differentiable physics (CPDeform) to address this important challenge.
CPDeform heuristically ﬁnd contact points for end effectors by using transport priorities computed from optimal transport to compare the current shape with the target shape, where soft-body manipulation is treated as a particle transportation problem.
After ﬁnding contact points, CPDeform can combine the differentiable physics solver to solve soft body manipulation tasks.
On single-stage tasks that do not require contact point switching, CPDeform can ﬁnd suitable initial contact points to ﬁnish the task.
On multi-stage tasks, using an example shown in Figure 1 (right) where the goal is to reshape a plasticine cube into an airplane, CPDeform can iteratively switch the contact points of end effectors based on transport priorities.
This iterative deformation process is motivated by how humans manipulate plasticine.
As shown in Figure 1 (left), when humans manipulate a plasticine dough, they tend to repeatedly focus on the point of interest and modify it towards the target shape.
CPDeform can mimic this process by iteratively switching contact points of interests based on transport priorities and deforming the soft bodies into the target shape with the help of the differentiable solver.
By integrating contact points discovery into the differentiable physics solver, CPDeform can skip over the local minima caused by contact switching and improve the performance of the stand-alone solver.
To evaluate the effectiveness of CPDeform, we introduce PlasticineLab-M that extends the existing differentiable physics benchmark PlasticineLab (Huang et al., 2021) to seven new challenging multistage soft body tasks.
Extensive experimental results suggest that: on single-stage tasks where the vanilla differentiable physics solver performs sub-optimally or near-optimally in PlasticineLab, we ﬁnd that the backbone of CPDeform, a contact point discovery method based on optimal transport, single-handedly performs better than or on par with the manipulation performance obtained with random-selected or human-deﬁned contact points.
On multi-stage tasks that are infeasible for the vanilla gradient-based solver, we ﬁnd that CPDeform performs reasonably well in practice and the iterative deformation method equipped with contact point discovery could serve as an alternative to the expensive long-horizon searching algorithm.
In summary, our work makes the following contributions:
• We perform an in-depth study of local optimum issues of differentiable physics solver for initial contact points and switching contact points.
• We propose a principled framework, CPDeform, that integrates optimal transport-based contact discovery into differentiable physics.
• We ﬁnd that the backbone of CPDeform, the contact point discovery method, can be directly employed by the stand-alone solver to ﬁnd better initial contact points for single-stage tasks.
• On multi-stage tasks, which are infeasible for the vanilla solver, CPDeform employs a heuristic searching approach to iteratively complete the tasks.
M OTIVATION
In this section, we provide an intuitive analysis of the drawback of the differentiable physics solver through motivating toy examples.
We start with a brief review of how the differentiable physics solver could be employed to optimize manipulation policies. We then demonstrate how initial contact
[Figure]
points affect the optimization performance.
Finally, we take a simple but representative multi-stage task as an example and discuss why contact switching would often lead to local minima.
We study a Writer task as shown in Figure 2(a).
In this task, an agent needs to manipulate the capsule “pen” to initiate contact with the yellow plasticine cube, and sculpt a line scribble on the plasticine surface.
The agent can move the tip of the pen along three dimensions.
To solve this task with differentiable physics, we manually initialize the end-effector “pen” near the suitable contact point that allows the "pen" to initiate contact with the plasticine.
We then parameterize the desired motion trajectory of the pen as a sequence of three-dimensional actions { a 1 , . .
. , a T } where T is the number of simulation steps.
Let { s t } 0 ≤ t ≤ T be simulation states at different time steps which include the state of the plasticine and manipulator.
The differentiable simulator starts from the initial state s 0 and completes the action sequences by repeatedly executing the forward function s t {++
+}
= φ ( s t , a t +1 ) until the ending state s T has been reached.
The objective of this optimizer is to minimize the distance between the current shape and the target shape.
We represent the objective as a loss function
L ( s T , g ) where g is the target shape.
Since the simulation forward function φ is fully-differentiable, we can compute gradients ∂
L /∂a t of the loss L with respect to action a t , and run multiple gradient descent steps to optimize the action sequences by a t = a t − α∂ L /∂a t where α is the learning rate.
As shown in Figure 2(b), we can see that the agent succeeds at sculpting the target scribble by moving the “pen” downwards.
We refer the readers to Algorithm 2 in Appendix D for more details on differentiable physics for controller optimization.
However, if we do not initialize the positions of the end effectors well, the solver would get stuck in the local minima.
For example, in the task as shown in Figure 3, we illustrate the optimization outcomes for end effectors with different contact points by showing their corresponding resulting shapes.
Even with an arbitrarily large number of steps T given, the gradient-based solver is unable to discover a policy that moves away from the local optimum to the a new contact point that allows for task completion.
Such phenomena are commonly observed across soft body manipulation tasks using the differentiable physics solver (Huang et al., 2021).
When end effectors are far away from the region of interest, it is unlikely that the gradient could push the end-effectors towards the desired region.
This observation poses the question of how to efﬁciently ﬁnd optimal contact points to place the end effectors.
The local minima problem caused by inappropriate contact points becomes a more serious issue in multi-stage tasks.
Taking the multi-stage writer task illustrated in Figure 2(c) as an example, the agent now needs to write an additional straight line on the plasticine surface by switching its contact point.
Even with a well-initialized contact point for the ﬁrst line, the solver is unable to relocate to the new region of interest for the upcoming line.
We observe that, differing from the vanilla differentiable physics solver, humans tend to employ an explicit “iterative deformation” schema to complete such a task.
Humans would decompose this task into two stages.
In each stage, we tend to iteratively derive the correspondence between the current shape and target shape to arrive at useful contact points from observations and then subsequently move the “pen” and write the lines.
This motivates us to combine contact point discovery with iterative deformation.
M ETHOD
In this section, we introduce CPDeform, a principled framework that integrates optimal transportbased contact discovery into differentiable physics for solving challenging soft-body manipulation tasks.
We ﬁrst describe our contact point discovery method in relation to the transport priorities found by optimal transport in Section 3.1.
In Section 3.2, we describe how transport priorities can be used to place end-effectors.
Finally, in Section 3.3, we show how CPDeform integrates contact points discovery with differentiable physics and iteratively deforms the soft bodies for multi-stage tasks.
3. O PTIMAL T RANSPORT AND CONTACT POINT D ISCOVERY
One way to consider soft-body manipulation is by treating it as a particle transportation problem.
By evaluating the cost of transporting the current state particles µ to the target state particles ν , optimal transport provides a useful framework for comparing differences between any given pair of shapes, which can guide us to discover contact points.
Let all particles be weighted equally in our simulator.
Given a cost matrix M , optimal transport ﬁnds a transportation plan P in transportation polytope U by minimizing the transportation cost min P ∈ U (cid:104) P, M (cid:105) .
Casting the problem into dual form, we have OT ( α, β ) :=
E µ [ f ]+ E ν [ g ] such that ∀
i, j , Lagrange multipliers f i , g j satisfy f i + g j ≤ M ij , where α, β are the mass vectors that for the particles in µ, ν respectively.
We refer the reader to Appendix B for more details on optimal transport.
We focus on the Lagrange multipliers f of the source particles, which we refer to as the dual potentials.
Since it represents the supports of the source measure, we interpret f as the transport priorities for the source particles µ .
Transport priorities are helpful for selecting contact points.
Given a pair of current and target softbody shapes, we intuitively would place the end-effectors around the region of the largest difference between the two shapes, in order to substantially modify the shapes.
This observation leads us to place the end-effectors at contact points whose corresponding optimal manipulation policies can minimize the difference between the current and target shapes.
However, given the computation budget, it is not easy to directly evaluate the optimality of the contact point by enumerating through a set of contact points.
Thus, we propose to use optimal transport priorities to heuristically identify contact points, based on a simple rule of selecting contact points with high transport priorities.
We observe that contact points with high transport priorities mostly correspond with superior optimization performances.
3. E ND E FFECTOR P LACEMENT W ITH T RANSPORT P RIORITIES
In this section, we describe how we place the manipulator at advantageous locations based on optimal transport priorities.
We ﬁrst describe the single manipulator case, and it can be extended to multiple manipulator environments by adding a heuristic.
Direct Single Manipulator Placement.
The transport priorities can be employed to directly place the single manipulator.
We identify the source particle µ ∗ corresponding to the largest potential f ∗ .
We then ﬁnd a suitable contact point around µ ∗ through grid search, where we create a 3D grid of size I × I ×
I centered at µ ∗ with each dimension evenly spaced into N i intervals.
The criterion of the grid search is as follows: We reject points whose placements of the manipulator lead to collisions with the particles.
And we iterate over each grid point x ∈ R 3 , and place the manipulator at the point with the maximal score according to the following criterion: score ( x )
= 1 N p (cid:80)
N p
i =
f i d ( µ i ,x ) 2 +1 where d ( µ i , x ) computes the signed distance between the particle µ i to the closest point on the manipulator placed at grid point x .
As shown in Figure 4 (top), the criterion leads us to discover contact points that allow the manipulator to cover high potential particles while reducing subsequent changes to the low potential particles.
Multiple Manipulator Placement With Heuristic.
For multiple manipulator environments, we need to consider a heuristically deﬁned candidate pose set T , where each pose corresponds with the desired manipulation strategy.
Taking Figure 4 (bottom) as an example, it is more advantageous to
[Figure]
Placement
Target Shape
S i ng l e
M an i pu l a t o r
M u l t i p l e
M an i pu l a t o r s use the highlighted pose than the ones drawn with low opacity. Each pose can be speciﬁed by the rotations of the manipulators as well as the direction vectors used to place the other manipulators with respect to the ﬁrst manipulator. For each pose in set T , we employ the single-manipulator placement strategy for the ﬁrst manipulator.
Using the ﬁrst placement position as a starting point, for the remaining manipulators, we search along their pose-speciﬁc directions to ﬁnd placement positions with minimal distances from the ﬁrst manipulator that do not result in collisions with soft bodies.
Since the differentiable physics solver is able to effectively adjust manipulator poses during optimization, in practice, the candidate pose set does not need to be exhaustively large.
In practice, we use three poses to cover left-right, top-bottom, and front-back grasping poses.
3. I TERATIVE D EFORMATION FOR M ULTI - STAGE S OFT B ODY M ANIPULATION
Continuing with the focus on multi-stage soft-body manipulation tasks, we now describe how
CPDeform combines the contact point discovery method with the use of differentiable physics for these tasks.
For every stage, we iteratively specify the contact points and perform stage-level manipulation with the differentiable physics solver.
For the multiple manipulator case, we search over contact plans corresponding to different poses, and choose the plan that achieves the lowest loss.
We present the detailed Algorithm 1 in Appendix C.
We start with the initial shape S 0 .
Our goal is to reach the target shape G within a given number of stage n stage .
Each stage contains n step simulation steps.
As described in Section 3.2, we consider a heuristically deﬁned candidate pose set T .
In each stage, we ﬁrst compute the optimal transport between the current shape S i and the target shape G to derive the transport priorities.
Next, we select the particle p with the largest transport priority.
For each pose, we search for the placement in relation to p that maximizes the aforementioned criterion.
E XPERIMENTS
We conduct multiple experiments to test the efﬁcacy of CPDeform on soft-body manipulation tasks to address the following questions:
• On multi-stage tasks that involve multiple contact switches, can CPDeform complete these tasks by iteratively manipulating the soft-bodies?
• How robust is the backbone of CPDeform, the contact point discovery method, when tested on single-stage tasks where it is only allowed to specify a single contact point in one shot?
• How is transport priority compared with other contact discovery methods?
[Figure]
4. D ATASETS
To extensively evaluate our method, we introduce PlasticineLab-M, a dataset that extends the existing differentiable physics benchmark PlasticineLab with seven new challenging multi-stage soft-body manipulation tasks, and contains the multi-stage environment Pinch in PlasticineLab.
We show these eight multi-stage tasks in Figure 5.
We refer the reader to Appendix A for detailed task descriptions.
We also use the remaining eight tasks in PlasticineLab to evaluate our contact point discovery method for single stage tasks.
For multi-stage environments, we use the Wasserstein-1 distance approximated by Sinkhorn iteration (Cuturi, 2013) between the source and target particles to quantify the ﬁne-grained difference between a state and the goal for all environments.
As for single-stage environments from PlasticineLab, we evaluate our approach using the IoU metric for a fair comparison with PlasticineLab.
4. E VALUATION OF CPD EFORM ON M ULTI -S TAGE T ASKS
We compare our approach with the following baselines:
PlasticineLab : The vanilla gradient-based solver does not come with any contact point discovery features.
It corresponds with the stand-alone differentiable physics solver described in Huang et al.
Reinforcement Learning (RL) :
We evaluate the performance of the existing RL algorithms on our tasks.
We use three model-free reinforcement learning algorithms: Soft Actor-Critic (SAC) (Haarnoja et al., 2017), Policy Proximal
Optimization (PPO) (Schulman et al., 2017), and TD3 (Fujimoto et al., 2018).
For each stage, we optimize for 200 episodes for differentiable physics-based approaches with a learning rate 0 .
For each environment, we modestly choose a horizon of 10 or 20 .
We restrict the number of environment steps used for optimization under 1 million.
We train each RL algorithm on each environment for 1000 episodes, with 1000 environment steps per episode, which accounts for the 1 million environment-step limit.
Our reward function is of the form R = −R shape − R grasp , where R shape is the Wasserstein-1 distance between the source and target particles for measuring shape differences.
And R grasp encourages the manipulators to be closer to the soft bodies.
For fair comparison with Pinch from PlasticineLab, we use the reward formulation described in Huang et al.
We show the quantitative results in Table 3 and the qualitative results in Figure 6.
We ﬁnd that our approach is capable of ﬁnishing these complex tasks, and signiﬁcantly outperforms the baselines.
We ﬁnd that with the discovered contact points, our approach is able to iteratively build and reﬁne the nose, tail, and wings of the Airplane .
In Chair , we ﬁnd that our approach guides the solver to ﬁrst create the general seat, then reﬁne the arm rest and back of the chair.
In Bottle , our approach ﬁrst pushes down the top of the plasticine cube to create the neck, before reﬁning the sides of the bottle.
For Move++ , our approach is able to complete the transportation tasks of the three cubes by selecting the most advantageous object to transfer at each stage.
In Rope++ , our approach ﬁrst moves the rope to form the general shape, before reﬁning the ends of the rope.
In Writer++ , our approach is capable
[Table]
A i r p l ane
C ha i r
W r i t e r ++
[Figure]
to iteratively guide the solver to print the "ICLR" letters on the plasticine cube.
Comparatively, we ﬁnd that the stand-alone differentiable physics solver fails completely on the multi-stage tasks, as it lacks the necessary exploration power to overcome the local minima.
Take Move++ as an example, the vanilla solver is unable to move away from the cube it ﬁrst initiates contact with after a stage.
Similar to PlasticineLab (Huang et al., 2021), we also observe that the RL approaches in general perform worse than the vanilla gradient-based method.
4. E VALUATION OF C ONTACT P OINT D ISCOVERY ON S INGLE S TAGE T ASKS
To further demonstrate the effectiveness of our approach, we compare the one-shot contact points discovered by the backbone of CPDeform with the human-deﬁned contact points on single-stage tasks from PlasticineLab.
Table 2 lists the normalized incremental IoU scores, together with the standard deviations of all approaches.
[Table]
From Table 2 we can see that on most tasks, CPDeform performs better than or on par with the manipulation performance obtained from the human-deﬁned initial end-effector positions from
PlasticineLab.
In Table , the agent needs to push one of the four legs of the plasticine.
The initial position of the agent in PlasticineLab does not directly establish contact between the end effectors and the plasticine.
The stand-alone differentiable physics solver from PlasticineLab with contact loss is inadequate for guiding the agent to ﬁnd the correct “leg“ to push.
Whereas in CPDeform, the optimal priorities is capable of identifying the “leg” of interest, resulting in a suitable placement of the manipulator that signiﬁcantly improves task completion.
However, we recognize that CPDeform is not a panacea.
We notice that CPDeform struggles on Chopsticks due to the limitation of transport priorities for capturing the dramatic topological change.
Speciﬁcally, we ﬁnd that the transport priorities cannot discover the transport plan that preserves the continuity of the topology, as discussed in Feydy (2020).
How to ﬁnd a continuous correspondence to map a source shape to target remains to be an interesting future direction to explore.
4. C OMPARISON WITH OTHER CONTACT DISCOVERY METHOD
[Figure]
In this section, we perform an ablation study to verify if transport priorities help ﬁnd accurate solutions efﬁciently.
We compare CPDeform with random sampling contact points on the surface of the soft bodies, while maintaining all other settings and hyperparameters, including the number of stages and candidate pose sets.
Figure 7 shows the comparison of randomly sampled contact points with CPDeform.
We observe that our method in CPDeform outperforms the surface-point sampling strategy by a large margin in terms of both accuracy and efﬁciency, proving the effectiveness of the transport-priority-based contact point discovery backbone employed by CPDeform.
R ELATED W ORK
Soft Body Manipulation Soft body manipulation has a long history in robotics across multiple ﬁelds, such as fabric manipulation (Liang et al., 2019; Wu et al., 2020; Ha & Song, 2021), rope controlling (Yan et al., 2020; Wu et al., 2020), and food preparation (Bollini et al., 2013; Heiden et al., 2019).
In this work, we study the elastoplastic materials in PlasticineLab (Huang et al., 2021) due to their wider applications than the pure elastic material model, leading to more general and realistic modeling of real-world soft bodies.
The high degrees of freedom of soft bodies (Hu et al., 2018;
Essahbi et al., 2012) limit the application of motion planning methods (Kuffner & LaValle, 2000;
Kavraki et al., 1996) and most works focus on linear models like rope (Saha & Isto, 2006; 2007;
Wakamatsu et al., 2006) or planar models like cloth (McConachie et al., 2020).
Control methods such as Hirai & Wada (2000); Wada et al.
; Smolen & Patriciu (2009) bypass the expensive global planning by approximating local models and manipulating soft bodies with local controllers, which sacriﬁces the ability to plan for the multi-stage tasks that we study in this work.
Recent development of Deep RL (Mnih et al., 2013; Schulman et al., 2017; Haarnoja et al., 2017;
Fujimoto et al., 2018) methods has enabled a uniﬁed approach to learn both perception module and manipulation policy in an end-to-end way (Lin et al., 2020; Wu et al., 2020; Nair et al., 2017; Wang et al., 2019).
However, policy learning suffers from exploration issue and usually require a huge amount of data or additional expert demonstrations to support imitation learning (Wu et al., 2020; Lee et al., 2015; Matas et al., 2018; Seita et al., 2019).
Another popular approach is to use neural networks to approximate soft body dynamics (Li et al., 2018; Lin et al., 2021; Hoque et al., 2021) and solve control problems with model predictive control (Rubinstein, 1999) or gradient-based optimization.
While these methods gain the generalizability by employing a shared local dynamic model, they need a way to use the model for planning.
Similar to our iterative contact-and-deform approach, methods like Lin et al.
; Seita et al.
; Li et al.
(2018) apply a pick-and-place action space to manipulate soft bodies like cloth and rope.
While this action space enables efﬁcient exploration and simpliﬁes learning tasks, it poses a restriction for the type of end effectors, making it unsuitable for the ﬁne-grained manipulation of soft bodies such as plasticine.
Comparatively, CPDeform leverages the differentiable physics solver to complete more sophisticated manipulation tasks.
Differentiable Physics for Trajectory Optimization Our method uses the differentiable simulator developed in PlasticineLab (Huang et al., 2021) for trajectory optimization.
Encouraged by the success of gradient descent in neural network learning, differentiable physics with analytic physics models (Geilinger et al., 2020; Degrave et al., 2016; de Avila Belbute-Peres et al., 2018;
Carpentier &
Mansard, 2018; Giftthaler et al., 2017; Heiden et al., 2019; 2020; 2021; Toussaint et al., 2018;
Hu et al., 2019; 2020; Qiao et al., 2020; Murthy et al., 2020; Millard et al., 2020; Werling et al., 2021; Du et al., 2021) has gained increasing popularity.
As mentioned in Sec. 2, applying gradient-optimization with an inappropriate contact point would get stuck in the local minima, especially in tasks where multiple contact switches are needed.
Previous research has explored various ways to handle contacts for rigid bodies.
Contact-invariant optimization (Mordatch et al., 2012) imposes variables as soft representation of objects’ contact relationship, and in (Posa et al., 2014; Sleiman et al., 2019), contacts are handled implicitly within analytical models.
However, due to the nonlinear and contact-rich nature of manipulation tasks, we often have to combine search and optimization to solve mixed-integer program (Han & Tedrake, 2020) or logic-geometric program (Toussaint et al., 2018) problems, which are inefﬁcient given the complexity of 3D soft bodies.
We explore a complementary direction of previous approaches.
By integrating visual cues into differentiable physics, we are able to skip many local minima and boost the performance in various soft body manipulation tasks.
Grasping and Rigid Body Manipulation
Our contact point discovery approach shares a similar spirit with grasp pose detection in rigid body manipulation.
Determining contact point or grasp pose for different end effectors is one of the everlasting topics in rigid body manipulation (Bohg et al., 2014; Miller & Allen, 2004; Dang & Allen, 2012;
Qin et al., 2020; Mahler et al., 2017).
By analyzing 3D geometry (Sahbani et al., 2012), one can ﬁnd antipodal grasps that satisfy force closure (Nguyen, 1988; Chen & Burdick, 1993), and grasp objects without running simulation.
Similarly, our contact point discovery method tries to ﬁnd contact points through geometrical analysis, which could be viewed as a step towards extending geometric analysis for more general soft body manipulation tasks.
C ONCLUSION AND F UTURE W ORK
In this paper, we propose a novel framework, CPDeform, that integrates optimal transport-based contact discovery into differentiable physics.
Our framework requires a properly deﬁned Wasserstein distance on the object’s representation.
The choice of material types does not affect our optimal transport heuristic because it relies on shape information only.
We ﬁnd that the contact point discovery method, can be directly employed by the differentiable physics solver and performs on par with or better than human-deﬁned initial contact points in single-stage tasks.
On multi-stage tasks that are infeasible for the vanilla solver, CPDeform employs a heuristic searching approach to iteratively solve the tasks.
Our work demonstrates the importance of contact point in policy learning with differentiable physics and the advantage of geometric-analysis methods as a heuristic.
Interesting avenues for future work include generalizing the discovery of useful contact points through learning methods for a diverse set of shapes, and applying a similar contact point discovery principle to dexterous rigid body manipulation, or combining it with other planning approaches.
A E NVIRONMENT DETAILS
• Airplane The agent needs to manipulate a pair of "spatulas" (represented as round boxes) to sculpt the nose, wing, and tail on the plasticine cube to match the target airplane shape.
• Chair With a pair of "spatulas", the agent needs to build the seat, back, and arm rest from the plasticine cube to reach the target chair shape.
• Bottle With a pair of "spatulas", the agent needs to build the neck and body from the plasticine cube to reach the target bottle shape.
Star The agent needs manipulate a pair of "spatulas" to build the tips of the star from the plasticine cube to reach the target bottle shape.
Move++ The agent needs to use a single pair of sphere manipulators to separately transport three plasticine cubes to fulﬁll the three destinations of demand.
Rope++ The agent needs to use a single pair of sphere manipulators to reshape the plasticine rope to reach the target polyline shape.
• Writer++ The agent manipulates a "pen" (represented using a vertical capsule), to print the letters "ICLR" on the plasticine cube.
• Pinch
In this task, the agent manipulates one rigid sphere to create dents speciﬁed by the target shape on the plasticine box.
B M ORE DETAILS ABOUT O PTIMAL T RANSPORT
Optimal transport is useful for comparing measures µ and ν in a Lagrangian framework, by accounting for the cost of transporting one measure to another.
Here, we consider the ﬁrst measure µ as the soft-body particles in the source state and ν as the soft-body particles in the target state that we wish to achieve through manipulation, where both input measures are on the space R N p × 3 .
Let α, β ∈ R N p be the mass vectors that for the particles in µ, ν respectively.
Since all particles are weighted equally in our simulator, α = β = 1 .
Let U ( α, β ) be the polytope that contains all transport plans, which are non-negative matrices whose rows and columns sum to α, β respectively.
We have
U ( α, β ) := { P ∈ R N p × N p + | P 1 = α, P T 1 = β } .
Given a cost function C : R 3 × R 3 (cid:55)→ [0 , ∞ ] , we create the cost matrix M ∈ R N p × N p , where each element M i,j = C ( µ i , ν j ) measures the cost of transporting the particle µ i to ν j .
Now, the cost of mapping µ to ν can be quantiﬁed as (cid:104) P, M (cid:105) .
We deﬁne the primal form of the optimal transport (OT) problem as OT ( α, β ) :
= min P ∈ U ( α,β ) (cid:104) P, M (cid:105) .
C P SEUDOCODE OF CPD EFORM
D C ONTROLLER OPTIMIZATION WITH DIFFERENTIABLE PHYSICS
Consider that the motion trajectory is parameterized as a sequence of three-dimensional actions { a 1 , . .
. , a T } , where T is the number of simulation steps.
Let { s t } 0 ≤ t ≤ T be simulation states at different time steps which include the state of the plasticine and manipulator.
{+The differentiable simulator starts from the initial state s 0 and completes the action sequences by repeatedly executing the forward function s t +
+}
= φ ( s t , a t +1 ) until the ending state s T has been reached.
The objective of the optimizer is to minimize the distance between the ﬁnal shape and the target shape.
We represent the objective as a loss function
L ( s T , g ) where g is the target shape.
Since the simulation forward function φ is fully-differentiable, we can compute gradients ∂
L /∂a t of the loss L with respect to each action a t , and run gradient descent steps to optimize the action sequences by a t = a t − α∂ L /∂a t where α is the learning rate.
We repeat this optimization loop for N iterations.
The overall process is reﬂected by Algorithm 2.
Input: Target shape g , action sequence { a 1 , . .
. , a T } , initial state s 0 , differentiable forward simulation function φ , number of iterations N , learning rate α 1: for 0 ≤
{+i < N do 2: for 0 ≤ j < T do 3: s t +
+}
= φ ( s t , a t +1 ) 4: end for 5:
Compute shape difference loss using L ( s T , g ) 6:
Update the action sequence by a t = a t − α∂ L /∂a t 7: end for
E A DDITIONAL B ASELINES
We compare CPDeform with two additional baselines, named Multiple-Restarts (Multi-Re) and
Bayesian-Optimization (Bayes-Op).
Investigation on multiple restarts For each stage, we randomly sample a collection of 15 contact plans.
We then use differentiable physic solver to optimize the action sequence.
The execution of the solver for each plan corresponds with a single restart or an "initial guess.
" We use the contact plan that achieves the lowest loss for manipulation.
Investigation on Bayesian optimization For each stage, we use Bayesian optimization with Gaussian process to optimize the contact plan.
We employ a black box function that takes a contact plan as its input and outputs the loss achieved by the differentiable physics solver.
We then perform Bayesian optimization for 15 iterations to optimize the contact plan for the given stage.
[Table]
In general, we observe that Multiple-Restarts and Bayesian-Optimization perform better than PlasticineLab, or the standalone differentiable physics solver.
CPDeform outperforms these two additinoal baselines.
Additionally, because both Multiple-Restarts and Bayesian-Optimization rely heavily on trials and errors to ﬁnd the suitable contact points, as illustrated in Figure 8, they are much more computationally expensive and less efﬁcient than CPDeform.
F R UNTIME
We show the runtime of CPDeform, PlasticineLab, Bayesian-Optimization, and Multiple-Restarts in
Figure 8.
RL approaches are not drawn because they cannot complete the manipulation tasks, and their loss curves are close to the starting loss.
We draw the wall-time in seconds on the x-axis and the corresponding Wasserstein-1 distance loss values on the y-axis.
Each data point represents the lowest loss achieved for that method at the end of a stage.
Bayesian-Optimization and Multiple-Restarts spends more time than the CPDeform and PlasticineLab on each stage, due to the number of Bayesian optimization iterations and the number of restarts performed respectively.
We observe that CPDeform is more efﬁcient than other approaches.
G L IMITATION AND F UTURE W ORKS
In this section, we discuss several implicit assumptions we made when constructing tasks.
These limitations are not common in our tasks and could be addressed by including additional techniques; we did not take those factors into account in this work.
We believe they constitute several interesting future directions to explore.
Density Assumption Low or non-uniform density could potentially impact the optimization steps due to varying optimal transport mapping.
For instance, the optimal transport loss might not be able to capture areas of the target shape whose particle distributions are extremely sparse (e.g., a few particles) and is likely unable to guide the differentiable physics solver to task completion.
In such a case, having low or non-uniform density might pose a limit on our method.
As a solution, we have uniformly sampled particles inside the target shapes.
By uniformly sampling particles inside a given 3D shape of mesh representation, we create its corresponding target template that the differentiable physics solver can use for the manipulation tasks.
We think that considerations of target shapes (e.g., a cuboid) with varying internal conﬁgurations (e.g., non-uniform density) is an interesting future direction.
Volume Assumption We assume that the volumes of the initial and target shapes are similar.
Since our tasks use manipulators that are tailored to compressing instead of expanding the soft materials, the initial shape cannot be signiﬁcantly smaller than the target shape.
Conversely, if the initial shape is signiﬁcantly larger than the target shape, it will require a lot of effort to compress the initial shape into the target shape.
As a solution, we can meet this condition by cutting and assembling the initial plasticines.
Topology Assumption We assume that the initial and target shapes do not have tremendous topological differences while requiring shape transformation that involves complex motion planning.
When this condition is false, the greedy algorithm that ﬁnds the best contact points at the current stage could be limited.
For example, consider a task where the initial shape is a straight rope, and the target shape is a rope with knots.
In order to complete the task, the rope needs to be folded and wrapped, which might lead to a temporary increment of shape-matching loss for a few stages.
The greedy approach is limited because it might be unwilling to accept the temporary increment of the loss.
It would potentially arrive at a solution whose general shape is matched with the target rope by omitting the detailed knots.
As a possible solution, if we incorporate RL-based algorithms into our approach, the ability to plan globally could allow the differentiable physics solver to overcome the temporary increment of the loss at an intermediate stage and act for the best completion of the task at the ﬁnal stage.
H I NVESTIGATION ON THE OPTIMALITY OF CPD EFORM
We use Writer to study the optimality of our optimal-transport-based heuristic.
Holding the height of the manipulator at a ﬁxed value, as illustrated in the leftmost ﬁgure in Figure 9, we create a 2D grid that covers the top surface of the plasticine cube.
The grid is of size 20 × 20 and its points are equidistant from each other on each row or column.
We then record the loss achieved by executing the differentiable physics solver at each grid point (i.e., contact point).
Having collected a grid of loss values, we use interpolation to draw the 2D loss landscape, as illustrated in Figure 9.
By exhaustively searching through each grid point, we observe that the higher transport priorities indicated by our heuristic tend to correlate with lower loss values, and vice versa.