FabricFlowNet: Bimanual Cloth Manipulation with a Flow-based Policy
Anonymous Author(s)
Abstract: We address the problem of goal-directed cloth manipulation, a challenging task due to the deformability of cloth.
Our insight is that optical ﬂow, atechnique normally used for motion estimation in video, can also provide an effective representation for corresponding cloth poses across observation and goalimages.
We introduce FabricFlowNet (FFN), a cloth manipulation policy thatleverages ﬂow as both an input and as an action representation to improve performance.
FabricFlowNet also elegantly switches between dual-arm and singlearm actions based on the desired goal.
We show that FabricFlowNet signiﬁcantlyoutperforms state-of-the-art model-free and model-based cloth manipulation policies.
We also present real-world experiments on a bimanual system, demonstrating effective sim-to-real transfer.
Finally, we show that our method generalizeswhen trained on a single square cloth to other cloth shapes, such as T-shirts andrectangular cloths.
Video and other supplementary materials are available at:https://sites.google.com/view/fabricﬂownet.
Introduction
Cloth manipulation has a wide range of applications in domestic and industrial settings.
However,it has posed a challenge for robot manipulation: compared to rigid objects, fabrics have a higherdimensional conﬁguration space, can be partially observable due to self-occlusions in crumpledconﬁgurations and do not transform rigidly when manipulated.
Early approaches for cloth manipulation relied on scripted actions; these policies are typically slow and do not generalize to arbitrarycloth goal conﬁgurations [27, 13, 3].
Recently, learning-based approaches have been explored for cloth manipulation [18, 33, 42, 29, 32] including model-free reinforcement learning to obtain a policy [21, 39].
For a cloth manipulationpolicy to be general to many different objectives, it must receive a representation of the currentfolding objective.
A standard approach for representing a goal-conditioned policy is to input animage of the current cloth conﬁguration together with an image of the goal [21, 32].
We will show a number of downsides to such an approach when applied to cloth manipulation.
First,the policy must learn to reason about the relationship between the current observation and the goal,while also reasoning about the action needed to obtain that goal.
These are both difﬁcult learningproblems; requiring the network to reason about them jointly exacerbates the difﬁculty.
Additionally,previous work has used reinforcement learning (RL) to try to learn such a policy [21, 39]; however, areward function is a fairly weak supervisory signal, which makes it difﬁcult to learn a complex clothmanipulation policy.
Finally, while many desirable folding actions are more easily and accuratelymanipulated with bimanual actions, previous learning-based methods for goal-conditioned clothmanipulation have been restricted to single-arm policies.
In this paper, we introduce FabricFlowNet (FFN), a goal-conditioned policy for bimanual clothmanipulation that uses optical ﬂow to improve policy performance (see Fig. 1).
Optical ﬂow hastypically been used for video-related tasks such as object tracking and estimating camera motion.
We demonstrate that ﬂow can also be used in the context of policy learning for cloth manipulation;we use an optical ﬂow-type network to estimate the relationship between the current observation anda sub-goal.
We use ﬂow in two ways: ﬁrst, as an input representation to our policy; second, after estimating the pick points for a pick-and-place policy, we query the ﬂow image to determine the place actions.
Our method is learned entirely with supervised learning, leveraging ground particlesfrom simulation. Our method learns purely from random actions without any expert demonstrations during training and without reinforcement learning.
Our learned policy can perform bimanual manipulation and switches easily between dual and singlearm actions, depending on what is most suitable for the desired goal.
Our approach signiﬁcantlyoutperforms our best efforts to extend recent single-arm cloth manipulation approaches to bimanualmanipulation tasks [18, 21].
We present experiments on a dual-arm robot system and in simulationevaluating our method’s cloth manipulation performance.
FabricFlowNet outperforms state-of-theart model-based and model-free baselines, and we provide extensive ablation experiments to demonstrate the importance of each component of our method to the achieved performance.
Our methodalso generalizes with no additional training to other cloth shapes and colors.
This paper contributes:
• A novel ﬂow-based approach for learning goal-conditioned cloth manipulation policies that can
perform dual-arm and single-arm actions
• A test suite for benchmarking goal-conditioned cloth folding algorithms encompassing and ex-
panding on goals used in previous literature [18, 21, 15]; we perform extensive experiments us-
ing this test suite to evaluate FabricFlowNet (FFN), baselines [18, 21] and ablations, demonstrating that FFN outperforms previous approaches.
• Experiments to demonstrate that FFN generalizes to other cloth colors and shapes, even withouttraining on such variations.
Related Work
Bimanual Manipulation .
A large body of research exists on dual-arm, or bimanual, manipulation [35].
Dual-arm systems allow for more complex behaviors than single-arm systems at thecost of greater planning complexity [14, 34], leading to research on closed kinematic chain planning [36, 4], composable skill learning [40, 7], and rewarding synergistic behavior [8].
Prior workhas also explored bimanual cloth manipulation [31], including establishing a diverse set of benchmark tasks [16].
Cloth manipulation is a highly underactuated task, and bimanual manipulationenables controlling multiple cloth points [5].
A common approach for cloth ﬂattening is to lifta cloth with one arm and regrasp it with the other arm until it reaches the ﬂattened conﬁguration [23, 10, 27, 13, 3].
Previous work in this direction uses hard-coded policies [27, 13, 3], whereaswe learn to achieve arbitrary folded conﬁgurations.
Tanaka et al.
[37] learn bimanual actions forgoal-conditioned folding, using a voxel-based dynamics model to predict how actions will changethe cloth state.
However, optimizing this dynamics model can slow down inference time comparedto our model-free approach.
Dynamic bimanual manipulation has also been explored in simulationfrom ground-truth keypoints [20] and for unfolding cloth in the real world [17]; we perform realworld bimanual folding using depth image observations.
Learning for Cloth Manipulation.
Prior works have proposed various hand-deﬁned representations for cloth manipulation, such as parameterized shape models [28] or binary occupancy fea- tures [22].
Recent approaches use contrastive learning to learn pixel-wise latent embeddings forcloth [15, 6].
Both contrastive learning [15] and goal-conditioned transporter networks [32] havebeen applied to imitate expert demonstrations. Our approach doesn’t require expert actions, justsub-goal states provided at test-time to deﬁne the task.
In contrast to these previous representations,our method uses a ﬂow-based representation, which we found to significantly outperform previousmethods for goal-based cloth manipulation. Other approaches have applied policy learning techniques to single-arm cloth smoothing [39, 33]. In contrast, we learn a policy that performs either single and dual-arm cloth manipulation; further,our focus is on goal-conditioned cloth folding, rather than smoothing.
For cloth manipulation, Lee etal.
[21] learns a model-free value function, but is limited by its discrete action space, and further,they do not use a ﬂow-based representation, which we show leads to large beneﬁts.
Prior methods forlearning goal-conditioned policies have used self-supervised learning to learn an inverse dynamicsmodel for rope [29, 30] but such approaches have not been demonstrated for cloth manipulation.
Lippi et al.
[25] plan cloth folding actions in latent space, but do not demonstrate generalization tounseen cloth shapes. Other papers use an online simulator [23], or learn a cloth dynamics modelin latent space [42], pixel-space [18], or over a graph of keypoints [26].
Unlike these model-basedmethods, our method is model-free and does not require online simulation or time-expensive CEMplanning, leading to much faster inference.
Further, we compare our approach to state-of-the-artapproaches for cloth manipulation [21, 18] and show signiﬁcantly improved performance.
Optical Flow for Policy Learning.
Optical ﬂow is the task of estimating per-pixel correspondencesbetween two images, typically for video-related tasks such as object tracking and motion estimation.
State-of-the-art approaches use convolutional neural networks (CNN) to estimate ﬂow [12, 19, 38].
Optical ﬂow between successive observations has previously been used as an input representationto capture object motion for peg insertion [11] or dynamic tasks [1].
Within the domain of clothmanipulation, Yamazaki et al.
[41] similarly use optical ﬂow on successive observations to identifyfailed actions.
We use ﬂow not to represent motion between successive images, but to correspondthe cloth pose between observation and goal images, and to determine the placing action for folding.
Argus et al.
[2] use ﬂow in a visual servoing task to compute residual transformations betweenimages from a demonstration trajectory and observed images.
In contrast, we learn a policy withﬂow to determine what cloth folding actions to take, not how to servo to a desired pose.
Learning a Goal-Conditioned Policy for Bimanual Cloth Manipulation3.1 Problem Formulation
Our objective is to enable a robot to perform cloth folding manipulation tasks.
Let each task bedeﬁned by a sequence of sub-goal observations G : { x g 1 , x g 2 , . . .
, x gN } , each of which can be achievedby a single (possibly bimanual) pick-and-place action from the previous sub-goal.
We require subgoals, rather than a single goal, because a folded cloth can be highly self-occluded such that a singlegoal observation fails to describe the full goal state.
Deﬁning a task using a sequence of sub-goalsis found in other recent work [30].
Similar to prior work [30, 29], even if the sub-goals are obtainedfrom an expert demonstration, we nonetheless do not assume access to the expert actions; this is arealistic assumption if the sub-goals are obtained from visual observations of a human demonstrator.
We assume that the agent does not have access to the sub-goal sequence G during training thatit must execute during inference.
Thus, the agent must learn a general goal-conditioned policya t = π ( x t , G ) , where x t is the current observation of the cloth and a t ∈ A is the action selected bythe policy.
In our approach, we input each sub-goal x gi sequentially to our policy: a t = π ( x t , x gi ) .
Agoal recognizer [30] can also be used to decide which sub-goal observation to input at each timestep.
For convenience, we will interchangeably refer to x gi as a goal or sub-goal.
3.2 Overview
A common approach for a goal-conditioned policy is to input the current observation x t and thegoal observation x gi directly into to a neural network representation of a policy [30, 21] or a Qfunction [39, 21].
However, the network must reason simultaneously about the relationship between [Figure] the observation and the goal, as well as the correct action to achieve that goal.
Our ﬁrst insight isthat we can improve performance by separating these two components: we will learn to reason aboutthe relationship between the observation and the goal, and separately use this relationship to reasonover actions.
Speciﬁcally, we represent this relationship using a “ﬂow image” f , which indicatesthe correspondence between the current observation x t and sub-goal x gi .
Thus we propose using theﬂow image f as an improved input representation of the policy, rather than directly inputting theobservation x t and goal observation x gi .
Our second insight is that we can also use ﬂow in the output representation of the policy.
We use apick and place action space; prior methods that learn pick and place policies for deformable objectmanipulation predict place points using the policy network, either explicitly [33, 32, 42, 29, 30, 39]or implicitly by transforming the inputs to a Q-function [21].
Instead, we simplify the problemby leveraging ﬂow: our policy network only learns to predict the pick points.
For the place point,we query the ﬂow image f for the ﬂow vector starting at the predicted pick location, and use theendpoint of that vector as the place point.
We demonstrate that using ﬂow in the two ways described above for our policy achieves signiﬁcantlyimproved performance compared to prior work.
Furthermore, our approach extends naturally todual-arm manipulation, allowing us to easily transition between single and dual-arm actions.
A schematic overview of our system can be found in Fig.
2b.
We ﬁrst compute the ﬂow f betweenthe current observation x t and goal x gi .
Next, we input the ﬂow f to a policy network (PickNet),which outputs pick points p i .
We then query the ﬂow image f
( p i ) to determine the place points foreach robot arm.
Further details of our approach are described below.
3.3 Estimating Flow between Observation and Goal Images
We learn ﬂow to use it as an input representation to our pick prediction network, and as an actionrepresentation for computing place points.
Given an observed depth image x t and desired goal depthimage x gi , we estimate the ﬂow f = ( f 1 , f 2 ) , mapping each pixel ( u, v ) in x t to its correspondingcoordinates ( u (cid:48) , v (cid:48) )
=
( u + f 1 ( u ) , v + f 2 ( v )) in x gi .
This task formulation differs from standardoptical ﬂow tasks as the input image pair ( x t , x gi ) are not consecutive images drawn from videoframes.
To capture the complex correspondences between x t and x gi , we train a convolutional neural networkto estimate the ﬂow image f
(see Appendix for details).
The training loss we use to supervise thenetwork is endpoint error (EPE), the standard error for optical ﬂow estimation.
EPE is the Euclideandistance between the predicted ﬂow vectors f and the ground truth f ∗ , averaged over all pixels:
L EPE = 1 N (cid:80) Ni =1 (cid:107) f ∗ − f (cid:107) 2 .
We use a cloth simulation to collect training examples with groundtruth ﬂow.
The simulator provides the ground-truth correspondence between the particles of thecloth in different poses.
The simulation cloth particles are not as dense as the depth image pixels;as a result, we only have ground-truth ﬂow supervision for a sparse subset of the pixels that alignwith the cloth particles.
Thus, we mask the loss to only supervise the ﬂow for the pixels that alignwith the location of the cloth particles.
We train the ﬂow network using data collected from randomactions.
See Sec.
3.6 for more details on the simulator, data collection, and network training.
3.4 Learning to Predict Pick Points
[Figure] Our bimanual action space A consists of actions a = ( p 1 , p 2 , q 1 , q 2 ) , where p and q are the pickand place points respectively, paired according to the subscripts.
We train a neural network called
PickNet to estimate the pick points p 1 , p 2 .
Crucially, the input to PickNet is a ﬂow image f , estimated between the current depth image x t and the desired goal depth image x g i , as described inthe previous section.
The ﬂow image indicates, for each pixel ( u, v ) in the current observation, thelocation f ( u, v ) that the pixel has moved to in the goal observation.
Our ﬂow network (Sec. 3.3above) reasons about the observation-goal relationship, so that the policy network (PickNet) onlyneeds to reason about the action, speciﬁcally the two pick points ( p 1 , p 2 ); computing the place pointsis described in Sec. 3.5.
For dual-arm actions, the pick points must be estimated conditionally, as the location of pick pointp 1 on the cloth inﬂuences the optimal location of pick point p 2 , and vice versa.
To decouple thisconditional estimation problem, we propose a two-network architecture, PickNet1 and PickNet2,to estimate the pick points (see Fig. 3).
This architecture was inspired by Wu et al.
[39], whichused two networks for pick-conditioned placing; we instead use two networks to condition dual-armpicking.
PickNet1 is a fully convolutional network that receives ﬂow image f as input and outputsa single heatmap H 1 estimating the optimal pick points for arm 1.
We compute the ﬁrst pick pointas p 1 = arg max p H 1 ( p ) .
The second network, PickNet2, predicts the second arm
’s pick point p 2conditioned on p 1 ; PickNet2 takes as input both the ﬂow image f and an additional image witha 2D Gaussian centered on p 1 , and is otherwise identical to PickNet1.
PickNet2 outputs heatmap
H 2 , from which we compute the second pick point: p 2 = arg max p H 2 ( p ) .
The two-networkarchitecture decouples the conditionally dependent pick point predictions and does not require usto resort to heuristics to extract two pick points from a single heatmap.
We refer to PickNet1 and
PickNet2 together as “PickNet.”
To train PickNet, we collect a dataset of random actions (see Sec.
3.6 for details) and record thecurrent observation x t , the bimanual action a = ( p 1 , p 2 , q 1 , q 2 ) , and the next observation x t +1 .
Wealso estimate the ﬂow f from x t to x t +1 , as explained in Sec. 3.3.
We create ground truth pickheatmaps H ∗ i for arm i using the recorded random action a , by placing a 2D Gaussian N ( p i , σ ) oneach ground truth pick location p i .
We then supervise PickNet using the binary cross-entropy (BCE)loss between predicted heatmaps H 1 , H 2 and ground truth heatmaps H ∗ 1 , H ∗ 2 .
However, it might beunclear to the network which pick point should be output by PickNet1 and which should be outputby PickNet2.
To allow for ﬂexibility, we compute the loss for both possible correspondences anduse the minimum:
[Equation]
At inference time, PickNet outputs the pick points p 1 , p 2 , computed from the argmax of H 1 , H 2respectively, as described above.
3.5 Estimating the Place Points from Flow
After estimating the pick points p 1 , p 2 from ﬂow, the remaining step to predict a bimanual pick andplace action a = ( p 1 , p 2 , q 1 , q 2 ) is to estimate the place points q 1 , q 2 .
A straightforward approach would be to train the network to predict place points q 1 , q 2 , similar to the pick points p 1 , p 2 asdescribed above (see Fig. 2a).
Instead, our approach uses the ﬂow image to ﬁnd the place points, sothat the place points do not have to be learned separately.
Our approach makes the assumption that, to achieve a desired subgoal conﬁguration, the point pickedon the cloth should be moved to its corresponding position in the goal image (which is estimatedby the ﬂow).
This is a simplifying assumption, since it is possible that the picked point will shiftslightly after it is released by the gripper; our method does not take into account such small movements.
Using this assumption, to compute the place points q 1 , q 2 , we query the ﬂow f at each pickpoint p 1 , p 2 to estimate the delta between the pick point location in the observation image and thecorresponding location of the pick points in the goal image.
We use these predicted correspondencesas the place points: q i = f ( p i )
+ p i , for each arm i .
Action predictions estimated by our approach can produce nearly overlapping pick and place points,indicating that arm 1 and arm 2 should perform identical actions.
We observe this behavior from PickNet when the goal is best achieved with a single-arm action, rather than a bimanual one.
Ona real robot, grippers are likely to collide if grasping points that are too close.
Therefore, to switchbetween executing a single-arm or bimanual action, we compute the L2 pixel distance between pickpoints d pick = (cid:107) p 1 − p 2 (cid:107) 2 and place points d place =
(cid:107) q 1 − q 2 (cid:107) 2 .
We use a single-arm action wheneither distance is smaller than a threshold α , which we set to 30 for all experiments.
3.6 Implementation Details
We use SoftGym [24], an environment for cloth manipulation built on the particle-based simulator
Nvidia Flex, to collect training datasets.
The simulator models cloth as particles connected bysprings.
We use pickers that simulate a grasping action by binding to the nearest cloth particlewithin a threshold to execute pick and place actions in SoftGym.
We collect data by taking randomactions, biased towards grasping corners of the cloth.
We demonstrate that we are able to train ourmethod in SoftGym and then transfer the policy to the real world.
Details on the data collection, aswell as the network architecture and training details, can be found in the Appendix.
Experiments
4.1 Simulation Experiments
Experiment Setup.
We evaluate our method and compare to state-of-the-art baselines in the SoftGym [24] simulator; real-world evaluations are below in Sec.
Our experiments focus on foldingtasks, and we assume that a cloth smoothing method ( e.g., [17, 33]) is used to ﬂatten the cloth beforefolding is executed.
Our error metric is the average particle position error between the achieved andgoal cloth conﬁguration.
We evaluate on two sets of goals: 40 one-step goals that can be achievedwith a single fold action, and 6 multi-step goals that require multiple folding actions.
The multi-stepgoals each consist of a sequence of sub-goal images, with the next sub-goal presented after eachaction.
This protocol follows from our problem formulation in Sec. 3.1, and is similar to the protocol in Nair et al.
Our goals include test goals from Ganapathi et al.
[15] and Lee et al.
[21]that are achievable with one arm, as well as additional goals more suitable for two-arm actions (see Appendix Fig.
S1 for the full set of goals).
We compare our method to Fabric-VSF [18], which learns a visual dynamics model and uses CEMto plan using the model.
We only use Fabric-VSF with RGB-D input, as depth-only input performspoorly for folding tasks [18].
FabricFlowNet only uses depth and does not rely on RGB, whichenables our method to transfer easily to the real world without extensive domain randomization.
We also compare to Lee et al.
[21], a model-free approach.
We extend the the original single-armmethod to a dual-arm variant and compare against both.
For both our method and the baselines, weonly allow each method to perform one pick-and-place action for each subgoal (e.g. one pick andplace action for each single-step goals).
Additional baseline details can be found in the Appendix.
4.1.1 Results
Table 1 contains our simulation results for all methods.
We report average particle distance error(in mm) for one-step goals only, multi-step goals only, and over both one-step and multi-step goals.
Our results show that our method, FabricFlowNet, achieves the lowest error over all goals and hasthe fastest inference time.
Images of goals, qualitative results, and additional results on baselinevariants, crumpled initial conﬁgurations, and end-to-end training can be found in the Appendix.
[Table]
Ablations
[Table] We run series of ablations to evaluate the importance of the components of our system; results averaged over all 46 goals can be found in Table 2, with additional results in the Appendix.
Our ablationsare designed to answer the following questions:
What is the beneﬁt of using ﬂow as input?
We modify PickNet to receive depth images of theobservation and goal as input to the network (“DepthIn”), as is commonly done in previous work ongoal-conditioned RL [21, 32].
In this ablation, the PickNet needs to reason about the relationshipbetween the observation and the goal as well as about the action.
In contrast, in our method the ﬂownetwork separately reasons about the relationship between the observation and the goal; the PickNetreceives the ﬂow image as input and thus only needs to reason about the action.
This separationleads to a 23.7% improvement.
What is the beneﬁt of using ﬂow to choose the place point?
In this ablation, we train a networkto predict the place points directly (“PredictPlace”); see Appendix for details.
This is in contrast toour approach where we use the ﬂow ﬁeld, evaluated at the pick point f
( p i ) , to compute the placepoint q i for arm i .
Our approach leads to a 32.4% improvement, showing the beneﬁt to using ﬂowas an action representation.
No ﬂow: We combine the above two ablations and remove ﬂow entirely, (“NoFlow”; ours has 60.4%improvement).
The above ablations all indicate the strong beneﬁt of using ﬂow as both an input andaction representation for cloth manipulation.
What is the beneﬁt of biasing the data collection to grasping corners?
Our method uses priorknowledge about cloth folding tasks to bias the training data and pick at corners of the cloth.
In thisablation, we choose pick points randomly (“NoCornerBias”, ours has 35.5% better performance).
Different architecture: We also compare our architecture for PickNet (See Sec. 3.4) to an alternate,simpler architecture that takes as input the ﬂow image I f and outputs two heatmaps correspondingto each of the pick points (“SinglePickNet”; ours has 2.1% better performance).
Does the loss formulation in Eq.
1 improve performance?
We compare our method to an ablationwhere the ﬁrst ground-truth heatmap is used to supervise PickNet1 and similarly for the second, i.e.
L Pick = l BCE ( H 1 , H 2 , H ∗ 1 , H ∗ 2 ) .
(”NoMinLoss”; ours has similar performance).
[Table]
4.2 Real World Experiments
We evaluate FabricFlowNet in the real world and demonstrate that our approach successfully manipulates cloth on a real robot system.
Experiment Setup.
Our robot system consists of two 7-DOF Franka Emika Panda arms and a singlewrist-mounted Intel RealSense D435 sensor (See Fig. 1).
We plan pick and place trajectories using MoveIt!
We evaluate on a 30x30 cm cloth, using 12 single-step and 6 multi-step goals (see
Fig. 4 for examples) that form a representative subset of our simulation test goals.
To transfer our method from simulation to the real world, we align the depth between real andsimulated images by subtracting the difference between the average depth of the real support surface [Figure] [Figure]
(i.e. the table) and the simulated surface.
We mask the cloth by color-thresholding the background;see Appendix for details.
To account for variations in initial cloth pose, we train our networks withrandom translations (up to 5 pixels) and rotations (up to 5 degrees) of the input images.
We foundthat these simple techniques were sufﬁcient to transfer the networks trained entirely in simulation tothe real world, since the simulated depth image matches reasonably well to the real depth image.
4.2.1 Results
[Figure]
Fig.
4 provides qualitative real world results, showing that we successfully achieve many of thegoals.
Our website contains videos of these trials (see link in the abstract).
Our method transferseasily to the real world because we use only depth images as input, which appear similar in bothsimulation and the real world, unlike RGB images.
We compare FabricFlowNet to the NoFlow ablation from Sec. 4.1.2.
Both methods were used thesame sim-to-real techniques described in the previous section.
While we do not have access to thetrue cloth position error in the real world, the Intersection-over-Union (IoU) metric on the achievedcloth masks serves as a reasonable proxy metric [21].
FFN achieves 0.857 mean IoU on 3 trials overthe real test goals, whereas NoFlow only achieves 0.523.
See the Appendix for additional details.
Generalization.
In addition to evaluating the folding policy on square cloth for various goal conﬁgurations, we also test the generalization of our method to other shapes of cloth.
We evaluate theperformance of FFN trained only on a square cloth on folding goals for a rectangular cloth as well asa T-shirt.
These fabrics are also thinner than the square blue towel used in the real world experimentsabove.
Fig.
5 shows that FFN trained on a square yellow cloth in simulation is able to generalize toother cloth shapes, textures, and colors (FFN only receives depth images as input).
See Appendixfor additional results.
Conclusion
In this work we present FabricFlowNet, a method which utilizes ﬂow to learn goal-conditioned fabricfolding.
We leverage ﬂow to represent the correspondence between observations and goals, and asan action representation.
The method is trained entirely using random data in simulation.
Our resultsshow that separating the correspondence learning and the policy learning can improve performanceon an extensive suite of single- and dual-arm folding goals in simulated and real environments.
Ourexperiments also demonstrate generalization to different fabric shapes, textures, and colors.
Futurework on ﬂow-based fabric manipulation could incorporate actions beyond pick and place, such asparameterized trajectories or dynamic actions.