Folding Deformable Objects using
Predictive Simulation and Trajectory Optimization
Yinxiao Li, Yonghao Yue, Danfei Xu, Eitan Grinspun, Peter K. Allen
Abstract— Robotic manipulation of deformable objects remains a challenging task. One such task is folding a garment
autonomously. Given start and end folding positions, what is an
optimal trajectory to move the robotic arm to fold a garment?
Certain trajectories will cause the garment to move, creating
wrinkles, and gaps, other trajectories will fail altogether. We
present a novel solution to find an optimal trajectory thatavoids such problematic scenarios. The trajectory is optimized
by minimizing a quadratic objective function in an off-line
simulator, which includes material properties of the garment
and frictional force on the table. The function measures the dissimilarity between a user folded shape and the folded garment
in simulation, which is then used as an error measurement tocreate an optimal trajectory. We demonstrate that our two-arm
robot can follow the optimized trajectories, achieving accurate
and efficient manipulations of deformable objects.

I. I NTRODUCTION
Robotic folding of a garment is a difficult task because it
requires sequential manipulations of a highly unconstrained,
deformable object. Given the garment shape, the robot canfold it by following a folding plan [14][13]. However, thelayout of the same folding action can vary in terms of the material properties such as cloth hardness and the environment
such as friction between the garment and the table. Given
the starting and ending folding positions, different folding
trajectories will lead to different results. In this paper, we
propose a novel method that learns optimal folding trajectory
parameters from predicted thin shell simulations of similar
garments, which can then be applied to a real garment folding
task (see Figure 1). The contributions of our paper are:
- A fast and robust algorithm that can detect garment
key points such as sleeve ends, collar, and waist corner,
automatically. These key points can be used for folding
plan generation.
- An online optimization algorithm that learns optimal
trajectories for manipulation from mathematical model
evolution combined with predictive thin shell simulation.
- A novel approach that adjusts the simulation environment to the robot working environment for the purposeof creating a similar manipulation result.
- The trajectories are general in that they can be scaled
to accommodate similar garments of different size.
- Experimental results with a Baxter robot showing successful folding trajectories for a number of different
garments including sweaters, pants, and towels.
All the authors are with Department Computer Science, Columbia University, New York, NY, USA {yli@cs., yonghao@cs., dx2143,

eitan@cs. allen@cs.}columbia.edu

Fig. 1. Comparison of our simulation of robotic manipulation (TOP) andreal robot implementation (BOTTOM). The green curves show the virtualand the real trajectories for folding.

Figure 2 shows the complete pipeline of garment manipulation, as well as the key steps of garment folding.
The garment folding is the final step of the entire pipelineof garment manipulation which contains grasping, visual
recognition, regrasping, unfolding, placing flat, and folding.
Our previous work [9][10][11] has successfully addressed
all the stages of the pipeline with the exception of the final
folding task. This paper specifically addresses the robotic
folding task (purple rectangle in Figure 2) with the goal offinding optimal trajectories to successfully fold garments.
We begin by assuming the garment is placed flat on
the table initially, as shown in our previous work [11]. By
detecting the key points of the garment (see section V-A), a
pre-defined folding plan is used to create optimal trajectories
for folding the garment. After several steps, we obtain adesired folding result in the real world using the Baxter robot,
which is comparable to the result from the simulation.
II. R ELATED W ORK
There are many challenges associated with the manipulation of a deformable object such as a garment. One of
the challenges is unfolding a garment from an arbitrarystate and placing it flat on a table. Many researchers startwith simple garments such as a towel [6]. By iteratively
looking for the lowest corner point of the towel, the robot

Regrasp

Improperpose

Proper pose

Initial state

Initial grasp

Pose estimation

UnfoldFold

Place flat

P1

P2

Trajectoryyq( y )yq( y )

P3

Real World

Trajectory

Simulation

P0

Iterative Trajectory Optimization

Adaptation

P2
P1
Trajectory
P3
P0

Fig. 2. T OP ROW: The entire pipeline of dexterous manipulation of deformable objects. In this paper, we are focusing on the phases of garment folding,as highlighted in the purple rectangle. B OTTOM ROW: Details of the folding procedure. We apply off-line simulation with iterative trajectory optimization
to find the best trajectory for a specific folding action by comparing the result (light blue contour) with template (black contour). Similar steps are repeateduntil the garment is folded in the simulator. Then all the folding trajectories are exported, adapted, and implemented on a real robot. Green arcs illustrate
the actual trajectories of robotic arms.

is able to unfold it and place flat on a table. Then the towel
can be quickly folded by symmetric information. For more
complicated garments such as sweaters and pants, their states(poses) have to be recognized first by either image-based
perception [21][9][4] or 3D shape matching [8][10], etc..
Deformable objects such as a garment have large dimensional
state spaces which are hard to track and recognize. Therefore,
for such tasks, many researchers employ a large databasewhich contains exemplars of different states of an object from
off-line simulation or real garments as the training data. By
using SIFT feature [21][9] or volumetric features [4][10], thestate of a garment is recognized and tracked. After regraspingseveral times, the garment can be unfolded and placed flaton a table by a dual-arm robot [3][18][11], a prerequisite forgarment folding.
With the garment fully spread on the table, attention isturned to parsing its shape. S. Miller et al. have designed aparametrized shape model for unknown garments [14][13].
Each set of parameters defines a certain type of garment suchas a sweater or a towel. The contour-based garment shapemodel was further improved by J. Stria et al. using polygonalmodels [17]. The detected garment contour is matched toa polygonal model by removing non-convex points using
a dynamic programming approach. Folding is the ultimate
goals of garment manipulation and only a few researchers
have achieved this. F. Osawa et al. used a robot to fold agarment with a special purpose table that contains a plate thatcan bend and fold the clothes assisted by a dual-arm robot.
Another folding method using a PR2 robot was implemented
by J. van den Berg et al. [19]. The core of their approach
was the geometry reasoning with respect to the cloth model
without any physical simulation. Similar work done by J.
Stria et al. [18] using two industrial arms and a polygonal
contour model. Kita, et al. [7] use a humanoid robot to folda garment starting from a random configuration.
None of the previous works focus on trajectory optimization for garment folding, which brings uncertainty to thelayout given the same folding plan. One possible case is thatthe garment shifts on the table during one folding action
so that the targeted folding position is also moved. Anothercase is that an improper folding trajectory causes additional
deformation of the garment itself, which can accumulate.

Fig. 3. Failure example with improper folding trajectories. F IRST ROW:
Folding trajectory is low and flat that causes drift to the towel and longsleeve T-Shirt. S ECOND ROW: Folding trajectory is too high when thegripper approaching the target folding position that piles up the towel.
T HIRD ROW: Dual-arm folding. If the distance between the two arms is
too close, the folding may fail.

Figure 3 shows a few failure examples with improper trajectories. We use green tape on the table to show the originalposition of the garments. The first two rows show that if themoving trajectory is too low and close to the garment, thefolded part will fall down, pull the rest, and cause drift of thewhole garment. These cases usually happen when the folding
step is lengthy without trajectory optimization. The third row
shows a case where the folding trajectory is too high, which
will cause extra wrinkles or even piling up. The last row
shows two cases using two arms to fold. If the arms areclose, the part in between loses tension, and will fall down
and pull the rest away. The focus of this paper is to create
trajectories for folding that will overcome these problems.

- Slowly put down the garment on a table and keep the
picking point and the lowest point in the previous stepat maximum spread condition. Measure the distance between these two points again as L2 . The shear resistancefraction is defined as the following
shear f rac = (L1 − L2 )/L2

(1)

- We then pick up and hang the virtual garment in Maya,
adjusting the Maya shear parameter such that the shear
fraction as calculated in the simulator is identical to thereal world.

III. S IMULATION E NVIRONMENT
A. Folding Pipeline in Simulation
In the model simulation, we use a physics engine [1]to simulate the movement and deformation of the garment
mesh models. We assume there is only one garment for eachfolding task, which has been placed flat on a table. A virtualtable is added to the scene which the garment lies on, as
shown in Figure 1, top.
During each folding step, the robot arm picks up a smallpart of the mesh, moves it to the target position following
a computed trajectory, and places it on the table to simulate
an entire folding scenario. If the part of the garment to befolded is relatively wide, then both left and right arms may
be involved. The trajectory is generated using a Bézier curve,
which will be discussed in section IV.
Most of the garment mesh models are built from our test
garments. A garment mesh is created by first extracting thecontour of the garment (see section V-A). Then by inserting
points on the inside of the garment contour, we triangulate amesh by connecting these points. Lastly, we mirror the meshto construct a two-sided garment mesh.
B. Parameter Adaptation
There are two key parameters needed to accurately simulate the real world folding environment. The first is thematerial properties of the fabric, and the second is thefrictional forces between the garment and the table.
1) Material properties: Through many experiments, wefound that the most important property for the garments inthe simulation environment is shear resistance. It specifies theamount the simulated mesh model resists shear under strain;when the garment is picked up and hung by gravity, the total
length will be elongated due to the balance between gravity
force and shear resistance. An appropriate shear resistancemeasure allows the simulated mesh to reproduce the sameelongation as the real garment. This parameter will bridge
the gap between the simulation and the real world for the
garment mesh model.
For each garment, we follow the steps described below tomeasure the shear resistance. Figure 4 shows an example.
- Manually pick one extremum part of the garment suchas the sleeve end of a T-shirt, the waist part of a pairof pants, and a corner of a towel.
- Hang the garment under gravity and measure the length
between the picking point and the lowest point as L1

Fig. 4. Method for measuring the shear resistance. L EFT: Diagonal length
measurement. M IDDLE : Zoomed in regions. R IGHT: The garment is hanging
under gravity.

2) Frictional forces: The surface of the table can be roughif covered by a cloth sheet or slippery if not covered, whichleads to variance in friction between the table and garment.
A shift of the garment during the folding can possibly
impair the whole process and cause additional repositioning.
Adjusting the frictional level in the simulation to the realworld is crucial and necessary for trajectory optimization.
To measure the friction between the table and the garment,
we do the following steps.
- Place a real garment on the real table of length Lt .
- Slowly lift up one side of the real table, until thegarment in the real world begins to slide. The lifted
height is Hs . The friction angle is computed as,∠F riction = sin−1 (Hs /Lt )(2)- In the virtual environment, the garment is placed flaton a table with gravity. Assign a relatively high friction
value to the virtual table. Lift up one side of the virtualtable to the angle of ∠F riction .
- Gradually decrease the frictional force in the virtualenvironment until the garment begins to slide. Use thisfrictional force in the virtual environment as it mirrors
the real world
With these two parameters, we obtain similar manipulation
results for both the simulation and the real garment.
IV. T RAJECTORY O PTIMIZATION
The goal of the folding task is specified by the initialand folded shapes of the garment, and by the starting andtarget positions of the grasp point (as in Figure 5). Given thesimulation parameters, we seek the trajectory that effects thedesired set of folds. We first describe how to optimize thetrajectory for a single end effector, and then discuss the caseof two end effectors.

tion. We define the dissimilarity term asZ
1kq(y) − ykdA,
D(St , Sx ) =|St | St

A. Trajectory parametrization

where |St | is the total surface area of the garment meshincluding both sides of the garment, y ∈ St is a point on thetarget folded shape St , q(y) ∈ Sx is the corresponding pointon the simulated folded shape, and dA is the area measure,see Figure 6. Our implementation assumes St and Sx are
given as triangle meshes, and discretizes (5) as
1 X
kqi − yi kAi ,(6)
D̃(St , Sx ) =|St | i

We use a Bézier curve [5] to describe the trajectory. An
n-th order Bézier curve T(u) has (n + 1) control points
Pk = (Pk,x , Pk,y , Pk,z )T ∈ R3 , defined byT(u) =nX

Bkn (u)Pk ,

(3)

k=0

where Bkn (u) =functions.

 n(1 − u)n−k uk are the Bernstein basis
kP2
P1
gripper

Trajectory
P3target position

starting position P0

Fig. 5. An example of the folding task: we want to fold a sleeve into theblue target position, by using a robotic gripper to move the tip of the sleeve(grasp point) from the starting position (P0 ) to the target position (P3 ),following a trajectory, shown as the red curve. P1 and P2 are knot points
that form the Bézier trapezoid.

We use n = 3 for simplicity, but our method can be easily
extended to deal with higher order curves. P0 and P3 are
fixed to the specified starting and target positions of the grasp
point (as in Figure 5). The intermediate control points x =(PT1 , PT2 )T can then be adjusted to define a new trajectoryusing the objective function defined below. The update ruleis described in section IV-B.
xopt = argmin{lx + αD(St , Sx )}2 .
{z
}
|
x

(4)

C(x)

Here C(x) is a cost function
with two terms. The first term
penalizes the trajectory lengthy q(y)
lx , thus preferring a folding paththat is efficient in time and energy. The second term seeks
Sx
St
the desired fold, by penalizing dissimilarity D(St , Sx ) be- Fig. 6.
The dissimilarity
tween the desired folded shape captures the misalignment beSt , compared to the shape Sx tween St and Sx by integrating
distance between the correobtained by the candidate fold- thesponding points y ∈ St anding trajectory x, as predicted by q(y) ∈ Sx over the garment.
a cloth simulation; we used aphysical simulation engine [1], for the cloth simulation. The
weight α balances the two terms; we used α = 103 in our
experiment. See section IV-B for optimization details.
Intuitively, dissimilarity measures the difference between
the desired folded shape and the folded garment in simula-(5)

where yi is the barycenter of i-th triangle on the target shape,qi is the (corresponding) barycenter of i-th triangle on thesimulated shape, and Ai is the area of the i-th triangle on
the target shape.
To compute the trajectory length lx , we use the De
Casteljau’s algorithm [5] to recursively subdivide the Bézier
curve T into a set of Bézier curves T(j) , until the deviation(j)(j)
between the chord length (kP0 − P3 k) and the total
P2
(j)
(j)
length between the control points ( i=0 kPi − Pi+1 k) for(j)each subdivided curve T is sufficiently small. Then, lx isapproximated by summing up the chord lengths of all the
P
(j)
(j)
subdivided curves: lx ≈ j kP0 − P3 k.
We initialize P1 and P2 as
1
2(7)
P1 = P0 + P3 + hkP0 − P3 kev ,3
3
1
2
P2 = P0 + P3 + hkP0 − P3 kev ,(8)
3
3
where ev is the unit vector in the upward vertical direction, his a constant value of 1/3, which means the initial trajectory
will have equal horizontal extent between knot points.
B. Optimization.
To optimize equation (4), we apply a secant version of
the Levenberg-Marquardt algorithm [12][15]. For the current
trajectory generated by x, we estimate the derivative ∇C(x)of the cost function C(x) numerically, by sampling slightly
modified trajectories x + δej , where ej , 1 ≤ j ≤ dim(x),are the orthonormal bases, and we used δ = 10−1 in our
implementation.
The secant version of Levenberg-Marquardt algorithm
iteratively builds a local quadratic approximation of {C(x)}2
based on the numerical derivative, and then takes a step
toward an improved state. The direction of the step is acombination of the steepest gradient descent direction and the
conjugate gradient direction. We use the specific approachdescribed by Madsen et al. [12] (see §3.5 therein). The
iterative procedure terminates when the improvement in{C(x)}2 becomes sufficiently small.
C. Multiple arms.
In the case of using multiple arms, we associate anindividual trajectory xi to each of the arms Ri . We thenextend the state variable to x = (xT1 , ...)T . The rest of the
optimization procedure is the same as the single arm case.

Note that both single and dual-arm trajectories are in 3D
space. The optimization for dual-arm trajectories is able tofind a solution which will overcome failures such as shownin Figure 3 bottom.
V. A PPLICATION TO G ARMENT F OLDING
A. Key Points Localization
We assume the garment is placed roughly in the centerof the table, as shown in the Figure 7, bottom. Our firststep is to segment the garment from the background. Since
we can easily obtain a table or a surface with homogeneouscolor [14][17], a fast color-based supervised image segmentation method is suitable for our task. We apply a markerbased watershed method [20] with the four corners of the table initialized as background and the center initialized as theforeground. More complicated scenes can employ advanced
image segmentation algorithms such as GrabCut [16].

where T is the target garment mask, S is the current
deformed 2D contour, and S̄ is a reference 2D contour, whichis from the previous state. Efit (S, T ) penalizes discrepancies
between the contour and the target mask. The last two terms
seek to limit and regularize the deformation of the contour
in order to preserve the angle features, which includes thestretching and bending energies, weighted by user specified
coefficients κ and β.
We represent the contour as a closed loop consisting of line
segments. In this discrete representation, Efit is computed as
X
2Ẽfit (S, T ) =(distT (xi )) li ,(10)iwhere li and xi are the length and the midpoint of the i-thline segment of the deformed contour. Estretch is computedas2
1 X li¯(11)Ẽstretch (S̄, x) =¯li − 1 li ,2iwhere ¯li is length of the i-th line segment of the reference
contour. Ebend is computed as2
1 X θj− 1 L̄j ,(12)Ẽbend (S̄, x) =2 j
θ̄jFig. 7.
Red dots are the predefined key points for the garment, suchas sleeve ends. Left column is a long-sleeve t-shirt example, and the rightcolumn is a pants example. T OP : Initialized contour. Each garment category
is initialized with a different contour. M IDDLE : Fitting results. The contour
shrinks onto the boundary of the garment according to the distance field.
B OTTOM : Fitting results mapped back to the original input image.

To register the feature points, such as the corner points ofsleeves, we employ a 2D registration technique to register a
pre-defined garment template (as in Figure 7 top row) withthe captured garment mask. Our 2D registration is based on
our 3D non-rigid registration code for thin shell models [11],and can deal with garment masks that have curved contours.
We first initialize a distance field using the segmented
mask from the previous step. The category of the garment canbe easily recognized by using a template matching algorithm,
which leads to an associated garment initial template. We
register the template with the garment mask by minimizing
the following energy function:
ET (S, S̄) = Efit (S, T ) + κEstretch (S̄, x) + βEbend (S̄, x),(9)where θj and θ̄j are the angles between the adjacent line
segments at j-th vertex of the deformed and the reference
contours, respectively. L̄j are the average length of theadjacent line segments at j-th vertex of the reference contour.
Our registration iteratively updates S by using the secant version of the Levenberg-Marquardt algorithm [12][15].
Initially, each vertex in the template polygon is marked asa feature point (red points in Figure 7, top row), and isassigned a unique ID to identify its semantic meaning. At
the end of each step, we subdivide each contour segment
if its length is larger than a threshold, and mark the newly
added vertex as non-feature points (green points in Figure 7,
middle row). Next, we merge any pair of adjacent segments
if they do not share a feature point and their length is below a
threshold. The subdivide and merge operations guarantee thatthe garment contour is sufficiently but not overly sampled.
Then, we update the reference contour S̄ by S.
We repeat the iteration until the reduction in ET (S, S̄)
becomes sufficiently small. Finally, the positions as well as
the semantic meanings of the feature points in the garment
mask are identified by retrieving the feature points via theunique point ID in the registered contour.
VI. E XPERIMENTAL R ESULTS
To evaluate our results, we tested our method on several
different garments such as long-sleeve t-shirts, pants, andtowels for multiple trials, as shown in Figure 8 left. These
garments require both single and dual-arm folds. A highresolution video of our experimental results is online at
http://www.cs.columbia.edu/˜yli/IROS2015.

A. Robot Setup
In our experiments, we use a Baxter research robot, whichis equipped with two arms with seven degrees of freedom.
We mount a Prime Sense Xtion range sensor [2] on top of
the Baxter head panel, which has been calibrated to the robot
base frame.To improve grasp stability and form a closed loop
controller, we add tactile sensors to the grippers.
B. Measurement of parameters
To make the off-line simulation better approximate the real
scenario, as described in Section III, we manually measure
the stretch resistance of each garment and friction on thetable. Figure 8, left shows a picture of all the test garments
we used in different colors, sizes, and material properties.
Figure 8, right table shows the measured parameters ofeach test garment, including stretch percentage and Friction
angle, and corresponding Maya parameters. For common
garments, these parameters do not have a significant variance.
Therefore, we suggest that if researchers use simulators such
as Maya, the average values of each column are a reasonably
good initialization.
C. Garment manipulation and folding
Figure 9 shows three successful folding examples fromthe simulation and the real world, including a long-sleeveshirt, a pair of pants, and a medium size towel. We show
six key frames for each folding task. The folding poses fromthe simulation are in the first row of each group with an
optimized trajectory. We also show corresponding resultsfrom the real world. The green tape contour on the tableindicates the original position of the garment.
Each garment is first segmented from the backgroundand key points are detected from the binary mask, whichtakes 3 − 5 secs on a regular CPU. The algorithm discussed
in Sec. V-A. Given the key points, a corresponding multistep folding plan is created For each garment, we have
optimized trajectories for each folding step. Here, we mapthese optimized trajectories to our scenario according to thegenerated folding plan. Then the Baxter robot follows thefolding plan with optimized trajectories to fold the garment.
We can see that the deformation of the real garment andthe simulated garment is very similar. Therefore, the final
folding outcome is comparable to the simulation.
Table I shows statistical results of the garment folding test.
Each time one or two robotic arms fold the garment counts
as one fold. We run 10 trials for each test garment. It turns
out that the folding performance of the Long-Sleeve T-Shirts
and Towels are very stable with our optimized trajectories.
Jeans and pants are less stable because the shear resistanceof the surface is relatively high, and sometimes is difficult
to bend, leading to unsuccessful folding. In the successful
folding cases for jeans and pants, we sometimes ended up
with small wrinkles, but the folding plan was still able tocomplete successfully. We also show the average time tofold a garment in the last row. The robot is able to fold most
garments in about 1.5 minutes.

Garment Type
L-S T-Shirt (large)
L-S T-Shirt (small)
Jeans
Pants
Large Towel
Medium Towel
Small Towel
Average

# of
folds
3
3
2
2
2
2
2
2.3

Success Rate
10/10
10/10
7/10
8/10
10/10
10/10
10/10
9.3/10Avg. Time
(sec)
121
118
88
88
90
88
83
97

TABLE I. Results of folding test for each garment . We show the number offolding steps, successful rate, and total time of each garment. Each garment
has been tested 10 times. L-S stands for Long-Sleeve. The time is theaverage over all successful trials for each garment.

There is a trade-off between doing contour fitting at eachstep and total time spent to fold a garment. In this work, westart with one template and then assume that each step afterthat the folded garment is close to that in the simulation.
Our results in Table I verify that this method works welland is able to save time since we only do the contour fitting
once. With our simulated trajectories, the Baxter robot isable to fold a garment under predefined steps correctly. An
alternative method could use the contour fitting at each step
but this would require more time and computation.
We note that some failures due to the motor control error
from the Baxter robot. When the robot executes an optimized
trajectory, its arm suffers from a sudden drop or jitter. Suchactions will raise pull forces to the garment, leading todrift and inaccurate folding. This can be solved by using an
industrial level robotic arm with more accurate control. We
also note that failures can be recognized with the correct
sensing suite, and we are currently investigating ways toeffect online error recovery for such failures. One difference
between the simulation and the real world we found is thatmoving a point on the mesh in the simulation is different
from using a gripper to grasp a small area of a real garment
and move it. In the future, we hope to be able to simulate asimilar grasp effect for the trajectory optimization.
VII. C ONCLUSION
In this paper, we propose a novel solution to find an optimal trajectory for manipulation of deformable garments. We
first create a simulation environment that is comparable to
the real world. By minimizing a quadratic objective functionthat measures dissimilarity between simulated folded shapeand user specified shape iteratively, we obtain an optimized
trajectory. The trajectory is then mapped to a real robot
and executed accordingly. Experimental results demonstratethat with our optimized trajectories, the Baxter robot canmanipulate the garment efficiently and accurately.
Acknowledgments We’d like to thank J. Weisz and J.
Varley, for many discussions. We’d also like to thank NVidia
Corporation, Intel Corporation, and Takktile LLC for the
hardware support. This material is based upon work supported by the National Science Foundation under Grant No.
1217904 and in part by the JSPS Postdoctoral Fellowships
for Research Abroad.
R EFERENCES[1] Maya, http://www.autodesk.com/products/autodesk-maya/.

Garment Type
Long-Sleeve T-Shirt (large)
Long-Sleeve T-Shirt (small)
Jeans
Pants
Large Towel
Medium Towel
Small Towel
Average

Stretch
(%)
2.9
2.9
2.9
1.7
2.2
3.1
1.1
2.4

FrictionAngle (◦ )
24.3
24.7
19.1
21.9
18.7
22.3
24.3
22.2

Maya Shear
Resistance
200
200
200
340
260
190
530
274

Maya
Friction
0.7
0.7
0.5
0.6
0.5
0.6
0.7
0.6

Fig. 8. L EFT: A picture of our test garments. R IGHT: Results for each unfolding test on the garments. We show the results of stretch percentage, Friction
angle of the table, and the corresponding parameters in Maya by each test. The last row shows the average of each measurement component.

Fig. 9. Successful folding examples with optimized folding trajectories from off-line simulation. The first row of each group is from the simulation andthe second row is from the real world (Green tape shows the original garment contour position). T OP G ROUP : Long-sleeve shirt folding with 3 steps.
M IDDLE G ROUP : Long pants folding with 2 steps. B OTTOM G ROUP : Medium size towel folding with 2 steps.

[2] Prime Sense, http://en.wikipedia.org/wiki/PrimeSense/.
[3] M. Cusumano-Towner, A. Singh, S. Miller, J. F. OBrien, and P. Abbeel.
Bringing clothing into desired configurations with limited perception.
In Proc. ICRA, 2011.
[4] A. Doumanoglou, T-K Kim, X. Zhao, and S. Malassiotis. Active
random forests: An application to autonomous unfolding of clothes.
In Proc. ECCV, 2014.
[5] G. Farin. Curves and Surfaces for Computer Aided Geometric Design.
Academic Press, 1988.
[6] J. Lei J. Maitin-Shepard, M. Cusumano-Towner and P. Abbeel. Cloth
grasp point detection based on multiple-view geometric cues withapplication to robotic towel folding. In Proc. ICRA, 2010.
[7] Y. Kita, F. Kanehiro, T. Ueshiba, and N. Kita. Strategy for folding
clothing on the basis of deformable models. 2014.
[8] Y. Kita, T. Ueshiba, E-S Neo, and N. Kita. Clothes state recognition
using 3d observed data. In Proc. ICRA, 2011.
[9] Y. Li, C-F Chen, and P. K. Allen. Recognition of deformable object
category and pose. In Proc. ICRA, June 2014.
[10] Y. Li, Y. Wang, M. Case, S-F Chang, and P. K. Allen. Real-timepose estimation of deformable objects using a volumetric approach.
In Proc. IROS, September 2014.
[11] Y. Li, D. Xu, Y. Yue, Y. Wang, S-F Chang, E. Grinspun, and P. K.
Allen. Recognition, regrasping, and unfolding of deformable objects
using predictive thin shell modeling. In Proc. ICRA, May 2015.
[12] K. Madsen, H. B. Nielsen, and O. Tingleff. Methods for non-linear least squares problems (2nd ed.). Technical report, Technical
University of Denmark, 2004.
[13] S. Miller, J. Berg, M. Fritz, T. Darrell, K. Goldberg, and P. Abbeel.
A geometric approach to robotic laundry folding. IJRR, 2012.
[14] S. Miller, M. Fritz, T. Darrell, and P. Abbeel. Parametrized shapemodels for clothing. In Proc. ICRA, Sept. 2011.
[15] J. Nocedal and S. Wright. Numerical Optimization Second Edition.
Springer, 2006.
[16] C. Rother, V. Kolmogorov, and A. Blake. ”grabcut”: Interactive
foreground extraction using iterated graph cuts. ACM Trans. Graph.,23(3):309–314, August 2004.
[17] J. Stria, D. Prusa, and V. Hlavac. Polygonal models for clothing. In
Proc. Towards Autonomous Robotic Systems, 2014.
[18] J. Stria, D. Prusa, V. Hlavac, L. Wagner, V. Petrik, P. Krsek, and
V. Smutny. Garment perception and its folding using a dual-arm robot.
In Proc. IROS, Sept. 2014.
[19] J. van den Berg, S. Miller, K. Goldberg, and P. Abbeel. Gravity-based
robotic cloth folding. In Proc. Intl. Workshop on the Algorithmic
Foundations of Robotics (WAFR), 2010.
[20] L. Vincent and P. Soille. Watersheds in digital spaces: An efficientalgorithm based on immersion simulations. IEEE Transactions on
Pattern Analysis and Machine Intelligence, 13(6):583–598, 1991.
[21] B. Willimon, I. Walker, and S. Birchfield. A new approach to clothingclassification using mid-level layers. In Proc. ICRA, 2013.