ISP: Multi-Layered Garment Draping with Implicit Sewing Patterns
Authors: Ren Li
Benoît Guillard
Pascal Fua
Venue: NeurIPS 2023 poster (2023)
Keywords: garment modeling
draping
deformation
human body modeling

Abstract: Many approaches to draping individual garments on human body models are realistic, fast, and yield outputs that are differentiable with respect to the body shape on which they are draped. However, they are either unable to handle multi-layered clothing, which is prevalent in everyday dress, or restricted to bodies in T-pose. In this paper, we introduce a parametric garment representation model that addresses these limitations. As in models used by clothing designers, each garment consists of individual 2D panels. Their 2D shape is defined by a Signed Distance Function and 3D shape by a 2D to 3D mapping. The 2D parameterization enables easy detection of potential collisions and the 3D parameterization handles complex shapes effectively. We show that this combination is faster and yields higher quality reconstructions than purely implicit surface representations, and makes the recovery of layered garments from images possible thanks to its differentiability. Furthermore, it supports rapid editing of garment shapes and texture by modifying individual 2D panels.

ISP: Multi-Layered Garment Draping with Implicit Sewing Patterns
Ren Li
Benoît Guillard
Pascal Fua
Computer Vision Lab, EPFL Lausanne, Switzerland
ren. www.1@@eff..co
benoit. pooett.guillard@ep1...t
pascal.fua@epfl.
Abstract
Many approaches to draping individual garments on human body models are realistic, fast, and yield outputs that are differentiable with respect to the body shape on which they are draped. However, they are either unable to handle multi- layered clothing, which is prevalent in everyday dress, or restricted bodies in T-pose. In this paper, we introduce parametric garment representation model that addresses these limitations. As in models used by clothing designers, each garment consists of individual 2D panels. Their 2D shape is defined by a Signed Distance Function and 3D shape by a 2D 3D mapping. The 2D parameterization enables easy detection of potential collisions and the 3D parameterization handles complex shapes effectively. We show that this combination is faster and yields higher quality reconstructions than purely implicit surface representations, and makes the recovery of layered garments from images possible thanks its differentiability. Furthermore, it supports rapid editing of garment shapes and texture by modifying individual 2D panels.
Introduction
Draping virtual garments on body models has many applications fashion design, movie-making, virtual try-on, virtual and augmented reality, among others. Traditionally, garments represented by 3D meshes and the draping relies on physics-based simulations (PBS) I 4
6
g
10 II [2] to produce realistic interactions between clothes and body. Unfortunately PBS is often computationally expensive and rarely differentiable, which limits the scope of downstream applications. Hence, many recent techniques use neural networks speed up the draping and
make it differentiable. The garments can be represented by 3D mesh templates [33 14
15
16]
17
18 19 20,
point clouds [2] (2), UV maps 23 24 25 26 27 implicit surfaces [28 29 30
Draping can then be achieved by Linear Blend Skinning (LBS) from the shape and pose parameters of body model, such as SMPL 31
Even though all these methods can realistically drape individual garments over human bodies, none handle multiple clothing layers, despite being prevalent in everyday dress. To address overlapping clothing layers such as those in Fig. [ while preserving expressivity, we introduce an Implicit Sewing Pattern (ISP), new representation inspired by the way fashion designers represent clothes. As shown in Fig. 2 a sewing pattern is made of several 2D panels implicitly represented by signed distance functions (SDFs) that model their 2D extent and conditioned information about how to stitch them into a complete garment. To each panel is associated 2D to 3D mapping representing its 3D shape, also conditioned on The 2D panels make it easy detect collisions between surfaces and to prevent interpenetrations. In other words, the garment is made of panels whose 2D extent is learned instead of having to be carefully designed by human and whose
latent vector z, along with
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

Figure 1: Multi-layered garment draping. Top: Draping multiple layers of garments over one body (left) and modifying the body's shape (right). Bottom: Draping the same set of 5 garments bodies with varying poses and shapes.
3D shape is expressed in unified wy space in which a loss designed to prevent interpenetratioos can easily be written.
This combination enables us to model layered garments such as those of Fig. | while preserving end-to-end differentiability. This lets us drape them realistically over bodies arbitrary poses, to recover them from images, and to edit them easily. Doing all this jointly is something that has not yet been demonstrated the Computer Vision Computer Graphics literature. Furthermore, most data driven draping methods rely on synthetic data generated with PBS for supervision purposes. In contrast, ISPs rely on physics-based approach can handle arbitrary body poses, while only requiring garments draped over bodies in canonical pose training time. Our code available at (ithtttt:/// h/ttps://githbb.. homm/irenn21111
supervisisn of [88 13.. As result, at inference time, our
2 Related Work
Garment Draping. Garment draping approaches can be classified physics-based data-driven.
demanding, while data-driven approaches produce high-quality results but are computationally faster, sometimes the cost of realism. Most data-
Physics-based ones 3]
33
34
35
[2]
18] 20 [6] 18, [3). with a triangulated mesh modeling specific garment and a draping function trained specifically for As this is impractical for large garment collections, some recent works [22] 21 36) use 3D point clouds to represent garments instead. Unfortunately, this either prevents differentiable changes in garment topology or loses the point connections with physical meanings. The approaches of [24] 2.5 replace the clouds by UV maps that encode the diverse geometry of garments and predict positional draping maps. These UV maps are registered to the body mesh, which restricts their interpretability and flexibility for garment representation and manipulation. The resulting garments follow the underlying body topology, and the ones that should not, such as skirts, must be post-processed to remove artifacts. Yet other algorithms 22 30 37 rely on learning 3D displacement fields or hierarchical graphs for garment deformation, which makes them applicable to generic garments.
driven methods are template-based [13
14
15
19

---

While many of these data deta-driven methods deliver good results, they are typically designed to handle single garment or a top and a bottom garment with only limited overlap. An exception the approach of (38] that augments SDFs with covariant fields to untangle multi-layered garments. However, the untangling is limited to garments T-pose. In contrast, our proposed method can perform multi-layered garment draping for bodies in complex poses and of diverse shapes.
Garments Sets of Panels. Sewing patterns are widely used in garment design and manufactur- 41
43]. Typically, flat sewing patterns are assembled and then draped using PBS. To automate pattern design, recent works introduce a parametric pattern space. For example, the methods of 44 45 introduce sparse set of parameters, such as sleeve length or chest circumference, while [27] relies principal component analysis (PCA) to encode the shape of individual panels. It requires hierarchical graphs on groups of panels to handle multiple garment styles By contrast, our approach relies the expressivity of 2D Sign Distance Functions (SDFs) to represent the panels.
ing [39 40
42
To promote the use of sewing patterns in conjunction with deep learning, a fully automatic dataset. generation tool is proposed in [46]. It randomly samples parameters to produce sewing patterns and uses PBS to drape them on a T-posed human body model to produce training pairs of sewing patterns and 3D garment meshes.
Garments Implicit Surfaces. SDFs have become very popular to represent 3D surfaces. How- ever, they are primarily intended for watertight surfaces. A standard way to use them to represent open surfaces such garments is to represent them thin volumes surrounding the actual surface [2] 29
which can be represented by SDFs but with an inherent accuracy loss. To address this issue, in 33) the SDFs replaced by unsigned distance functions (UDFs) while relying on the differentiable
approach of [47] to meshing, in case an actual mesh is required for downstream tasks. However, this requires extra computations for meshing and makes training more difficult because surfaces lie at singularity of the implicit field. This can result in unwanted artifacts that our continuous UV parameterization over 2D panels eliminates.
3 Method
In clothing software used industrially 33 40
garments are made from sets of 2D panels cut from pieces of cloth which are then stitched together. Inspired by this real-world practice, we introduce the Implicit Sewing Patterns (ISP) model depicted by Fig. [ It consists of 2D panels whose shape is defined by the zero crossings of function that takes as input 2D location x (Zu, ze) and a latent vector z, which is specific to each garment. A second function that also takes x and z as arguments maps the flat 2D panel to the potentially complex 3D garment surface within the panel, while enforcing continuity across panels. Finally, we train draping networks to properly drape multiple garments human bodies of arbitrary shapes and poses, while avoiding interpenetrations of successive clothing layers.
3.1 Modeling Individual Garments
We model garments as sets of 2D panels stitched together in a pre-specified way. Each panel turned into a 3D surface using parameterization and the networks that implement this parameterization are trained to produce properly stitched 3D surfaces. To create the required training databases, we use the PBS approach of [46]. Because it also relies on 2D panels, using its output to train our networks has proved straightforward, with only minor modifications required.
Flat 2D Panels. We take a pattern P to be a subset of Ω = [-1, 1]2 whose boundary are the zero crossings of an implicit function. More specifically, we define
Is : Ω x R | + R x N, (x, (s,c),
where Ze is implemented by a fully connected network. It takes as input local 2D location and a global latent vector z and returns two values. The first, s, should approximate the signed distance to the panel boundary. The second, c, is label associated to boundary points and is used when assembling the garment from several panels. Then, boundaries with the same labels should be stitched together. Note that the values are only relevant at the panel boundaries, that is, when s = However, training the network to produce such values only there would be difficult because this would involve a very sparse supervision. So, instead, we train network to predict the label
3

---

Front
Back
Serving
Latent Space Interpolstion
[d]
Pattorn Parameterization
UV Parameterization
(5y-g) 5))
Mr.
Fr
UV queries 12
UV queriee 4M
Sewing
An
Laleni code
Lalent code
(5g Cp)
[e]
Figure 2: Implicit Sewing Pattern (ISP). (a) The 3D mesh surface for shirt with the front surface in gray and the back one in blue. (b) The front and back 2D panels of the ISP. The numbers denote the labels of seamed edges Eg, and indicate how to stitch them when O. (c) We representation for the signed distance and for the edge labels, denoted here by the different colors. (d) Interpolation in the latent space allows topology changes, here from sleeveless shirt long-sleeve open jacket. The top rows show the front and back panels, the bottom row the reconstructed meshes. (e) To parameterize a two-panel garment, train the implicit network Ze predict the signed distance field (18/)) and the edge label field (cf/cs) that represent the two panels. They are mapped 3D surfaces by the network Aa.
implicit neural
of the closest boundary for all points, as shown in Fig. [e), which yields much better behaved minimization problem at training time.
In this paper, we model every garment G' using two panels ps and pb one for the front and one for the back. E and E labels assigned to boundary points. Label denotes unstitched boundary points like collar, waist, or sleeve ends. Labels other than zero denote points in the boundary of one panel that should be stitched a point with the same label in the other.
In practice given the database of garments and corresponding 2D panels we generated using publicly available software designed for this purpose [46), Ia and to associate a latent vector z to each training garment. To this end, we minimize the loss function obtained by summing over all training panels
auto-decoding approach jointly train
Li Σ (((x2))
5** + AcECE(c(xx e0 (x)) + Xreg |/2||
(2)
XEN
with respect to separate latent code per garment and the network weights θ that are used for all garments. CE is the cross-entropy loss, sh and e are the ground-trut truth signed distance value and the label of the closest seamed edge of x, and XCE and Aveg different terms. We handle each garment having a front and back panel pf and pb by two separate network Te, and Ie, with shared latent codes so that it can produce two separate sets of (s, c) values at each (I.) location, one for the front and one for the back.
scalars balancing the influence of the
From 2D panels to 3D Surfaces. The sewing patterns described above flat panels whose role is to provide stitching instructions. To turn them into 3D surfaces that can be draped on a human body, learn mapping from the 2D panels the 3D garment draped on the neutral SMPL body. To this end, we introduce a second function, similar to the one used in AtlasNet (48
As 52 R |z] + R (x, +
(3)

---

It is also implemented by fully connected network and takes as input local 2D location x and a global latent vector z. It returns a 3D position X for every 2D location x in the pattern. A key difference with AtlasNet is that we only evaluate As for points within the panels, that is points for which the signed distance returned by Ia of Eq. (1) is not positive. Hence, there is no need to deform or stretch ue patterns. This is in contrast to the square patches of AtlasNet that had to be, which simplifies the training.
Given the latent codes z learned for each garment in the training database, as described above, we train a separate set of weights & y and i for the front and back of the garments. To this end, we minimize a sum over all front and back training panels of
L LOOD + An Coormat + Achooonsist
(4)
Lconsist =
-ΣΣ I|A0, (x, Ao. (x, eS0XEE
(5)
where LOHD, Enormat, and Cconsist are the Chamfer distance, normal consistency loss, and a loss term whose minimization ensures that the points on the edges of the front and back panels sewn together- E. for c > 0 are aligned when folding the panels in 3D, An and de are scalar weights. This assumes that the front and back panels have their seamed edges aligned in 2D, ie. for > 0 and x E Q2 if has label e on the front panel, then it should also have label e the back panel, and vice-versa. Our experiments show that Lemsiis reduces the gap between the front and back panels.
Meshing and Preserving Differentiability. Ag continuously deforms 2D patches that are implicitly defined by Ie. To obtain triangulated meshes from these, we lift a regular triangular 2D mesh defined on f2 by querying Aq on each of its vertex, as in [48]. More specifically, we first create a square 2D mesh T = {V Fn) for Ω where vertices Vn are grid points evenly sampled along orthogonal axis of Ω and faces Fg are created with Delaunay triangulation. Given the latent code z of a specific garment, for each vertex
Vn, we obtain its signed distance value and edge label e with (s, c) = Ia z). We construct the 2D panel mesh Tp = (Vr, Fr} by discarding vertices of T whose SDF is positive. To get cleaner panel borders, we also keep vertices v E Vi2 with s(v, z) 0 that belong to the mesh edges that cross the 0 issoleeel,, and adjust their positions to (U, z) Vu to project them to the zero level set 49] The front and back panel 2D meshes are then lifted 3D by querying Ag on each vertex. During post-processing, their edges are sewn together to produce the final 3D garment mesh (z).). More details can be found in the Supplementary Material.
Te performs as an indicator function to keep vertices with s S O, which breaks automatic differen- tiability. To restore it, we rely on gradients derived in [50 51 for implicit surface extraction. More formally, assume v is a vertex of the extracted mesh TG and x is the point on the UV space that satisfies As (x z), then, proved in the Supplementary Material,
dv
0.A. (x,
0.
Ōs
z)
Vs(x. z)
Oz (x, z)
(6)
dz Oz
0x
Hence, ISP can be used to solve inverse problems using gradient descent, such as fitting garments to partial observations.
3.2 Garment Draping
We have defined mapping from a latent vector z to a garment draped over the neutral SMPL. model [31], which we will refer to as a rest pose. We now need to deform the garments to conform to bodies of different shapes and in different poses. To this end, we first train a network that can deform single garment. We then train a second one that handles the interactions between multiple garment layers and prevents interpenetrations as depicted in Fig.
Single Layer Draping. In SMPL, body templates are deformed using LBS. As in 19 29 30, we first invoke an extended skinning procedure with the diffused body model formulation to an initial rough estimate of the garment shape. More specifically, given the parameter vectors B and θ that control the body shape and pose respectively, each vertex v of garment mesh Te is transformed by
Uimit W (v(B,)), B, θ w())
((B,)) = + w. (B) + B,,)),,
where W--) is the SMPL skinning function with skinning weights W E Na with NB being the number of vertices of the SMPL body mesh and (B) Nax3 and (0) N x3 are the
5

---

shape and pose displacements returned by SMPL. w(r) E IR are diffused weights returned by neural network, which generalizes the SMPL skinning to any point in 3D space. The training of w(v) is achieved by smoothly diffusing the surface values, as in [29]
This yields initial deformation, such that the garment roughly fits the underlying body. A displace- ment network D3 is then used to refine it. D. is conditioned on the body shape and pose parameters B, O and on the garment specific latent code z. It returns a displacement map D. ( θ z) in UV space. Hence, the vertex Vinit with UV coordinates (x,, F.) in 52 is displaced accordingly and becomes Vinat + Da [ Fe], where [; denotes standard array addressing. D, is implemented as multi-layer perceptron (MLP) that outputs a vector of size 6NN², where N is the resolution of UV mesh T. The output is reshaped to two N x N 3 arrays to produce the front and back displacement maps.
To learn the deformation for various garments without collecting any ground-truth data and train D, in self-supervised fashion, we minimize the physics-based loss from [8]
Sphy Estrain + Lbend + ggravity + LBGcol
(8)
where Estrain is the membrane strain energy caused by the deformation, Lrend the bending energy raised from the folding of adjacent faces, Lgranity the gravitational potential energy, and L the penalty for mody-garment collisions.
Multi-Layer Draping. When draping independently multiple garments worn by the same person using the network D. introduced above, the draped garments can intersect, which is physically impossible and must be prevented. We now show that our ISP model makes that straightforward first in the case of two overlapping garments, and then in the case of arbitrarily many.
Consider outer garment TG and an inner garment T6 with rest state vertices Vo and V. We first drape them independently on a target SMPL body with 2z, as described for single layer draping. This yields the deformed outer and underlying garments with vertices V. and V. respectively. We then rely a second network Dm predict corrective displacements, conditioned on the outer garment geometry and repulsive virtual forces produced by the intersections with the inner garment.
In ISP, garments are represented by mapping 2D panels to 3D surfaces. Hence their geometry can be stored on regular 2D grids on which a convolutional network can act. In practice, we first encode the rest state of the outer garment TV into 2D position map M,. The grid M, records the 31) location of the vertex Uo E V, at its (Fa, coordinate as [M _v] = %% within the panel boundaries, M, _ (0, 0, 0) elsewhere. Concatenating both front and back panels yields N x N x 6 array, that is, 2D array of spatial dimension with 6 channels. After draping, the same process is applied to encode the geometry of Te into position map Ma, using vertices V. instead of V, this time. Finally, for each vertex ig E V.. we take the repulsive force acting on it to be
((,) max (0, (ū: -io) nn:.n;,
where ig is the closest vertex in V. n, is the normal of ü; and represents the dot product. The repulsive force is also recorded in the UV space to generate the 2D force map My. Note that it is 0 for vertices that are already outside of the inner garment, for which no collision occurs. Given the forces My. the garment geometry in the rest state M, and after draping Ma, the network Dm predicts a vertex displacements map Dm = Dm,,,,, (M,, Ma, My) for the outer garment to resolve intersections, as shown in Fig. We replace the vertex To E Vo with coordinates (I), by i = J, + Dm zz]. Dm is implemented as a CNN, and it can capture the local geometry and force information of vertices from the input 2D maps. The training of Dm is self-supervised by
LDm = =Epbay + XyLoGee + XpFreg
(10)
LGGeal max(0, < (1 - 0z) nz)3 and Creg = | (M,, Ma, M2)
where Cphy is the physics-based loss of Eq. 8 Ag and A are weighting constants, and e is a safety margin to avoid collisions. LCGGol penalizes the intersections between the outer and underlying garments, and Freg is an L2 regularization on the output of D...
Given pair of garments, our layering network D,, only deforms the outer garment to resolve collisions, leaving the inner one untouched. Given more than 2 overlapping garments, the process can be iterated to layer them in any desired order over body, as detailed in the Supplementary Material.

---

Multi-Layer Draping
Dos
MG
M
Gammont Force
UDe
Figure 3: Multi-layer Draping. The net- work Dm uses the garment geometry and forces encoded in the input UV maps to resolve garment intersections for multi- layered draping.
Figure 4: Comparison between UDF and our method in resolution 512, 256 and 128. Our reconstructions (top row) are more accurate than those of UDF (bottom row) which have artifacts and uneven borders.
4 Experiments
4.1 Dataset, Evaluation Metrics, and Baseline
To create training and test sets, used the software of 46 to generate sewing patterns and the corresponding 3D garment meshes in their rest state, that is draped over T-Posed body. Our training set comprises 400 shirts, 300 skirts, and 200 pairs of trousers. For testing, use 20 shirts, 20 skirts, and 20 pairs of trousers. As discussed in Section 3.2 we trained the draping models using SMPL body poses O randomly sampled from the AMASS [22 dataset and body shapes B uniformly sampled from 2, 2)10. We use the Chamfer Distance (CHD) and Normal Consistency (NC) to quantify garment reconstruction accuracy, along with the percentage of the garment mesh area that undergoes interpenetrations (Intersection), as (30]
We compare approach against recent and state-of-the-an methods DIG [29] and DrapeNet [30], which, like ours, can drape various garments over bodies of different shapes in arbitrary poses. Like ISPs, DrapeNet is self-supervised using a physics- based loss and can prevent unwanted intersections between top (shirts) and bottom (trousers) garments. By contrast, DIG is a fully supervised method and cannot handle garment intersections.
4.2 Garment Reconstruction
Train
(x10 ())
NC (%.
Time (ms, J)
Test
CHD (x100 (110
NC (%,
UDF 128
0.641
98.86
601
UDF 128
0.803
97.71
UDF 256
0.338
99.20
2379
UDF 256
0,493
98.44
UDF- 512
0.262
98.77
13258
UDF 512
0.424
98.09
Ours Ours 256
0.454
0.579
98.70
0.290
99.37
77
Ours - 256
0.392
98.83
Ours - 512
0.250
99.41
261
Ours - 512
0.349
98.89
Table I: Comparison of our method to UDF on shirts under the resolutions of 128, 256 and 512.
We first consider 3D garments in their rest pose and compare the accuracy of our ISPs against that of the UDF-based representation [30]. In Fig. we provide qualitative results in specific case and for resolutions of Marching Cubes (53) ranging from 512 to 128. Our result is visually superior and more faithful the ground-truth garment, with UDF producing more artifacts and uneven borders, especially at resolution 512. The quantitative results reported in Tab. Too the training and test sets confirm this. For the test set garments, we reconstruct them by optimizing the latent code minimize the Chamfer distance between the predicted and ground truth meshes, using the gradients of Eq. (6). Our approach consistently outperforms UDF all resolutions while being faster, mostly because we evaluate our network 2D implicit field while UDF does it a 3D one. For example, at resolution 256, our reconstruction time is ms on an Nvidia V100 GPU, while UDF requires 2379 ms. Interestingly, increasing the resolution from 256 to 512 improves the reconstruction accuracy

---

Figure 5: Realistic garment draping with method, for different combinations of garments.
of our method, whereas that of UDF's drops. This is because precisely learning the 0 iso-surface of a 3D UDF is challenging, resulting in potentially inaccurate normals near the surface [54] 47 We present similar results for trousers and skirts in the supplementary material.
4.3
Garment Draping
12.31%
None DrapeNet
25.00%
62. 69%
Ours
(a)
Intersection (8t, !)
DrapeNet Ours
0.536
0.276
(b)
(c)
(d)
Figure 6: Draping evaluation. (a) Human evaluation results. None refers to no preference. (b) Percentage of intersecting areas. (c) For each method, left is the draping results for a shirt and a pair of trousers, and right with only the pants. (d) The draping results of our method, DIG and DrapeNet.
Figs. I and 5 showcase our method's ability to realistically drape multiple garments with diverse geometry and topology the body. Our approach handle multi layei- garments, which is not achievable with DrapeNet. Additionally, unlike the method of (38) that is limited to multi-layered garments the T-posed body, our method be applied to bodies in arbitrary poses.
As pointed out in 13 there are no objective metrics for evaluating the realism of draping. Therefore, conducted human evaluation. As in [30], designed website displaying side-by- side draping results generated by our method and DrapeNet for the same shirts and trousers, on the same bodies. Participants were asked to select the option that seemed visually better them, with the third option being "I cannot decide". 64 participants took part in the study, providing total of 884 responses. As shown in Fig. 6 a), the majority of participants preferred 25.00%), demonstrating the higher fidelity of our results. The lower intersection ratio reported in Fig. b further demonstrates the efficacy of our draping model. In Figs. (cc) and (d), compare qualitatively our method against DrapeNet and DIG.
method (62.69% vs.
4.4 Recovering Multi-Layered Garments from Images
Thanks to their differentiability, our ISPs can used to recover multi-layered garments from image data, such as 2D garment segmentation masks. Given image of a clothed person, we can obtain the estimation of SMPL body parameters (B, Q) and the segmentation mask using the algorithms of [56] 57). To each detected garment, associate latent code and reconstruct garment meshes

---

Ours
SMPLicit
ClotwWild
DrapeNet
Ours
SMPLick
ClothwWlld
Figure 7: Garment recovery from images. We compare the meshes recovered by our method and the state of the art methods SMPLicit [28]. ClothWild [55]. DrapeNet 30
(unavailable for skirts).
Madtfed
Medffer
Short Body
Figure 8: Shape editing. Garment attributes can be edited by modifying the sewing pattern.
by minimizing
Zi. argmin LioU (R(G M(B, O)).
(11)
C 2IN)) - D(B, O, zi, To(z))) @ + ( θ zN, To(ZN)),
where Lua is the loU loss [58] over the rendered and the given mask, R(-) is a differentiable renderer [59] N is the number of detected garments, and represents the operation of mesh concatenation. M the SMPL. body mesh, while g θ, 21:N) ) is the concatenation of garment meshes reconstructed from the implicit sewing pattern (2,)) and then draped by draping model 'D Dn DD, In practice, the minimization is performed from the outermost garment to the innermost, one by one. Further details can be found in the Supplementary Material.
Fig. 7 depicts the results of this minimization. Our method outperforms the state-of-the-an methods SMPLicit (28). Wild [55] and DrapeNet (30), given the same garment masks. The garments recover exhibit higher fidelity and have no collisions between them or with the underlying body.
4.5 Garment Editing
As the panel shape of the sewing pattern determines the shape of the garment, we can easily manipulate the garment mesh by editing the panel. Fig. depicts this editing process. We begin by moving the sleeve edges of the panel inwards to shorten the sleeves of the mesh, then move the bottom edges up to shorten the length the jacket, and finally remove the edges for the opening to close it. This achieved by minimizing
[(z)= d(E()), É). where F(z) [x|s(x, 2) = 0, 0.x
(12)
w.r.t. d(-) computes the chamfer distance, E represents the edges of the modified panel, and Fl() represents the iso-level extracted from Ie, that is, the edges of the reconstructed panel. Our sewing pattern representation makes it easy to specify new edges by drawing and erasing lines in

---

the 2D panel images, whereas the fully implicit garment representation of [30 requires an auxiliary classifier identify directions in the latent space for each garment modification. As shown in the supplementary material, the texture of the garment can also be edited by drawing on the UV panels.
5
Conclusion
We have introduced a novel representation for garments to be draped on human bodics. The garments made of flat 2D panels whose boundary is defined by 2D SDF. To each panel is associated a 3D surface parameterized by the 2D panel coordinates. Hence, different articles clothing are represented in standardized way. This allows the draping of multi-layer clothing bodies and the recovery such clothing from single images. Our current implementation assumes quasi-static garments and only deforms the outer garment solve collisions in multi-layer draping. In future work, we will introduce garment dynamics and focus on more accurate physical interactions between the outer and inner garments.
Acknowledgement. This project was supported in part by the Swiss National Science Foundation.
References
[1] Xavier Provot et Deformation constraints in a mass-spring model to describe rigid cloth behaviour. In Graphics interface, 1995.
[2] X. Provot. Collision and Self-Collisie Handling in Cloth Model Dedicated to Design Garments. In Computer Animation and Simulation, 1997.
[3] D. Baraff and A. Witkin. Large Steps in Cloth Simulation. In ACM SIGGRAPH, pages 43-54, 1998.
[4] T. Vassilev, B. Spanlang, and Chrysanthou. Fast cloth animation on walking avatars, In Computer Graphics Forum, 2001.
[5] C. Zeller. Cloth simulation the gpu. In ACM SIGGRAPH, 2005.
[6] M. Tang, R. Tong. R. Narain, C. Meng, and D. Manocha. A GPU-based streaming algorithm for high-resolution cloth simulation. In Computer Graphics Forum, 2013.
[7] T. Liu, S. Bouaziz, and L. Kavan. Quasi-newton methods for real-time simulation of hyperelastic materials, ACM Transactions on Graphics, 2017.
[8] Nvidia. Nveloth, 2018.
[9] Optitext Fashion Design Software, 2018 httpp://optitex..com
[10] Nvidia. NVIDIA Flex, 2018 Lutp:://teveloper..vddia com/flex
[11] M. Designer, 2018. https:: / ptttps://www.marrte www.marvallosddsiigerrrrr
[12] Tongkui Su, Yan Zhang, Yu Zhou, Yao Yu, and Sidan Du. GPU-based Real-time Cloth Simulation for Virtual Try-on. Pacific Conference on Computer Graphics and Applications, 2018.
[13] H. Bertiche, M. Madadi, and S. Escalera. PBNS: Physically Based Neural Simulation for Unsupervised Garment Pose Space Deformation. ACM Transactions on Graphics, 2021.
[14] B. L. Bhatnagar, G. Tiwari, C. Theobalt, and G. Pons-Moll. Multi-Garment Net: Learning to Dress 3D People from Images. In International Conference Computer Vision, 2019.
[15] B. Jiang, Zhang, Y. Hong. J. Luo, L. Liu, and H. Bao. Benet: Learning body and cloth shape from a single image. In European Conference on Computer Vision, 2020.
[16] X. Pan, J. Mai, X. Jiang, D. Tang, J. Li, T. Shao, K. Zhou, X. Jin, and D. Manocha. Predicting loose-fitting garment deformations using bone-driven motion networks. ACM SIGGRAPH, 2022.
10

---

[17] C. Patel, Z. Liao, and G. Pons-Moll. Tailomet: Predicting clothing in 3d as function of human pose, shape and garment style. In Conference on Computer Vision and Pattern Recognition. 2020.
[18] I. Santesteban, M.A. Otaduy, and D. Casas. SNUG: Self-Supervised Neural Dynamic Garments. In Conference on Computer Vision and Pattern Recognition, 2022.
[19] I. Santesteban, N. Thuerey, M. A. Otaduy, and D. Casas. Self-Supervised Collision Handling via Generative 3D Garment Models for Virtual Try-On. In Conference on Computer Vision and Pattern Recognition, 2021.
[20] G. Tiwari, B. L. Bhatnagar, T. Tung, and G. Pons-Moll. Sizer: A Dataset and Model for Parsing 3D Clothing and Learning Size Sensitive 3D Clothing. In European Conference on Computer Vision, 2020.
[21] H. Bertiche, M. Madadi, E. Tylson, and S. Escalera. DeePSD: Automatic Deep Skinning and Pose Space Deformation for 3D Garment Animation. In International Conference on Computer Vision, 2021.
[22] E. Gundogdu, V. Constantin, S. Parashar, A. Seifoddini, M. Dang, M. Salzmann, and P. Fua. Garnet++: Improving Fast and Accurate Static 3D Cloth Draping by Curvature Loss. IEEE Transactions on Pattern Analysis and Machine Intelligence, ) 181):1811195, 2022.
2022.
[23] Z. Lahner, D. Cremers, and T. Tung. Deepwrinkles: Accurate and Realistic Clothing Modeling. In European Conference on Computer Vision. September 2018.
[24] Y. Shen, J. Liang, and M.C. Lin. Gan-based garment generation using sewing pattern images. European Conference on Computer Vision, 2020.
[25] Z. Su, T. Yu, Y. Wang, and Y. Liu. DeepCloth: Neural Garment Representation for Shape and Style Editing. IEEE Transactions Pattern Analysis and Machine Intelligence, 45(2):1881- 1593, 2022.
[26] M. Zhang, D. Ceylan, and N. Mitra. Motion Guided Deep Dynamic 3D Garments. ACM Transactions on Graphics, 4 41(6):1-12, 2022.
[27] X. Chen, G. Wang, D. Zhu, X. Liang, P. Torr, and I., Lin. Strueture-Preserving 3D Garment Modeling with Neural Sewing Machines. In Advances in Neural Information Processing Systems, 2022.
[28] E. Corona, A. Pumarola, G. Alenya, G. Pons-Moll, and F. Moreno-Noguer. Smplicit: Topology- Aware Generative Model for Clothed People. In Conference on Computer Vision and Pattern Recognition, 2021.
[29] R. Li, B. Guillard, E. Remelli, and P. Fua. DIG: Draping Implicit Garment over the Human Body. In Asian Conference on Computer Vision, 2022.
[30] Luca DeLuigi, Ren Li, Benoît Guillard, Mathieu Salzmann, and Pascal Fua. DrapeNet: Gener- ating Garments and Draping them with Self- self-Supervision. In Conference on Computer Vision and Pattern Recognition, 2023.
[31] Bogo, A. Kanazawa, C. Lassner, Gehler, J. Romero, and M. J. Black. Keep SMPL: Automatic Estimation of 3D Human Pose and Shape from Single Image. In European Conference on Computer Vision, 2016.
[32] Y. Li, M. Habermann, B. Thomaszewski, S. Coros, T. Beeler, and C. Theobalt. Deep physics- aware inference of cloth deformation for monocular human performance capture. In Interna- tional Conference on 3D Vision, 2021.
[33] J. Liang, M. Lin, and V. Koltun. Differentiable Cloth Simulation for Inverse Problems. In Advances Neural Information Processing Systems, 2019.
[34] R. Narain, Samii, and J.F. O'brien. Adaptive anisotropic remeshing for cloth simulation. ACM Transactions Graphics, 2012.

---

[35] R. Narain, T. Pfaff, and J.F. O'Brien. Folding and crumpling adaptive sheets. ACM Transactions on Graphics, 2013.
[36] I. Zakharkin, K. Mazur, A. Grigorev, and V. Lempitsky. Point- modeling of human clothing. In International Conference on Computer Vision, 2021.
[37] A. Grigorev, B. Thomaszewski, M.J. Black, and O. Hilliges. HOOD: Hierarchical Graphs for Generalized Modeling of Clothing Dynamics, In Conference Computer Vision and Pattern Recognition, 2023.
[38] I. Santesteban, M.A. Otaduy, N. Thuerey, and D. Casas. Ulnef: Untangled layered neural fields for mix-and-match virtual try-on, In Advances in Neural Information Processing Systems, 2022.
[39] Clo 3d. attps://www.clon comm://mcooddddccm/// com/en/
[40] Browzwear. https://browzwear com/
[41] Marvellous designer.
http///wwmareelosusssssiieeeeeee com
[42] N. Umetani, D. M. Kaufman, T. Igarashi, and E. Grinspun. Sensitive Couture for Interactive Garment Editing and Modeling. ACM SIGGRAPH, 30(4), 2011.
[43] A. Kaspar, K. Wu, Y. Luo, L. Makatura, and W. Matusik. Knit Sketching: from Cut & Sew Patterns to Machine- Knit Garments. ACM Transactions on Graphics, 40(4):1-15, 2021.
[44] T. Y. Wang, D. Ceylan, J. Popovic, and N. J. Mitra. Learning a Shared Shape Space for Multimodal Garment Design. In ACM SIGGRAPH Asia, 2018.
[45] R. Vidaurre, I. Santesteban, E. Garces, and D. Casas. Fully Convolutional Graph Neural Networks for Parametric Virtual Try-On. In Computer Graphics Forum, pages 145-156, 2020.
[46] M. Korosteleva and Lee. Generating Datasets of 3D Garments with Sewing Patterns. In Advances in Neural Information Processing Systems, 2021.
[47] B. Guillard, F. Stella, and P. Fua. MeshUDF: Distance Field Networks. In European Conference on Computer Vision, 2022.
Fast and Differentiable Meshing of Unsigned
[48] T. Groueix, M. Fisher, V. Kim, B. Russell, and M. Aubry. Atlasnet: A Rapier-Mäché Approach Learning 3D Surface Generation. Conference on Computer Vision and Puttern Recognition, 2018.
[49] Chibane, A. Mir, and G. Pons-Moll. Neural Unsigned Distance Fields for Implicit Function Learning. In Advances in Neural Information Processing Systems, 2020.
[50] E. Remelli, A. Lukoianov, S. Richter, B. Guillard, T. Bagautdinov, P. Baque, and P. Fua. Meshsdf: Differentiable Iso-Surface Extraction. Advances in Neural Information Processing Systems, 2020.
[51] B. Guillard, E. Remelli, A. Lukoianov, S. Richter, Bagautdinov, P. Baque, and P. Fua. Deepmesh: Differentiable Iso-Surface Extraction. In arXiv Preprint, 2022.
[52] N. Mahmood, N. Ghorbani, N. F. Troje, G. Pons-Moll, and M. J. Black. AMASS: Archive of Motion Capture as Surface Shapes. In International Conference on Computer Vision, pages 5442 5442-5451, 2019.
[53] W.E. Lorensen and H.E. Cline. Marching Cubes: High Resolution 3D Surface Construction Algorithm. In ACM SIGGRAPH, pages 163-169, 1987.
[54] R. Venkatesh, T. Karmali, S. Sharma, A. Ghosh, R. V. Babu, L., A. Jeni, and M. Singh. Deep Implicit Surface Point Prediction Networks. In International Conference on Computer Vision, 2021.
[55] G. Moon, H. Nam, Shiratori, and K.M. Lee. 3d clothed human reconstruction the wild. In European Conference Computer Vision, 2022.
12

---

[56] Yu Y. Rong, T. Shiratori, and H. Joo. Frankmocap: Fast monocular 3d hand and body mo- tion capture by regression and integration. In International Conference on Computer Vision Workshops, 2021.
[57] P. Li, Y. Xu, Y. Wei, and Y. Yang. Self-Correction for Human Parsing, IEEE Transactions Pattern Analysis and Machine Intelligence, 44(6):3260 -011100--11--- 2020.
[58] R. Li, M. Zheng, S. Karanam, T. Chen, and Z. Wu. Everybody Is Unique: Towards Unbiased Human Mesh Recovery. In British Machine Vision Conference, 2021.
[59] Nikhila Ravi, Jeremy Reizenstein, David Novotny, Taylor Gordon, Yen Lo, Justin Johnson, and Georgia Gkioxari. PyTorch3D, htttg////gtttbbb.cco//ccbbeeeeee pytorch3d 2020.
[60] M. Loper, Mahmood, J. Romero, Pons-Moll, and M.J. Black. SMPL: A Skinned Multi- Person Linear Model. ACM SIGGRAPH Asia, 34(6), 2015.
13

---

Appendix
Additional Results
Texture Editing
A.2 Garment Reconstruction
A.3 Garment Draping
A.4
Multi-Layer Draping
B
Technical Details
B. I
Sewing Patterns for Trousers and Skirts
B.2
Mesh Triangulation
B.3
Proof of the Differentiability ISP
B 4
Garment Draping
B.5
Recovering Multi- Garments from Images
B.6
Loss Terms, Network Architectures and Training
C Extension to Sewing Patterns with More Panels
D Failure Cases
A
Additional Results
A.1 Texture Editing
(1]
(-)
Figure 9: Texture editing, (a) A smiling face drawn on the front panel. (b) A flower pattern painted on both the front and the back panels. (c) A pocket drawn for the left shirt can be transferred to different garments.
As illustrated in Fig. 9 with our ISP, the texture of the garment can be easily edited by drawing on the UV panels. The figures drawn the panel of one garment can be directly transferred to others as shown by Fig. c). since panels are defined on the same UV space.
A.2
Garment Reconstruction
A.2.1
Evaluation Results
In Tabs. and 3
we report the reconstruction results for skirts and trousers the training and the test set. Similar to the results on shirts shown in the main paper, our method achieves better reconstruction quality than UDF [30) with lower CHD and higher NC at all resolutions, and needs less time to
14

---

Truin
CHD (x1001.. ()
NC (5c, f)
Time (ms, !)
Test
CHD (x10*...
NC (%;, 1)
UDF 128
0.714
98.53
690
UDF 128
0.734
97.79
UDF 256
0.408
98,93
2784
UDF-256
0.403
98
UDF 512
0.331
98.43
17362
UDF 512
0,324
98.27
Ours 128
0.538
98.80
25
Ours - 128
0,583
98.62
Ours 256
0.321
99.08
256
Ours 512
0.267
99.14
262
Ours 512
0.304
98.89
Table Comparison of our method to UDF on skirts under the resolutions of 128, 256 and 512.
Train
CHD (x10-1.)) .)
NC (Sc, 1)
Time (ms, J)
Test
CHD (x (x10-)..)
NC (Si, 1>
UDF 128
0.758
98.22
661
UDF 128
0.752
97.41
UDF 256
0.425
UDF 512
0.342
98.33
17141
UDF 512
0.350
97.63
Ours 128
0.545
98.28
25
Ours 128
0.529
98.03
Ours 256
0.363
98.59
Ours 256
0,346
98.31
Ours 512
0.317
98.62
269
Ours
0.300
98.32
Table 3: Comparison of our method to UDF on trousers under the resolutions of 128, 256 and 512.
reconstruct a single mesh. Figs. 10 12 shirts, skirts and trousers respectively.
show the qualitative results reconstructed by our method for
A.2.2 Latent Space Interpolation
In Figs. to we display the results of interpolation in the latent space of shirts, skirts and trousers respectively. We observe smooth transformation in both the reconstructed sewing patterns and the garment meshes, despite the different topology and geometry of the given garments.
A.2.3 Comparison with AtlasNet
which learns to deform square patch. AtlasNet struggles to learn mapping function capable of accurately deforming the square patch to produce a surface that matches the ground truth, especially in the collar region. In contrast, our method leverages the pattern parameterization network Ie to simplify the training of our mapping function Aq. Specifically, our approach only requires learning the mapping for points within the panels, resulting in reconstruction that is more faithful the ground truth.
In Fig. we compare our method with AtlasNet 48
A.2.4 Ablation Study
To investigate the impact of the loss terms of Eq. (4) in the main paper reconstruction quality, we performed an ablation study. We report the results of reconstructing 300 skirts using models trained with and without Enormal in Tab. 4 and increases the NC, thus improving the reconstruction accuracy. In contrast, without Lnormat- the
We observe that training with Lnormal reduces the CHD
Figure 10: Reconstruction samples of shirts.
15

---

Figure 11: Reconstruction samples of skirts.
Figure 12: Reconstruction samples of trousers.
b
Figure 13: Interpolation. We interpolate the latent code from a sleeveless shirt to long-sleeve jacket. (a) and (b) show the reconstructed front and back panels, where the colors on them denote the edge label fields. (c) shows the reconstructed mesh.
CHD (x10-4. ()
NC (%, f)
w/o Lnormat
0.324
98.69
w/ Enormal
0.321
99.08
Table 4: Comparison of the results reconstructed by the models trained w/o and w/ Lnormat-
16

---

(a)
[b]
(c)
Figure 14: Interpolation. interpolate the latent code from a short tight skirt to a long loose skirt. (a) and (b) show the reconstructed front and back panels, where the colors them denote the edge label fields. (c) shows the reconstructed mesh.
(a)
WMMMA
(b)
(c)
Figure 15: Interpolation. We interpolate the latent code from a pair of short trousers a pair of long trousers. (a) and (b) show the reconstructed front and back panels, where the colors on them denote the edge label fields. (c) shows the reconstructed mesh.
17

---

TTT GT
AtlasNet
Ours
Figure 16: Comparison with AtlasNet. The meshes in black boxes are the source patches where the mapping function is applied: a square patch for AtlastNet, a garment panel for our method.
O
GT
Enorm
foorm
Figure 17: Ablation on Cnormat. Train-Figure 18: Ablation on Lconsist- The skirts reconstructed ing without yields flipped triangle by the models trained (a) with Cconsist and (b) without faces (shown in dark grey).
Lconsiat- The latter has gap between its front and back panels.
model fails to learn the correct parameterization for the garment, resulting in a mesh with inverted faces as illustrated in Fig. 17 Fig. 18 shows a comparison of the results obtained by training models without and with Lconsist to help stitch the front and back panels. We can notice that without Lconsists a spatial gap exists between the front (in gray) and back (in cyan) surfaces.
A.2.5 Vizualization of the Chamfer Distance
A visualization of the spatial error distribution be found in Fig. 19 In this figure, compare the error distribution between our reconstructions and those produced by UDFs. Our reconstructions exhibit lower error across the entire surface compared to UDFs.
A.3 Garment Draping
Fig. and Fig. 21 aa present additional comparisons of draping results for our method, DIG 29 and
DrapeNet [30] Our results show higher fidelity and fewer artifacts compared the other methods.
We also compare our method with SNUG [18), a self-supervised method that relies on mesh templates for garment representation and trains one network for each clothing item. In Fig. b)),), show the qualitative results on the same shirt. Our results are either comparative or visually superior to those of SNUG, despite using a single draping network for a whole garment category.
A.4 Multi-Layer Draping
To assess the effectiveness of our the intersection ratio between different layers of garments, and between the garments and the body. We used the 673 unseen poses from the test set of AMASS dataset [52] and 673 shape parameters uniformly sampled from the continuous space [-2, 20 to generate 673 unique unseen bodies. For each body, we randomly generated five unseen garments (one skirt or one pair of trousers, with four shirts) by interpolating garment latent codes and draped them on the body. The first layer contained the skirt trousers, while the shirts were the following layers.
model, we conducted an experiment to measure
18

---

A
(x)
(b)
(c)
(d)
(c)
Figure 19: Comparative analysis of mesh reconstruction and spatial error distribution for shirt, skirt and trousers. (a) represents the ground truth mesh. (b) and (d) illustrate the mesh reconstruction and corresponding spatial error distribution for our proposed method. (c) and (e) display the same for UDFs. Each row provides both front (upper half) and back (lower half) views. In (d) and (e), error magnitude is indicated by color gradation, with white representing large errors and blue small errors.
The corresponding results are shown in Tabs. and 6 where the diagonal values represent the intersection ratio between the body and the i-th (2 1, 2, 3, 4, 5) layer, and the other values represent the intersection between different layers garments. is important to note that we cannot fairly compare these numbers to any other method, they do not support layering. The left and right sides of the tables correspond to the results obtained with and without the layering procedure, respectively. As shown in the tables, the significantly lower intersection ratios the left side demonstrate the effectiveness our model in handling intersections with the body and between different garments. Another noteworthy observation is that the intersections tend increase we layer more garments. This behavior explained in Appendix [B.4 where we clarify that our multi-layer draping model D,,. trained solely on garments obtained through single-layer draping, which relatively close the body. However, it worth mentioning that even after five layers, the intersections still remain at low level (~2%).
Technical Details
B.1 Sewing Patterns for Trousers and Skirts
In Fig. 22 show the sewing patterns used in experiments for trousers and skirts.
19

---

Ours
DrapeNet
Figure 20: The comparison of draping results of our method and DrapeNet.
Ours
Ours
DIG
SNUG
Figure 21: The comparison of draping results for (a) our method vs. DIG, and (b) our method vs. SNUG.
w/ D...
2
3
4
w/o D...
1
2
5
0.000
0.067
0.001
0.000
0.000
1
0.000
0.456
0.340
0.357
0.233
0.003
0.633
0.608
0.636
2
0.004
28.033
26.378
26.246
0.028
1.224
1.158
0.046
24.729
24.679
0.019
2.449
0.073
27.821
0.035
5
0.052
Table 5: Evaluation of intersections with the underlying body and between garments on different layers (the 1st layer is skirt). Left and right are the results of draping with and without the multi-layering network D... respectively. The unit is %.
20

---

w/ Dm
2
3
4
5
w/o Dm
2
5
0.185
0.048
0.000
0.000
0.000
1
0.185
0.225
0.259
0.223
0.134
0.003
0.631
0.588
0.637
2
0.004
28.033
26.378
26.246
0.030
1.227
1.161
0.046
24.729
24.679
0.020
2.426
0.073
27.821
0.028
5
0.052
Table 6: Evaluation of intersections with the underlying body and between garments on different layers (the 1st layer is pair of trousers). Left and right the results of draping with and without the multi-layering network D,, respectively. The unit is %.
Front
Back
3
3
Front
2
Back
I
Sewing Pattern
Sewing Pattern
(a)
Figure 22: The sewing patterns for (a) trousers and (b) skirts. The front mesh surfaces are in gray and the back ones in blue.
<uugcco
Tr,
Ts
Front
100-3 -0
As
Ta
Ts.
Back
Aa
Meshing Process. Starting with square 2D mesh T, first extract the front and back panel meshes Tp, and Tp, using the implicit function Ia. Then we lift Tp, and Tp, to 3D to get the surface meshes Ts, and Ts, by querying As, and As, their vertices respectively.
Figure 23:
B.2 Mesh Triangulation
In this section, we detail the meshing process of ISP. We first create a square 2D mesh T for Ω as shown on the left of Fig. 23 Given the latent code z of a specific garment, for each vertex E Vn, we compute its signed distance value and edge label with (s, panel meshes Tp, and Tp, are constructed by keeping vertices of with negative signed distance (the blue region of the colored gird in Fig. 23 and those that have positive signed distance but belong to the edges crossing the 0 iso-level (the gray region of the colored gird in Fig. 23 For the later ones, adjust their positions from to i blue line). Finally, we query As, and As, on each vertex of T P, and T Tr respectively to lift them 3D, giving the front and back surfaces Ts, and Ts, as shown the right of Fig. 23
Ie (v, z). The 2D front and back
s(v, Vs(c, z) to project them to the zero level set (the
To sew the lifted front and back surfaces Ts, and T S. we perform triangulation with the help of panel meshes Tp, and Tp. As illustrated in Fig. 24 group the boundary vertices of Tp, and
21

---

Berore
E
Triangulation
Merge
Tn
Ty. and Ts.
Serving
Figure 24: Sewing process. Left: the boundary vertices belonging to the sewing edges E and E marked in red. Middle: The triangulation is performed between Ey and E to create new faces Fee. Right: The mesh generated by merging the front and back surfaces Ts, and Tss with Fsew-
Tr, whose predicted labels are the same (c, with >> 0) to form the sewing edges E and E for the front and back panels separately. Then we create faces Fsew between the vertices of E/ and E and use F Feew to merge the meshes of Ts, and Ts, which gives us the final assembled garment mesh.
B.3 Proof of the Differentiability of ISP
According to the Theorem I of [50], for an SDF & and the point X lying on the iso-level / = {q|>(q.2) z) 0, q E 22, have
Oxo -s((x) z).
(13)
Ōs
For point x lying on the a iso-level, we can have s(x, z) = a, where a is constant. Let 8a = a, then x lies on the 0 iso-level of 8a. Based on Eq. 13 we can have
8x asa
V8a (x, (s(
(14)
Assume is a vertex of the mesh Tc reconstructed by ISP and x is the point on the UV space that satisfies (x z), then holds that
Ov
dAs
+ 0.D Ox
(x,
δx Oz (x. 2),
(15)
dz
Oz
dAp
OAp 0x asa Ōs
(x, 2) +
(x, 2).
(16)
Oz
Ox Osn
Os Oz
Since Den = 1, we substitute Eq. 14 into Eq.
16 to derive that
8s
Ov
AAp
0Ag Vs(x, 2)
Os (x, (x.
(x, z)
(17)
Oz
Oz
8x
Oz
B.4
Garment Draping
Single Layer Draping. In Fig. 25 illustrate the pipeline for the single layer draping, which relies on the diffused LBS of SMPL. [60] to get the initial rough estimate of the garment shape and the displacement map output by D. refine it.
Multi-Layer Draping. Our layering network Dm can be applied to multiple garments iteratively to resolve collisions between them. More specifically, consider K garments [G,, G2 Gx] that are already draped individually by single layer draping network D.. Their subscripts denote their draping order, with smaller ones being closer to the body. We obtain their rest state maps [M, MM..... M as described in Sec. 3.2 of the main paper, and apply Algorithm for layering them by iterating on garments following their draping order.
Note that we only train D,, on one pair of garments individually draped by D,. Therefore, it can only resolve the intersections happening at the same layer, which leads to an extra inner loop in Algorithm
22

---

Single Layer Draging
Diffused LBS
D
Single layer draping. The rest-state garment is first deformed by the diffused LBS to get the initial shape, and then refined by the displacement maps predicted by D..
Figure 25:
Figure 26: Layering. (a) We drape red and an orange shirts on the body. A third one (yellow) is draped by (b) Algorithm I and (c) by naively applying Dn, it.
Algorithm 1: Multi-layer Draping
Require:
Function ForceMap 6) that computes the force map for by taking b the underlying layer; Function PositionMap (a) that computes the 2D position map of a; Layering network Dm.-
Input :An ordered set of garments G2, Gx]; Rest state maps for each garemnts [M,., Mf,.., ..., M.
Output :layered garments (G1, G2, Gx} without intersections.
for 1 to K do
G G;;
for j f + to K do
M2 ForceMap (G2, G.)
/* Eq. (9) in the main paper +/
Ma f PositionMap (G,);
Update vertex positions of Gj by Dm (MF, MA. Mf):
that moves all the subsequent garments to the same j-th layer. Otherwise, intersections cannot be completely resolved when applying Dm to two garments lying on the different layers as illustrated in Fig. c).
B.5 Recovering Multi-Layered Garments from Images
Figure 27: Segmentation masks obtained from [57]. Jackets are marked in gray, shirts in purple, skirts dark purple, trousers in red, and body parts in other colors.
In Fig. 27 we show the segmentation masks used for the optimization Eq. (11) in the main paper. The opumization is performed from the outer garment to the inner one, i.e., jacket (if detected) - shirt + trousers (or skirt). More specifically, for the first example shown in Fig. 27 we have detected segmentation mask S, S2 and S3 for the jacket, the shirt and the trousers respectively. We first initialize latent code 2₁ for the jacket and perform the following optimization recover its mesh
z arg min Lio (BG M(B, O)),
(18)
23

---

Ta(Zi))
(19)
Note that the rendered mask is obtained by setting the colors of the jacket and the body mesh vertices to white and black respectively. Then we fix 2 and initialize a new latent code 2g for the shirt and perform
z arg min LoU (R(G θ Z1:2
(20)
(B O 212) D(B O 21, Ta(zi)) D(B, 22, Tz(2))),
(21)
recover the mesh of the shirt. The optimization for the trousers is similar: fixing 21 and 22: initializing the latent code 23 for the trousers; performing
Z3 = arg min LiU [ O 213) @ O)).
(22)
G [B O (1:3 D(B, θ, 21, Ta(21)) @ D(B, 0, Ta(2))) @ D(B, 0, To(zs)).
(23)
B.6 Loss Terms, Network Architectures and Training
Loss Terms. The Chamfer distance loss LOHD of Eq. (4) in the main paper is formulated as
LCHD
E min min [x, x XES
min
[A,
(24)
xEP
XEs
where the panel and is the ground truth surface mesh.
The normal consistency loss fonormat of Eq. (4) in the main paper is formulated as
Lnormat
Σ ny (x 2)) nx-) + Σ (1 (x((xx"",
2)) nx),
(25)
XEFep
XePes
X argmin
X - Ax z)
x argmin
X - (x,
(26)
XEFes
xEFep
where Fep and Fes are the face centers of the panel mesh and the surface mesh, and nx represents the normal X. ny (.) is the function that computes the normal for the face that Ap (x, z) belongs
to.
Network Architectures. For each garment category, i.e. shirts, skirts, and trousers, we train one separate set of networks [Ia.Ao, D.). Dm is shared by all garment categories. Our models implemented as the following.
Pattern parameterization network Ie: We use two separate networks Ie, and Ie, to learn the pattern parameterization for the front and back panels. Each of them is implemented as MLP with Softplus activations.
UV parameterization network As: We use two separate networks As, and As, learn the UV parameterization for the front and back surfaces. Both of them have the same architecture, which a 7-layer MLP with skip connection from the input layer to middle and Softllus activations.
Latent code 2: The dimension is 32.
Diffuse skinning weight model A 9-layer MLP with leaky ReLU activations and an extra Softmax layer at the end normalize the output.
Displacement network D.: 10-layer MLP with a skip connection from the input layer to the middle and leaky ReLU activations.
Layering network Dm A U-Net with 4 convolution blocks and 4 deconvolution blocks.
Training. We use the Adam ? optimizer for training our networks. The batch sizes, the learning rates and the numbers of iterations for training are summarized Table. 7 The hyperparameters of the training losses are summarized in Table. 8 As. w and D,, are trained with a TESLA V100 GPU, while D, is trained with GPUs. During the training of Dm, we randomly select two garments as the outer and inner layers, and let the model Iearn to resolve intersections between them.

---

Network
Te
A
Dm
Learning Rate
10-4
10-1
10-4
x 10
10-4
Batch Size
50
50
6000
30
6
Iterations
70000
70000
2000
20000
30000
Table 7: Training hyperparameters.
LI
A
Dm
Loss
ACE
Xreg
An
de
Xg
X
Value
0.01
0.001
0.01
0.5
0.005
Table 8: Training loss hyperparameters for LI, LA and LD...
rbsleeve
rfsleeve
back
front
Ibsleeve
Ifsleeve
(a)
(b)
Figure 28: The 6-panel sewing pattern. (a) The 3D mesh surface for shirt. The corresponding surface for each panel is denoted in different colors. (b) The six 2D panels for the front, the back, the right front/back sleeves and the left front/back sleeves.
Figure 29; Reconstruction with panels. For both the left and right examples, we show the reconstructed panels the top and the reconstructed surfaces and the sewed meshes at the bottom. Colors on panels denote edge labels predicted by Ie-
C Extension to Sewing Patterns with More Panels
In experiments, each garment's sewing pattern consists of panels, the front and the back. However, our ISP can be extended to patterns with any number of panels. For example, we can train Ie and Ag database of sewing patterns with six panels shown in Figure 28 using the same training protocol described in the main paper. After training, we can use them to reconstruct the panels and surfaces and produce the sewed mesh illustrated Fig. 29 Adding more panels does not result in better reconstructions. For this reason and for the sake of simplicity, we use model with panels our default setting.
D Failure Cases
Fig- 30 presents draping results as the number of shirts increases. We observe that the model produces unrealistic deformation when the number of shirts is greater than four. This behavior occurs because our multi-layer draping model D,, only trained on garments obtained by single layer draping as described in Section 3.2 of the main paper. In this scenario, the garments relatively close to
25

---

Figure 30: Draping increasingly many shirts: from 1 (left) to 7 (right).
the body. When applied to cases with more shirts (typically over four), the model may generate unpredictable results with the shirts moving far away from the body. However, we note that this issue can be resolved by finetuning the model progressively on layered garments. We also consider that wearing more than four shirts is not common scenario.
26


Reviews:
[{"review_id": "eeygB6soh0", "reviewer": "Reviewer_kV1W", "summary": "The article introduces a 3D clothing generation and driving method inspired by the traditional clothing production process, which has made certain contributions to the reconstruction of clothing, especially multi-layer clothing. This method can achieve the reconstruction of multiple pieces of clothing, the editing of clothing shape and texture, as well as the driving and collision detection processes of clothing through two-dimensional to three-dimensional mapping. The feasibility of this method can be demonstrated through training and validation on the synthesized dataset.", "questions": "According to the description in section 2.5 of the supporting materials, it seems that a potential code is needed to recover the human body from the image. How was this potential code obtained?", "limitations": "No extra.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- A method has been proposed to solve the problem of multi-layer clothing threading, converting the three-dimensional threading problem to the two-dimensional template level for processing, improving the efficiency and effectiveness of collision detection, and providing technological innovation for the reconstruction of multi-layer clothing.\n- A method flow inspired by the designer's clothing production process has been proposed, providing a new approach for the reconstruction of multi-layer clothing.", "weaknesses": "- According to the description in line 226, the training set (400 shirts, 300 skirts, and 200 pants) and the test set (20 shirts, 20 skirts, and 20 pants) of the experiment are both composed of the same three types of clothing. Perhaps fewer types may lead to limitations in the generalization ability of methods?\n- According to the description in section 3.1, the style of clothes is controlled by Potential space interpolation. This means that the style type and change direction of clothes seem to be displayed by the size of Potential space, and the existing data size seems difficult to support enough Potential space.\n- According to the description in line 96, the stitching method of clothing seems to be pre-defined, which limits the generalization of clothing styles. If more flexible and variable stitching methods can be chosen, it seems that better universality can be achieved.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank you for your acknowledgement of our contribution in cloth modeling and multi-layer clothing. Below are our responses to your comments and questions.\n\n1. *Fewer types may lead to limitations in the generalization ability.*\n\nWe could train on more types if we had the data for them, which would improve the generalization abilities of our model. This expansion is feasible because, in industrial practice, the majority of garments are decomposed into flat sewing patterns, and our representation would effectively apply to such patterns.\n\n2. *Existing data size seems difficult to support enough Potential space.*\n\nOur dataset is generated using the software of [Korosteleva2021] with sewing pattern parametrization. The specific parameters we use to generate shirts are \\{length, width, hem width, collar width, front/back collar depth, sleeve connection width, sleeve opening width, sleeve length\\}. We uniformly sample values from pre-defined ranges for these parameters, enabling us to generate the sewing patterns and their corresponding 3D meshes. We follow the same process to generate data for trousers and skirts, each with their own set of parameters. Finally, we get 400 shirts, 300 skirts and 200 pairs of trousers for training, and 20 shirts, 20 skirts and 20 pairs of trousers for testing.\n\nWhile our current dataset captures variations in style and shape, we acknowledge that it may not cover the entire range of garment shapes and styles due to their vast diversity. However, this can be mitigated by designing and training on additional patterns. As stated above, we ackonwledge that expanding the dataset to encompass a broader range of garment variations would enhance the robustness and generalizability of our method.\n\nFurthermore, as demonstrated in Section 3 of the supplementary material, our approach can be easily extended to sewing patterns with more panels. This implies that our method has the potential to model complex garments beyond those represented in the current dataset. We believe this scalability and adaptability further strengthen the applicability and value of our proposed method.\n\n3. *The stitching method of clothing seems to be pre-defined.*\n\nIndeed, the stitching information is built-into the sewing patterns, but our implicit formulation allows us to handle garment types with different stitching patterns using a single neural network. The reviewer is right to point out that a more flexible stitching method could potentially enhance the generalization of clothing styles. This is an interesting direction for future research. \n\n4. *How was the potential code of human body obtained from images?*\n\nWe utilize the SMPL model, which incorporates shape parameter $\\mathbf{B}$ and pose parameter $\\mathbf{\\Theta}$, to model the human body in our approach. As stated in Line 268-270 of the main paper, we estimate these parameters $\\mathbf{B}$ and $\\mathbf{\\Theta}$ from the input images using the algorithm proposed by [Rong2021]. The latent code for the garments are randomly initialized and then optimized with gradient descent to fit the observations as Eq. (11).\n\n### References\n\n*M. Korosteleva and S. Lee. Generating Datasets of 3D Garments with Sewing Patterns. In Advances in Neural Information Processing Systems, 2021.*  \n*Y. Rong, T. Shiratori, and H. Joo. Frankmocap: Fast monocular 3d hand and body motion capture by regression and integration. In International Conference on Computer Vision Workshops, 2021.*"}, {"review_id": "r2nMUzlXxv", "reviewer": "Reviewer_cVr6", "summary": "This paper introduces ISP, a novel system that can, for the first time, drape a 3D human body with multi-layer garments, without the need of physics-based simulation. Several technical contributions addresses the key aspects of garment draping. To enable learning of garment sewing patterns, the authors cleverly formulate the 2D patterns with 2D signed-distance fileds (SDFs) and label fields. An AtlasNet-style network maps the 2D sewing pattern to 3D, thereby draping clothing to the body. To achieve multi-layer draping, a intuitive mechanism derives a virtual repulsive force based on the 3D locations of draped garments, thereby resolving the collision of cloth.\nExtensive experiments are performed to characterize the model in garment reconstruction, draping, image-based recovery and editing. In all the tasks, ISP has a clear advantage over existing methods, showing its great potential for many downstream tasks in visual computing.", "questions": "- Out of curiosity, how would the model perform when applied to garments that are too complex to be cut into simple front and back pieces (such as a robe or a double-breasted suits)? \n\n- Currently the garment shape vector z is a single, global one for each garment. Would it increase the representation power (hence helpful  to reconstruction tasks) if the z-vector becomes pixel-aligned with the uv map, as done in e.g. NeuralActor (Liu et al., SIGGRAPH Asia 2021) and PoP (Ma et al., ICCV 2021)?\n\n- Following the arguments in L.135, it is only necessary to deform the areas within the front and back panels instead of the full square of the UV map. However in single layer draping (L.184), the displacement MLP outputs the NxNx3 arrays, i.e. covering the entire square, which seems a bit inconsistent to me. To my understanding, in this case the values for out-of-panel areas are null or zero, is that right? If so, how to ensure that?\n\n- Perhaps adding the dimensionality of z to Fig 2 to make it consistent with x?", "limitations": "The technical limitations are discussed in the conclusions and the authors provide an outlook to the solutions.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "I appreciate this paper for its strengths in many aspects:\n\n- Significance of the problem. The paper addressed an important yet long-overlooked problem: automatic modeling, editing and perception of multi-layer clothing. This is a technically challenging problem. Being the first method tackling this (to my knowledge), this paper can potentially have a good impact in various relevant fields in both research and industry (fashion, movie-making etc.).\n\n- Novelty. The technical contributions of this work are strong. It is clever design to leverage neural fields as a formulation for the seemingly discrete 2D sewing patterns, and to use AtlasNet for the draping problem -- both being creative ways to use recent powerful tools. The layer-wise draping is another great example of combining the power of CNNs with physics-inspired losses. In addition, the method provides a unified pipeline for garments of different topology, such that different garment pieces can be modeled in a standardized way -- another favorable property for machine-learning based approaches.\n\n- Evaluation soundness. The experiments holistically validate the characteristics of the proposed model across multiple relevant tasks in vision and graphics, showing the versatility of the method: 3D garment reconstruction, garment draping, image-based garment reconstruction, and shape/texture editing. The implimentation details, runtime benchmarking are well documented. The user study of the draping quality makes the study more compelling. \n\n- Model performance. To my knowledge, this is the first learning-based work that can drape multi-layer clothing on the bodies and handles cloth interpenetration adequately. Even in the single-layered setting, he proposed method has advantages over prior work in terms of less cloth-body intersecting, draping realism, reconstruction accuracy, and inference speed, all being important aspects of garment draping. \n\n- Presentation. The paper is well-written, easy to follow, and technically sound. The math formulations are specific and clear. The illustrations are clear and well complements the text elaboration. \n\nOverall, this is a very strong submission from which I've learned much, and I believe it can be highly impactful.", "weaknesses": "I do not spot major weaknesses in this paper but one observation. In the image-based garment recovery experiment (Sec. 4.4), although recovered garments are correct in their topology and coarse dimensions (e.g. length), but they do not match the details (overall deformation and wrinkles) as that in the image. This inherits the limitation from previous methods such as DrapeNet. Probably in future work a combination with photometric losses can result in more accurate wrinkle estimation?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your appreciation of our work. We address your questions and comments as follows:\n\n1. *A combination with photometric losses can result in more accurate wrinkle estimation.*\n\nTrue, incorporating such a loss could significantly enhance our current method. However, formulating an effective photometric loss is not easy:  The appearance of a garment in a single image is influenced not only by its geometry but also by factors such as texture and shading. The is something we plan to do  in future work.\n\n2. *How would the model perform when applied to garments that are too complex to be cut into simple front and back pieces?*\n\nAs demonstrated in Section 3 of the supplementary material, our method naturally extends to sewing patterns with multiple panels. This allows us to effectively handle the modeling of garments that are more complex than those represented by just two panels, i.e. the front and back panels. Thus, by incorporating additional panels, our method can accommodate intricate garment designs and accurately capture their details. However, it is worth noting that most common garments can easily be modeled by only two panels, and our model excels at handling them.\n\n3. *Pixel-aligned latent codes would increase the representation power.*\n\nWe agree that employing a pixel-aligned encoding strategy would enhance the representation power of our method. By adopting such a strategy, we would be able to preserve more intricate local details of the garment than just using a single global latent code. On the downside, however, a model using pixel aligned latent codes would a) require training images, that is, for each garment geometry we would need a realistic rendering, and b) be more suited for 3D reconstruction from images and less for tasks such as fitting 3D scans.\n\n4. *In single layer draping (L.184), the displacement MLP outputs the NxNx3 arrays, i.e. covering the entire square, which seems a bit inconsistent to me. To my understanding, in this case the values for out-of-panel areas are null or zero, is that right? If so, how to ensure that?*\n\nIndeed, in our single layer draping model, the output is a $N\\times N \\times 3$ array representing the displacement $D_s$ that covers the entire square. To ensure that the out-of-panel areas, denoted as $x$, have zero values, we employ the signed distance function (SDF) of the corresponding garment. Specifically, for areas where the signed distance values are positive (i.e., $s(x,\\textbf{z})>0$), we set $D_s(x)=0$. This corresponds to masking the output of the displacement network outside the panels, using the predicted SDF. We will clarify the paper.\n\n5. *Adding the dimensionality of z to Fig 2.*\n\nWe will revise Fig. 2 to include the dimensionality of $\\textbf{z}$."}, {"review_id": "4Jk0ymduo0", "reviewer": "Reviewer_5F53", "summary": "The authors address the task of draping individual multi-layer garments on human body models. In this context, they introduce a respective garment representation suitable for this task. Garments are represented as a set of individual 2D panels whose shape is defined based on a signed distance function (in more detail, the zero-crossing of a function with 2D location and latent vector specific to each garment as inputs). For each 2D panel, a 2D-to-3D mapping (conditioned on the 2D location and latent vector) is used to map the 2D panel to the 3D garment surface, while enforcing continuity across panels. In addition, draping networks are trained to allow draping multiple garments on human bodies (represented based on SMPL model) in different poses. \n\nThe authors provide quantitative and qualitative comparisons that indicate some potential of the proposed approach.\n", "questions": "Please address my comments under ‘Weaknesses’.", "limitations": "Limitations and failure cases have only been discussed in the supplemental.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Technical soundness:\n- The approach seems reasonable and the results indicate some improvement over the competing techniques.\n\nEvaluation:\n- The authors provide quantitative and qualitative evaluations with comparisons to alternatives.\n- The supplemental provides further experiments as well as limitations.\n\nExposition:\n- The paper is well-structured and good to follow. Figures/tables and respective captions are informative.\n- Text quality is good.\n- The authors provide a comprehensive supplemental.\n\nReferences:\n- References seem ok, but I am not an expert in this field.\n\n", "weaknesses": "Technical soundness:\n- Table 1 only shows results for shirts. What are the results for the other categories like?\n- The variations in the used dataset are not discussed in detail. More details on this would be interesting and allow a better assessment on the potential and remaining challenges of the presented approach.\n\nEvaluation:\n- The evaluation could have been improved by also showing examples or highlighting what works particularly well in comparison to other approaches and what still remains challenging for the presented method.\n- Providing a visualization of the spatial distribution of the Chamfer distance across the surface would show where the errors are larger and where the errors are small. This could help to show the accuracy a bit better, also when comparing to others.\n- The user study is quite simple regarding the fact that only direct preferences have been checked for. More detailed subquestions could have provided insights on what exactly the participants in the study did not yet like and where further improvement is needed.\n- For Figure 6, zoom-ins could be used to highlight improvements/differences between methods.\n\nReproducibility:\n- The paper presents a relatively complex system. This might complicate reproducibility. It is not clear whether code and data will be made available.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank you for your valuable reviews. Below are our responses to your comments.\n\n1. *Table 1 only shows results for shirts. What are the results for the other categories like?*\n\nThe results for trousers and skirts can be found in Table 1 and Table 2 of the supplementary material respectively, where our method shows similar performance with higher accuracy and faster inference speed than UDF.\n\n2. *The variations in the used dataset are not discussed in detail.*\n\nWe apologize for the oversight regarding the detailed discussion about the dataset in our paper.\n\nFor the generation of sewing patterns and corresponding 3D garment meshes, we used the software developed by [Korosteleva2021]. Our dataset encompasses three garment categories: shirts, skirts, and trousers. In [Korosteleva2021], shirts are parameterized by \\{length, width, hem width, collar width, front/back collar depth, sleeve connection width, sleeve opening width, sleeve length\\}, while skirts and trousers are parameterized by \\{length, width, front/back curve\\} and \\{length, crotch depth, hem width\\}, respectively. The values of these parameters were uniformly sampled from pre-defined ranges.\n\nTo create our training set, we generated 400 shirts, 300 skirts, and 200 pairs of trousers. Additionally, we prepared a separate testing set, consisting of 20 shirts, 20 skirts, and 20 pairs of trousers. In the supplementary material, Figures 2, 3, and 4 showcase the variations in garment shape and style that were incorporated into the dataset.\n\nIn line with the methodology described in [Santesteban2022], we relied on the AMASS dataset [Mahmood2019] for training and evaluating our draping models. The AMASS dataset offers a diverse range of poses, containing activities such as walking, running, jumping, arm movements, torso movements, dancing, and more, which ensures comprehensive coverage of various human motions.\n\n3. *The evaluation could have been improved by also showing examples or highlighting what works particularly well in comparison to other approaches and what still remains challenging for the presented method.*\n\nWe conducted comparisons between our method and prior approaches in garment reconstruction, draping, and recovery. Our results demonstrate that our method is more accurate and faster than UDF in the context of reconstruction. Additionally, our method is able to handle multi-layered draping, which is not achieved by any other existing works. A more detailed comparison and comprehensive analysis can be found in the supplementary material (Sec. 1 and 4), where we provide further insights, limitations, and discussion of failure cases. \n\n4. *Providing a visualization of the spatial distribution of the Chamfer distance.*\n\nThe visualization of the spatial error distribution can be found in Figure 1 of the attached PDF file. In this figure, we compare the error distribution between our reconstructions and those produced by UDF. Our reconstructions exhibit lower error across the entire surface compared to UDF's. We will include this figure in the revised version of our paper.\n\n5. *The user study is quite simple.*\n\nTrue but the simple interface and instructions we used allowed us to query many users from diverse backgrounds at a reasonably low cost. In future research, we will work on collecting finer feedback, while keeping a low user friction.\n\n6. *For Figure 6, zoom-ins could be used to highlight improvements/differences between methods.*\n\nWe will revise Fig. 6 for better visualization of the improvements.\n\n7. *Reproducibility.*\n\nAs mentioned in Line 44, we will release our codes and trained models.\n\n8. *Limitations and failure cases have only been discussed in the supplemental.*\n\nWe will revise our paper to incorporate the discussion of limitations and failure cases, as shown in Sec. 4 and Fig. 21 of the supplementary material, in the main paper.\n\n\n### References  \n*M. Korosteleva and S. Lee. Generating Datasets of 3D Garments with Sewing Patterns. In Advances in Neural Information Processing Systems, 2021.*  \n*I. Santesteban, M.A. Otaduy, and D. Casas. SNUG: Self-Supervised Neural Dynamic Garments. In Conference on Computer Vision and Pattern Recognition, 2022.*  \n*N. Mahmood, N. Ghorbani, N. F. Troje, G. Pons-Moll, and M. J. Black. AMASS: Archive of Motion Capture as Surface Shapes. In International Conference on Computer Vision, pages 5442–5451, 2019.*"}, {"review_id": "Tu2DAFMiZb", "reviewer": "Reviewer_LvBx", "summary": "This paper proposes to drape multi-layered garments on SMPL-based human bodies with different poses and shapes. The method is inspired by the commonly adopted sewing pattern and learns to map garments from 2D panels, i.e. front and back panels, to 3D surfaces. The author first deforms the single-layered garments in rest pose according to SMPL skinning function, followed by the corrections of interpenetration between layers of garments. Experiments show the effectiveness and efficiency of the method. The author further shows some possible applications such as garment reconstructions from images and garment editing.", "questions": "* For each garment, should we learn a unique latent vector z? Do we need to train different models for different garments?", "limitations": "While multi-layered settings are more challenging, this paper need more experiments and further explanations to support the contributions in terms of multi-layered garments.\n\nFirst, though the method shows better qualitative results, some more experiments are needed to support the strengths of the method in terms of multi-layered settings, the efficiency, and even the robustness.\nMore comparisons between the existing methods mentioned at L62 are needed to prove the effectiveness of this method.\n\nSecond, as for the dataset, the number of samples seems not enough. The main concern is that whether the insufficient data would lead to some bias or overfitting and thus achieve better qualitative results. More quantitative results with clear explanations are needed.\n\nFinally, the idea of getting interpenetration-free garments seems more like a post-processing step, where another model is trained and specifically aims to move the penetrated vertices out of the inner mesh. I am concerned regarding this idea. Since this makes the settings of multi-layered garments as a simple extension of single-layered garments, weakening the contributions in the field of multi-layered garments.", "rating": 3, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "* An efficient method mapping implicit 2D sewing patterns to 3D garments with good performance.\n* The qualitative results of draping multi-layered garments show less interpenetration.\n* The method is differentiable and is able to be applied to inverse problems.", "weaknesses": "1. While this paper focuses on multi-layered garments, the experiments lack some quantitative results to support the effectiveness. For example, what is the rate of interpenetration between human body and multi-layered garments? What about the interpenetration between different layers of garments? Will different types of garments, such as dress or shirt, lead to different collision rate? How will the collision rates mentioned above change when the number of layers increase? In the meanwhile, please also provide more detail information about the test set for the above evaluation, such as the number of vertices or the resolution of the mesh.\n2. As for the efficiency, the author provides comparisons during training process in Table 1. Could you provide similar comparisons about reconstruction time or model forward time during test? \n3. The distribution of the human parameters in the dataset is unknown. How many different human bodies are sampled? What is the distribution of different gender in both training and test set?\n4. Is there any overlapping between training set and test set? How many unseen garments and human bodies are included in test set?\n5. While the previous method mentioned at L62 is only able to drape multi-layered garments in T-pose, could the author provide some comparisons also in T-pose? Since the method in this paper supports different poses of human and previous method at L62 did very similar work, the comparisons in T-pose are needed to support the effectiveness of the method.\n6. The writings can be further improved. Some descriptions and captions are not clear.\n7. Please change the tone of the sentence at L3. As discussed in related work at L62, previous method is able to drape multi-layered garments.\n8. The layering network at L191-221 seems like a post-processing step, making the multi-layered draping more like a simple extension of single-layered draping.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable reviews. Below are our responses.\n\n1. *The experiments of layering.*\n\nWe conducted an experiment to further evaluate our layering model by measuring the intersection ratio between garment layers and between garments and the body. We generated 673 unseen bodies with unseen poses from [1] and shape parameters uniformly sampled from $[-2, 2]^{10}$. Each body was paired with 5 randomly generated unseen garments (1 skirt or 1 trousers with 4 shirts), where the 1st layer consisted of the skirt or trousers.\n\nThe results, presented in Table 1 of the attached PDF file, display the intersection ratios. Diagonal values are the intersection ratio between the body and the $i$-th ($i=1,2,3,4,5$) layer, while other values are intersections between different garment layers. As prior works do not support layering, a direct comparison is impractical. The numbers inside and outside the bracket denote the results obtained with and without the layering procedure $\\mathcal{D}_m$ respectively. The significantly lower intersection ratios with layering demonstrate the effectiveness of our model in handling intersections with the body and garments, even after layering five garments (~2%). The increase in intersections as more garments are layered is explained in L150-155 of the supplementary material, where we clarify the training of $\\mathcal{D}_m$ is on single-layer draped garments, which are closer to the body. The average vertex numbers for shirts/skirt/trousers are 9.2k/9.4k/10.6k respectively.\n\n2. *Similar comparisons about reconstruction time or model forward time during test.*\n\nAs stated in L247, the time reported in the left of Table 1 in the main paper refers to the reconstruction time (or model forward time) for the garments from the training set, rather than the time taken for the training process. Furthermore, the labels 'Train'/'Test' in Table 1 indicate that the results are evaluated on the training/test set respectively, and not refer to the training/testing process. As shown in Table 2 of the attached PDF file, the reconstruction time during testing is comparable to those in the left of Table 1 of the main paper.\n\n3. *The distribution of the human parameters.*\n\nWe use SMPL to model the human bod. Its usual parameters $\\mathbf{\\Theta}$ and $\\mathbf{B}$ control the body pose and shape respectively. \n\nOur training and evaluation protocol follows prior art [1,2], using the AMASS dataset. With 6519 poses for training and 673 unseen poses for testing, the dataset covers a wide range of human motions, including walking, running, jumping, arm/torso movements and dancing. Importantly, our self-supervised draping method, driven by a physics-based loss, eliminates the need of simulated or scanned garment data for training. This allows our model to easily generalize and extend to more pose data. During training and testing, we uniformly sample $\\mathbf{B}$ from $[-2, 2]^{10}$.\n\nSMPL offers 3 body models: female, male and neutral. While our study focuses on the female model, as do many prior works [2-5], there is nothing specific about it and our approach applies just as well to the other two.\n\n4. *Is there any overlapping between training set and test set?*\n\nNo, there is no overlapping between training set and test set. As stated in L226-227 of the main paper, for the experiments of garment reconstruction, the test set comprises 20 shirts, 20 skirts, and 20 pairs of trousers that are not part of the training set (unseen). For garment draping, the test set contains 673 unseen $\\mathbf{\\Theta}$, along with $\\mathbf{B}$ sampled from $[-2, 2]^{10}$ . \n\n5. *Comparisons to [5].*\n\nThere unfortunately is not easy way to generalize to [5] because the code is unavailable. In addition, it would only work in T-pose, whereas we generalize to different ones.\n\n6. \\& 7. *The writings and the tone of L3.*\n\nWe will revise our paper to make it clearer, and rephrase L3 to '*However, they are either unable to handle multi-layered clothing, which is prevalent in everyday dress, or restricted to bodies in T-pose*'.\n\n8. *The multi-layered draping like an extension of single-layered draping.*\n\nThe layering network is indeed a straightforward extension of single-layered draping precisely because our single-layered model is designed to make this extension easy, which we view as part of our contribution. Single-layer draping provides a good starting point for our layering model. By predicting corrective displacement for vertices, rather than directly regressing vertex positions, we can ease the training process. This is important because our model is *self-supervised* using a physics-based loss, and directly regressing vertex positions would cause the training to collapse and not converge.\n\n9. *The latent vector and model for different garments.*\n\nWe use a single network to handle a wide range of garments of varying topology and geometry. Each garment is associated to a unique latent vector $\\textbf{z}$ learned by auto-decoding, as in [6]. During training, the latent vectors for garments in the database are randomly initialized and optimized alongside the weights of our ISP network $\\mathcal{I}_{\\Theta}$ using backprop. For unseen garments, we randomly initialize a new $\\textbf{z}$, and optimize it while keeping the network frozen by fitting it to observations like image segmentations or 3D meshes.\n\nFor our draping models $\\mathcal{D}_s$ and $\\mathcal{D}_m$, we learn a single generic network separately that can handle various garments rather than multiple garment-specific networks. This makes our approach more scalable, without having to train and maintain a separate network for each garment.\n\n### References  \n*[1] N. Mahmood, et al. AMASS. ICCV 2019.*\n*[2] I. Santesteban, et al. SNUG. CVPR 2022.*\n*[3] L. Luigi, et al. DrapeNet. CVPR 2023.*\n*[4] I. Santesteban, et al. VTO Garment Collisions. CVPR 2021.*\n*[5] I. Santesteban, et al. Ulnef. NIPS 2022.*\n*[6] J. Park, et al. Deepsdf. CVPR 2019.*"}]

Author rebuttal:
We would like to thank all reviewers for their valuable suggestions and constructive comments. We have carefully considered all of the suggestions and concerns raised by each reviewer and responded to each of them below. We will implement these suggestions in our revised paper. 

The attached PDF file contains the figure, illustrating the spatial error distribution as suggested by Reviewer 5F53, and the tables presenting the experimental results as recommended by Reviewer LvBx.

Once again, we sincerely thank all reviewers for their expertise and the time they spent in reviewing our paper.

Meta-review: The paper presents an interesting solution to the multi-layer garment draping problem. This is not the first paper exploring this problem setup. The rebuttal helped to clarify several of the concerns raised in the original reviews. In the post-rebuttal discussion, most of the reviewers were confident about the work. There was one negative reviewer, but in the end, the AC championed the paper. The authors engaged in a useful post-rebuttal discussion that helped to highlight the improvements over existing methods. 

Please incorporate the additional details (evaluations) in the final version or in the supplementary. The additional experiments are valuable for the readers.
Decision: Accept (poster)