Title: ISP: Multi-Layered Garment Draping with Implicit Sewing Patterns
Venue: NeurIPS 2023
Accepted: yes
Keywords: garment modeling

draping

deformation

human body modeling
Abstract: Many approaches to draping individual garments on human body models are realistic, fast, and yield outputs that are differentiable with respect to the body shape on which they are draped. However, they are either unable to handle multi-layered clothing, which is prevalent in everyday dress, or restricted to bodies in T-pose. In this paper, we introduce a parametric garment representation model that addresses these limitations. As in models used by clothing designers, each garment consists of individual 2D panels. Their 2D shape is defined by a Signed Distance Function and 3D shape by a 2D to 3D mapping. The 2D parameterization enables easy detection of potential collisions and the 3D parameterization handles complex shapes effectively. We show that this combination is faster and yields higher quality reconstructions than purely implicit surface representations, and makes the recovery of layered garments from images possible thanks to its differentiability. Furthermore, it supports rapid editing of garment shapes and texture by modifying individual 2D panels.

ISP: Multi-Layered Garment Draping with Implicit Sewing Patterns

Abstract

Many approaches to draping individual garments on human body models are realistic, fast, and yield outputs that are differentiable with respect to the body shape on which they are draped. However, they are either unable to handle multilayered clothing, which is prevalent in everyday dress, or restricted to bodies in T-pose. In this paper, we introduce a parametric garment representation model that addresses these limitations. As in models used by clothing designers, each garment consists of individual 2D panels. Their 2D shape is defined by a Signed Distance Function and 3D shape by a 2D to 3D mapping. The 2D parameterization enables easy detection of potential collisions and the 3D parameterization handles complex shapes effectively. We show that this combination is faster and yields higher quality reconstructions than purely implicit surface representations, and makes the recovery of layered garments from images possible thanks to its differentiability. Furthermore, it supports rapid editing of garment shapes and texture by modifying individual 2D panels.

1 Introduction

Draping virtual garments on body models has many applications in fashion design, movie-making, virtual try-on, virtual and augmented reality, among others. Traditionally, garments are represented by 3D meshes and the draping relies on physics-based simulations (PBS) [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12] to produce realistic interactions between clothes and body. Unfortunately PBS is often computationally expensive and rarely differentiable, which limits the scope of downstream applications. Hence, many recent techniques use neural networks to speed up the draping and to make it differentiable. The garments can be represented by 3D mesh templates [13, 14, 15, 16, 17, 18, 19, 20], point clouds [21, 22], UV maps [23, 24, 25, 26, 27], or implicit surfaces [28, 29, 30]. Draping can then be achieved by Linear Blend Skinning (LBS) from the shape and pose parameters of a body model, such as SMPL [31].

Even though all these methods can realistically drape individual garments over human bodies, none can handle multiple clothing layers, despite being prevalent in everyday dress. To address overlapping clothing layers such as those in Fig. 1 while preserving expressivity, we introduce an Implicit Sewing Pattern (ISP), a new representation inspired by the way fashion designers represent clothes. As shown in Fig. 2, a sewing pattern is made of several 2D panels implicitly represented by signed distance functions (SDFs) that model their 2D extent and are conditioned on a latent vector z , along with information about how to stitch them into a complete garment. To each panel is associated a 2D to 3D mapping representing its 3D shape, also conditioned on z . The 2D panels make it easy to detect collisions between surfaces and to prevent interpenetrations. In other words, the garment is made of panels whose 2D extent is learned instead of having to be carefully designed by a human and whose pascal.fua@epfl.ch

3D shape is expressed in a unified uv space in which a loss designed to prevent interpenetrations can easily be written.

This combination enables us to model layered garments such as those of Fig. 1 while preserving end-to-end differentiability. This lets us drape them realistically over bodies in arbitrary poses, to recover them from images, and to edit them easily. Doing all this jointly is something that has not yet been demonstrated in the Computer Vision or Computer Graphics literature. Furthermore, most data driven draping methods rely on synthetic data generated with PBS for supervision purposes. In contrast, ISPs rely on physics-based self-supervision of [18, 13]. As a result, at inference time, our approach can handle arbitrary body poses, while only requiring garments draped over bodies in a canonical pose at training time. Our code is available at https://github.com/liren2515/ISP .

2 Related Work

Garment Draping. Garment draping approaches can be classified as physics-based or data-driven. Physics-based ones [3, 32, 33, 34, 35, 12] produce high-quality results but are computationally demanding, while data-driven approaches are faster, sometimes at the cost of realism. Most datadriven methods are template-based [13, 14, 15, 17, 19, 18, 20, 16, 18, 13], with a triangulated mesh modeling a specific garment and a draping function trained specifically for it. As this is impractical for large garment collections, some recent works [22, 21, 36] use 3D point clouds to represent garments instead. Unfortunately, this either prevents differentiable changes in garment topology or loses the point connections with physical meanings. The approaches of [24, 25] replace the clouds by UV maps that encode the diverse geometry of garments and predict positional draping maps. These UV maps are registered to the body mesh, which restricts their interpretability and flexibility for garment representation and manipulation. The resulting garments follow the underlying body topology, and the ones that should not, such as skirts, must be post-processed to remove artifacts. Yet other algorithms [29, 30, 37] rely on learning 3D displacement fields or hierarchical graphs for garment deformation, which makes them applicable to generic garments.

While many of these data-driven methods deliver good results, they are typically designed to handle a single garment or a top and a bottom garment with only limited overlap. An exception is the approach of [38] that augments SDFs with covariant fields to untangle multi-layered garments. However, the untangling is limited to garments in T-pose. In contrast, our proposed method can perform multi-layered garment draping for bodies in complex poses and of diverse shapes.

Garments as Sets of Panels. Sewing patterns are widely used in garment design and manufacturing [39, 40, 41, 42, 43]. Typically, flat sewing patterns are assembled and then draped using PBS. To automate pattern design, recent works introduce a parametric pattern space. For example, the methods of [44, 45] introduce a sparse set of parameters, such as sleeve length or chest circumference, while [27] relies on principal component analysis (PCA) to encode the shape of individual panels. It requires hierarchical graphs on groups of panels to handle multiple garment styles. By contrast, our approach relies on the expressivity of 2D Sign Distance Functions (SDFs) to represent the panels.

To promote the use of sewing patterns in conjunction with deep learning, a fully automatic dataset generation tool is proposed in [46]. It randomly samples parameters to produce sewing patterns and uses PBS to drape them on a T-posed human body model to produce training pairs of sewing patterns and 3D garment meshes.

Garments as Implicit Surfaces. SDFs have become very popular to represent 3D surfaces. However, they are primarily intended for watertight surfaces. A standard way to use them to represent open surfaces such as garments is to represent them as thin volumes surrounding the actual surface [28, 29], which can be represented by SDFs but with an inherent accuracy loss. To address this issue, in [30], the SDFs are replaced by unsigned distance functions (UDFs) while relying on the differentiable approach of [47] to meshing, in case an actual mesh is required for downstream tasks. However, this requires extra computations for meshing and makes training more difficult because surfaces lie at a singularity of the implicit field. This can result in unwanted artifacts that our continuous UV parameterization over 2D panels eliminates.

3 Method

In clothing software used industrially [39, 40, 41], garments are made from sets of 2D panels cut from pieces of cloth which are then stitched together. Inspired by this real-world practice, we introduce the Implicit Sewing Patterns (ISP) model depicted by Fig. 2. It consists of 2D panels whose shape is defined by the zero crossings of a function that takes as input a 2D location x = ( x u , x v ) and a latent vector z , which is specific to each garment. A second function that also takes x and z as arguments maps the flat 2D panel to the potentially complex 3D garment surface within the panel, while enforcing continuity across panels. Finally, we train draping networks to properly drape multiple garments on human bodies of arbitrary shapes and poses, while avoiding interpenetrations of successive clothing layers.

3.1 Modeling Individual Garments

We model garments as sets of 2D panels stitched together in a pre-specified way. Each panel is turned into a 3D surface using a uv parameterization and the networks that implement this parameterization are trained to produce properly stitched 3D surfaces. To create the required training databases, we use the PBS approach of [46]. Because it also relies on 2D panels, using its output to train our networks has proved straightforward, with only minor modifications required.

Flat 2D Panels. We take a pattern P to be a subset of Ω = [ -1 , 1] 2 whose boundary are the zero crossings of an implicit function. More specifically, we define

where I Θ is implemented by a fully connected network. It takes as input a local 2D location x and a global latent vector z and returns two values. The first, s , should approximate the signed distance to the panel boundary. The second, c , is a label associated to boundary points and is used when assembling the garment from several panels. Then, boundaries with the same labels should be stitched together. Note that the c values are only relevant at the panel boundaries, that is, when s = 0 . However, training the network to produce such values only there would be difficult because this would involve a very sparse supervision. So, instead, we train our network to predict the label

2D SDF Field

2D Label Field

(c)

(a)

Sewing Pattern

Front

Back

0

1

2

0

3

0

4

0

5

6

0

1

2

0

3

0

4

0

5

6

(b)

Latent code

Pattern Parameterization

UV queries

Latent code

UV queries

Sewing

𝑀

!

UV Parameterization

(e)

(d)

Latent Space Interpolation

𝐱 ∈ ℝ

!

!

×#

𝐳

𝐱 ∈ ℝ

!

!

×#

𝐳

(𝑠

"

, 𝑐

"

)

(𝑠

#

, 𝑐

#

)

ℐ

%

𝒜

&

c of the closest boundary for all points, as shown in Fig. 2(c), which yields a much better behaved minimization problem at training time.

In this paper, we model every garment G using two panels P f and P b , one for the front and one for the back. E f and E b are labels assigned to boundary points. Label 0 denotes unstitched boundary points like collar, waist, or sleeve ends. Labels other than zero denote points in the boundary of one panel that should be stitched to a point with the same label in the other.

In practice, given the database of garments and corresponding 2D panels we generated using publicly available software designed for this purpose [46], we use an auto-decoding approach to jointly train I Θ and to associate a latent vector z to each training garment. To this end, we minimize the loss function obtained by summing over all training panels

with respect to a separate latent code z per garment and the network weights Θ that are used for all garments. CE is the cross-entropy loss, s gt and c gt are the ground-truth signed distance value and the label of the closest seamed edge of x , and λ CE and λ reg are scalars balancing the influence of the different terms. We handle each garment having a front and a back panel P f and P b by two separate network I Θ f and I Θ b with shared latent codes so that it can produce two separate sets of ( s, c ) values at each ( x u , x v ) location, one for the front and one for the back.

From 2D panels to 3D Surfaces. The sewing patterns described above are flat panels whose role is to provide stitching instructions. To turn them into 3D surfaces that can be draped on a human body, we learn a mapping from the 2D panels to the 3D garment draped on the neutral SMPL body. To this end, we introduce a second function, similar to the one used in AtlasNet [48]

It is also implemented by a fully connected network and takes as input a local 2D location x and a global latent vector z . It returns a 3D position X for every 2D location x in the pattern. A key difference with AtlasNet is that we only evaluate A Φ for points x within the panels, that is points for which the signed distance returned by I Θ of Eq. (1) is not positive. Hence, there is no need to deform or stretch uv patterns. This is in contrast to the square patches of AtlasNet that had to be, which simplifies the training.

Given the latent codes z learned for each garment in the training database, as described above, we train a separate set of weights Φ f and Φ b for the front and back of the garments. To this end, we minimize a sum over all front and back training panels of

where L CHD , L normal , and L consist are the Chamfer distance, a normal consistency loss, and a loss term whose minimization ensures that the points on the edges of the front and back panels sewn togetherE c for c > 0 -are aligned when folding the panels in 3D, λ n and λ c are scalar weights. This assumes that the front and back panels have their seamed edges aligned in 2D, ie. for c > 0 and x ∈ Ω , if x has label c on the front panel, then it should also have label c in the back panel, and vice-versa. Our experiments show that L consist reduces the gap between the front and back panels.

Meshing and Preserving Differentiability. A Φ continuously deforms 2D patches that are implicitly defined by I Θ . To obtain triangulated meshes from these, we lift a regular triangular 2D mesh defined on Ω by querying A Φ on each of its vertex, as in [48]. More specifically, we first create a square 2D mesh T = { V Ω , F Ω } for Ω , where vertices V Ω are grid points evenly sampled along orthogonal axis of Ω and faces F Ω are created with Delaunay triangulation. Given the latent code z of a specific garment, for each vertex v ∈ V Ω , we can obtain its signed distance value s and edge label c with ( s, c ) = I Θ ( v, z ) . We construct the 2D panel mesh T P = { V P , F P } by discarding vertices of T whose SDF is positive. To get cleaner panel borders, we also keep vertices v ∈ V Ω with s ( v, z ) > 0 that belong to the mesh edges that cross the 0 iso-level, and adjust their positions to v -s ( v, z ) ∇ s ( v, z ) to project them to the zero level set [49]. The front and back panel 2D meshes are then lifted to 3D by querying A Φ on each vertex. During post-processing, their edges are sewn together to produce the final 3D garment mesh T G ( z ) . More details can be found in the Supplementary Material.

I Θ performs as an indicator function to keep vertices with s ≤ 0 , which breaks automatic differentiability. To restore it, we rely on gradients derived in [50, 51] for implicit surface extraction. More formally, assume v is a vertex of the extracted mesh T G and x is the point on the UV space that satisfies v = A Φ ( x , z ) , then, as proved in the Supplementary Material,

Hence, ISP can be used to solve inverse problems using gradient descent, such as fitting garments to partial observations.

3.2 Garment Draping

We have defined a mapping from a latent vector z to a garment draped over the neutral SMPL model [31], which we will refer to as a rest pose . We now need to deform the garments to conform to bodies of different shapes and in different poses. To this end, we first train a network that can deform a single garment. We then train a second one that handles the interactions between multiple garment layers and prevents interpenetrations as depicted in Fig. 3.

Single Layer Draping. In SMPL, body templates are deformed using LBS. As in [19, 29, 30], we first invoke an extended skinning procedure with the diffused body model formulation to an initial rough estimate of the garment shape. More specifically, given the parameter vectors B and Θ that control the body shape and pose respectively, each vertex v of a garment mesh T G is transformed by

where W ( · ) is the SMPL skinning function with skinning weights W ∈ R N B × 24 , with N B being the number of vertices of the SMPL body mesh, and B s ( B ) ∈ R N B × 3 and B p ( Θ ) ∈ R N B × 3 are the

shape and pose displacements returned by SMPL. w ( v ) ∈ R N B are diffused weights returned by a neural network, which generalizes the SMPL skinning to any point in 3D space. The training of w ( v ) is achieved by smoothly diffusing the surface values, as in [29].

This yields an initial deformation, such that the garment roughly fits the underlying body. A displacement network D s is then used to refine it. D s is conditioned on the body shape and pose parameters B , Θ and on the garment specific latent code z . It returns a displacement map D s = D s ( B , Θ , z ) in UV space. Hence, the vertex v init with UV coordinates ( x u , x v ) in Ω is displaced accordingly and becomes ˜ v = v init + D s [ x u , x v ] , where [ · , · ] denotes standard array addressing. D s is implemented as a multi-layer perceptron (MLP) that outputs a vector of size 6 N 2 , where N is the resolution of UV mesh T . The output is reshaped to two N × N × 3 arrays to produce the front and back displacement maps.

To learn the deformation for various garments without collecting any ground-truth data and train D s in a self-supervised fashion, we minimize the physics-based loss from [18]

where L strain is the membrane strain energy caused by the deformation, L bend the bending energy raised from the folding of adjacent faces, L gravity the gravitational potential energy, and L BGcol the penalty for body-garment collisions.

Multi-Layer Draping. When draping independently multiple garments worn by the same person using the network D s introduced above, the draped garments can intersect, which is physically impossible and must be prevented. We now show that our ISP model makes that straightforward first in the case of two overlapping garments, and then in the case of arbitrarily many.

Consider an outer garment T o G and an inner garment T i G with rest state vertices V o and V i . We first drape them independently on a target SMPL body with D s , as described for single layer draping. This yields the deformed outer and underlying garments with vertices ˜ V o and ˜ V i , respectively. We then rely on a second network D m to predict corrective displacements, conditioned on the outer garment geometry and repulsive virtual forces produced by the intersections with the inner garment.

In ISP, garments are represented by mapping 2D panels to 3D surfaces. Hence their geometry can be stored on regular 2D grids on which a convolutional network can act. In practice, we first encode the rest state of the outer garment T o G into a 2D position map M r . The grid M r records the 3D location of the vertex v o ∈ V o at its ( x u , x v ) coordinate as M r [ x u , x v ] = v o within the panel boundaries, M r [ x u , x v ] = (0 , 0 , 0) elsewhere. Concatenating both front and back panels yields a N × N × 6 array, that is, a 2D array of spatial dimension N with 6 channels. After draping, the same process is applied to encode the geometry of T o G into position map M d , using vertices ˜ V o instead of V o this time. Finally, for each vertex ˜ v o ∈ ˜ V o , we take the repulsive force acting on it to be

where ˜ v i is the closest vertex in ˜ V i , n i is the normal of ˜ v i , and · represents the dot product. The repulsive force is also recorded in the UV space to generate the 2D force map M f . Note that it is 0 for vertices that are already outside of the inner garment, for which no collision occurs. Given the forces M f , the garment geometry in the rest state M r and after draping M d , the network D m predicts a vertex displacements map D m = D m ( M r , M d , M f ) for the outer garment to resolve intersections, as shown in Fig. 3. We replace the vertex ˜ v o ∈ ˜ V o with coordinates ( x u , x v ) by ˜ v ∗ o = ˜ v o + D m [ x u , x v ] . D m is implemented as a CNN, and it can capture the local geometry and force information of vertices from the input 2D maps. The training of D m is self-supervised by

where L phy is the physics-based loss of Eq. (8), λ g and λ r are weighting constants, and ϵ is a safety margin to avoid collisions. L GGcol penalizes the intersections between the outer and underlying garments, and L reg is an L 2 regularization on the output of D m .

Given a pair of garments, our layering network D m only deforms the outer garment to resolve collisions, leaving the inner one untouched. Given more than 2 overlapping garments, the process can be iterated to layer them in any desired order over a body, as detailed in the Supplementary Material.

Garment State

& Force

𝑀𝑟

𝑀𝑑

𝑀𝑓

Multi-Layer Draping

𝐷𝑚

D

m

Ours

512

128

UDF

GT

4 Experiments

4.1 Dataset, Evaluation Metrics, and Baseline

To create training and test sets, we used the software of [46] to generate sewing patterns and the corresponding 3D garment meshes in their rest state, that is draped over a T-Posed body. Our training set comprises 400 shirts, 300 skirts, and 200 pairs of trousers. For testing, we use 20 shirts, 20 skirts, and 20 pairs of trousers. As discussed in Section 3.2, we trained the draping models using SMPL body poses Θ randomly sampled from the AMASS [52] dataset and body shapes B uniformly sampled from [ -2 , 2] 10 . We use the Chamfer Distance (CHD) and Normal Consistency (NC) to quantify garment reconstruction accuracy, along with the percentage of the garment mesh area that undergoes interpenetrations (Intersection), as in [30].

We compare our approach against recent and state-of-the-art methods DIG [29] and DrapeNet [30], which, like ours, can drape various garments over bodies of different shapes in arbitrary poses. Like our ISPs, DrapeNet is self-supervised using a physics-based loss and can prevent unwanted intersections between top (shirts) and bottom (trousers) garments. By contrast, DIG is a fully supervised method and cannot handle garment intersections.

4.2 Garment Reconstruction

We first consider 3D garments in their rest pose and compare the accuracy of our ISPs against that of the UDF-based representation [30]. In Fig. 4, we provide qualitative results in a specific case and for resolutions of Marching Cubes [53] ranging from 512 to 128. Our result is visually superior and more faithful to the ground-truth garment, with UDF producing more artifacts and uneven borders, especially at resolution 512. The quantitative results reported in Tab. 1 for the training and test sets confirm this. For the test set garments, we reconstruct them by optimizing the latent code to minimize the Chamfer distance between the predicted and ground truth meshes, using the gradients of Eq. (6). Our approach consistently outperforms UDF at all resolutions while being faster, mostly because we evaluate our network on a 2D implicit field while UDF does it on a 3D one. For example, at resolution 256, our reconstruction time is 77 ms on an Nvidia V100 GPU, while UDF requires 2379 ms. Interestingly, increasing the resolution from 256 to 512 improves the reconstruction accuracy

Y

of our method, whereas that of UDF's drops. This is because precisely learning the 0 iso-surface of a 3D UDF is challenging, resulting in potentially inaccurate normals near the surface [54, 47]. We present similar results for trousers and skirts in the supplementary material.

4.3 Garment Draping

62.69%

25.00%

12.31%

0

20

40

60

80

None

DrapeNet

Ours

(a)

Intersection (%,

↓

)

DrapeNet

0.536

Ours

0.276

(b)

Ours

DrapeNet

(c)

Ours

DIG

DrapeNet

(d)

Figs. 1 and 5 showcase our method's ability to realistically drape multiple garments with diverse geometry and topology over the body. Our approach can handle multi-layered garments, which is not achievable with DrapeNet. Additionally, unlike the method of [38] that is limited to multi-layered garments on the T-posed body, our method can be applied to bodies in arbitrary poses.

As pointed out in [13, 30], there are no objective metrics for evaluating the realism of a draping. Therefore, we conducted a human evaluation. As in [30], we designed a website displaying side-byside draping results generated by our method and DrapeNet for the same shirts and trousers, on the same bodies. Participants were asked to select the option that seemed visually better to them, with the third option being "I cannot decide". 64 participants took part in the study, providing a total of 884 responses. As shown in Fig. 6(a), the majority of participants preferred our method (62.69% vs. 25.00%), demonstrating the higher fidelity of our results. The lower intersection ratio reported in Fig. 6(b) further demonstrates the efficacy of our draping model. In Figs. 6(c) and (d), we compare qualitatively our method against DrapeNet and DIG.

4.4 Recovering Multi-Layered Garments from Images

Thanks to their differentiability, our ISPs can be used to recover multi-layered garments from image data, such as 2D garment segmentation masks. Given an image of a clothed person, we can obtain the estimation of SMPL body parameters ( B , Θ ) and the segmentation mask S using the algorithms of [56, 57]. To each detected garment, we associate a latent code z and reconstruct garment meshes

Ours

SMPLicit

ClothWild

DrapeNet

Ours

SMPLicit

ClothWild

Original

Modified

Original

Modified

Original

Modified

Short Sleeves

Short Body

Closed

by minimizing

where L IoU is the IoU loss [58] over the rendered and the given mask, R ( · ) is a differentiable renderer [59], N is the number of detected garments, and ⊕ represents the operation of mesh concatenation. M is the SMPL body mesh, while G ( B , Θ , z 1: N ) is the concatenation of garment meshes reconstructed from the implicit sewing pattern as T G ( z i ) and then draped by our draping model D = D m ◦ D s . In practice, the minimization is performed from the outermost garment to the innermost, one by one. Further details can be found in the Supplementary Material.

Fig. 7 depicts the results of this minimization. Our method outperforms the state-of-the-art methods SMPLicit [28], ClothWild [55] and DrapeNet [30], given the same garment masks. The garments we recover exhibit higher fidelity and have no collisions between them or with the underlying body.

4.5 Garment Editing

As the panel shape of the sewing pattern determines the shape of the garment, we can easily manipulate the garment mesh by editing the panel. Fig. 8 depicts this editing process. We begin by moving the sleeve edges of the panel inwards to shorten the sleeves of the mesh, then move the bottom edges up to shorten the length of the jacket, and finally remove the edges for the opening to close it. This is achieved by minimizing

w.r.t. z . d ( · ) computes the chamfer distance, ˜ E represents the edges of the modified panel, and E ( z ) represents the 0 iso-level extracted from I Θ , that is, the edges of the reconstructed panel. Our sewing pattern representation makes it easy to specify new edges by drawing and erasing lines in

the 2D panel images, whereas the fully implicit garment representation of [30] requires an auxiliary classifier to identify directions in the latent space for each garment modification. As shown in the supplementary material, the texture of the garment can also be edited by drawing on the UV panels.

5 Conclusion

We have introduced a novel representation for garments to be draped on human bodies. The garments are made of flat 2D panels whose boundary is defined by a 2D SDF. To each panel is associated a 3D surface parameterized by the 2D panel coordinates. Hence, different articles of clothing are represented in a standardized way. This allows the draping of multi-layer clothing on bodies and the recovery of such clothing from single images. Our current implementation assumes quasi-static garments and only deforms the outer garment to solve collisions in multi-layer draping. In future work, we will introduce garment dynamics and focus on more accurate physical interactions between the outer and inner garments.

Acknowledgement. This project was supported in part by the Swiss National Science Foundation.

References

Xavier Provot et al. Deformation constraints in a mass-spring model to describe rigid cloth behaviour. In Graphics interface , 1995.

X. Provot. Collision and Self-Collision Handling in Cloth Model Dedicated to Design Garments. In Computer Animation and Simulation , 1997.

D. Baraff and A. Witkin. Large Steps in Cloth Simulation. In ACM SIGGRAPH , pages 43-54, 1998.

T. Vassilev, B. Spanlang, and Y. Chrysanthou. Fast cloth animation on walking avatars. In Computer Graphics Forum , 2001.

C. Zeller. Cloth simulation on the gpu. In ACM SIGGRAPH , 2005.

M. Tang, R. Tong, R. Narain, C. Meng, and D. Manocha. A GPU-based streaming algorithm for high-resolution cloth simulation. In Computer Graphics Forum , 2013.

T. Liu, S. Bouaziz, and L. Kavan. Quasi-newton methods for real-time simulation of hyperelastic materials. ACM Transactions on Graphics , 2017.

Nvidia. Nvcloth, 2018.

Optitext Fashion Design Software, 2018. https://optitex.com/ .

Nvidia. NVIDIA Flex, 2018. https://developer.nvidia.com/flex .

M. Designer, 2018. https://www.marvelousdesigner.com .

Tongkui Su, Yan Zhang, Yu Zhou, Yao Yu, and Sidan Du. GPU-based Real-time Cloth Simulation for Virtual Try-on. In Pacific Conference on Computer Graphics and Applications , 2018.

H. Bertiche, M. Madadi, and S. Escalera. PBNS: Physically Based Neural Simulation for Unsupervised Garment Pose Space Deformation. ACM Transactions on Graphics , 2021.

B. L. Bhatnagar, G. Tiwari, C. Theobalt, and G. Pons-Moll. Multi-Garment Net: Learning to Dress 3D People from Images. In International Conference on Computer Vision , 2019.

B. Jiang, J. Zhang, Y. Hong, J. Luo, L. Liu, and H. Bao. Bcnet: Learning body and cloth shape from a single image. In European Conference on Computer Vision , 2020.

X. Pan, J. Mai, X. Jiang, D. Tang, J. Li, T. Shao, K. Zhou, X. Jin, and D. Manocha. Predicting loose-fitting garment deformations using bone-driven motion networks. In ACM SIGGRAPH , 2022.

C. Patel, Z. Liao, and G. Pons-Moll. Tailornet: Predicting clothing in 3d as a function of human pose, shape and garment style. In Conference on Computer Vision and Pattern Recognition , 2020.

I. Santesteban, M.A. Otaduy, and D. Casas. SNUG: Self-Supervised Neural Dynamic Garments. In Conference on Computer Vision and Pattern Recognition , 2022.

I. Santesteban, N. Thuerey, M. A. Otaduy, and D. Casas. Self-Supervised Collision Handling via Generative 3D Garment Models for Virtual Try-On. In Conference on Computer Vision and Pattern Recognition , 2021.

G. Tiwari, B. L. Bhatnagar, T. Tung, and G. Pons-Moll. Sizer: A Dataset and Model for Parsing 3D Clothing and Learning Size Sensitive 3D Clothing. In European Conference on Computer Vision , 2020.

H. Bertiche, M. Madadi, E. Tylson, and S. Escalera. DeePSD: Automatic Deep Skinning and Pose Space Deformation for 3D Garment Animation. In International Conference on Computer Vision , 2021.

E. Gundogdu, V. Constantin, S. Parashar, A. Seifoddini, M. Dang, M. Salzmann, and P. Fua. Garnet++: Improving Fast and Accurate Static 3D Cloth Draping by Curvature Loss. IEEE Transactions on Pattern Analysis and Machine Intelligence , 22(1):181-195, 2022.

Z. Lahner, D. Cremers, and T. Tung. Deepwrinkles: Accurate and Realistic Clothing Modeling. In European Conference on Computer Vision , September 2018.

Y. Shen, J. Liang, and M.C. Lin. Gan-based garment generation using sewing pattern images. In European Conference on Computer Vision , 2020.

Z. Su, T. Yu, Y. Wang, and Y. Liu. DeepCloth: Neural Garment Representation for Shape and Style Editing. IEEE Transactions on Pattern Analysis and Machine Intelligence , 45(2):15811593, 2022.

M. Zhang, D. Ceylan, and N. Mitra. Motion Guided Deep Dynamic 3D Garments. ACM Transactions on Graphics , 41(6):1-12, 2022.

X. Chen, G. Wang, D. Zhu, X. Liang, P. Torr, and L. Lin. Structure-Preserving 3D Garment Modeling with Neural Sewing Machines. In Advances in Neural Information Processing Systems , 2022.

E. Corona, A. Pumarola, G. Alenya, G. Pons-Moll, and F. Moreno-Noguer. Smplicit: TopologyAware Generative Model for Clothed People. In Conference on Computer Vision and Pattern Recognition , 2021.

R. Li, B. Guillard, E. Remelli, and P. Fua. DIG: Draping Implicit Garment over the Human Body. In Asian Conference on Computer Vision , 2022.

Luca DeLuigi, Ren Li, Benoît Guillard, Mathieu Salzmann, and Pascal Fua. DrapeNet: Generating Garments and Draping them with Self-Supervision. In Conference on Computer Vision and Pattern Recognition , 2023.

F. Bogo, A. Kanazawa, C. Lassner, P. Gehler, J. Romero, and M. J. Black. Keep It SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image. In European Conference on Computer Vision , 2016.

Y. Li, M. Habermann, B. Thomaszewski, S. Coros, T. Beeler, and C. Theobalt. Deep physicsaware inference of cloth deformation for monocular human performance capture. In International Conference on 3D Vision , 2021.

J. Liang, M. Lin, and V. Koltun. Differentiable Cloth Simulation for Inverse Problems. In Advances in Neural Information Processing Systems , 2019.

R. Narain, A. Samii, and J.F. O'brien. Adaptive anisotropic remeshing for cloth simulation. ACM Transactions on Graphics , 2012.

R. Narain, T. Pfaff, and J.F. O'Brien. Folding and crumpling adaptive sheets. ACM Transactions on Graphics , 2013.

I. Zakharkin, K. Mazur, A. Grigorev, and V. Lempitsky. Point-based modeling of human clothing. In International Conference on Computer Vision , 2021.

A. Grigorev, B. Thomaszewski, M.J. Black, and O. Hilliges. HOOD: Hierarchical Graphs for Generalized Modeling of Clothing Dynamics. In Conference on Computer Vision and Pattern Recognition , 2023.

I. Santesteban, M.A. Otaduy, N. Thuerey, and D. Casas. Ulnef: Untangled layered neural fields for mix-and-match virtual try-on. In Advances in Neural Information Processing Systems , 2022.

Clo 3d. https://www.clo3d.com/en/ .

Browzwear. https://browzwear.com/ .

Marvellous designer. https://www.marvelousdesigner.com/ .

N. Umetani, D. M. Kaufman, T. Igarashi, and E. Grinspun. Sensitive Couture for Interactive Garment Editing and Modeling. ACM SIGGRAPH , 30(4), 2011.

A. Kaspar, K. Wu, Y. Luo, L. Makatura, and W. Matusik. Knit Sketching: from Cut & Sew Patterns to Machine-Knit Garments. ACM Transactions on Graphics , 40(4):1-15, 2021.

T. Y. Wang, D. Ceylan, J. Popovic, and N. J. Mitra. Learning a Shared Shape Space for

Multimodal Garment Design. In ACM SIGGRAPH Asia , 2018.

R. Vidaurre, I. Santesteban, E. Garces, and D. Casas. Fully Convolutional Graph Neural Networks for Parametric Virtual Try-On. In Computer Graphics Forum , pages 145-156, 2020.

M. Korosteleva and S. Lee. Generating Datasets of 3D Garments with Sewing Patterns. In Advances in Neural Information Processing Systems , 2021.

B. Guillard, F. Stella, and P. Fua. MeshUDF: Fast and Differentiable Meshing of Unsigned Distance Field Networks. In European Conference on Computer Vision , 2022.

T. Groueix, M. Fisher, V. Kim, B. Russell, and M. Aubry. Atlasnet: A Papier-Mâché Approach to Learning 3D Surface Generation. In Conference on Computer Vision and Pattern Recognition , 2018.

J. Chibane, A. Mir, and G. Pons-Moll. Neural Unsigned Distance Fields for Implicit Function Learning. In Advances in Neural Information Processing Systems , 2020.

E. Remelli, A. Lukoianov, S. Richter, B. Guillard, T. Bagautdinov, P. Baque, and P. Fua. Meshsdf: Differentiable Iso-Surface Extraction. In Advances in Neural Information Processing Systems , 2020.

B. Guillard, E. Remelli, A. Lukoianov, S. Richter, T. Bagautdinov, P. Baque, and P. Fua. Deepmesh: Differentiable Iso-Surface Extraction. In arXiv Preprint , 2022.

N. Mahmood, N. Ghorbani, N. F. Troje, G. Pons-Moll, and M. J. Black. AMASS: Archive of Motion Capture as Surface Shapes. In International Conference on Computer Vision , pages 5442-5451, 2019.

W.E. Lorensen and H.E. Cline. Marching Cubes: A High Resolution 3D Surface Construction Algorithm. In ACM SIGGRAPH , pages 163-169, 1987.

R. Venkatesh, T. Karmali, S. Sharma, A. Ghosh, R. V. Babu, L. A. Jeni, and M. Singh. Deep Implicit Surface Point Prediction Networks. In International Conference on Computer Vision , 2021.

G. Moon, H. Nam, T. Shiratori, and K.M. Lee. 3d clothed human reconstruction in the wild. In European Conference on Computer Vision , 2022.

Yu Y. Rong, T. Shiratori, and H. Joo. Frankmocap: Fast monocular 3d hand and body motion capture by regression and integration. In International Conference on Computer Vision Workshops , 2021.

P. Li, Y . Xu, Y . Wei, and Y . Yang. Self-Correction for Human Parsing. IEEE Transactions on Pattern Analysis and Machine Intelligence , 44(6):3260-3271, 2020.

R. Li, M. Zheng, S. Karanam, T. Chen, and Z. Wu. Everybody Is Unique: Towards Unbiased Human Mesh Recovery. In British Machine Vision Conference , 2021.

Nikhila Ravi, Jeremy Reizenstein, David Novotny, Taylor Gordon, Wan-Yen Lo, Justin Johnson, and Georgia Gkioxari. PyTorch3D. https://github.com/facebookresearch/ pytorch3d , 2020.

M. Loper, N. Mahmood, J. Romero, G. Pons-Moll, and M.J. Black. SMPL: A Skinned MultiPerson Linear Model. ACM SIGGRAPH Asia , 34(6), 2015.

Appendix

A Additional Results

A.1 Texture Editing

A.2 Garment Reconstruction

A.3 Garment Draping

A.4 Multi-Layer Draping

B Technical Details

B.1 Sewing Patterns for Trousers and Skirts

B.2 Mesh Triangulation

B.3 Proof of the Differentiability of ISP

B.4 Garment Draping

B.5 Recovering Multi-Layered Garments from Images

B.6 Loss Terms, Network Architectures and Training

C Extension to Sewing Patterns with More Panels

D Failure Cases

A Additional Results

A.1 Texture Editing

(a)

(b)

(c)

As illustrated in Fig. 9, with our ISP, the texture of the garment can be easily edited by drawing on the UV panels. The figures drawn on the panel of one garment can be directly transferred to others as shown by Fig. 9(c), since panels are defined on the same UV space.

A.2 Garment Reconstruction

A.2.1 Evaluation Results

In Tabs. 2 and 3 we report the reconstruction results for skirts and trousers on the training and the test set. Similar to the results on shirts shown in the main paper, our method achieves better reconstruction quality than UDF [30] with lower CHD and higher NC at all resolutions, and needs less time to

reconstruct a single mesh. Figs. 10 to 12 show the qualitative results reconstructed by our method for shirts, skirts and trousers respectively.

A.2.2 Latent Space Interpolation

In Figs. 13 to 15, we display the results of interpolation in the latent space of shirts, skirts and trousers respectively. We observe a smooth transformation in both the reconstructed sewing patterns and the garment meshes, despite the different topology and geometry of the given garments.

A.2.3 Comparison with AtlasNet

In Fig. 16, we compare our method with AtlasNet [48] which learns to deform a square patch. AtlasNet struggles to learn a mapping function capable of accurately deforming the square patch to produce a surface that matches the ground truth, especially in the collar region. In contrast, our method leverages the pattern parameterization network I Θ to simplify the training of our mapping function A Φ . Specifically, our approach only requires learning the mapping for points within the panels, resulting in a reconstruction that is more faithful to the ground truth.

A.2.4 Ablation Study

To investigate the impact of the loss terms of Eq. (4) in the main paper on reconstruction quality, we performed an ablation study. We report the results of reconstructing 300 skirts using models trained with and without L normal in Tab. 4. We observe that training with L normal reduces the CHD and increases the NC, thus improving the reconstruction accuracy. In contrast, without L normal , the

AOMAMAA

(b)

(c)

(a)

(b)

(c)

(a)

(b)

(c)

(a)

IWVVV

AAANI

GT

Ours

AtlasNet

GT

w/

ℒ𝑛𝑜𝑟𝑚

w/o

ℒ𝑛𝑜𝑟𝑚

(a)

(b)

M

model fails to learn the correct parameterization for the garment, resulting in a mesh with inverted faces as illustrated in Fig. 17. Fig. 18 shows a comparison of the results obtained by training models without and with L consist to help stitch the front and back panels. We can notice that without L consist , a spatial gap exists between the front (in gray) and back (in cyan) surfaces.

A.2.5 Vizualization of the Chamfer Distance

A visualization of the spatial error distribution can be found in Fig. 19. In this figure, we compare the error distribution between our reconstructions and those produced by UDFs. Our reconstructions exhibit lower error across the entire surface compared to UDFs.

A.3 Garment Draping

Fig. 20 and Fig. 21(a) present additional comparisons of draping results for our method, DIG [29] and DrapeNet [30]. Our results show higher fidelity and fewer artifacts compared to the other methods. We also compare our method with SNUG [18], a self-supervised method that relies on mesh templates for garment representation and trains one network for each clothing item. In Fig. 21(b), we show the qualitative results on the same shirt. Our results are either comparative or visually superior to those of SNUG, despite using a single draping network for a whole garment category.

A.4 Multi-Layer Draping

To assess the effectiveness of our multi-layering model, we conducted an experiment to measure the intersection ratio between different layers of garments, and between the garments and the body. We used the 673 unseen poses from the test set of AMASS dataset [52] and 673 shape parameters uniformly sampled from the continuous space of [ -2 , 2] 10 to generate 673 unique unseen bodies. For each body, we randomly generated five unseen garments (one skirt or one pair of trousers, with four shirts) by interpolating the garment latent codes and draped them on the body. The first layer contained the skirt or trousers, while the shirts were on the following layers.

(a)

(e)

(d)

(c)

(b)

W

The corresponding results are shown in Tabs. 5 and 6, where the diagonal values represent the intersection ratio between the body and the i -th ( i = 1 , 2 , 3 , 4 , 5 ) layer, and the other values represent the intersection between different layers of garments. It is important to note that we cannot fairly compare these numbers to any other method, as they do not support layering. The left and right sides of the tables correspond to the results obtained with and without the layering procedure, respectively. As shown in the tables, the significantly lower intersection ratios on the left side demonstrate the effectiveness of our model in handling intersections with the body and between different garments. Another noteworthy observation is that the intersections tend to increase as we layer more garments. This behavior is explained in Appendix B.4, where we clarify that our multi-layer draping model D m is trained solely on garments obtained through single-layer draping, which are relatively close to the body. However, it is worth mentioning that even after five layers, the intersections still remain at a low level ( ∼ 2%).

B Technical Details

B.1 Sewing Patterns for Trousers and Skirts

In Fig. 22, we show the sewing patterns used in our experiments for trousers and skirts.

Ours

DrapeNet

Ours

DIG

Ours

SNUG

(a)

(b)

Front

0

1

2

3

0

4

0

Back

0

1

2

3

0

4

0

(a)

Front

1

2

0

0

Back

1

2

0

0

(b)

Sewing Pattern

Sewing Pattern

𝒗

ෝ 𝒗

𝑠 𝑣, 𝑧

< 0

𝑠 𝑣, 𝑧

> 0

𝑠 𝑣, 𝑧

= 0

Front

Back

𝒗

ෝ 𝒗

𝑠 𝑣, 𝑧

< 0

𝑠 𝑣, 𝑧

> 0

𝑠 𝑣, 𝑧

= 0

TPs

TSs

A

T

T

TPb

AΦb

TSb

Le

B.2 Mesh Triangulation

In this section, we detail the meshing process of ISP. We first create a square 2D mesh T for Ω as shown on the left of Fig. 23. Given the latent code z of a specific garment, for each vertex v ∈ V Ω , we compute its signed distance value s and edge label c with ( s, c ) = I Θ ( v, z ) . The 2D front and back panel meshes T P f and T P b are constructed by keeping vertices of T with negative signed distance (the blue region of the colored gird in Fig. 23) and those that have positive signed distance but belong to the edges crossing the 0 iso-level (the gray region of the colored gird in Fig. 23). For the later ones, we adjust their positions from v to ˆ v = v -s ( v, z ) ∇ s ( v, z ) to project them to the zero level set (the blue line). Finally, we query A Φ f and A Φ b on each vertex of T P f and T P b respectively to lift them to 3D, giving us the front and back surfaces T S f and T S b as shown on the right of Fig. 23.

To sew the lifted front and back surfaces T S f and T S b , we perform triangulation with the help of panel meshes T P f and T P b . As illustrated in Fig. 24, we group the boundary vertices of T P f and

Triangulation

Before Sewing

After Sewing

Merge

and

𝐹 𝑠𝑒𝑤

𝐸𝑐

𝑓

𝐸𝑐

𝑏

T

TPo

T P b whose predicted labels are the same ( c , with c > 0 ) to form the sewing edges E f c and E b c for the front and back panels separately. Then we create faces F sew between the vertices of E f c and E b c , and use F sew to merge the meshes of T S f and T S b , which gives us the final assembled garment mesh.

B.3 Proof of the Differentiability of ISP

According to the Theorem 1 of [50], for an SDF s and the point x 0 lying on the 0 iso-level l = { q | s ( q , z ) = 0 , q ∈ Ω } , we have

For point x lying on the α iso-level, we can have s ( x , z ) = α , where α is a constant. Let s α = s -α , then x lies on the 0 iso-level of s α . Based on Eq. 13, we can have

Assume v is a vertex of the mesh T G reconstructed by ISP and x is the point on the UV space that satisfies v = A Φ ( x , z ) , then it holds that

Since ∂s α ∂s = 1 , we can substitute Eq. 14 into Eq. 16 to derive that

B.4 Garment Draping

Single Layer Draping. In Fig. 25, we illustrate the pipeline for the single layer draping, which relies on the diffused LBS of SMPL [60] to get the initial rough estimate of the garment shape and the displacement map output by D s to refine it.

Multi-Layer Draping. Our layering network D m can be applied to multiple garments iteratively to resolve collisions between them. More specifically, consider K garments [ G 1 , G 2 , ..., G K ] that are already draped individually by single layer draping network D s . Their subscripts denote their draping order, with smaller ones being closer to the body. We can obtain their rest state maps [ M 1 r , M 2 r , ..., M r K ] as described in Sec. 3.2 of the main paper, and apply Algorithm 1 for layering them by iterating on garments following their draping order.

Note that we only train D m on one pair of garments individually draped by D s . Therefore, it can only resolve the intersections happening at the same layer, which leads to an extra inner loop in Algorithm

(𝜽, 𝜷, 𝒛)

Diffused LBS

Single Layer Draping

D

8

Algorithm 1: Multi-layer Draping

Require : Function ForceMap( a , b ) that computes the force map for a by taking b as the underlying layer; Function PositionMap( a ) that computes the 2D position map of a Layering network D .

Input : An ordered set of garments [ G 1 , G 2 , ..., G K ] [ M 1 , M 2 , ..., M ] .

Output : Layered garments { ˜ G , ˜ G , ..., ˜ G }

1 that moves all the subsequent garments to the same j -th layer. Otherwise, intersections cannot be completely resolved when applying D m to two garments lying on the different layers as illustrated in Fig. 26(c).

B.5 Recovering Multi-Layered Garments from Images

In Fig. 27, we show the segmentation masks used for the optimization in Eq. (11) in the main paper. The optimization is performed from the outer garment to the inner one, i.e., jacket (if detected) → shirt → trousers (or skirt). More specifically, for the first example shown in Fig. 27, we have the detected segmentation mask S 1 , S 2 and S 3 for the jacket, the shirt and the trousers respectively. We first initialize a latent code z 1 for the jacket and perform the following optimization to recover its mesh

(a)

(b)

(c)

Note that the rendered mask is obtained by setting the colors of the jacket and the body mesh vertices to white and black respectively. Then we fix z 1 and initialize a new latent code z 2 for the shirt and perform

to recover the mesh of the shirt. The optimization for the trousers is similar: fixing z 1 and z 2 ; initializing the latent code z 3 for the trousers; performing

B.6 Loss Terms, Network Architectures and Training

Loss Terms. The Chamfer distance loss L CHD of Eq. (4) in the main paper is formulated as

where P is the panel and S is the ground truth surface mesh.

The normal consistency loss L normal of Eq. (4) in the main paper is formulated as

where Fc P and Fc S are the face centers of the panel mesh and the surface mesh, and n X represents the normal of X . n f ( · ) is the function that computes the normal for the face that A Φ ( x , z ) belongs to.

Network Architectures. For each garment category, i.e. shirts, skirts, and trousers, we train one separate set of networks {I Θ , A Φ , D s } . D m is shared by all garment categories. Our models are implemented as the following.

Pattern parameterization network I Θ : We use two separate networks I Θ f and I Θ b to learn the pattern parameterization for the front and back panels. Each of them is implemented as an MLP with Softplus activations.

UV parameterization network A Φ : We use two separate networks A Φ f and A Φ b to learn the UV parameterization for the front and back surfaces. Both of them have the same architecture, which is a 7-layer MLP with a skip connection from the input layer to the middle and Softplus activations.

Latent code z : The dimension is 32.

Diffuse skinning weight model w : A 9-layer MLP with leaky ReLU activations and an extra Softmax layer at the end to normalize the output.

Displacement network D s : A 10-layer MLP with a skip connection from the input layer to the middle and leaky ReLU activations.

Layering network D m : A U-Net with 4 convolution blocks and 4 deconvolution blocks.

Training. We use the Adam [ ? ] optimizer for training our networks. The batch sizes, the learning rates and the numbers of iterations for training are summarized in Table. 7. The hyperparameters of the training losses are summarized in Table. 8. I Θ , A Φ , w and D m are trained with a TESLA V100 GPU, while D s is trained with 3 GPUs. During the training of D m , we randomly select two garments as the outer and inner layers, and let the model learn to resolve intersections between them.

Loss

L

I

L

A

L

D

m

λ

CE

λ

reg

λ

n

λ

c

λ

g

λ

r

ϵ

Value

0.01

0.001

0.01

1

0.5

1

0.005

(a)

(b)

rbsleeve

rfsleeve

back

front

Ibsleeve

Ifsleeve

HXL

HYY

m

C Extension to Sewing Patterns with More Panels

In our experiments, each garment's sewing pattern consists of two panels, the front and the back. However, our ISP can be extended to patterns with any number of panels. For example, we can train I Θ and A Φ on a database of sewing patterns with six panels as shown in Figure 28, using the same training protocol described in the main paper. After training, we can use them to reconstruct the panels and surfaces and produce the sewed mesh as illustrated in Fig. 29. Adding more panels does not result in better reconstructions. For this reason and for the sake of simplicity, we use a model with 2 panels as our default setting.

D Failure Cases

Fig. 30 presents draping results as the number of shirts increases. We observe that the model produces unrealistic deformation when the number of shirts is greater than four. This behavior occurs because our multi-layer draping model D m is only trained on garments obtained by single layer draping as described in Section 3.2 of the main paper. In this scenario, the garments are relatively close to

the body. When applied to cases with more shirts (typically over four), the model may generate unpredictable results with the shirts moving far away from the body. However, we note that this issue can be resolved by finetuning the model progressively on layered garments. We also consider that wearing more than four shirts is not a common scenario.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes to drape multi-layered garments on SMPL-based human bodies with different poses and shapes. The method is inspired by the commonly adopted sewing pattern and learns to map garments from 2D panels, i.e. front and back panels, to 3D surfaces. The author first deforms the single-layered garments in rest pose according to SMPL skinning function, followed by the corrections of interpenetration between layers of garments. Experiments show the effectiveness and efficiency of the method. The author further shows some possible applications such as garment reconstructions from images and garment editing.\n\nSTRENGTHS:\n* An efficient method mapping implicit 2D sewing patterns to 3D garments with good performance.\n* The qualitative results of draping multi-layered garments show less interpenetration.\n* The method is differentiable and is able to be applied to inverse problems.\n\nWEAKNESSES:\n1. While this paper focuses on multi-layered garments, the experiments lack some quantitative results to support the effectiveness. For example, what is the rate of interpenetration between human body and multi-layered garments? What about the interpenetration between different layers of garments? Will different types of garments, such as dress or shirt, lead to different collision rate? How will the collision rates mentioned above change when the number of layers increase? In the meanwhile, please also provide more detail information about the test set for the above evaluation, such as the number of vertices or the resolution of the mesh.\n2. As for the efficiency, the author provides comparisons during training process in Table 1. Could you provide similar comparisons about reconstruction time or model forward time during test? \n3. The distribution of the human parameters in the dataset is unknown. How many different human bodies are sampled? What is the distribution of different gender in both training and test set?\n4. Is there any overlapping between training set and test set? How many unseen garments and human bodies are included in test set?\n5. While the previous method mentioned at L62 is only able to drape multi-layered garments in T-pose, could the author provide some comparisons also in T-pose? Since the method in this paper supports different poses of human and previous method at L62 did very similar work, the comparisons in T-pose are needed to support the effectiveness of the method.\n6. The writings can be further improved. Some descriptions and captions are not clear.\n7. Please change the tone of the sentence at L3. As discussed in related work at L62, previous method is able to drape multi-layered garments.\n8. The layering network at L191-221 seems like a post-processing step, making the multi-layered draping more like a simple extension of single-layered draping.\n\nQUESTIONS:\n* For each garment, should we learn a unique latent vector z? Do we need to train different models for different garments?\n\nLIMITATIONS:\nWhile multi-layered settings are more challenging, this paper need more experiments and further explanations to support the contributions in terms of multi-layered garments.\n\nFirst, though the method shows better qualitative results, some more experiments are needed to support the strengths of the method in terms of multi-layered settings, the efficiency, and even the robustness.\nMore comparisons between the existing methods mentioned at L62 are needed to prove the effectiveness of this method.\n\nSecond, as for the dataset, the number of samples seems not enough. The main concern is that whether the insufficient data would lead to some bias or overfitting and thus achieve better qualitative results. More quantitative results with clear explanations are needed.\n\nFinally, the idea of getting interpenetration-free garments seems more like a post-processing step, where another model is trained and specifically aims to move the penetrated vertices out of the inner mesh. I am concerned regarding this idea. Since this makes the settings of multi-layered garments as a simple extension of single-layered garments, weakening the contributions in the field of multi-layered garments.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-06-19"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors address the task of draping individual multi-layer garments on human body models. In this context, they introduce a respective garment representation suitable for this task. Garments are represented as a set of individual 2D panels whose shape is defined based on a signed distance function (in more detail, the zero-crossing of a function with 2D location and latent vector specific to each garment as inputs). For each 2D panel, a 2D-to-3D mapping (conditioned on the 2D location and latent vector) is used to map the 2D panel to the 3D garment surface, while enforcing continuity across panels. In addition, draping networks are trained to allow draping multiple garments on human bodies (represented based on SMPL model) in different poses. \n\nThe authors provide quantitative and qualitative comparisons that indicate some potential of the proposed approach.\n\nSTRENGTHS:\nTechnical soundness:\n- The approach seems reasonable and the results indicate some improvement over the competing techniques.\n\nEvaluation:\n- The authors provide quantitative and qualitative evaluations with comparisons to alternatives.\n- The supplemental provides further experiments as well as limitations.\n\nExposition:\n- The paper is well-structured and good to follow. Figures/tables and respective captions are informative.\n- Text quality is good.\n- The authors provide a comprehensive supplemental.\n\nReferences:\n- References seem ok, but I am not an expert in this field.\n\nWEAKNESSES:\nTechnical soundness:\n- Table 1 only shows results for shirts. What are the results for the other categories like?\n- The variations in the used dataset are not discussed in detail. More details on this would be interesting and allow a better assessment on the potential and remaining challenges of the presented approach.\n\nEvaluation:\n- The evaluation could have been improved by also showing examples or highlighting what works particularly well in comparison to other approaches and what still remains challenging for the presented method.\n- Providing a visualization of the spatial distribution of the Chamfer distance across the surface would show where the errors are larger and where the errors are small. This could help to show the accuracy a bit better, also when comparing to others.\n- The user study is quite simple regarding the fact that only direct preferences have been checked for. More detailed subquestions could have provided insights on what exactly the participants in the study did not yet like and where further improvement is needed.\n- For Figure 6, zoom-ins could be used to highlight improvements/differences between methods.\n\nReproducibility:\n- The paper presents a relatively complex system. This might complicate reproducibility. It is not clear whether code and data will be made available.\n\nQUESTIONS:\nPlease address my comments under ‘Weaknesses’.\n\nLIMITATIONS:\nLimitations and failure cases have only been discussed in the supplemental.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces ISP, a novel system that can, for the first time, drape a 3D human body with multi-layer garments, without the need of physics-based simulation. Several technical contributions addresses the key aspects of garment draping. To enable learning of garment sewing patterns, the authors cleverly formulate the 2D patterns with 2D signed-distance fileds (SDFs) and label fields. An AtlasNet-style network maps the 2D sewing pattern to 3D, thereby draping clothing to the body. To achieve multi-layer draping, a intuitive mechanism derives a virtual repulsive force based on the 3D locations of draped garments, thereby resolving the collision of cloth.\nExtensive experiments are performed to characterize the model in garment reconstruction, draping, image-based recovery and editing. In all the tasks, ISP has a clear advantage over existing methods, showing its great potential for many downstream tasks in visual computing.\n\nSTRENGTHS:\nI appreciate this paper for its strengths in many aspects:\n\n- Significance of the problem. The paper addressed an important yet long-overlooked problem: automatic modeling, editing and perception of multi-layer clothing. This is a technically challenging problem. Being the first method tackling this (to my knowledge), this paper can potentially have a good impact in various relevant fields in both research and industry (fashion, movie-making etc.).\n\n- Novelty. The technical contributions of this work are strong. It is clever design to leverage neural fields as a formulation for the seemingly discrete 2D sewing patterns, and to use AtlasNet for the draping problem -- both being creative ways to use recent powerful tools. The layer-wise draping is another great example of combining the power of CNNs with physics-inspired losses. In addition, the method provides a unified pipeline for garments of different topology, such that different garment pieces can be modeled in a standardized way -- another favorable property for machine-learning based approaches.\n\n- Evaluation soundness. The experiments holistically validate the characteristics of the proposed model across multiple relevant tasks in vision and graphics, showing the versatility of the method: 3D garment reconstruction, garment draping, image-based garment reconstruction, and shape/texture editing. The implimentation details, runtime benchmarking are well documented. The user study of the draping quality makes the study more compelling. \n\n- Model performance. To my knowledge, this is the first learning-based work that can drape multi-layer clothing on the bodies and handles cloth interpenetration adequately. Even in the single-layered setting, he proposed method has advantages over prior work in terms of less cloth-body intersecting, draping realism, reconstruction accuracy, and inference speed, all being important aspects of garment draping. \n\n- Presentation. The paper is well-written, easy to follow, and technically sound. The math formulations are specific and clear. The illustrations are clear and well complements the text elaboration. \n\nOverall, this is a very strong submission from which I've learned much, and I believe it can be highly impactful.\n\nWEAKNESSES:\nI do not spot major weaknesses in this paper but one observation. In the image-based garment recovery experiment (Sec. 4.4), although recovered garments are correct in their topology and coarse dimensions (e.g. length), but they do not match the details (overall deformation and wrinkles) as that in the image. This inherits the limitation from previous methods such as DrapeNet. Probably in future work a combination with photometric losses can result in more accurate wrinkle estimation?\n\nQUESTIONS:\n- Out of curiosity, how would the model perform when applied to garments that are too complex to be cut into simple front and back pieces (such as a robe or a double-breasted suits)? \n\n- Currently the garment shape vector z is a single, global one for each garment. Would it increase the representation power (hence helpful  to reconstruction tasks) if the z-vector becomes pixel-aligned with the uv map, as done in e.g. NeuralActor (Liu et al., SIGGRAPH Asia 2021) and PoP (Ma et al., ICCV 2021)?\n\n- Following the arguments in L.135, it is only necessary to deform the areas within the front and back panels instead of the full square of the UV map. However in single layer draping (L.184), the displacement MLP outputs the NxNx3 arrays, i.e. covering the entire square, which seems a bit inconsistent to me. To my understanding, in this case the values for out-of-panel areas are null or zero, is that right? If so, how to ensure that?\n\n- Perhaps adding the dimensionality of z to Fig 2 to make it consistent with x?\n\nLIMITATIONS:\nThe technical limitations are discussed in the conclusions and the authors provide an outlook to the solutions.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe article introduces a 3D clothing generation and driving method inspired by the traditional clothing production process, which has made certain contributions to the reconstruction of clothing, especially multi-layer clothing. This method can achieve the reconstruction of multiple pieces of clothing, the editing of clothing shape and texture, as well as the driving and collision detection processes of clothing through two-dimensional to three-dimensional mapping. The feasibility of this method can be demonstrated through training and validation on the synthesized dataset.\n\nSTRENGTHS:\n- A method has been proposed to solve the problem of multi-layer clothing threading, converting the three-dimensional threading problem to the two-dimensional template level for processing, improving the efficiency and effectiveness of collision detection, and providing technological innovation for the reconstruction of multi-layer clothing.\n- A method flow inspired by the designer's clothing production process has been proposed, providing a new approach for the reconstruction of multi-layer clothing.\n\nWEAKNESSES:\n- According to the description in line 226, the training set (400 shirts, 300 skirts, and 200 pants) and the test set (20 shirts, 20 skirts, and 20 pants) of the experiment are both composed of the same three types of clothing. Perhaps fewer types may lead to limitations in the generalization ability of methods?\n- According to the description in section 3.1, the style of clothes is controlled by Potential space interpolation. This means that the style type and change direction of clothes seem to be displayed by the size of Potential space, and the existing data size seems difficult to support enough Potential space.\n- According to the description in line 96, the stitching method of clothing seems to be pre-defined, which limits the generalization of clothing styles. If more flexible and variable stitching methods can be chosen, it seems that better universality can be achieved.\n\nQUESTIONS:\nAccording to the description in section 2.5 of the supporting materials, it seems that a potential code is needed to recover the human body from the image. How was this potential code obtained?\n\nLIMITATIONS:\nNo extra.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-07"}