SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Fig. 20 and Fig. 21(a) present additional comparisons of draping results for our method, DIG [29] and DrapeNet [30]. Our results show higher fidelity and fewer artifacts compared to the other methods. We also compare our method with SNUG [18], a self-supervised method that relies on mesh templates for garment representation and trains one network for each clothing item. In Fig. 21(b), we show the qualitative results on the same shirt. Our results are either comparative or visually superior to those of SNUG, despite using a single draping network for a whole garment category.
A.4 Multi-Layer Draping
To assess the effectiveness of our multi-layering model, we conducted an experiment to measure the intersection ratio between different layers of garments, and between the garments and the body. We used the 673 unseen poses from the test set of AMASS dataset [52] and 673 shape parameters uniformly sampled from the continuous space of  $[-2,2]^{10}$  to generate 673 unique unseen bodies. For each body, we randomly generated five unseen garments (one skirt or one pair of trousers, with four shirts) by interpolating the garment latent codes and draped them on the body. The first layer contained the skirt or trousers, while the shirts were on the following layers.
Figure 19: Comparative analysis of mesh reconstruction and spatial error distribution for shirt, skirt and trousers. (a) represents the ground truth mesh. (b) and (d) illustrate the mesh reconstruction and corresponding spatial error distribution for our proposed method. (c) and (e) display the same for UDFs. Each row provides both front (upper half) and back (lower half) views. In (d) and (e), error magnitude is indicated by color gradation, with white representing large errors and blue small errors.
The corresponding results are shown in Tabs. 5 and 6, where the diagonal values represent the intersection ratio between the body and the  $i$ -th ( $i = 1,2,3,4,5$ ) layer, and the other values represent the intersection between different layers of garments. It is important to note that we cannot fairly compare these numbers to any other method, as they do not support layering. The left and right sides of the tables correspond to the results obtained with and without the layering procedure, respectively. As shown in the tables, the significantly lower intersection ratios on the left side demonstrate the effectiveness of our model in handling intersections with the body and between different garments. Another noteworthy observation is that the intersections tend to increase as we layer more garments. This behavior is explained in Appendix B.4, where we clarify that our multi-layer draping model  $\mathcal{D}_m$  is trained solely on garments obtained through single-layer draping, which are relatively close to the body. However, it is worth mentioning that even after five layers, the intersections still remain at a low level ( $\sim 2\%$ ).
B.1 Sewing Patterns for Trousers and Skirts
In Fig. 22, we show the sewing patterns used in our experiments for trousers and skirts.
Figure 20: The comparison of draping results of our method and DrapeNet.
(b)
Figure 21: The comparison of draping results for (a) our method vs. DIG, and (b) our method vs. SNUG.
w/ Dm	1	2	3	4	5
1	0.000	0.067	0.001	0.000	0.000
2	-	0.003	0.633	0.608	0.636
3	-	-	0.028	1.224	1.158
4	-	-	-	0.019	2.449
5	-	-	-	-	0.035
w/o Dm	1	2	3	4	5
1	0.000	0.456	0.340	0.357	0.233
2	-	0.004	28.033	26.378	26.246
3	-	-	0.046	24.729	24.679
4	-	-	-	0.073	27.821
5	-	-	-	-	0.052
Table 5: Evaluation of intersections with the underlying body and between garments on different layers (the 1st layer is a skirt). Left and right are the results of draping with and without the multi-layering network  $\mathcal{D}_m$ , respectively. The unit is %.
w/ Dm	1	2	3	4	5
1	0.185	0.048	0.000	0.000	0.000
2	-	0.003	0.631	0.588	0.637
3	-	-	0.030	1.227	1.161
4	-	-	-	0.020	2.426
5	-	-	-	-	0.028
w/o Dm	1	2	3	4	5
1	0.185	0.225	0.259	0.223	0.134
2	-	0.004	28.033	26.378	26.246
3	-	-	0.046	24.729	24.679
4	-	-	-	0.073	27.821
5	-	-	-	-	0.052
Table 6: Evaluation of intersections with the underlying body and between garments on different layers (the 1st layer is a pair of trousers). Left and right are the results of draping with and without the multi-layering network  $\mathcal{D}_m$ , respectively. The unit is %.
Figure 22: The sewing patterns for (a) trousers and (b) skirts. The front mesh surfaces are in gray and the back ones in blue.
Figure 23: Meshing Process. Starting with a square 2D mesh  $\mathbf{T}$ , we first extract the front and back panel meshes  $\mathbf{T}_{P_f}$  and  $\mathbf{T}_{P_b}$  using the implicit function  $\mathcal{I}_{\Theta}$ . Then we lift  $\mathbf{T}_{P_f}$  and  $\mathbf{T}_{P_b}$  to 3D to get the surface meshes  $\mathbf{T}_{S_f}$  and  $\mathbf{T}_{S_b}$  by querying  $\mathcal{A}_{\Phi_f}$  and  $\mathcal{A}_{\Phi_b}$  on their vertices respectively.
B.2 Mesh Triangulation
In this section, we detail the meshing process of ISP. We first create a square 2D mesh  $\mathbf{T}$  for  $\Omega$  as shown on the left of Fig. 23. Given the latent code  $\mathbf{z}$  of a specific garment, for each vertex  $v\in V_{\Omega}$ , we compute its signed distance value  $s$  and edge label  $c$  with  $(s,c) = \mathcal{I}_{\Theta}(v,\mathbf{z})$ . The 2D front and back panel meshes  $\mathbf{T}_{P_f}$  and  $\mathbf{T}_{P_b}$  are constructed by keeping vertices of  $\mathbf{T}$  with negative signed distance (the blue region of the colored gird in Fig. 23) and those that have positive signed distance but belong to the edges crossing the 0 iso-level (the gray region of the colored gird in Fig. 23). For the later ones, we adjust their positions from  $v$  to  $\hat{v} = v - s(v,\mathbf{z})\nabla s(v,\mathbf{z})$  to project them to the zero level set (the blue line). Finally, we query  $\mathcal{A}_{\Phi_f}$  and  $\mathcal{A}_{\Phi_b}$  on each vertex of  $\mathbf{T}_{P_f}$  and  $\mathbf{T}_{P_b}$  respectively to lift them to 3D, giving us the front and back surfaces  $\mathbf{T}_{S_f}$  and  $\mathbf{T}_{S_b}$  as shown on the right of Fig. 23.
To sew the lifted front and back surfaces  $\mathbf{T}_{S_f}$  and  $\mathbf{T}_{S_b}$ , we perform triangulation with the help of panel meshes  $\mathbf{T}_{P_f}$  and  $\mathbf{T}_{P_b}$ . As illustrated in Fig. 24, we group the boundary vertices of  $\mathbf{T}_{P_f}$  and
Figure 24: Sewing process. Left: the boundary vertices belonging to the sewing edges  $E_{c}^{f}$  and  $E_{c}^{b}$  are marked in red. Middle: The triangulation is performed between  $E_{c}^{f}$  and  $E_{c}^{b}$  to create new faces  $F_{sew}$ . Right: The mesh generated by merging the front and back surfaces  $\mathbf{T}_{S_f}$  and  $\mathbf{T}_{S_b}$  with  $F_{sew}$ .
$\mathbf{T}_{P_b}$  whose predicted labels are the same  $(c,$  with  $c > 0$  ) to form the sewing edges  $E_{c}^{f}$  and  $E_{c}^{b}$  for the front and back panels separately. Then we create faces  $F_{sew}$  between the vertices of  $E_{c}^{f}$  and  $E_{c}^{b}$ , and use  $F_{sew}$  to merge the meshes of  $\mathbf{T}_{S_f}$  and  $\mathbf{T}_{S_b}$ , which gives us the final assembled garment mesh.
B.3 Proof of the Differentiability of ISP
According to the Theorem 1 of [50], for an SDF  $s$  and the point  $\mathbf{x}_0$  lying on the 0 iso-level  $l = \{\mathbf{q}|s(\mathbf{q},\mathbf{z}) = 0,\mathbf{q}\in \Omega \}$ , we have
$$
\frac {\partial \mathbf {x} _ {0}}{\partial s} = - \nabla s (\mathbf {x} _ {0}, \mathbf {z}). \tag {13}
$$
For point  $\mathbf{x}$  lying on the  $\alpha$  iso-level, we can have  $s(\mathbf{x},\mathbf{z}) = \alpha$ , where  $\alpha$  is a constant. Let  $s_{\alpha} = s - \alpha$ , then  $\mathbf{x}$  lies on the 0 iso-level of  $s_{\alpha}$ . Based on Eq. 13, we can have
$$
\frac {\partial \mathbf {x}}{\partial s _ {\alpha}} = - \nabla s _ {\alpha} (\mathbf {x}, \mathbf {z}) = - \nabla s (\mathbf {x}, \mathbf {z}). \tag {14}
$$
Assume  $v$  is a vertex of the mesh  $\mathbf{T}_G$  reconstructed by ISP and  $\mathbf{x}$  is the point on the UV space that satisfies  $v = \mathcal{A}_{\Phi}(\mathbf{x},\mathbf{z})$ , then it holds that
$$
\frac {\partial v}{\partial \mathbf {z}} = \frac {\partial \mathcal {A} _ {\Phi}}{\partial \mathbf {z}} (\mathbf {x}, \mathbf {z}) + \frac {\partial \mathcal {A} _ {\Phi}}{\partial \mathbf {x}} \frac {\partial \mathbf {x}}{\partial \mathbf {z}} (\mathbf {x}, \mathbf {z}), \tag {15}
$$
$$
= \frac {\partial \mathcal {A} _ {\Phi}}{\partial \mathbf {z}} (\mathbf {x}, \mathbf {z}) + \frac {\partial \mathcal {A} _ {\Phi}}{\partial \mathbf {x}} \frac {\partial \mathbf {x}}{\partial s _ {\alpha}} \frac {\partial s _ {\alpha}}{\partial s} \frac {\partial s}{\partial \mathbf {z}} (\mathbf {x}, \mathbf {z}). \tag {16}
$$
Since  $\frac{\partial s_{\alpha}}{\partial s} = 1$ , we can substitute Eq. 14 into Eq. 16 to derive that
$$
\frac {\partial v}{\partial \mathbf {z}} = \frac {\partial \mathcal {A} _ {\Phi}}{\partial \mathbf {z}} (\mathbf {x}, \mathbf {z}) - \frac {\partial \mathcal {A} _ {\Phi}}{\partial \mathbf {x}} \nabla s (\mathbf {x}, \mathbf {z}) \frac {\partial s}{\partial \mathbf {z}} (\mathbf {x}, \mathbf {z}). \tag {17}
$$
B.4 Garment Draping
Single Layer Draping. In Fig. 25, we illustrate the pipeline for the single layer draping, which relies on the diffused LBS of SMPL [60] to get the initial rough estimate of the garment shape and the displacement map output by  $\mathcal{D}_s$  to refine it.
Multi-Layer Draping. Our layering network  $\mathcal{D}_m$  can be applied to multiple garments iteratively to resolve collisions between them. More specifically, consider  $K$  garments  $[G_1, G_2, \dots, G_K]$  that are already draped individually by single layer draping network  $\mathcal{D}_s$ . Their subscripts denote their draping order, with smaller ones being closer to the body. We can obtain their rest state maps  $[M_r^1, M_r^2, \dots, M_r^K]$  as described in Sec. 3.2 of the main paper, and apply Algorithm 1 for layering them by iterating on garments following their draping order.
Note that we only train  $\mathcal{D}_m$  on one pair of garments individually draped by  $\mathcal{D}_s$ . Therefore, it can only resolve the intersections happening at the same layer, which leads to an extra inner loop in Algorithm
Figure 25: Single layer draping. The rest-state garment is first deformed by the diffused LBS to get the initial shape, and then refined by the displacement maps predicted by  $\mathcal{D}_s$ .
Figure 26: Layering. (a) We drape a red and an orange shirts on the body. A third one (yellow) is draped by (b) Algorithm 1 and (c) by naively applying  $\mathcal{D}_m$  to it.
Algorithm 1: Multi-layer Draping
Require: Function ForceMap  $(a,b)$  that computes the force map for  $a$  by taking  $b$  as the underlying layer; Function PositionMap  $(a)$  that computes the 2D position map of  $a$ ; Layering network  $\mathcal{D}_m$ .  
Input: An ordered set of garments  $[G_1,G_2,\dots,G_K]$ ; Rest state maps for each garemnts  $[M_r^1,M_r^2,\dots,M_r^K]$ .
Output : Layered garments  $\{\tilde{G}_1,\tilde{G}_2,\dots,\tilde{G}_K\}$  without intersections.
1 for  $i\gets 1$  to  $K$  do   
2  $\tilde{G}_i\gets G_i;$    
3 for  $j\gets i + 1$  to  $K$  do   
4  $\begin{array}{l}M_f^j\gets \mathrm{ForceMap}(G_j,\tilde{G}_i);\\ M_d^j\gets \mathrm{PositionMap}(G_j);\\ \mathrm{Update~vertex~positions~of~}G_j\mathrm{~by~}\mathcal{D}_m(M_r^j,M_d^j,M_f^j); \end{array}$    
5   
6
1 that moves all the subsequent garments to the same  $j$ -th layer. Otherwise, intersections cannot be completely resolved when applying  $\mathcal{D}_m$  to two garments lying on the different layers as illustrated in Fig. 26(c).
B.5 Recovering Multi-Layered Garments from Images
Figure 27: Segmentation masks obtained from [57]. Jackets are marked in gray, shirts in purple, skirts in dark purple, trousers in red, and body parts in other colors.
In Fig. 27, we show the segmentation masks used for the optimization in Eq. (11) in the main paper. The optimization is performed from the outer garment to the inner one, i.e., jacket (if detected)  $\rightarrow$  shirt  $\rightarrow$  trousers (or skirt). More specifically, for the first example shown in Fig. 27, we have the detected segmentation mask  $\mathbf{S}_1$ ,  $\mathbf{S}_2$  and  $\mathbf{S}_3$  for the jacket, the shirt and the trousers respectively. We first initialize a latent code  $\mathbf{z}_1$  for the jacket and perform the following optimization to recover its mesh
$$
\mathbf {z} _ {1} ^ {*} = \arg \min  _ {\mathbf {z} _ {1}} L _ {\mathrm {I o U}} \left(R \left(\mathcal {G} (\mathbf {B}, \boldsymbol {\Theta}, \mathbf {z} _ {1}) \oplus \mathcal {M} (\mathbf {B}, \boldsymbol {\Theta})\right), \mathbf {S} _ {1}\right), \tag {18}
$$
$$
\mathcal {G} (\mathbf {B}, \boldsymbol {\Theta}, \mathbf {z} _ {1}) = \mathcal {D} \left(\mathbf {B}, \boldsymbol {\Theta}, \mathbf {z} _ {1}, \mathbf {T} _ {G} (\mathbf {z} _ {1})\right), \tag {19}
$$
Note that the rendered mask is obtained by setting the colors of the jacket and the body mesh vertices to white and black respectively. Then we fix  $\mathbf{z}_1$  and initialize a new latent code  $\mathbf{z}_2$  for the shirt and perform
$$
\mathbf {z} _ {2} ^ {*} = \underset {\mathbf {z} _ {2}} {\arg \min } L _ {\mathrm {I o U}} \left(R \left(\mathcal {G} (\mathbf {B}, \boldsymbol {\Theta}, \mathbf {z} _ {1: 2}) \oplus \mathcal {M} (\mathbf {B}, \boldsymbol {\Theta})\right), \mathbf {S} _ {2}\right), \tag {20}
$$
$$
\mathcal {G} (\mathbf {B}, \boldsymbol {\Theta}, \mathbf {z} _ {1: 2}) = \mathcal {D} (\mathbf {B}, \boldsymbol {\Theta}, \mathbf {z} _ {1}, \mathbf {T} _ {G} (\mathbf {z} _ {1})) \oplus \mathcal {D} (\mathbf {B}, \boldsymbol {\Theta}, \mathbf {z} _ {2}, \mathbf {T} _ {G} (\mathbf {z} _ {2})), \tag {21}
$$
to recover the mesh of the shirt. The optimization for the trousers is similar: fixing  $\mathbf{z}_1$  and  $\mathbf{z}_2$ ; initializing the latent code  $\mathbf{z}_3$  for the trousers; performing
$$
\mathbf {z} _ {3} ^ {*} = \arg \min  _ {\mathbf {z} _ {3}} L _ {\mathrm {I o U}} \left(R \left(\mathcal {G} (\mathbf {B}, \boldsymbol {\Theta}, \mathbf {z} _ {1: 3}) \oplus \mathcal {M} (\mathbf {B}, \boldsymbol {\Theta})\right), \mathbf {S} _ {3}\right), \tag {22}
$$
$$
\mathcal {G} (\mathbf {B}, \boldsymbol {\Theta}, \mathbf {z} _ {1: 3}) = \mathcal {D} (\mathbf {B}, \boldsymbol {\Theta}, \mathbf {z} _ {1}, \mathbf {T} _ {G} (\mathbf {z} _ {1})) \oplus \mathcal {D} (\mathbf {B}, \boldsymbol {\Theta}, \mathbf {z} _ {2}, \mathbf {T} _ {G} (\mathbf {z} _ {2})) \oplus \mathcal {D} (\mathbf {B}, \boldsymbol {\Theta}, \mathbf {z} _ {3}, \mathbf {T} _ {G} (\mathbf {z} _ {3})). \tag {23}
$$
B.6 Loss Terms, Network Architectures and Training
Loss Terms. The Chamfer distance loss  $\mathcal{L}_{CHD}$  of Eq. (4) in the main paper is formulated as
$$
\mathcal {L} _ {C H D} = \sum_ {\mathbf {x} \in P} \min  _ {\mathbf {X} \in \mathcal {S}} | | \mathcal {A} _ {\Phi} (\mathbf {x}, \mathbf {z}) - \mathbf {X} | | _ {2} ^ {2} + \sum_ {\mathbf {X} \in \mathcal {S}} \min  _ {\mathbf {x} \in P} | | \mathcal {A} _ {\Phi} (\mathbf {x}, \mathbf {z}), \mathbf {X} | | _ {2} ^ {2}, \tag {24}
$$
where  $P$  is the panel and  $\mathcal{S}$  is the ground truth surface mesh.
The normal consistency loss  $\mathcal{L}_{\text{normal}}$  of Eq. (4) in the main paper is formulated as
$$
\mathcal {L} _ {\text {n o r m a l}} = \sum_ {\mathbf {x} \in F c _ {P}} \left(1 - n _ {f} \left(\mathcal {A} _ {\Phi} (\mathbf {x}, \mathbf {z})\right) \cdot \mathbf {n} _ {\mathbf {X} ^ {*}}\right) + \sum_ {\mathbf {X} \in F c _ {\mathcal {S}}} \left(1 - n _ {f} \left(\mathcal {A} _ {\Phi} \left(\mathbf {x} ^ {*}, \mathbf {z}\right)\right) \cdot \mathbf {n} _ {\mathbf {X}}\right), \tag {25}
$$
$$
\mathbf {X} ^ {*} = \underset {\mathbf {X} \in F c _ {S}} {\operatorname {a r g m i n}} \| \mathbf {X} - \mathcal {A} _ {\Phi} (\mathbf {x}, \mathbf {z}) \| _ {2}, \quad \mathbf {x} ^ {*} = \underset {\mathbf {x} \in F c _ {P}} {\operatorname {a r g m i n}} \| \mathbf {X} - \mathcal {A} _ {\Phi} (\mathbf {x}, \mathbf {z}) \| _ {2} \tag {26}
$$
where  $Fc_{P}$  and  $Fc_{S}$  are the face centers of the panel mesh and the surface mesh, and  $\mathbf{n}_{\mathbf{X}}$  represents the normal of  $\mathbf{X}$ .  $n_f(\cdot)$  is the function that computes the normal for the face that  $\mathcal{A}_{\Phi}(\mathbf{x},\mathbf{z})$  belongs to.
Network Architectures. For each garment category, i.e. shirts, skirts, and trousers, we train one separate set of networks  $\{\mathcal{I}_{\Theta},\mathcal{A}_{\Phi},\mathcal{D}_s\}$ .  $\mathcal{D}_m$  is shared by all garment categories. Our models are implemented as the following.
- Pattern parameterization network  $\mathcal{I}_{\Theta}$ : We use two separate networks  $\mathcal{I}_{\Theta_f}$  and  $\mathcal{I}_{\Theta_b}$  to learn the pattern parameterization for the front and back panels. Each of them is implemented as an MLP with Softplus activations.
- UV parameterization network  $\mathcal{A}_{\Phi}$ : We use two separate networks  $\mathcal{A}_{\Phi_f}$  and  $\mathcal{A}_{\Phi_b}$  to learn the UV parameterization for the front and back surfaces. Both of them have the same architecture, which is a 7-layer MLP with a skip connection from the input layer to the middle and Softplus activations.
- Latent code z: The dimension is 32.
- Diffuse skinning weight model  $w$ : A 9-layer MLP with leaky ReLU activations and an extra Softmax layer at the end to normalize the output.
- Displacement network  $\mathcal{D}_s$ : A 10-layer MLP with a skip connection from the input layer to the middle and leaky ReLU activations.
- Layering network  $\mathcal{D}_m$ : A U-Net with 4 convolution blocks and 4 deconvolution blocks.
Training. We use the Adam [?] optimizer for training our networks. The batch sizes, the learning rates and the numbers of iterations for training are summarized in Table. 7. The hyperparameters of the training losses are summarized in Table. 8.  $\mathcal{I}_{\Theta}$ ,  $\mathcal{A}_{\Phi}$ ,  $w$  and  $\mathcal{D}_m$  are trained with a TESLA V100 GPU, while  $\mathcal{D}_s$  is trained with 3 GPUs. During the training of  $\mathcal{D}_m$ , we randomly select two garments as the outer and inner layers, and let the model learn to resolve intersections between them.
Network	IΘ	AΦ	w	Ds	Dm
Learning Rate	10-4	10-4	10-4	5 × 10-5	10-4
Batch Size	50	50	6000	30	6
Iterations	70000	70000	2000	20000	30000
Table 7: Training hyperparameters.
Loss	L1λCEλreg	L2λnλc	L3λg	L4mλrε
Value	0.01 0.001	0.01 1	0.5 1	0.005
Table 8: Training loss hyperparameters for  ${\mathcal{L}}_{\mathcal{I}},{\mathcal{L}}_{\mathcal{A}}$  and  ${\mathcal{L}}_{{\mathcal{D}}_{m}}$  .
Figure 28: The 6-panel sewing pattern. (a) The 3D mesh surface for a shirt. The corresponding surface for each panel is denoted in different colors. (b) The six 2D panels for the front, the back, the right front/back sleeves and the left front/back sleeves.
Figure 29: Reconstruction with 6 panels. For both the left and right examples, we show the reconstructed panels on the top and the reconstructed surfaces and the sewed meshes at the bottom. Colors on panels denote edge labels predicted by  $\mathcal{I}_{\Theta}$ .
C Extension to Sewing Patterns with More Panels
In our experiments, each garment's sewing pattern consists of two panels, the front and the back. However, our ISP can be extended to patterns with any number of panels. For example, we can train  $\mathcal{I}_{\Theta}$  and  $\mathcal{A}_{\Phi}$  on a database of sewing patterns with six panels as shown in Figure 28, using the same training protocol described in the main paper. After training, we can use them to reconstruct the panels and surfaces and produce the sewed mesh as illustrated in Fig. 29. Adding more panels does not result in better reconstructions. For this reason and for the sake of simplicity, we use a model with 2 panels as our default setting.
D Failure Cases
Fig. 30 presents draping results as the number of shirts increases. We observe that the model produces unrealistic deformation when the number of shirts is greater than four. This behavior occurs because our multi-layer draping model  $\mathcal{D}_m$  is only trained on garments obtained by single layer draping as described in Section 3.2 of the main paper. In this scenario, the garments are relatively close to
Figure 30: Draping increasingly many shirts: from 1 (left) to 7 (right).
the body. When applied to cases with more shirts (typically over four), the model may generate unpredictable results with the shirts moving far away from the body. However, we note that this issue can be resolved by finetuning the model progressively on layered garments. We also consider that wearing more than four shirts is not a common scenario.