Reconstruction of Manipulated Garment with Guided Deformation Prior
Authors: Ren Li
Corentin Dumery
Zhantao Deng
Pascal Fua
Venue: NeurIPS 2024 poster (2024)
Keywords: Garment reconstruction
Deformation priors
Geometric deformations
Garment manipulation
3D to UV mapping
Non-rigid reconstruction
Primary area: machine_vision

Abstract: Modeling the shape of garments has received much attention, but most existing approaches assume the garments to be worn by someone, which constrains the range of shapes they can assume. In this work, we address shape recovery when garments are being manipulated instead of worn, which gives rise to an even larger range of possible shapes. To this end, we leverage the implicit sewing patterns (ISP) model for garment modeling and extend it by adding a diffusion-based deformation prior to represent these shapes. To recover 3D garment shapes from incomplete 3D point clouds acquired when the garment is folded, we map the points to UV space, in which our priors are learned, to produce partial UV maps, and then fit the priors to recover complete UV maps and 2D to 3D mappings. Experimental results demonstrate the superior reconstruction accuracy of our method compared to previous ones, especially when dealing with large non-rigid deformations arising from the manipulations.

Reconstruction of Manipulated Garment with Guided Deformation Prior
Ren Li
Corentin Dumery
Zhantao Deng
Pascal Fua
Computer Vision Lab, EPFL
Lausanne, Switzerland zhantao zhantao.deng@eptl.co ch mascal.fu@@epl...
ren. ren. amn.1iepfl.com corentin. dumerydepfl.cl ch
Abstract
Modeling the shape of garments has received much attention, but most existing approaches assume the garments to be worn by someone, which constrains the range of shapes they can assume. In this work, address shape recovery when garments are being manipulated instead of worn, which gives rise to an even larger range of possible shapes. To this end, leverage the implicit sewing patterns (ISP) model for garment modeling and extend it by adding diffusion- prior represent these shapes. To recover 3D garment shapes from incomplete 3D point clouds acquired when the garment is folded, we map the points to UV space, in which our priors are learned, to produce partial UV maps, and then the priors to recover complete UV maps and 2D to 3D mappings. Experimental results demonstrate the superior reconstruction accuracy our method compared to previous ones, especially when dealing with large non-rigid deformations arising from the manipulations.
deformation
Introduction
Garments play an important role in our daily lives, as we interact with them through wearing, folding, and manipulating them. Therefore, the ability to recover their 3D shape is important in many fields, including virtual try-on, VR/AR, and robotic manipulation. However, since garments non-rigid thin structures with a near-infinite number of degrees of freedom, accurate reconstruction remains a challenge, especially in the presence of massive self-occlusions caused by folding crumpling.
Most existing techniques focus on reconstructing garments being worn by someone and therefore constrained by the body shape. This limits the amount of crumpling and provides a shape prior that can exploited. this paper, we address the even more challenging problem of recovering the shape of garments not being worn and therefore possibly assuming arbitrary shapes, such as those of Fig. TT
To this end, we start from the Implicit Sewing Patterns (ISP) model [ As in models used by clothing designers, each garment consists of individual 2D panels. Their 2D shape is defined by Signed Distance Function and 3D shape by a 2D to 3D mapping. We chose this formalism because it can handle complex garments with various geometries, while preserving differentiability with respect to observations. However, its 3D parameterization is limited to single rest state for each garment and designed to draped on a human body.
To handle garments unconstrained by the wearer's body, we introduce a prior to represent the many plausible deformations, including folding and crumpling. It is learned using a generative diffusion model that generates 2D positional UV maps that are applicable to many different garments. We use recover 3D garment shapes from incomplete 3D point clouds, such those that are acquired using a laser scanner when the garment is folded.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Print Crout Irput
Recontuaiion
Figure I: Recovering the 3D shape of folded and crumpled garments from incomplete point clouds. Top: The input point clouds (green) overlaid the ground truth meshes (gray). Bottom: Our reconstructions.
In practice, given that every panel in the ISP model registered to a unified 2D space parameterized by its UV coordinates, we train a UV mapper assign the points to individual panels and project them into the corresponding UV-space, as shown at the top of Fig. 2 This yields partial UV maps for each panel. We then fit the 2D panels and use guided reverse diffusion process generate complete UV maps from the partial ones.
We validate approach the data from the VR-Folding dataset [2], where point clouds generated from multi-view RGBD images. As shown in Fig. our approach accurately reconstructs 3D garment meshes under high levels of deformation and sell-occlusion. We also demonstrate that our algorithm can handle real point cloud data. Notably, our method achieves this without requiring explicit prior knowledge of the garment geometry. further demonstrating its practical applicability. This goes well-beyond prior diffusion- based work [3] that can only model the deformations of single specific ganment worn the human body. Our implementation and model weights are available at https:/ tegithub..a wwtp:/cce//.lenieeemmmmmaa 0att:255555/.rmmeeeooo.iiggg
2 Related Work
Non-Rigid 3D Reconstruction. Reconstructing non-rigid deforming objects has been a longstand-
ing rescarch area in computer vision and graphics. One line of work [4 6 7 g 9 I0 [I] 12] 13
[5] 16), known as Shape from Template (SfT) and 3D registration, assumes the availability 3D surface templates. These methods aim minimize the difference between the observations, such as captured images or point clouds, and the given template surface to recover the deformed state of the objects with geometric constraints differentiable physies simulators. Unfortunately 3D templates are rarely available. Thus, there are many approaches that rely on free-form techniques without any geometric prior [17
[8 19 20), which warp and accumulate different observations across frames into a 3D volume. However, these methods face challenges when reconstructing regions that remain occluded throughout the sequence. While the algorithms of 21 22 23
25 26 can recover full 24
object geometry from RGB-D vidcos, the shape representations they use are designed for watertight surfaces, and thus not suitable for garments that are thin open surfaces.
In contrast, GarmentNets 27 recovers the full 3D surfaces of previously unseen garments from point clouds. It leverages the Normalized Object Coordinate Space (NOCS) 28 category-specific canonical representation for garments. The garment mesh is reconstructed by mapping the predicted canonical mesh the deformed one. However, GarmentNets only handles garments being grasped. To cover the much wider range of possible states that garments can be in, a large-scale garment manipulation dataset is introduced in (2). It relies model for estimating the complete pose of a given garment. Despite promising results, requiring canonical geometry for garment and an initial shape estimate in the first frame imposes limitations similar to those of template-based methods. In contrast, our proposed method is not subject to these and can recover 3D meshes of garments with unknown geometry.
VR system and uses
learn a tracking
On-Body Garment Reconstruction. Clothed human reconstruction has received significant at- tention recent years. However, the majority of methods primarily focus on clothing that tightly adheres to the body. In these, garments are represented either explicitly using template meshes

---

Point Cloud
Mapping
ROM Fanms
Mook
Fosed Fistums
Maudtad
dul
Resllwen
Po
Pesliional Exbodding
tV Cooetinalos
Up
Spars UV Mip
Deformation Ptior
Λ-
ISP Fitting
Guided Reverse Dilfusion Process
Recevesed Mesh
Coneerted UV Map.A4
Reswend Panal Misk
Figure 2: Our framework. Given a point cloud, we first map to UV space to obtaín sparse UV maps M and panel masks Ó. We recover complete UV maps M and panel masks O from them using ISP and deformation prior, enabling the reconstruction of the deformed garment's 3D mesh.
To handle 38 leverage complex physics simulation steps or feature line estimation to align the garment surface with the input image. The one of [39 reconstructs garments from point clouds by predicting displacement and principal component analysis (PCA) coefficients for the mesh registered to base templates. While effective, these methods' reliance on garment templates limits their generality. To address this limitation, (40) uses the Implicit Sewing Patterns (ISP) model from [ as the garment representation and fits an image-conditioned deformation model to the normal estimation of the garments.
29 30 31 32) or implicitly by signed and unsigned distance functions 33 34
35
loose-fitting garments such as skirts and dresses, the methods of [36] 37
However, all these methods rely on the articulated body shape model 41 as prior. This makes them unsuitable for the perception task in the context of robot manipulations where body is involved and garments can exhibit higher levels of crumpled deformation and occlusions, which is the focus of our work.
Diffusion Model. Diffusion models 42 43
class of generative models that excel at learning complex data distributions through score matching. By iteratively denoising the data, these models generate high-quality samples. They have achieved state-of-the-arr performance in wide range of image-based generative tasks [44
45 46 47). Additionally, diffusion models have found
application in various 3D tasks, such as text-to-31D generation 48 49 48
image-to-3D generation 3 introduced a diffusion-baser shape
(50 48 51 [22. and point cloud synthesis 53 54
Recently,
prior for on-body garment registration, employing UV maps to parameterize the garment. However, its prior is specific to single garment piece and requires coarse registration of the input point clouds. In contrast, our proposed method is capable of handling garments with diverse geometries and does not impose any registration requirements.
3 Method
Given a point cloud E Nx3 that partially represents previously unknown garment, we want to reconstruct an accurate 3D model that captures both its geometry and deformation. Instead of doing this directly in 3D space, we first use a trained UV mapper to map 3D points to the unified 2D UV space in which individual ISP [ garment panels represented. In practice, we typically use two such panels, one for the front, and one for the back. the resulting UV maps, each pixel can either be empty or contain the 3D location of point. We then fit the 2D panels to these and use reverse diffusion process to fill the potentially large holes in the UV maps, which yields a complete reconstruction. Fig, 2 depicts this process.
In this section, we first briefly describe the Implicit Sewing Pattern (ISP) garment model [ which represents the garment geometry using one UV map for each 2D panel and which we extend by adding a diffusion-bann and potentially large garment deformations. We then introduce the UV mapper that maps the point
42 43 deformation model. This makes possible to model plausible

---

cloud data to UV space of the individual panels and discuss our approach to fitting the augmented ISP model to the resulting UV maps.
3.1 ISP Garment Model
Formalization. ISP [T] is a garment model inspired by the sewing patterns used the fashion industry to design and manufacture clothes. Such pattern consists of several 2D panels along with stitch information for assembling them. They are implicitly modeled using 2D signed distance field (SDF) and 2D label field, respectively. For specific garment, its corresponding latent code and a point u in the 2D UV space Ω = [-1, 1]". the ISP model outputs the signed distance 8 to the panel boundary and label using fully connected network Ie
(s. (u.
(I)
The zero crossing of the SDF defines the shape of the panel, with < 0 indicating that u is within the panel and s > 0 indicating that u outside the panel. The label I encodes the stitch information, indicating which panel boundaries should be stitched together. To map the 2D sewing patterns to 3D surfaces, a UV parameterization function As learned to perform the 2D-to-3D mapping. is written
(2) where X E RR' represents the 3D position of u. ISP effectively registers different garments onto unified UV space and establishes the mapping functions between points in UV space and the garments' 3D surfaces. Crucially, this is differentiable representation. Given masks or contours of the panels, we can easily the latent code z to recover the corresponding garment geometry.
X Ao
Training. Training ISP requires the 2D sewing patterns of 3D garments in a rest state, which are not available in most garment datasets, such as CLOTH3D [55]. Following the garment flattening approach described in 40 (66. we cut the garment mesh into front and a back piece according to predefined cutting rules and then unfold them into 2D panels by minimizing an as-rigid-as prrigiddas-possiiiee possible energy [57] to ensure local area preservation. For each garment in the dataset, we generate front and back panel as sewing pattern. By pairing these 2D sewing patterns with their corresponding 3D meshes, we follow the training procedure of []] to learn the weights of the ISP model (Ie. Aq)).
3.2 Modeling Deformations
Although the UV parameterization described above is good at representing garments in their rest state, it does not capture the various deformations that can occur when the garment is subjected to external forces, such as folding or creasing. To address this and model the possibly large deformations of garments, we incorporate deformation prior into ISP.
Given set of deformed garments whose rest states are modeled by ISP, write the corresponding UV maps as
V, Ø if > 0
su 00
M u, w]
(3)
where V E IR3 is the corresponding position on the deformed mesh surface for the UV point (u, v). Su is the SDF value of u, -] denotes standard array addressing and Ø (-1,-1, Note that Ø indicates that u is out of the panel and has no corresponding 3D point. Each M represents specific deformed state for a particular garment. To capture the distribution of plausible deformations represented in this way, we learn a deformation prior using a standard diffusion model 42 43
42 prises a forward and reverse process. The forward process perturbs the clean data X, ~ (xx)) by iteratively adding Gaussian noise (O, I) to it. This is written as
Diffusion.
The popular Denoising Diffusion Probabilistic Model (DDPM) framework
/1 B.x-11 + VBe.
where is the noised intermediate state at step = 1, 2, T, and B, E (0. 1) denotes the variance schedule. The reverse process recovers the clean data from random noise with a trained neural network ep
Be
co(xx))) 11 + oz
X
X;
(5)
Vo
VI
aj

---

where 04 = St, 03 = -I 02 2 N)O.T) and o; = / V 3n-
Training. In our context, concatenate the UV map M generated by Eq. 3 with the panel mask O along the channel dimension to form the training samples X, = [M, O] where
1, if Su Σ 0.
O u, 2)
(6)
sf 0
The panel mask O encodes the shape of the panels as well as the 3D geometry the canonical garment. The network is trained on corrupted Xo to predict the noise by minimizing the loss
L=
cæ (Va,xo + v1 d,c,t) ll2
Once the diffusion model is trained, it learns the deformation prior, enabling it to generate recover realistic and diverse deformations for different garments.
3.3 Mapping Point Cloud to UV Space
To relate input 3D point cloud P of the garment to the UV space in which the deformation prior is learned, we rely on the UV mapper G shown at the top of Fig. 2 For each 3D point, it predicts o, the probability of belonging to either the front back panel, along with the u, coordinates of the pixel where the 3D point should be stored in the UV map. As in [2], we use sparse 3D convolution network (58 to extract features for each point p; P. These raw features are then passed through a transformer encoder with self-attention, producing fused per-point features f, that capture relationships across points. These are fed to an MLP that predicts the per-point UV coordinates. It outputs probability distributions &u E R2 and do E IRK over K discrete values for the u- and v-axes, along with a. is trained by minimizing
LG = C'' d..) + C'E go) + BE(a,
(8)
where denotes the ground-truth values, and CE and BE are the cross entropy and the binary cross entropy, respectively.
Once trained, 9 assigns each point p; a UV coordinate u, (ui, 0z) in the front (a, 2 0.5) or the back (o; < 0.5) panel with
2k
2k
U1 =
+
Ui = + K 1
K
where ku argmax te and ho argmax
Ke (O.....X-1)
AE (0.....-1))
We then combine these predictions with M[u,, vi] = p: and [uu v/] = pixels where a 3D point projected, and M vi] = and (lu Vi} elsewhere, producing the assembled UV map M and the panel mask O.
3.4 Fitting the Model
When the garment deformations are severe, there are many occlusions, and M and Ô are typically sparse. Nevertheless, we can use the deformation model of Section 3.2 to fill-in the holes and recover complete UV maps. To this end, we first recover the 2D shape of the 2D panels and then their individual 3D surfaces, shown at the bottom of Fig.
Panel Recovery. To recover the 2D shape of the panels, we find the latent code z of Eq. that yields patterns matching O as well as possible. We take it to be
argmin Σ R(- R(-su(z)) Aerea O
Su(z) Az |z|l2
(10)
where O4 = Ōu|O, = 1, u E Ω}. R(-) is the ReLU function, sn (z) the SDF value of u computed by ISP, and Aorea and X are the weighting constants. Since the second item of the objective function
5

---

Print
Bark
Root
Sperse Punel Merk
Sparse Funal Mask
Sparse UV Map
Sparse Map
(a)
The projected sparse masks Ô and UV maps M of the point clouds with (a) the maximum volume and (b) the minimum volume. The point clouds are color coded by their 3D positions.
Figure 3:
in Eq. 10 penalizes large panel area, this yields panels O whose contours the zero crossings of the SDF-surround the non-zero point of Ô as closely as possible, as shown in the bottom right of Fig. 2
Given point-cloud sequence, solve Eq. TOolly once, specifically at the frame where the point cloud occupies the maximum volume in 3D space. This choice is motivated by the fact that a large volume of the point cloud indicates less deformation and occlusion of the garment, resulting in more visible and informative mask Ô as shown in Fig. Using this frame for the optimization of leads to more accurate fitting results as demonstrated in the Appendix.
Revease Difiusion Process
Hocoeedd/hho Maske
Guidknce Signess
Suod Madoo
Figure 4: The guided reverse diffusion process. The UV maps of the deformed garment generated by using the observations from the input point cloud as guidance to direct the reverse diffusion process.
Deformation Recovery. The deformation model of Section 3.2 has been trained to generate UV maps representing plausible deformations of garments of many different geometries. To align the generation process with the observation of the sparse UV map M and the Iecoverde mask O introduced in the previous paragraph, we use them as manifold guidance 45 46 diffusion process. We write
in the reverse
(x(x),)) dt
Vx, 1,,00
pV x. d(x0 M.O,O)
(11)
ar 1- ca (xt,
X0
(12)
Va
āi
where p is the guidance step size. The function d measures the difference between the generated result and the observations
M, 16 - -M)l2 | O11
(13)
where & and &, refer to the generated UV map and panel mask respectively, and denotes the elementwise multiplication. When sequential information is available, we additionally refine our method by using the previous prediction Mprev the regularization for the unobserved part
d(E, M, 6, O) = 11ō x (Xo.M -M)l2 + | Ol1 + A]| - Õ) (Xo.M Mprew)lle- (14)
where À is weighting constant. Finally, the garment mesh is inferred from the generated UV map using the mapping function ISP. illustrated in Fig. Ml this process finally produces garment mesh that aligns with the point cloud observation.

---

Experiments
4.1 Dataset, Evaluation Metrics, and Baseline
We train models using data from the VR-Folding 2 and CLOTH3D [55] datasets. The VR- Folding dataset is collected using a VR-based recording system, where participants manipulate garments (i.e., folding and flattening) in simulator through VR interface. The dataset features pants, shirts, tops, and skirts selected from the CLOTH3D, and each category covers a wide shape range. VR-Folding comprises 9767 manipulation videos and 790K multi-view RGB-D frames, which are used to generate point clouds. For the training of ISP, we use the same garments from CLOTH3D those selected in VR-Folding and generate their sewing patterns described in Sec. 3.1] We generate UV maps of deformed garments and the corresponding UV coordinates point clouds using the mapping function of ISP. These UV maps and coordinates serve to train the diffusion model and the UV mapper, respectively. For each garment category, we train separate set of models, using the same training and test splits as [2].
As in 27,, we employ the Chamfer Distance Def and the Correspondence Distance (Der, Ad) evaluation metrics. Def measures the surface reconstruction quality by calculating the Chamfer distance centimeters from the reconstructed mesh to the ground truth. Der represents the point-wise L2 distance in centimeters between the reconstruction and the ground truth, which evaluates the accuracy of garment pose estimation. Note that the correspondences of Der are established by finding the closest point of the ground truth in canonical space instead of the deformed one Des- Finally, we take Aa to be the ratio of frames with Der < cm.
We compare our method against state-of-the- art approaches: GarmentNets 27 and Garmen[Tracking 2] GarmentTracking estimates the per-vertex garment pose based on the given canonical garment mesh and the initialization of the first frame. GarmentNets is single-frane frame garment shape estimation method that utilizes the winding number field for garment meshing. Like our method, GarmentNets does not require ground truth geometry.
4.2 Quantitative Results
Table 1: Quantitative comparisons of our method to GarmentNets and GarmentTracking on VR-
Folding dataset.
Folding
Flattening
Type
Method
Init.
Ag
D
Def
As 1
A
Der
Def 1
GarmentNets
2
N/A
0.8%
21.5%
6.40
1.58
13.2%
59.4%
10.54
3.54
GarmenfTracking
GT
29.8%
85.8%
3.88
1.16
30.7%
83.4%
8.63
1.78
Shirt
GarmentTracking
Pert.
29.0%
85.9%
3.88
1.18
25.4%
81.6%
8.94
1.85
GarmentTracking
GN.
25.4%
78.9%
4.04
1.18
Ours
N/A
84.7%
97.9%
2.36
0.77
78.8
95.2%
4.19
1.08
GarmentNets
27
N/A
16.2%
69.5%
4.43
1.30
1.5%
42.4%
12.54
4.19
GarmentTracking
GT
47.3%
94.0%
3.26
1.07
31.3%
78.2%
8.97
Pants
Garmen(Tracking
Pert.
42.8%
93.6%
3.35
1.10
30.7%
76.9%
9.55
2.71
GarmentTracking
GN
45.1%
92.2%
3.33
1.16
Ours
N/A
75.9%
97.9%
2.69
0.70
60.8%
91.4%
5.32
1.16
GarmentNets
27
N/A 10.3%
53.8%
5.19
1.51
13.1%
42.5%
12.11
2.85
Garmen(Tracking
GT 37.9%
85.9%
3.75
0.99
54.6%
82.8%
6,59
1.15
Тop
GarmentTracking
Pert. 36.6%
86.1%
3.76
1.00
54.2%
82.6%
7.80
2.59
GarmentTracking
GN
21.1%
61.9%
4.82
1.11
Ours
N/A 71.2%
93.5%
2.65
0.74
70.2%
86.2%
5.24
1.08
GarmentNets
N/A 1.1%
30.3%
6.95
1.89
0.1%
7.9%
18.48
5.99
GarmenfTracking
GT 23.5%
71.3%
1.33
5.4%
39.4%
16.09
2.02
Skirt
GarmenfTracking
Pert.
22.8%
70.6%
4.72
1.36
2.3%
35.5%
16.55
2.15
GarmenfTrackinng
GN
14.7%
65.9%
5.36
1.46
Ours
N/A
32.5%
76.5%
4.70
1.04
5.1%
33.1%
14.26
1.75
Table T ssows the quantitative results obtained on the VR-Folding dataset. In the third column, the abbreviation "N/A" indicates the absence of any initialization, while "GT", "Pert." and "GN" represent
7

---

the results of Garment GarmentTracking using the ground ground-truth truth mesh, the noise- tatss--erturbed ground bround-truth mesh, and the estimation of GarmentNets as the initialization, respectively. Our method outperforms the baselines by large margin for both the Folding and Flattening sets. GarmentNets can handle garments without prior knowledge of their geometry but has the lowest reconstruction accuracy. GarmentTracke Tracking is more accurate and benefits from using the given garment mesh prior, but its performance highly influenced by the choice of initialization. When using the network- prredce-de result (GN) instead of the ground truth, its performance drops substantially, particularly for Shirt, Skirt and Top. This greatly restricts its applicability in real-world scenarios where obtaining the ground truth garment mesh in advance is rarely possible. In contrast, our method has such limitation while still achieving the highest reconstruction accuracy. The performance disparity between our method and the baselines is particularly significant for challenging metrics such as Ag (on Folding) and As (on Flattening), for instance, 84.7% vs 29.8% and 78.0% vs 24.5% on Shirt.
We also notice that both our method and the baseline models achieve relatively higher Der and lower Ad values for Skirt compared to other categories. This discrepancy arises from the ambiguous definition of skirt sides due to its rotational symmetry. When the skirt is rotated by specific amount around the medial axis, the resulting shape is nearly identical to the original one, which can yield high Der and low Aa values because they are computed using correspondence between the estimated canonical mesh and the ground truth. Consequently, this symmetrical ambiguity makes these metrics unsuitable for assessing the reconstruction quality of skirts, whereas the Chamfer distance Def does not have this issue, on which we obtain the lowest values. An illustrative example is provided in the Appendix.
Folding
Flattening
Purtss
Top
Shirt
Slart
Input
GT
CammetTrscking
GT
Ous
GammnrTacking
Figure 5: Qualitative comparisons of our method GarmentTracking (initialized with ground truth meshes) on VR-Folding dataset for the categories of Pants, Top, Shirt and Skirt.
4.3 Qualitative Results
In Fi we show the qualitative comparison with GarmentTracking which uses the ground truth garment mesh as the initialization. Garmen(Tracking produces results with inaccurate size and deformation, and unrealistic artifact can show up on the reconstructed surfaces. contrast, our method can recover garment meshes from input point clouds faithfully with correct shape and deformations. Fig. [6) we further show the reconstructed results for folding and a flattening sequences, which demonstrates method can consistently produce accurate results compared with GarmentTracking, More qualitative comparisons can be found in the Appendix.
4.4 Evaluation on Real-World Data
To evaluate our method on real-world data, we capture RGB images of pair of pants and sweater, and compute dense point clouds using the nerfstudio library [59] them, as illustrated Fig. (a). We remove background points and use the resulting point cloud downsampled as input to our method.

---

Folding Sequaroo
Ous
Flatanigg Commertroddng
Chrs
Figure 6: Qualitative comparisons of our method to GarmentTracking (initialized with ground truth meshes) VR-Folding dataset for the sequences of Folding and Flattening.
Side
Side view
Side viev
(a)
(b)
Figure 7: Real-worid evaluation. (a) The captured image and point cloud of the pants. (b) Our reconstructed results.
Fig. 7 (b) shows the qualitative results for the pants (the results of the sweater are included in the Appendix). Despite being trained on simulated data, our method is able to reconstruct 3D meshes for both flat and folded garments in real-world scenarios.

---

Conclusion
We have proposed a method that addresses the challenges of reconstructing garment that not being worn and can be manipulated in complex ways. It leverages the Implicit Sewing Patterns (ISP) model for geometry modeling, a generative diffusion model for learning deformation prior, and UV mapping network to relate the 3D point cloud observations to the UV space where the priors are learned. We have demonstrated the effectiveness of our fitting approach in accurately reconstructing garment meshes in the presence of severe self-occlusion and unknown garment geometries. In future work, we will incorporate accumulated point-cloud information across time to improve the accuracy of UV mapping and mesh reconstruction.
Acknowledgement.
This project was supported in part by the Swiss National Science Foundation.
References
[1] R. Li, B. Guillard, and P. Fua. ISP: Multi-Layered Garment Draping with Implicit Sewing Patterns. In Advances Neural Information Processing Systems, 2023.
[2] H. Xue, W. Xu, J. Zhang. T. Tang, Li, W. Du, R. Ye, and C. Lu. Garmenttracking: Category- Level Garment Pose Tracking. Conference on Computer Vision and Pattern Recognition. pages 11233-21243 2023.
[3] Guo, Prada, D. Xiang, J. Romero, C. Wu, H. S. Park, T. Shiratori, and Saito. Diffusion Shape Prior for 2024.
Cloth Registration. In International Conference on 3D Vision,
[4] D. Ngo, S. Park. A. Jorstad, A. Crivellaro, C. Yoo, and P. Fua. Dense Image Registration and Deformable Surface Reconstruction in Presence of Occlusions and Minimal Texture. In International Conference on Computer Vision, 2015.
[5] M. Perriollat, Hartley, and A. Bartoli. Monocular Template- Based Reconstruction of Inexten- sible Surfaces. International Journal of Computer Vision, 95(2): 2011.
[6] M. Salzmann, J. Pilet, S. Ilié, and P. Fua. Surface Deformation Models for Non-Rigid 3D Shape Recovery. IEEE Transactions on Patlern Analysis and Machine Intelligence, 1481-1448 February 2007.
[7] R. Yu, C. Russell, N. Campbell, and L. Agapito. Direct, Dense, and Deformable: Template- Based RigiR 3D Reconstruction from RGB Video. In International Conference Com- puter Vision, 2015.
[8] A. Pumarola, A. Agudo, L. Porzi, A. Sanfeliu, V. Lepetit, and F. Moreno- Geometry- Aware Network for Non-Rigid Shape Prediction from Single View. In Conference on Computer Vision and Pattern Recognition, June 2018.
[9] N. Kairanda, E. Tretschk, M. Elgharib, C. Theobalt, and V. Golyanik. F-Sft: Shape-From- Template with Physics- Based Deformation Model. In Conference Computer Vision and Pattern Recognition, pages
2022.
[10] B. Amberg, S. Romdhani, and T. Vetter. Optimal Step Nonrigid ICP Algorithms for Surface Registration. In Conference on Computer Vision and Pattern Recognition, 2007.
[11] Y. Furukawa and J. Ponce. Dense 3D Motion Capture from Synchronized Video Streams. In Conference on Computer Vision and Pattern Recognition, pages 1-8, 2008.
[12] M. Salzmann and P. Fua. Linear Local Models for Monocular Reconstruction of Deformable Surfaces. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(5):931-944, 2011.
[13] G. Pons-Moll, Pujades, S. Hu, and M.J. Black. Clotheap: Seamless 4D Clothing Capture and Retargeting, ACM SIGGRAPH, 36(4))311111 July 2017.
10

---

[14] D. Xiang, F. Prada, C. Wu, and J. Hodgins. Monoclotheap: Towards Temporally Coherent Clothing Capture from Monocular RGB Video. In arXiv Preprint, 2020.
[15] Taylor, Shotton, Sharp, and A. Fitzgibbon. The Vitruvian Manifold: Inferring Dense Correspondences for One-Shot Human Pose Estimation. In Conference on Computer Vision and Pattern Recognition, pages 103-110 10,, 2012.
[16] T. Groueix, M. Fisher, V. G. Kim, B. C. Russell, and M. Aubry. 3D-CODED :DD Correspon- dences by Deep Deformation. In European Conference on Computer Vision, 2018.
[17] R. A. Newcombe, D. Fox, and S. M. Seitz. Dynamicfusion: Reconstruction and Tracking of Non-Rigid Scenes in Real-Time. In Conference on Computer Vision and Pattern Recognition, 2015.
[18] A. Bozic, M. Zollhofer, C. Theobalt, and M. NieBner. Deepdeform: Learning Non-Rigid Rgb-D Reconstruction with Semi-Supervised Data. In Conference Computer Vision and Pattern Recognition, pages 7002-7012, 2020.
[19] A. Bozic, P. Palafox, M. Zollhöfer, A. Dai, J. Thies, and M. Nießner. Neural Non-Rigid Tracking. In Advances in Neural Information Processing Systems, pages 18727-18737, 2020.
[20] S. Parashar, Y. Long, M. Salzmann, and P. Fua. A Closed-Form, Pairwise Solution Local Non-Rigid Structure-From From-Motion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024.
[21] Mingsong Dou, Sameh Khamis, Yury Degtyarev, Philip Davidson, Sean Ryan Fanello, Adarsh Kowdle, Sergio Orts Escolano, Christoph Rhemann, David Kim, Jonathan Taylor, et al. Fu- sion4D: Real-Time Performance Capture of Challenging Scenes. ACM Transactions on Gruph- ics, 3- 2016.
[22] M. Dou, P. Davidson, S. R. Fanello, S. Khamis, A. Kowdle, C. Rhemann, V. Tankovich, and S. Izadi. Motion2fusion: Real-Time Volumetric Performance Capture. ACM Transactions on Graphics, 36(6): 2017.
[23] M. Niemeyer, L. Mescheder, M. Oechsle, and A. Geiger. Occupancy Flow: 4D Reconstruction by Learning Particle Dynamics. In International Conference on Computer Vision, 2019.
[24] P. Palafox, A. Božič, J. Thies, M. Nießner, and A. Dai. NPMs: Neural Parametric Models for 3D Deformable Shapes. In International Conference on Computer Vision, 2021.
[25] Y. Li, H. Takehara, T. Taketomi, B. Zheng, and M. Nießner. 4Dcomplete: Non-Rigid Motion Estimation Beyond the Observable Surface. In International Conference on Computer Vision, pages 12706-12716, 2021.
[26] W. Lin, C. Zheng, J.-H. Yong, and F. Xu. Occlusionfusion: Occlusion-Aware Motion Estimation for Real-Time Dynamic 3D Reconstruction. In Conference Computer Vision and Pattern Recognition, pages 1736-1745, 2022.
[27] C. Chi and S. Song. Garmentnets: Category-Level Pose Estimation for Garments via Canonical Space Shape Completion. In International Conference on Computer Vision, pages 3324-3333, 2021.
[28] H. Wang, S. Sridhar, J. Huang, J. Valentin, S. Song, and L. J. Guibas. Normalized Object Coordinate Space for Category- Level 6d Object Pose and Size Estimation. In Conference on Computer Vision and Pattern Recognition, pages 2642-2651 2019.
[29] R. Danerek, E. Dibra, C. öztireli, R. Ziegler, and M. Gross. Deepgarment: 3D Garment Shape Estimation from a Single Image. Eurographics, 2017.
[30] B. L. Bhatnagar, G. Tiwari, C. Theobalt, and G. Pons-Moll. Multi-Garment Net: Learning to Dress 3D People from Images. International Conference on Computer Vision, 2019.
[31] B. Jiang, J. Zhang, Y. Hong, Luo, L. Liu, and Bao. Benet: Learning Body and Cloth Shape from Single Image. In European Conference Computer Vision, 2020.
11

---

[32] X. Liu, J. Li, and G. Lu. Modeling Realistic Clothing from Single Image Under Normal Guide. IEEE Transactions on Visualization and Computer Graphics, 2023.
[33] E. Corona, A. Pumarola, G. Alenyå, and F, Moreno-Noguer. Context-Aware Human Motion Prediction. In Conference on Computer Vision and Pattern Recognition, 2020.
[34] R. Li, B. Guillard, E. Remelli, and P. Fua. DIG: Draping Implicit Garment over the Human
Body. In Asian Conference on Computer Vision, 2022.
[35] I.. DeLuigi, R. Li, B. Guillard, M. Salzmann, and P. Fua. Drapenet: Generating Garments and Draping Them with Seff-suuervision. In Conference on Computer Vision and Pattern Recognition, 2023.
[36] S. Yang, Z. Pan, T. Amert, K. Wang, L. Yu, T. Berg, and M. Lin. Physics- Garment Recovery from a Single-Viet View Image. ACM Transactions on Graphics, 37(5):1-14, 2018.
[37] H. Zhu, Y. Cao. H. Jin, W. Chen, D. Du, Z. Wang, S. Cui, and X. Han. Deep Fashion3D: A Dataset and Benchmark for 3d Garment Reconstruction from Single Images. In European Conference Computer Vision, pages 512-530, 2020.
[38] H. Zhu, L. Qiu, Y. Qiu, and X. Han. Registering Explicit to Implicit: Towards High-Fidelif Garment Mesh Reconstruction from Single Images. In Conference on Computer Vision and Pattern Recognition, pages
2022.
[39] Hong, L. Pan, Z. Cai, and Z. Liu. Garment4D: Garment Reconstruction from Point Cloud Sequences. In Advances in Neural Information Processing Systems, pages 27940-27951, 2021.
[40] R. Li, C. Dumery, B. Guillard, and P. Fua. Garment Recovery with Shape and Deformation Priors. In Conference on Computer Vision and Pattern Recognition, 2024.
[41] M. Loper and M.J. Black. Opendr: An Approximate Differentiable Renderer. In European Conference Computer Vision, pages 154 169,169,10 2014.
[42] Jonathan Ho, Ajay Jain, and Picter Abbeel. Denoising diffusion probabilistic models. In Advances in Neural Information Processing Systems, 2020.
[43] J. Song, C. Meng, and S. Ermon. Denoising Diffusion Implicit Models. In International Conference on Learning Representations, 2021.
[44] Prafulla Dhariwal and Alexander Nichol. Diffusion Models Beat GANs on Image Synthesis. In Advances in Neural Information Processing Systems, volume 34, pages 8780-8794, 2021.
[45] H. Chung, Kim, M. T. Mccann, M. L. Klasky, and C. Ye. Diffusion Posterior Sampling for General Noisy Inverse Problems. In International Conference on Learning Representations, 2022.
[46] H. Chung, B. Sim, D. Ryu, and J. C. Ye. Improving Diffusion Models for Inverse Problems Using Manifold Constraints. In Advances in Neural Information Processing Systems, pages 15683-25696, 2022.
[47] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In Conference on Computer Vision and Pattern Recognition, pages 10684-10699 2022.
[48] Yinghao Xu, Hao Tan, Fujun Luan, Sai Bi, Peng Wang. Jiahao Li, Zifan Shi, Kalyan Sunkavalli, Gordon Wetzstein, Zexiang Xu, et al. Dmv3D: Denoising Multi Diffusion Using 3d Large Reconstruction Model. In International Conference Learning Representations, 2024.
[49] Ben Poole, A. Jain, J. T. Barron, and Ben Mildenhall. Dreamfusion: Text-To-3D Using 2D Diffuusin. In International Conference on Learning Representations, 2022.
[50] N. Müller, Y. Siddiqui, L. Porzi, S. R. Bulo, P. Kontschieder, and M. Nießner. Diffrf: Rendering- Guided 3D Radiance Field Diffusion. In Conference on Computer Vision and Pattern Recogni- tion, pages 4328-4338. 2023.
12

---

[51] T. Anciukevičius, Z. Xu, M. Fisher, P. Henderson, H. Bilen, N. J. Mitra, and P. Guerrero. Renderdiffusion: Image Diffusion for 3D Reconstruction, Inpainting and Generation. In Conference Computer Vision and Pattern Recognition, pages 12608-12618, 2023.
[52] Ruoshi Liu, Rundi Wu, Basile Van Hoorick, Pavel Tokmakov, Sergey Zakharov, and Carl Vondrick. Zero-1-To-3: Zero-Shot One Image to 3D Object. In International Conference on Computer Vision, pages 9298-93099 2023.
[53] M. Tyszkiewic, P. Fua, and E. Trulls. GECCO: SiametrrcallyyConditiinee Point Diffusion Models. In International Conference on Computer Vision, 2023.
[54] L. Melas-Kyriazi, C. Rupprecht, and A. Vedaldi. Pe2: Ionjection-Condttione Point Cloud Diffusion for Single-Image 3D Reconstruction. In Conference on Computer Vision and Pattern Recognition, pages 12923-12932, 2023.
[55] H. Bertiche, M. Madadi, and S. Escalera. CLOTH3D: Clothed 3D Humans. In European Conference on Computer Vision, pages 344-359, 2020.
[56] N. Pictroni, C. Dumery, R. Falque, M. Liu, T. Vidal-Calleja, and O. Sorkine- Hornung. Compu- tational Pattern Making from 3D Garment Models. ACM Transactions Graphics, 41(4):1-14, 2022.
[57] L. Liu, L. Zhang, Y. Xu, C. Gotsman, and S. J. Gortler. A Local/global Approach to Mesh Parameterization, In Proceedings of the Symposium on Geometry Processing, pages 1495-1504, 2008.
[58] C. Choy, J. Park, and V. Koltun. Fully Convolutional Geometric Features. In International Conference on Computer Vision, pages 8958 88886,9666668 2019.
[59] M. Tancik, E. Weber, E. Ng. R. Li, B. J. Kerr, T. Wang, A. Kristoffersen, Austin, K. Salahi, A. Ahuja, D. McAllister, and A. Kanazawa. Nerfstudio: A Modular Framework for Neural Radiance Field Development. In ACM SIGGRAPH,, 2023.
[60] Blender, 2018. http://www.blender.ogg.
[61] O. Ronneberger, P. Fischer, and T. Brox. U-Net: Convolutional Networks for Biomedical Image Segmentation. In Conference Medical Image Computing and Computer Assisted Intervention, pages 234-241, 2015.
[62] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin. Attention All You Need. In Advances in Neural Information Processing Systems, 2017.
[63] D. P. Kingma and J. Ba. Adam: A Method for Stochastic Optimisation. In International Conference on Learning Representations, 2015.
13

---

Appendix
Additional Results
Qualitative Comparisons
A.2
Evaluation on Real- World Data
A.3
Evaluation of Intersections
Ablation Study
Robustness
A.6
Generative Samples
A.7
Evaluation of Panel Mask Fitting
A.8
Rotational Symmetry
A.9
Simulation
B
Implementation Details
Limitations
A
Additional Results
A.1
Qualitative Comparisons
t
Owas
CarnenEhwcingg
Figure The comparison of reconstructed results for the flattening sequence of Skirt.
Fig. we provide additional qualitative comparisons between the results our method and Garment Tracking [2] initialized with the ground truth garment mesh. Our reconstructions demonstrate higher accuracy and greater fidelity to the ground truth.
A.2 Evaluation on Real-World Data
Fig. 16 shows the qualitative results for the sweater. Similar to the results of the pants shown in Fig. of the main paper, our method is able to reconstruct 3D meshes for both the flat and folded sweaters. However, some discrepancies still exist between the input and our reconstruction, which attributed to the sim-to-real gap. Closing this domain gap will be an important direction of future work.
14

---

CanvrElnnoog
Figure 9: The comparison reconstructed results for the flattening sequence of Top.
Commanttaiion Oues Gr
Figure 10: The comparison of reconstructed results for the flattening sequence of Pants.
Cr
Ous
CeekenChhcknng
Figure 11: The comparison of reconstructed results for the flattening sequence of Shirt,
A.3 Evaluation Intersections
In Table we evaluate the intersections our reconstructions and compare them with those of CarmentTracking 2 using the ground-truth initialization. We compute the average ratio of faces with intersection as the evaluation metric. Notably, our results exhibit fewer intersections compared to Garmen(Tracking on Pants, Top and Skirt.
15

---

CT
Ouas
Commandiadinn
Figure 12: The comparison of reconstructed results for the folding sequence of Skirt.
Camerttadon
Figure 13: The comparison of reconstructed results for the folding sequence of Top.
A M
B
Oue
MODiIU
Figure 14: The comparison of reconstructed results for the folding sequence of Pants.
16

---

GT
Figure 15: The comparison of reconstructed results for the folding sequence of Shirt.
Figure 16: Qualitative results for folding real sweater. Top: the images of the sweater. Bottom: our reconstructed results.
Table 2:
The average intersection ratio of faces intersecting another face our reconstructions compared to GarmentTracking [2]. GarmentTracking is initialized with the ground-truth mesh.
Ours
Garmen(Tracking [2]
Pants
1.9%
3.8%
Shirt
2.2%
1.6 %
Top 1.8%
6.4%
Skirt
1.5%
2.7%
A.4 Ablation Study
Table 3: Ablation study. +M, +O and +MMree denote using the guidance of the sparse UV maps, the recovered panel mask and the recovery previous frames, respectively.
Ag t
Aş 1
D
Def 4
+M +M, +0 +M, +0, +Mrre
61.5%
85.0%
3.55
1.12
68.7%
91.2%
2.81
0.80
71.3%
94.2%
2.62
0.72

---

Table presents the evaluation results of our fitting method of Eq. 14 of the main paper with different combinations of guidance on the test set of Top. Utilizing only the sparse UV maps M as guidance results the lowest reconstruction accuracy. However, incorporating the guidance of the ISP-recovered panel mask O, which provides garment geometry information, improves the results significantly. Moreover, by incorporating the recovery of the previous frame Myrees we further reduce the reconstruction error by introducing additional regularization the unobserved surface.
A.5 Robustness
Robustness uncer Differntt Number of Points
Robustness against Norse
3322
1500
3000
25.00
3300
3000
1032
0.0
0.5
1.0
2.
2.0 25
30
Standard Dessation (cm)
(a)
(b)
Figure 17: Quantitative results of (a) using different numbers points as input and (b) under different noise levels with 4000 input points. Blue: the Correspondence Distance Der- Red: the Chamfer Distance Des-
To evaluate the influence of point quantity, we analyze the reconstruction errors by varying the number of points used as input on the subset of Folding Pants. The results are reported in Fig. 17 (a). reduction in points correlates with increased error. However, even with 2000 points, we maintain relatively low error margin.
To evaluate the influence of input point noise, add per-point Gaussian noise the input with varying standard deviation. Fig. 17 (b) shows the results on the subset of Folding Pants. It illustrates that the noise level rises, does the reconstruction error; nonetheless, the errors remain relatively low across different noise levels.
A.6 Generative Samples
In Fig. 18 we show the deformed garment mesh generated by using the diffusion model to denoise the randomly sampled noise images. The resulting plausible deformations of the garment surfaces demonstrate the effectiveness of our diffusion model in capturing meaningful deformation priors.
A.7 Evaluation of Panel Mask Fitting
Fig. 19 shows the curve of the mean Intersection over Union (mloU) between the ground truth panel masks and the masks fitted using Eq, 10 of the main paper. The r-axis represents the volume of the input point cloud, computed as the occupancy of the voxelized 3D space [- 1]3 and normalized by the min-mat max normalization. As shown, larger point cloud volume corresponds better fitting result reflected by a higher mloU. This validates our choice of using the fitted panel mask of the frame with the maximum volume for the entire sequence.
A.8 Rotational Symmetry
As mentioned in Sec. 4.2 of the main paper, the rotational symmetry of skirt can result relatively large Correspondence Distance (Dar, A,). Fig. prooviddes an illustrative example for this issue. Specifically, the front and back sides of the ground truth mesh are defined as Fig. (c). However, our model mistakenly identifies the facing-up up points of Fig. 20 (b) as the front surface,
18

---

Folding
Flattening
Figure 18: The generative samples for the categories of Skirt, Top, Pants and Shirt (top to bottom).
002
m100
479
Figure 19: The evaluation of the panel mask fitting results.
yielding Der of 19.67 cm. Despite this, as illustrated by Fig. 20(c), (d), our reconstruction maintains high quality, with small Chamfer Distance (Def) of 1.21 em.
Predicted UV maps
Ground truth
Back
Front
(8)
(b)
(c)
(d)
Figure 20: (a) The predicted sparse UV maps and the corresponding ground truth for (b) the input point cloud. (c) The ground truth mesh. (d) Our reconstruction. The front and back sides of the meshes are labeled with 'Front' and 'Back', respectively.
19

---

Figure 21: We use the reconstructed shirt meshes (left) to simulate the process dropping them onto bar (right).
A.9 Simulation
Our recovered meshes can be used for animation and simulation directly. Fig. 21 we show the simulated results for the recovered shirts using Blender [60], where we drop them onto a horizontal bar.
B Implementation Details
Following 40 [1], the pattern parameterization network Te and the UV parameterization network As of ISP are implemented as MLPs with the latent code size 128. We train Ia and As jointly for 9000 iterations with batch size of 50. For the diffusion model, we adopt U-Net architecture [6]] takes the concatenated front and back UV maps and panel maps with the dimensions of 128 x 256 x input. The diffusion model is trained for 100 epochs, with learning rate of lc-4, batch size of 64, and T = 1000 steps. The UV mapper G consists a sparse 3D CNN implemented (2) 6-layer Transformer encoder [62], and a 7-layer MLP for UV coordinate prediction. We choose = 128 and train G for 100 epochs, using a learning rate of le-4 and a batch size 128. To augment the data, apply random rotations both the point cloud and the mesh represented as UV maps. All the models are trained using the Adam optimizer
(33 on NVIDIA A100 GPUs.
C Limitations
Due to the thin structure and the close proximity of surfaces during manipulation, self-intersections can occur. An interesting direction for future research is the exploration of learning intersection-free deformation priors with physics- based constraints.
The UV mapper currently relies on single-frame information for predictions. To enhance its per- formance, intend to incorporate accumulated point cloud information across frames in future improvements.
Another limitation lies in the dataset used to train models, which was recorded under controlled settings where participants followed specific manipulation procedures. However, this can be ad- dressed by leveraging the recording platform of [ to introduce wider range of potential garment deformations.
20

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: See Section D
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how be expected to generalize other settings.
much the results
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: See Section C in the Appendix.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
21

---

Justification:
Not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: See Section 4.1 and Section B
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
22

---

Answer:
[No]
Justification:
We will release our codes and models upon acceptance.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.cm
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ //ippe.cc/pullic/guiii wwwticccguie//eeeeeeeeeii CoodeSubeiiieeee ssi ssionnolli onnolicy)))) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
See Section 4.1 and Section B
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: Not available.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, at least for the experiments that support main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
23

---

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of enrors is not verified.
For asymmetric distributions, the authors should be careful figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
show in tables or
error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
See Section B
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics http://tps.//aaa.o..cccc//aaaaaa
Answer: [Yes]
Justification: We have read the ethics review guidelines and ensured that our paper conforms to them.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative socictal impacts of the work performed?
Answer: [NA]
Justification: Our work is about modeling the garment deformation which has no societal impact.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no socictal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: See Section 4.1
Guidelines:
The answer NA means that the paper does existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
25

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets,
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
26


Reviews:
[{"review_id": "FNnh1OAvXe", "reviewer": "Reviewer_2QwG", "summary": "This paper presents a method for reconstructing folded and crumpled garments from point cloud data. It uses the implicit sewing pattern (ISP) model to represent the 3D shape in 2D uv-maps. The proposed method converts a 3D point cloud to sparse uv-maps and corresponding masks for front and back side using an encoder structure followed by a MLP. The incomplete masks are filled and used to guide the completion of the uv-maps via a diffusion process. Finally, the deformed mesh can be recovered from the filled uv-map.", "questions": "How many points do the input point cloud contain?\nDid you test how many points are necessary and how accurate do they have to be to produce a high-quality reconstruction?", "limitations": "The limitations are just mentioned very briefly. Some quantitative evaluation on the number of intersections in the reconstructed mesh or more animated reconstructions would show how large these limitations are. An analysis might even benefit the method as e.g. the number of intersections seems to be low based on the qualitative results.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "This paper improves state-of-the-art reconstruction of point cloud data for folded garments in visual quality as well as 3D accuracy. Notably, this is done while no prior knowledge of the garment geometry is needed. The usage of a diffusion network to fill the sparse 2D data of the ISP model is a clever idea and matches the network characteristics very well.", "weaknesses": "The comprehensibility of the paper could be improved by discussing the different parts of the pipeline in order and clearly pointing out the result of each stage and its purpose for the next stage. Some intermediate results for different scenes might be helpful to follow the pipeline.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank you for your acknowledgement of our contribution in manipulated garment reconstruction. Below are our responses to your comments and questions.\n\n1. *Comprehensibility.*\n\nThank you for pointing this out. At the end of each stage and to enhance comprehensibility, we will revise our paper and refer the reader to the intermediate results in our main framework figure (Fig. 2 of the main paper).\n\n2. *How many points do the input point cloud contain?*\n\nWe use 4000 points randomly sampled from the captured point clouds as the input. To evaluate the influence of point quantity, we analyze the reconstruction errors by varying the number of points used as input on the subset of Folding Pants. The results are reported in Fig. 1 (a) of the attached PDF file. A reduction in points correlates with increased error. However, even with 2000 points, we maintain a relatively low error margin. We will include this experiment in our final version of the paper.\n\n3. *How accurate do the points have to be to produce a high-quality reconstruction?*\n\nTo evaluate the influence of input point noise, we add per-point Gaussian noise to the input with varying standard deviation. Fig. 1 (b) of the attached PDF file shows the results on the subset of Folding Pants. It illustrates that as the noise level rises, so does the reconstruction error; nonetheless, the errors remain relatively low across different noise levels. We will include this experiment in our final version of the paper. Additionally, the evaluation of real-world data in Sec. 4.4 of the main paper also demonstrates the robustness of our method, where the input point cloud is generated using Nerf which is noisy and inaccurate. \n\n4. *Quantitative evaluation on the number of intersections or more animated reconstructions.*\n\nIn Table 1 of the attached PDF file, we evaluate the intersections of our reconstructions and compare them with those of GarmentTracking [2] using the ground-truth initialization. We compute the average ratio of faces with intersection as the evaluation metric. Notably, our results exhibit fewer intersections compared to GarmentTracking on Pants, Top and Skirt. We will revise our paper to include this evaluation and more reconstruction results.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "cskk3z9JPx", "reviewer": "Reviewer_4mMP", "summary": "The pape addresses the challenge of accurately reconstructing the 3D shape of garments that are manipulated rather than worn. The authors leverage the Implicit Sewing Patterns model and introduce a diffusion-based deformation prior to recover 3D garment shapes from incomplete 3D point clouds. The method maps these points to UV space, generates partial UV maps, and uses a reverse diffusion process to produce complete UV maps and 2D to 3D mappings. The approach demonstrates superior accuracy compared to previous methods, especially in handling large non-rigid deformations.", "questions": "Can this method handle noisy or highly sparse point clouds effectively, and have you tested its robustness in such scenarios?", "limitations": "While the method shows promise, its ability to generalize across a wide variety of garment types and materials without retraining is not fully explored.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The focus on reconstructing manipulated garments rather than worn ones addresses a significant gap in current research, as most existing methods assume garments are worn and thus have less complex deformations.\n\nCombining ISP with a diffusion-based deformation prior is a strong methodological contribution, enabling the modeling of complex deformations that were previously challenging to capture.", "weaknesses": "The accuracy of the reconstruction heavily depends on the quality of the input point clouds. Incomplete or noisy point clouds might still pose a challenge.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your appreciation of our work. We address your questions and comments as follows:\n\n1. *Handling noisy or highly sparse point clouds.*\n\nTo evaluate performance under noisy conditions, we add per-point Gaussian noise to the input data, varying the standard deviation. As shown in Fig. 1 (b) of the attached PDF file, the results on the Folding Pants subset indicate that reconstruction error increases with noise levels; however, the errors remain relatively low across different noise levels. Additionally, the evaluation of real-world data in Sec. 4.4 demonstrates the robustness of our method, even when the input point cloud, generated using Nerf, is noisy and inaccurate.\n\nRegarding sparsity, the captured points are generally dense in visible areas. Instead of using all available points, we randomly sample 4000 points from them as the input. We also evaluate the influence of point quantity on reconstruction quality by analyzing errors with varying input point numbers on the Folding Pants subset. The results, shown in Fig. 1 (a) of the attached PDF file, reveal that while a reduction in points leads to increased error, we maintain a relatively low error margin even with only 2000 points.\n\n2. *Generalization across a wide variety of garment types and materials without retraining is not fully explored.*\n\nDue to the challenging nature of our task where only a portion of the garment is observed, we consider category-level garment reconstruction as in prior art GarmentNets [27] and GarmentTracking [2]. However, we acknowledge the reviewer's point that investigating the generalization across types and materials is an important direction for future work.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Ke1wjONhQK", "reviewer": "Reviewer_J5PZ", "summary": "The paper aims to recover garments that are manipulated instead of worn. The method first generates the UV mappings from point clouds, followed by ISP to recover the complete mapping. A diffusion model is used to extract the deformation priors and guide the recovery from UV mappings to 3D mesh. Experiments show that the proposed method delivers lower reconstruction errors and outperforms the baselines.", "questions": "1. Are the recovered garments meshes able to be used for further animations?\n2. How to connect the edges in the recovered mesh from point clouds? Are the edges fixed or dynamically connected? Since the point clouds do not include any information about the connectivities, how to define the edges?\n3. In the qualitative results, such as Figure 5, the recovered garments seem to be smoother than the ground truth. Is this because of the “auto smooth” option during rendering? Could you provide some visual results of the smoothed ground truth garments? \n4. What is the averaged number of points for different garments? Is the model able to deal with large number of points?\n5. While the proposed method is able to handle garments in more complex poses, is it possible to compare with other baselines using the garments worn by the human body, e.g. the quantitative and qualitative results on CLOTH3D dataset?", "limitations": "Please refer to the weaknesses and questions.", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "* This method is able to recover garments in a more general and complex poses.\n* The proposed model achieves robust performance as shown in the experiments.", "weaknesses": "1. This method seems to be garment-specific design, or even topology-depend. For example, to recover the shirt and pants, one need to train different models to recover the garments, leading to limited generalisation abilities.\n2. While the ground truth may include too much details, i.e. too many wrinkles, and even looks a bit noisy, the recovered garments are oversmooth. The proposed model fails to recover high frequency details of the garments.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable reviews. \n\nTo provide a context,  we would first like to briefly describe our pipeline. Given the point cloud, we first map each point to the UV space using the UV mapper. This yields a sparse UV map and a sparse panel mask. We then use Eq. (10) to fit the optimal latent code $\\textbf{z}^*$ for the ISP model from the sparse panel mask. Note that with $\\textbf{z}^*$, we can recover a complete panel mask and a rest-state garment mesh that defines the vertices and faces of the garment. Then, we leverage the diffusion model to recover the complete UV map, utilizing the sparse UV map and the complete panel mask as the guidance in the reverse diffusion process.\n\nThis being said, below is the actual response to your comments. \n\n1. *Garment-specific design.*\n\nOur approach is not specific to a particular garment. However, due to the challenging nature of the task where only a partial garment is observed, we consider category-level garment reconstruction as in much prior art such as GarmentNets [27] and GarmentTracking [2]. Since a folded shirt and a pair of folded trousers can exhibit similar shapes as shown in Fig. 2 of the attached PDF file, we need to use different models for them.\n\n2. *The recovered garments are smooth.*\n\nThe qualitative results in our paper are rendered without smoothing. Both the ground truth meshes and our reconstructed meshes are visualized in their raw form. The fact that our reconstructions seem smoother than the ground truth meshes can be attributed to the tendency of neural networks to learn low-frequency functions [Rahaman2019], which yields smooth reconstructed UV maps. Furthermore, our guided denoising process finds the expected reconstructions $\\hat{x}$ given observations $y$ and the starting noise $ x_T $, where $\\hat{x} = E(x|y,x_T) = \\sum_x xP\\lbrace x|y, x_T\\rbrace$. In areas where the data is missing, it provides only weak guidance and the reconstructions are naturally smooth. In future work, we will explore methods to enhance our diffusion model to capture finer details. \n\n3. *Are the recovered garments meshes able to be used for further animations?*\n\nYes, our recovered meshes can be used for animation and simulation directly. In Fig. 3 of the attached PDF file, we show the simulated results for the recovered shirts using Blender, where we drop them onto a horizontal bar.\n\n4. *How to connect the edges in the recovered mesh from point clouds?*\n\nWe do not compute edges for the point cloud to generate the garment mesh. Instead, we use Eq. (10) to fit the optimal latent code $\\textbf{z}^*$ for ISP model from the sparse panel mask. Using $\\textbf{z}^*$ alongside the ISP meshing process, we reconstruct a garment mesh in rest state as illustrated in the bottom-right of Fig. 2 of the main paper. This mesh defines the vertices and faces of the garment. To reconstruct the garment in the observed deformed state, we update the vertex positions by $\\textbf{V}=\\mathcal{M}[u,v]$, where $\\mathcal{M}$ is the recovered UV map and $(u,v)$ is the corresponding UV coordinate of $\\textbf{V}$.\n\n5. *What is the averaged number of points for different garments? Is the model able to deal with large number of points?*\n\nThe averaged numbers of points provided by VR-Folding dataset [2] are 30K for Shirt/Pants/Skirt and 27K for Top. However, instead of using all points, we randomly sample 4000 of them as the input. Therefore, we are able to handle large number of points.\n\n6. *Garments worn by the human body.*\n\nUsing ISP model to recover on-body garments has been shown to work in [1,40]. Consequently, we focus on the more challenging task of recovering garments **not** being worn. Our method differs from [1, 40] by utilizing a diffusion model as the deformation prior and leveraging UV mapping along with the proposed fitting method (Sec. 3.3 and 3.4) to recover complete garment meshes from partial point clouds. With appropriate training data, our method can also handle garments worn on the human body. However, due to time constraints, we are unable to present results for this specific scenario.\n\n\n## References\n\n*N. Rahaman, A. Baratin, D. Arpit, F. Draxler, M. Lin, F. Hamprecht, Y. Bengio, and A.Courville. On the spectral bias of neural networks. In International Conference on Machine Learning, 2019.*", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We would like to thank all reviewers for their valuable suggestions and constructive comments. We have carefully considered and addressed each of the suggestions and questions raised. We will incorporate these suggestions into our revised paper.

The attached PDF file includes the following additions: 
- figures of error curves under varying numbers of points and noise levels as suggested by Reviewers 2QwG and 4mMP;
- a table of intersection evaluation as recommended by Reviewer 2QwG; 
- illustrative examples for Reviewer J5PZ.

Once again, we sincerely thank all reviewers for their expertise and the time they spent in reviewing our paper.

Meta-review: This paper received mixed ratings, with two positive and one negative review following the rebuttal. The authors provided detailed responses to all the reviewers' questions. The primary critique from the negative review focused on the method being limited to a specific garment and the insufficient recovery of high-frequency wrinkles. However, category-specific models are a common assumption in this field and should not be grounds for rejection. Moreover, the paper achieves state-of-the-art results compared to previous approaches and is considered technically sound.

The AC concurs with the positive reviews, believing that the strengths of the paper outweigh the critiques, and therefore recommends acceptance.
Decision: Accept (poster)