DreamVTON: Customizing 3D Virtual Try-on  with Personalized Diffusion Models

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-07 20:36:12

Abstract: Image-based 3D Virtual Try-ON aims to sculpt the 3D human according to person and clothes images, which is data-efficient (i.e., getting rid of expensive 3D data) but challenging. Recent text-to-3D methods achieve remarkable improvement in high-fidelity 3D human generation, demonstrating its potential for 3D virtual try-on. Inspired by the impressive success of personalized diffusion models (e.g., Dreambooth and LoRA) for 2D VTON, it is straightforward to achieve 3D VTON by integrating the personalization technique into the diffusion-based text-to-3D framework. However, employing the personalized module in a pre-trained diffusion model (e.g., StableDiffusion (SD)) would degrade the model's capability for multi-view or multi-domain synthesis, which is detrimental to the geometry and texture optimization guided by Score Distillation Sampling (SDS) loss. In this work, we propose a novel customizing 3D human try-on model, named \textbf{DreamVTON}, to separately optimize the geometry and texture of the 3D human. Specifically, a personalized SD with multi-concept LoRA is proposed to provide the generative prior about the specific person and clothes, while a Densepose ControlNet is exploited to guarantee consistent prior about body pose across various camera views. Besides, to avoid the inconsistent multi-view priors from the personalized SD dominating the optimization, DreamVTON introduces a template-based optimization mechanism, which employs mask templates for geometry shape learning and normal/RGB templates for geometry/texture details learning. Furthermore, for the geometry optimization phase, DreamVTON integrates a normal-style LoRA into personalized SD to enhance normal map generative prior, facilitating smooth geometry modeling. Extensive experiments show that DreamVTON can generate high-quality 3D Humans with the input person, clothes images, and text prompt, outperforming existing methods.

Main paper: 
 introduction: 
 The task of Virtual Try-ON (VTON), to transfer a clothing item onto a specific person, has been explored a lot in recent years due to its promising potential to revolutionize the industry of ecommerce and fashion design. The image-based 2D solutions take person and clothes images as inputs and achieve virtual try-on via 2D generative models [13, 22, 48, 54] . Although the advanced 2D solutions [2, 11, 14, 20, 33, 57, 67] can synthesize compelling results within particular viewpoint (e.g., front view), they fail to display try-on results for arbitrary observed viewpoint, which is commonly required in the real-world scenarios. Traditional 3D solutions model the try-on results in the 3D space, thus providing a more comprehensive and attractive perception of clothes fitting. However, most of these solutions [3, 15, 17, 32, 42] rely upon the 3D scanning equipment or labor-intensive manual annotation, making them much resource-hungry compared with the image-based 2D counterparts. The pros and cons of the existing 2D and 3D solutions inspire us to rethink whether sculpting the 3D try-on human by simply using the person and clothes images is possible.Recently, the extraordinary success of diffusion models [46, 48, 50] for text-to-image (T2I) has largely prompted the development of high-quality 3D content generation [8, 36, 43, 45, 52, 56] , whose optimization of the 3D representation is guided by 2D generative priors from the pre-trained T2I diffusion model (e.g., StableDiffusion(SD) [48] ) by using Score Distillation Sampling (SDS) loss [43] . Previous 3D human generation works explore this diffusion-based 3D generation framework to sculpt a 3D human according to textual descriptions [7, 25, 26, 28, 30, 35, 62] or reference human images [27, 61] . Despite the significant advancement in high-quality 3D modeling, these methods can not be directly adapted to 3D virtual try-on, because they neither take the clothing items as input nor consider the clothing manipulation during the 3D modeling procedure. Some diffusion-based methods have explored the potential of employing lightweight personalized modules (i.e., LoRA [24] ) in SD for 2D virtual try-on. As shown in Figure 2 Considering the benefits of 2D and 3D generation, it is straightforward to achieve 3D virtual try-on by integrating the personalized SD with the diffusion-based 3D generation framework. However, introducing LoRA into SD would degrade the model's capability for multi-view generation, since it is trained using rare images within limited viewpoints. As shown in Figure 2 (b), for some observed viewpoints, given the same prompt, integrating SD with LoRA would generate wrong results or directly crash, while SD without LoRA can generate realistic results conforming to the input densepose [16] . The degraded ability for multi-view synthesis results in inconsistent generative priors across various viewpoints and further influences the 3D optimization procedure, leading to coarse 3D geometry and blurred texture. Therefore, it is non-trivial to integrate the personalized SD into a diffusion-based 3D generation framework for image-based 3D virtual try-on.To target the challenges, we propose a novel diffusion-based 3D human generation framework, named DreamVTON, to sculpt the 3D human by simply taking several person images, clothes images, and a text prompt as inputs (see Figure 1 ). Specifically, our DreamVTON inherits the advanced two-stage 3D generation framework [8, 25, 27] , the first stage optimizes the DMTet-based [10, 51] 3D representation, while second stage optimizes the texture. During the geometry and texture optimization procedures, DreamV-TON introduces a multi-concept LoRA to provide generative priors about the specific person and clothes. Besides, inspired by Avatar-Verse [62] , DreamVTON employs a Densepose-guided Control-Net [63] to provide consistent priors about body pose across various viewpoints. To avoid inconsistent generative priors from personalized SD dominating the 3D optimization procedure, DreamVTON proposes a template-based optimization mechanism, which employs mask templates for precise geometry shape learning and normal/RGB templates for precise geometry/texture details learning. The personalized SD generates the RGB templates within several pre-defined viewpoints, while the mask and normal templates are derived from the RGB templates by using the off-the-shelf mask and normal predictors. Moreover, to enhance the 3D geometry perception of the personalized SD during the geometry optimization procedure, DreamVTON introduces a normal-style LoRA which can facilitate the personalized SD to provide more powerful prior about the normal map, leading to smoother geometry modeling.Overall, the main contributions can be summarized:  
related works 2.1 2d/3d virtual try-on: 
 Most Virtual Try-ON (VTON) methods [2, 14, 19, 20, 33, 55, 57, 59 ] explore 2D VTON and aim to transfer an in-shop garment onto a specific person. Generally, they employ a two-stage framework to process garment deformation and try-on generation separately, in which the former uses the Thin Plate Splines [5] (TPS) or flowbased [65] network to model geometry deformation, while the latter employs generative models like Generative Adversarial Network [13] or Diffusion Model [48] to synthesize the try-on results. TryonDiffusion [67] introduces an implicit warping mechanism and processes clothes warping and try-on generation within a single diffusion network. Traditional 3D VTON methods [6, 15, 17, 32, 42] relies on the 3D scan equipment or cloth simulation to generate geometric representations of high precision. Learning-based methods [4, 39, 40, 66] employ differentiable rendering to dress the SMPL [37] model with desired garment mesh. M3D-VTON [64] proposes a depth-based 3D VTON framework to lift the 2D VTON results to 3D. Differently, we handle image-based 3D VTON by using the powerful generative model, which can integrate the complementary advantages of 2D and 3D VTON.Diffusion-based 3D human generation methods [7, 28, 61] aim to generate 3D humans, using text prompts or reference images as input. They apply SDS-based optimization [43] to progressively generate 3D humans from initial shape often parameterized by SMPL [37] . TADA [35] and TeCH [27] deploys SMPL-X [41] expressing 3D human with more detail. Pose-aware neural human representation imGHUM [1] used to generate the human body in DreamHuman [30] .AvatarBooth [61] employs DreamBooth [49] to inject specific identity information into SD, enhancing identity consistency in the personalized 3D human body generation process. Both DreamWaltz [26] and AvatarVerse [62] leverage Pose ControlNet [63] to obtain detailed human body models. HumanNorm [25] introduces a normalaligned diffusion model that allows for custom identities and poses using normal maps in specific regions. Our DreamVTON is a poseaware 3D VTON pipeline that keeps face identity and clothes style.Dreambooth [49] proposes fine-tuning the network using a small set of subject-specific images, which learns specific objectives of the object with a unique identifier. Textual inversion [9] achieves efficient personalization by optimizing text embeddings, which is used to guide the creation of personalized images during inference. SVDiff [18] introduces an innovative approach by optimizing the singular values of weight matrices within the model. Custom Diffusion [31] focuses on fine-tuning the key and value projection matrices of cross-attention layers, and can jointly train for multiple concepts or combine multiple fine-tuned models through closedform constrained optimization. LoRA [23] introduces novel styles or concepts into pre-trained text-to-image models by optimizing lowrank approximations of weight residuals. However, DreamVTON addresses the 3D personalized challenge, enabling the generation of diverse clothes while preserving identity. 
methodology: 
 The image-based 3D Virtual Try-ON aims to sculpt the 3D digital human using several images of a specific person and clothes items.To achieve this, we propose DreamVTON, a personalized 3D human generation framework (Sec. 3.1) that collaboratively employs multiconcept LoRA and Densepose-guided ControlNet to provide the particular generative priors for the 3D optimization procedures of geometry and texture. To avoid the inferior generative priors from personalized modules (i.e., multi-concept LoRA) dominating the optimization procedure, DreamVTON employs a template-based optimization mechanism (Sec. 3.2) to facilitate realistic geometry and texture modeling. Besides, to further enhance the perception of the 3D geometry, DreamVTON introduces a normal-style LoRA (Sec. 3.3) into the personalized SD. An overview of DreamVTON is displayed in Figure 3 .Two-stage 3D generation framework. To efficiently model highquality 3D try-on digital human, our DreamVTON inherits the advanced two-stage 3D human generation framework [8, 25, 27] , in which the 3D geometry and texture are optimized separately by using Score Distillation Sampling (SDS) [43] to distill the generative priors from the pre-trained Stable Diffusion (SD) [48] 𝜖 𝜙 . For geometry modeling, DreamVTON utilizes a MLP network Ψ g to parameterize the DMTet-based [10, 51] geometry representation (𝑉 𝑇 ,𝑇 ), in which Ψ g is trained to predict the Signed Distance Function (SDF) value 𝑠 𝑖 and the deformation offset △𝑣 𝑖 for each vertex 𝑣 𝑖 ∈ 𝑉 𝑇 in tetrahedral grid 𝑇 . During training, DreamVTON first employs an initialization phase to fit 𝑇 onto an A-pose SMPL [37] mesh 𝑀 𝑠𝑚𝑝𝑙 , by using the following object function:EQUATION)where P is a point set randomly sampled around the surface of 𝑀 𝑠𝑚𝑝𝑙 , and 𝑆𝐷𝐹 (𝑝 𝑖 ) is the pre-calculated SDF value. Then, DreamV-TON employs the SDS-based optimization mechanism to sculpt the geometry details. To be specific, DreamVTON conducts differentiable rendering onto DMTet mesh to obtain a normal map n, which will then be passed to the pre-trained SD to calculate the normal map SDS loss as follows:EQUATIONwhere z n t is the latent code of n with t-step noising, c n is the embedding of normal map prompt extracted by CLIP [44] , and 𝜓 𝑔 is the parameters of Ψ g .For texture modeling, DreamVTON utilizes another MLP network Ψ t to parameterize the material model and uses the Physically-Based Rendering derived from Fantasia3D [8] to obtain the rendered RGB image x. During training, DreamVTON feeds x into pre-trained SD to calculate the image SDS loss as follows:EQUATIONwhere z x t is the latent code of x, c x is the embedding of image prompt, and 𝜓 𝑡 is the parameters of Ψ t .Personalized SD for image-based 3D VTON. Although existing methods [25, 27] based on the above two-stage framework can obtain high-quality 3D digital human, they can not be adapted to image-based 3D VTON, because they are incapable of handling clothes inputs. To address this problem, DreamVTON introduces a multi-concept LoRA to inject the knowledge of the specific person and clothes into pre-trained SD, which will provide the generative priors of clothes and person for 3D optimization. The multi-concept LoRA is trained by jointly using person images and clothes images. Person images and clothes images separately provide the identity and clothes information for virtual try-on. As shown in Figure 4 (a), the person images contain several person images of the same person, while the clothes image contains in-shop clothes and fashion models wearing the particular clothes. As for text prompts, DreamVTON employs the visual-language model BLIP [34] to generate captions for each training image. During the inference stage, the text prompt is constructed by extracting the principal concept of each image set, such as "a woman wears a white shirt and grey skirt. " We display additional examples of the text prompts used for training and inference in the supplementary material. Besides, inspired by AvatarVerse [62] , DreamVTON exploits a Densepose-guided ControlNet [63] to provide consistent generative priors about body pose across various viewpoints. Therefore, by employing multiconcept LoRA and Densepose-guided ControlNet, the pre-trained SD item in Eq. 2 and Eq. 3 should be modified to ε𝜙 z n t ; c n , t, p , where ε𝜙 refers to SD with LoRA and ControlNet, while p refers to DensePose rendered from a SMPL mesh within current camera pose and translation. SDS-based 3D optimization mechanism commonly samples camera poses uniformly distributed in the 3D space, which enables the generative model (e.g., SD) to provide generative priors from as many viewpoints as possible, and thus facilitates comprehensive 3D optimization. However, when employing the multi-concept LoRA into SD, the capability of multi-view synthesis largely degrades (as shown in Figure 2 ), since LoRA is trained by using rare images within limited viewpoints. The degradation of personalized SD in multi-view synthesis results in inconsistent generative priors across various viewpoints, which is detrimental to 3D optimization. Therefore, simply using the SDS loss from personalized SD would not be an optimal optimization method for image-based 3D VTON.To prevent inconsistent generative priors across multiple views from dominating the optimization of 3D models, our DreamVTON introduces a template-based optimization mechanism to facilitate precise geometry and texture learning. Specifically, DreamVTON first employs personalized SD (i.e., with multi-concept LoRA and Densepose-guided ControlNet) to generate RGB results within 𝑁 pre-defined viewpoints, which can synthesize realistic results. The generated results are regarded as the RGB templates { xi } 𝑁 𝑖=1 , which will then be passed into the off-the-shelf parsing predictor [12] and normal map predictor [58] to obtain the mask templates { mi } 𝑁 𝑖=1 and normal templates { ni } 𝐾 𝑖=1 . During geometry learning, to guarantee DMTet is optimized into the correct geometry shape, DreamVTON calculate Mean Square Error (MSE) loss L 𝑚 𝑔 between the template masks { mi } 𝑁 𝑖=1 and their corresponding rendered masks {m i } 𝑁 𝑖=1 (rendered under the same camera poses with those of templates), which can be formulated as follows:EQUATIONBesides, to facilitate learning of geometry detail, DreamVTON introduces reconstruction losses between { ni } 𝑁 𝑖=1 and their corresponding rendered normal maps {n i } 𝑁 𝑖=1 , which consist of a MSE loss 𝐿 𝑚𝑠𝑒 𝑔 and a perceptual loss [29] 𝐿 𝑝𝑒𝑟 𝑔 , and can be formulated as follows:EQUATIONEQUATIONwhere 𝛾 𝑗 denotas the 𝑗-th feature map in a pre-trained VGG [53] network. Similarly, during texture learning, to facilitate learning of texture detail, DreamVTON exploits the MSE loss 𝐿 𝑚𝑠𝑒 𝑡 and perceptual loss 𝐿 𝑝𝑒𝑟 𝑡 between { xi } 𝑁 𝑖=1 and their corresponding rendered RGB images {x i } 𝑁 𝑖=1 . It is worth noting that, since the normal and RGB templates are derived from the same generated results, the geometry and texture details on each normal-RGB template pair (i.e., templates rendered under the same camera pose) are strictly aligned. By using the detailaligned templates for geometry and texture learning, DreamVTON is capable of generating geometry-texture consistent 3D human.By jointly using the SDS-based and template-based optimization mechanisms, the overall object functions for geometry and texture optimization can be formulated as follows:EQUATIONEQUATIONwhere 𝜆 * 𝑔 and 𝜆 * 𝑡 are the trade-off hyperparameters.During the geometry optimization stage, since DreamVTON employs the rendered normal map to calculate the SDS loss, the personalized SD is designed to provide the normal-style generative prior for geometry optimization. However, when introducing the multi-concept LoRA, the personalized SD's capability for normal map synthesis degrades a lot, since LoRA is trained by using seldom RGB images. To address this issue, our DreamVTON introduces normal-style LoRA into personalized SD to enhance the capability for normal map synthesis. Specifically, the normal-style LoRA is trained with 2,000 text-annotated normal maps, in which the normal maps are rendered from the THUman2.0 dataset [60] , while the text prompts are extracted by BLIP [34] . Once trained, the normalstyle LoRA is integrated into the personalized SD and jointly used for geometry optimization. As shown in Figure 4 (c), given the same prompt "a woman wears a white shirt and grey skirt, with normal map style. ", multiconcept LoRA can only generate realistic RGB images, while normalstyle can generate results with normal map style, demonstrating that introducing the normal-style LoRA, could improve the the personalized SD's normal synthesis capability. 
experiments: 
 We first introduce the implementation details of DreamVTON (Sec. Training configuration. The geometry network Ψ g and texture network Ψ t are trained 15000 and 3000 iterations, respectively. The training procedure of Ψ 𝑔 can be further divided into 2000 iterations SDF initialization phase and 13000 iterations DMTet optimization phase. During training, the batch size on each GPU is set to 1 and both networks are trained by using AdamW [38] optimizer. The learning rate for Ψ g and Ψ t are set to 1e-3 and 1e-2, respectively. Both Ψ g and Ψ t are trained on 1 NVIDIA 4090 GPUs. Camera sampling strategy. For geometry learning, at the beginning of DMTet optimization, the randomly sampled cameras are located in a position that can cover the full human body. To sculpt the geometry details, after 1200 iterations, the cameras are posed to positions that focus on various local regions (i.e., head, upper body, and lower body), within which SD can provide more detailed generative prior for geometry optimization. For texture learning, the local cameras are employed at the beginning to enhance the learning of texture details. Selection of geometry/texture templates. For geometry learning, we employ eight mask templates { mi } 8 𝑖=1 and two normal templates { ni } 2 𝑖=1 for optimization, in which { mi } 8 𝑖=1 are sampled uniformly around the human body while { ni } 2 𝑖=1 is composed of the front and back view normal maps. For texture learning, we utilize three RGB templates { xi } 3 𝑖=1 , which contain front and back views of full body images and one head image. The head image is used to enhance the texture detail around the face region.We compare our DreamVTON with three existing 3D human generation methods, namely DreamWaltz [26] , TEXTure [47] , and TeCH [27] . Since DreamWaltz and TEXTure take as input merely the text prompt, we use the constructed text prompts (used by our DreamVTON) for them. For TeCH, since it receives the text and image inputs, except for the constructed text prompt, we also feed the front-view RGB template (used by our DreamVTON) for it. Figure 5 displays the qualitative comparison of DreamVTON with the baselines. By simply receiving the text prompt as model input, DreamWaltz, and TEXTeure can generate 3D humans with similar clothes types to the input clothes images. However, they fail to preserve the texture detail or clothes color. For example, for the white-tshirt-grey-skirts case in the first case, they fail to generate grey skirts. TeCH [27] takes both text and images as inputs, thus is capable of preserving the clothes details in front view. However, it fails to generate realistic texture in the back view and face region, since it ignores the guidance about the back view and face. In comparison, DreamVTON can not only preserve the clothes information in arbitrary view but also generate a realistic face, demonstrating outperforms the compared methods.Inspired by HumanNorm [25] , we choose CLIP-similarity [44] and FID [21] to evaluate the generated quality of the 3D try-on results, in which FID measures the realism of rendered results while CLIP-Similarity measures the similarity between the rendered results from arbitrary viewpoint and the particular reference images. Specifically, for each test case, we first employ the personalized SD (Densepose-based ControlNet + Multi-concept LoRA) to generate 8 We evaluate our proposed DreamVTON's performance against other methods using the user study. As reported in Table 2 , a higher score for user evaluation indicates that humans preferred the performance, our proposed DreamVTON outperforms all the compared methods. In particular, our proposed DreamVTON significantly outperforms the compared methods in terms of texture realism and ID fidelity, with 91.5% and 94% of users, preferring to choose our model. Regarding the quality of the geometry, 78.8% of users still prefer to choose our method. Overall, Table 2 demonstrates the effectiveness of our method, which outperforms the other methods on texture, ID fidelity, and geometry, respectively. This means that our proposed DreamVTON can generate more realistic-looking 3D humans that are preferred by users, wearing different clothes with accurate 3D geometry and detailed textures.We conduct ablation studies to validate the effectiveness of templatebased optimization mechanisms and normal-style LoRA for geometry and texture optimization, using visual comparison when excluding the component. For geometry optimization, as shown in Figure 6 , without using normal-style LoRA or normal templates as optimization constraints, the surface of the learned geometry is coarse with artifacts. Either adding the normal-style LoRA or using the normal templates in the geometry optimization procedure can smooth the geometry surface. By jointly using normal-style LoRA and normal templates, the geometry surface can be smoother while preserving the clothes characteristics in the input images.For texture optimization, as shown in Figure 7 , without using the RGB templates as optimization constraints, the learned texture is noisy (e.g., unclear face region) and fails to preserve the characteristic of inputs image (e.g., incorrect clothes color). By using the RGB templates during optimization, DreamVTON can generate 3D humans with high-quality texture and retain the input images' characteristics (i.e., person identity, clothes style). 
limitation: 
 The texture detail of clothes in DreamVTON's result is derived from the generative priors from multi-concept LoRA. However, existing LoRA fails to preserve the texture detail for complicated logos, (as shown in Figure 8 ), thus preventing DreamVTON from generating detailed textures that are completely consistent with input images. This problem could be alleviated by balancing the personalization and generalizability of personalized SD, which is widely explored in the diffusion-based models. 
conclusion: 
 We propose a new method for 3D virtual try-on task, named DreamV-TON, which is capable of generating the 3D human using person images, clothes images, and a text prompt as inputs.  
abstract: 
 Image-based 3D Virtual Try-ON aims to sculpt the 3D human according to person and clothes images, which is data-efficient (i.e., getting rid of expensive 3D data) but challenging. Recent text-to-3D methods achieve remarkable improvement in high-fidelity 3D human generation, demonstrating its potential for 3D virtual try-on. Inspired by the impressive success of personalized diffusion models (e.g., Dreambooth and LoRA) for 2D VTON, it is straightforward to achieve 3D VTON by integrating the personalization technique into the diffusion-based text-to-3D framework. However, employing the personalized module in a pre-trained diffusion model (e.g., StableDiffusion (SD)) would degrade the model's capability for multi-view or multi-domain synthesis, which is detrimental to the geometry and texture optimization guided by Score Distillation Sampling (SDS) loss. In this work, we propose a novel customizing 3D human try-on model, named DreamVTON, to separately optimize the geometry and texture of the 3D human. Specifically, a personalized SD with multi-concept LoRA is proposed to provide the generative prior about the specific person and clothes, while a Denseposeguided ControlNet is exploited to guarantee consistent prior about 
 
 Appendix:  


Decision: All reviewers lean to accept the paper. Most concerns are cleared in the rebuttal. AC agrees with the reviewers and recommends acceptance.

Reviews: review: paper_summary: paper_summary: - This paper introduces DreamVTON, a custom 3D human try-on model. The author utilizes multi-concept LoRA and Densepose-guided ControlNet for creating a personalized SD. Additionally, a template-based optimization mechanism is introduced to enhance geometry and shape details. Lastly, a normal-style LoRA is integrated to improve the normal map. 
- DreamVTON seems to be an enhanced version of HumanNorm for personalized people and clothes.
strength_weakness: strength_weakness: - The paper is well-written and easy to follow. 
- I appreciate the motivation experiment in Fig. 2, which indicates a potential issue when using LoRA diffusion in multi-view generation. 
- The proposed DreamVTON seems to resolve this issue in Fig. 4, showing high-quality 3D try-on results and outperforming other 3D human generation methods.
limitations: limitations: - In Sec. 3.1, the authors claim that “the multi-concept LoRA is trained by jointly using person images and clothes images”, which is not sufficient for reproduction. More details of the multi-concept LoRA are recommended to add.

- The authors point out the multi-view generation issue when using LoRA diffusion. Is this issue mitigated by the proposed multi-concept LoRA? The solution and discussion are missing.

- Fig. 4 shows the sample results of multi-concept LoRA and Normal-style LoRA.  However, the normal map seems not aligned with the image (e.g., hair, skirt, and expression). Will the misalignment supervision aggravate the over-smoothing problem in a generation?
paper_summary: paper_summary: This paper introduces DreamVTON, a novel method for customizing 3D human try-on models. DreamVTON generates both the geometry and texture of a human figure wearing selected clothes.
strength_weakness: strength_weakness: 1) The method is based on LoRA, making it lightweight.

2) The visual results are impressive.

3) In terms of quantitative metrics and user evaluations, DreamVTON outperforms the baselines.
limitations: limitations: 1) The FID score is too high, indicating a need for more results to make the findings convincing. I suggest the authors calculate the FID at least five times with different samples and report the mean and standard deviation.

2) The CLIP-score for both TeCH and DreamVTON is the same, which is unusual. It is possible that the human pose itself introduces a high degree of CLIP-score similarity, making this metric less meaningful for this task. I would like to see more detailed research on this metric.
paper_summary: paper_summary: This paper proposes a 3D Virtual Try-On model that employs multiple personalized diffusion models to separately generate the geometry and texture of digital humans. The experimental results demonstrate the superior effectiveness of the DreamVTON model.
strength_weakness: strength_weakness: 1. The final results significantly surpass the baseline model in terms of both geometry and texture. The alignment between geometry and texture is commendably achieved, showing a notable improvement over existing methods.
2. The adaptation of the 3D generation model for the virtual try-on domain is comprehensive and well-executed, making a decent contribution to the field.
limitations: limitations: 1. The selection of human images for evaluation and demonstration shows an uneven distribution in terms of skin color and gender. Additionally, a greater variety of clothing choices could have been included to better demonstrate the robustness of the method.
2. There are problems of the deformation of hands in the generated results, and the clothing colors tend to mistakenly appear on the hands, which is more evident in the demo video.
3. There are still several questions that the authors did not mention: 1) How does this model compare in performance with traditional 3D virtual try-on solutions? 2) What is the time required to generate a single instance throughout the entire pipeline?
4. I notice that a similar article titled "AvatarFusion: Zero-shot Generation of Clothing-Decoupled 3D Avatars Using 2D Diffusion" was presented at MM2023 last year, which could be relevant for reference in further discussions of this topic.
paper_summary: paper_summary: This paper proposes an image-based 3D Virtual Try-On method, based on the DMTet geometry representation and SDS loss to perform two-stage 3D human generation. Specifically, the proposed method utilizes a Densepose-based ControlNet and multi-concept LoRA to generate pose-conditioned RGB/normal images with desired clothes and faces, which serve as the templates to optimize the 3D human. It also proposes a normal-style LoRA to enhance the geometry. The experiments validate the effectiveness of the proposed method.
strength_weakness: strength_weakness: 1. Without the need for a large image-3D dataset for training, this method can be used to generate 3D virtual try-on results given several images through LoRA fine-tuning.
2. This paper is well-structured and easy to read.
limitations: limitations: 1.  Almost all the 3D humans of the video demos provided in the supplementary material show inconsistent textures at the seams between the front and back views. 
2.  Although this work is designed to solve 3D virtual try-on, it cannot specify the user's body shape with merely face images. I notice that the results demonstrated in the paper all have mean body shapes. It seems that using lora combined with Text-to-3D human reconstruction for virtual try-on is limited, as it performs worse in preserving garment texture and identity features. Experiments lack comparisons with other VTON methods to demonstrate its effectiveness on this task.