Magic Clothing: Controllable Garment-Driven Image Synthesis

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-01 09:48:00

Abstract: We propose Magic Clothing, a latent diffusion model (LDM)-based network architecture for an unexplored garment-driven image synthesis task. Aiming at generating customized characters wearing the target garments with diverse text prompts, the image controllability is the most critical issue, i.e., to preserve the garment details and maintain faithfulness to the text prompts. To this end, we introduce a garment extractor to capture the detailed garment features, and employ self-attention fusion to incorporate them into the pretrained LDMs, ensuring that the garment details remain unchanged on the target character. Then, we leverage the joint classifier-free guidance to balance the control of garment features and text prompts over the generated results. Meanwhile, the proposed garment extractor is a plug-in module applicable to various finetuned LDMs, and it can be combined with other extensions like ControlNet and IP-Adapter to enhance the diversity and controllability of the generated characters. Furthermore, we design Matched-Points-LPIPS (MP-LPIPS), a robust metric for evaluating the consistency of the target image to the source garment. Extensive experiments demonstrate that our Magic Clothing achieves state-of-the-art results under various conditional controls for garment-driven image synthesis. Our source code is publicly available (for the review process, please refer to our supplementary material).

Main paper: 
 introduction: 
 Latent diffusion model (LDM)-based generative approaches [7, 9, 13, 15, 30, 33, 41] have achieved great success in text-to-image synthesis in recent years. Besides the textual condition, many other forms of conditional control have been explored for LDM-based image synthesis, such as pose, sketch and facial conditions [29, 43, 51] . Among these researches, little attention has been paid to image synthesis conditioned on a specific garment so far, which is a promising task with enormous application prospects for e-commerce and metaverse, etc. Such a garment-driven image synthesis task aims to generate a character wearing the target garment according to the customized text prompt, where the image controllability is the most critical issue. More specifically, the main challenges of garmentdriven image synthesis include preserving the garment details and maintaining faithfulness to the text prompts.Previous approaches to traditional subject-driven image synthesis [12, 19, 20, 25, 35, 38, 42, 54] usually focus on overall conditional information like appearance and structure, and invert the subject image into the text-embedding space, which are insufficient to capture the complicated fine-grained features of the garments. Another popular task similar to garment-driven image synthesis is virtual try-on (VTON) [8, 14, 21, 27, 28, 45, 48] , aiming to generate a specific person wearing the target clothes. However, VTON is more of an image-inpainting task, which only requires faithfully preserving the target garment features without any creative capability of following arbitrary customized text prompts.In view of the aforementioned challenges, we introduce Magic Clothing, an LDM-based network architecture that focuses on character generation conditioned on the given garments and text prompts. To preserve the fine-grained garment details, we propose a garment extractor with the UNet architecture [34] . By leveraging the power of pretrained LDMs [33] , it allows smooth incorporation of detailed garment features into the denoising UNet through selfattention fusion. To maintain faithfulness to arbitrary customized text prompts, we randomly drop garment features and text prompts from a joint distribution in training to enable the joint classifierfree guidance, which effectively balances the control of garment features and text prompts over the generated results. Our garment extractor is also a plug-in module compatible with various finetuned LDMs or extensions like ControlNet [51] and IP-Adapter [50] to further control the pose, face and even style of the character without degrading the garment details. In practice, we propose a novel robust metric, namely Matched-Points-LPIPS (MP-LPIPS) to quantify the garment-driven image synthesis quality. It measures the consistency of the target image to the source garment by comparing the patches obtained from point matching, thus mitigating the undesired effect of pose and background on the evaluation.In summary, the main contributions of this work are as follows:• Magic Clothing is, to the best of our knowledge, the first LDM-based work to investigate the unexplored garmentdriven image synthesis task. • We propose a garment extractor to incorporate garment features into the denoising process via self-attention fusion.And the joint classifier-free guidance is applied to balance the control of garment features and text prompts. • Our garment extractor is a plug-in module applicable to various finetuned LDMs, which can be easily combined with other powerful extensions, such as ControlNet [51] or IP-Adapter [50] , to employ additional conditions. • We develop a robust metric namely MP-LPIPS to evaluate the consistency of the target image to the source garment.Qualitative and quantitative experiments demonstrate our state-of-the-art performance on garment-driven image synthesis with high controllability. 
related work 2.1 latent diffusion models: 
 Latent diffusion models (LDMs) [33] have been successfully used for text-to-image generation tasks. Based on this robust foundational model, various LDM-based researches and applications have emerged. To add spatial conditioning controls to the pretrained LDMs, ControlNet [51] incorporated trainable encoder blocks into the original UNet. Meanwhile, T2I-Adapter [29] proposed a compact network design that provided the same functionality as ControlNet but with reduced complexity. To further reduce the training cost, Uni-ControlNet [55] proposed a unified framework that handles different conditional controls in a flexible and composable manner within one single model. On the other hand, LDMs have also played a significant role in the domain of image editing. InstructPix2Pix [3] retrained the UNet of LDMs by adding extra input channels to the first convolutional layer on a large dataset of image editing examples to make it follow the edit instructions. MasaCtrl [4] converted the self-attention in diffusion models into mutual self-attention to enable consistent image generation and complex non-rigid image editing simultaneously without additional training cost. InfEdit [46] performed consistent and faithful image editing for both rigid and We propose a garment extractor that captures the garment features and incorporate these features into the denoising process in self-attention layers. Besides the paired garment and character images, we obtain the text prompts for training through BLIP [23] . Only the garment extractor requires additional training, which is a plug-in module compatible with other useful extensions like ControlNet [51] or IP-Adapter [50] .non-rigid semantic changes through the denoising diffusion consistent model and attention control mechanisms. In this paper, we make full use of the power of pretrained LDMs in text-to-image generation, and introduce an extra garment extractor for garmentdriven image synthesis.Given an image of a particular subject, the subject-driven image synthesis task is to generate a novel image in different contexts while maintaining high fidelity to its key visual features. Textual Inversion [12] learned to represent the concept through new "words" in the embedding space of a frozen text-to-image model, which can be composed into natural language sentences and generate customized images. DreamBooth [35] finetuned the pretrained LDMs model with the newly designed class-specific prior preservation loss to make it learn to bind a unique identifier with that specific subject. However, finetuning the entire UNet for each subject raises significant computational cost. To address this, HyperDreamBooth [36] proposed a hypernetwork capable of efficiently generating an initial prediction of a subset of network weights and significantly accelerated the finetuning process. BLIP-Diffusion [22] introduced a new multimodal encoder which is pretrained to provide subject representation. By leveraging such visual representation, diffusion models can yield a wide range of subject variations. IP-Adapter [50] designed a decoupled cross-attention mechanism that separates cross-attention layers for text features and image features. Despite its simplicity, this method achieved comparable or even better performance than specifically finetuned models. Versatile Diffusion [47] expanded the existing single-flow diffusion pipeline into a multitask multimodal network to enable cross-modal generality. Break-A-Scene [2] extracted a distinct text token for each concept from a single image by introducing a two-phase customization process that optimizes a set of dedicated textual embedding and the model weights. In this work, we focus on an unexplored garmentdriven image synthesis task, which requires much better detail preservation together with the creative capability following diverse text prompts. 
method 3.1 preliminary: 
 Unlike other pixel-space based diffusion models, latent diffusion models (LDMs) [33] are designed to perform the denoising process in the latent space for reducing the computational cost. Our approach is based on the LDM framework, which consists of three main networks, i.e. Vector Quantized Variational AutoEncoder (VQ-VAE) [11] , CLIP VIT-L/14 [32] , and a denosing UNet [34] . The VQ-VAE enables image representation in the latent space, whose encoder E compresses the original image x into the image latent z L = E E (x),T y ,𝜖∼N (0,1),𝑡 ∥𝜖 -𝜖 𝜃 (z 𝑡 , 𝑡, T y )∥ 2 2 .(1) I 𝐺 ∈ R 3×𝐻 ×𝑊 into the image latents z 𝐶 , z 𝐺 ∈ R 4× 𝐻 8 × 𝑊8 , respectively, using VAE Encoder E. On the other hand, we caption I 𝐶 with BLIP [23] to obtain the text prompt y and transform it into token embeddings T y . Transferring knowledge from the image synthesis to feature extraction, we introduce a garment extractor E 𝐺 that has the same UNet architecture as the denoising UNet to extract the detailed garment features. Inspired by the powerful spatial-attention mechanisms [6, 18, 48, 49] , we incorporate the extracted garment features into the original denoising process smoothly through selfattention fusion. More concretely, 𝛼 𝑖 and 𝛽 𝑖 refer to the normalized attention hidden states of the 𝑖-th self-attention block in the original denoising UNet 𝜖 𝜃 and the garment extractor E 𝐺 , respectively. The calculation of self-attention in 𝜖 𝜃 after incorporation of garment features is modified as:Attention(𝛼 𝑖 , 𝛽 𝑖 ) = Softmax( W 𝑄 𝛽 𝑖 (W 𝐾 [𝛼 𝑖 , 𝛽 𝑖 ]) 𝑇 √ 𝑑 )W 𝑉 [𝛼 𝑖 , 𝛽 𝑖 ],(2) where [•] denotes concatenation operation, 𝑑 is the feature dimension, and W 𝑄 , W 𝐾 , W 𝑉 are linear projection weights of query, key, and value in self-attention layers.To maintain the text-to-image synthesis capabilities of the original LDM and reduce the training cost, we keep the weights of 𝜖 𝜃 frozen. Then we only train our garment extractor with its weights initialized by the weights of 𝜖 𝜃 , which further speeds up the training process. Our training objective is summarized as:EQUATIONwhere z 𝐶 𝑡 is obtained by adding noise to the character image latent z 𝐶 at time step 𝑡, and 𝛽 is the overall garment features from selfattention blocks of our garment extractor E 𝐺 . During inference, given an input garment image and a text prompt for the character, Magic Clothing is capable of generating an image of the character wearing the target garment. To add more conditional controls, our garment extractor can be utilized with other LDMs and extensions like ControlNet and IP-Adapter, which will be discussed in Section 3.4. Meanwhile, the garment features are shared across all denoising steps, adding minimal computational cost to the original LDM inference process.Classifier-free guidance [16] is a method that helps the diffusion models to attain a trade-off between sample quality and diversity "A fair-skinned old woman in a scarf and a long skirt" Input EQUATIONwhere 𝑠 ⩾ 1 represents the strength of conditional controls. As for our garment-driven image synthesis task, two variant conditional controls should be considered: detailed garment features 𝑐 𝐺 and text prompts 𝑐 𝑇 . If we consider them as independent controls, we can naively modify the independent classifier-free guidance in [24] for our use. Specifically, garment features c 𝐺 and text prompts c 𝑇 can be independently set to ∅ with some probability during training. Then at inference time, we introduce the garment guidance scale 𝑆 𝐺 and text guidance scale 𝑆 𝑇 to adjust the strengths of conditional controls from the garment and text prompts, respectively. The score estimate with independent classifier-free guidance is calculated as:EQUATIONDespite its simplicity, fusing two denoising scores as done in Equation 5 could lead to undesired results since this method ignores the fact that our two conditional controls may have overlapping semantics. Drawing inspiration from [3] , we leverage joint classifier-free guidance to balance these two conditional controls. Here joint indicates that we set the drop rate of garment features and text prompts according to a joint distribution. More precisely, we randomly set 5% of training samples with c 𝐺 = ∅ 𝐺 , 5% with c 𝑇 = ∅ 𝑇 and another 5% with both c 𝐺 = ∅ 𝐺 and c 𝑇 = ∅ 𝑇 . Our joint classifier-free guidance score estimate is:EQUATIONIn Figure 3 , we show the effects of these two parameters on generated samples with the random seed fixed. It is noticeable that the garment in the character become more similar to the input garment with a larger 𝑆 𝐺 and a larger 𝑆 𝑇 will make the generated result follow the text prompt more precisely. Since a large gap between 𝑆 𝑇 and 𝑆 𝐺 may distort the garment details, we empirically set 𝑆 𝑇 = 7.5 and 𝑆 𝐺 = 2.5 in our experiments.Since we keep the weights of the pretrained LDM [33] frozen and only train our garment extractor E 𝐺 , we can treat E 𝐺 as a plug-in module and combine it with various finedtuned LDMs to enhance the diversity of generated characters. For instance, when combined with various LoRA [17] or full-parameter finetuned LDMs, we are able to create characters in different styles, such as science fiction, realistic, and anime styles, etc. Moreover, our controllable image synthesis process is compatible with other advanced LDM extensions. To generate characters wearing the given garments with target poses, we can combine ControlNet-Openpose [51] with our Magic Clothing. And with ControlNet-Inpaint [51] , our model is able to perform the virtual try-on (VTON) task [8, 21, 27, 45, 48] and generate high-fidelity results. For the purpose of generating a specific person wearing the target garment, we can combine Magic Clothing with IP-Adapter-FaceID [50] , which takes the given portrait as an input condition. We remark that the finetuned LDMs and multiple extensions like ControlNet [51] and IP-Adapter [50] can be simultaneously combined with our Magic Clothing. In this way, we achieve comprehensive conditional controls over the image synthesis process, including text prompts, garments, styles, faces and postures, etc. The superior performance of our plug-in models on controllable image synthesis under various conditions is shown in Section 4.3.3.  
experiments 4.1 experimental setup: 
 4.1.1 Dataset. To train our Magic Clothing for this new task of garment-driven image synthesis, we need not only the paired garment and character images, but also text prompts containing the character descriptions. Since the VITON-HD dataset [8] only contains the paired data of frontal half-body characters and corresponding upper-body garments, we obtain the text prompts by captioning these characters using BLIP [23] .To evaluate the model performance, we provide 5 test text prompts covering different ages, appearances, ornaments and backgrounds of the target characters using GPT-4 [1] . And we select the garments in the test set of the VITON-HD dataset as our test garments, which contains 2,032 upper-body garments. We generate 5 character images for each test garment with the provided 5 text prompts, resulting in a total of 10,160 images for evaluation.4.1.2 Compared Methods. We compare our results with three stateof-the-art LDM-based subject-driven image synthesis models, IP-Adapter [50] , BLIP-Diffusion [22] , and Versatile Diffusion [47] . In addition, we train a ControlNet [51] that takes the garment directly as an input condition and generates images according to text prompts, namely ControlNet-Garment. For fair comparison, all the experiments are conducted at the resolution of 768 × 576.To evaluate the garment fidelity, the details of the garment on the generated character must be compared with those of the original garment. Following previous subjectdriven methods [35, 36] , we use CLIP-I [32] to compute the similarity of the same subject in different images. However, this metric is sensitive to background and the character pose in our task. Recently, S-LPIPS [53] is designed to tackle the problem of changes in the shape and posture of generated characters, but it still relies on human pose estimation [5, 39, 44] and paired images. Surprisingly, it is reported in [40] that geometric corresponding points can be accurately predicted from the intermediate layer activations of pretrained LDMs, even though they are not trained with any explicit geometry supervision. Driven by the above prospects and challenges, we propose Matched-Points-LPIPS (MP-LPIPS) that measures the LPIPS distance [52] between pixels on garment I 𝐺 and matched pixels on generated character I 𝐶 . Specifically, we uniformly select N points P 𝐺 on each garment, calculating their diffusion features [40] F t,l (P 𝐺 ) at time step 𝑡 and layer 𝑙. Then we find the matched points P 𝐶 on the target character by calculating cosine similarity as:EQUATIONand finally define MP-LPIPS distance as:EQUATIONWe measure the average LPIPS distance of the patches P 𝐺 , P 𝐶 ∈ R 𝑁 ×𝐻 ×𝑊 centred on N pairs of points P 𝐺 and P 𝐶 , where 𝐻 and 𝑊 indicate the height and width of the patch, respectively. As shown in Figure 4 , MP-LPIPS effectively measures the consistency of the garment to the character without the need for manual annotation. More importantly, it is robust to factors such as the background and posture. More detailed settings are to be provided in the supplementary material.In addition, we use CLIP-T [32] to measure the faithfulness of results to text prompts and CLIP aesthetic score [37] to evaluate the general image quality.In our experiments, we initialize the weights of our garment extractor by inheriting the pretrained weights of the UNet in Stable Diffusion v1.5 [33] , and only finetune its weight while keeping the weights of other modules frozen. Our model is trained on the paired images from VITON-HD [8] dataset at the resolution of 768 × 576 and the corresponding captions that obtained from BLIP [23] . We adopt the AdamW optimizer [26] with a fixed learning rate of 5e-5. The model is trained for 100,000 steps on a single NVIDIA A100 GPU with a batch size of 16. At inference time, the images are generated with the UniPC sampler [56] for 20 sampling steps.3.1 Qualitative Results. Figure 5 presents qualitative results with our text prompts and garment images from the VITON-HD dataset [8] . We can see that IP-Adapter [50] closely follows the textual conditions and preserves the general appearance of the garment. However, it struggles to preserve the garment details such as printed patterns or texts. BLIP Diffusion [22] and Versatile Diffusion [47] are weak at following text prompts and also have difficulty maintaining garment details. To make matters worse, they tend to generate garment variations instead of characters when the garment details are complex, as shown in the last line of the Figure 5 . These traditional subject-driven image synthesis methods typically embed the image in the CLIP text embedding space, which emphasis on overall structural similarity rather than preservation of fine-grained details. However, humans are sensitive to subtle variations in garment features such as printed patterns or text, making these conventional methods inadequate for the garment-driven image synthesis task.Although the results of ControlNet [51] follow the text prompts reasonably well, they almost ignore the conditional control from the garments, resulting in the garments on the generated characters being completely different from the input garments. We believe that this is mainly due to ControlNet failing to learn the spatial correlations between the garments and the characters during its training process. Compared to other methods, the results from our Magic Clothing not only maintain faithfulness to the text prompts but also preserve details of the given garment, demonstrating the best performance in garment-driven image synthesis.. Table 1 shows the quantitative results with our text prompts and garment images from the VITON-HD dataset [8] . ControlNet [51] achieves high scores on CLIP-T and CLIP-AS, which demonstrates its ability to follow text prompts and generate images with high aesthetic quality. However, it fails to retain garment details due to the spatial mismatch between the given garment and target character. According to their CLIP-I scores, Versatile Diffusion [47] , Blip Diffusion [22] and IP-Adapter [50] can generate characters that are more similar to the person wearing the garment in the VITON-HD dataset. Nonetheless, they extract image features using the CLIP image encoder [32] , which captures only semantic information and loses garment details, resulting in inferior performance on MP-LPIPS. In comparison, our Magic Clothing generates characters with fine-grained garment details and text prompt fidelity, significantly outperforming other methods on all the evaluation metrics. To verify the effectiveness of our joint classifier-free guidance, we train a model with the same network architecture but using independent classifier-free guidance as described in Section 3. conditional controls can hardly be independent from each other and are better considered jointly.Despite the state-of-the-art performance in garment-driven image synthesis, limitations remain in our Magic Clothing. For instance, the quality of our generated images are highly dependent on the base diffusion models. Further improvement can be achieved by using more powerful pretrained models like SDXL [31] and Stable Diffusion 3 [10] . Another limitation is that due to the limited training samples in the VITON-HD dataset [8] , Magic Clothing may fail to generate perfect results for complicated garments such as down jackets and coats. A possible solution in the future is to collect more comprehensive dataset for training our garment extractor. 
conclusion: 
 This paper presents Magic Clothing, an LDM-based network architecture for the unexplored garment-driven image synthesis task. A garment extractor is employed to incorporate the garment details via self-attention fusion, and the joint classifier-free guidance is applied to balance the control of garment features and text prompts. Our garment extractor is a plug-in module which can be easily combined with other useful extensions for additional conditions. Comprehensive experiments demonstrate our superiority in generating diverse images controlled by the given garments and text prompts, implying the tremendous potential of controllable garment-driven image synthesis.  
: 
 Figure 1 : Examples of our garment-driven image synthesis results. Given a target garment and a text prompt, our Magic Clothing is able to generate photorealistic (1st row) and anime-style (2nd row) characters with different finetuned latent diffusion models (LDMs) [33] . Besides, more of conditional controls can be combined with our Magic Clothing such as ControlNet [51] and IP-Adapter [50] (3rd and 4th rows).Unpublished working draft. Not for distribution. 
 
 Appendix:  


Decision: This paper introduces a latent diffusion model-based framework for virtual try-on, which can process diverse generation conditions such as texts, poses, and subject models. A garment extractor is designed to extract garment features, which are fused with features of other conditions for generations via attention modules and joint classifier-free guidance. The proposed framework is validated on the VITON-HD dataset.

Overall, the proposed framework is interesting and straightforward. The pre-rebuttal ratings of this paper were mixed, including two borderline rejects and one weak accept. The authors did an excellent job during the rebuttal and addressed most of the concerns of the reviewers. The final ratings of this paper are all positive (two borderline accepts and one weak accept). The AC confirmed that this paper can be revised and meet the acceptance bar of the conference. Hence, the final recommendation of this paper is accepted (poster).

Reviews: review: paper_summary: paper_summary: This work introduces a novel network architecture based on the Latent Diffusion Model (LDM), ‘Magic Clothing’, which is tailored for the task of clothing-driven image synthesis. This work introduces a garment extractor to capture detailed garment features and incorporate them into a pre-trained LDM using self-attention fusion to preserve garment details during the synthesis process. For evaluating the consistency between the target image and the source garment, this work introduces a robust metric called Matching Point-LPIPS (MP-LPIPS). Experiments demonstrate that Magic Clothing can achieve good performance under various conditions of control, displaying its effectiveness in garment-driven image synthesis.
strength_weakness: strength_weakness: 1. this work is well-written and logical, and the author's work can be easily understood.
2. the author clearly describes the methodology designed in the paper, giving details of the experimental set-up.
3. this work performs a large number of qualitative experiments that demonstrate the excellent qualitative performance of the method. In addition, the authors have placed source code in the supplementary material, which increases the reproducibility of this work.
limitations: limitations: 1. the template used in the paper does not quite match this year's conference and there are watermarks in the paper which the authors should check carefully.
2. this work is not more improved from the common Diffusion+Clip-Prompt approach to VTON and the novelty of this work is lacking.
3. there are only two tables of quantitative experiments in the main text, lack of more comparative experiments and ablation analysis of key components. The quantitative analysis of this work is insufficient.
4. the description and details of the Joint Classifier-free Guidance are lacking in the paper, and the section is lacking in the contribution and innovation of this work.
5. This work lacks experimental comparisons with some recently developed open source Diffusion-based methods, such as IDM-VTON, StableVITON, OOTDiffusion, and other related work.
paper_summary: paper_summary: This paper introduces "Magic Clothing", a novel latent diffusion model (LDM) based network architecture designed for garment-driven image synthesis. Key contributions include: a garment extractor that captures detailed garment features and integrates them into pretrained LDMs via self-attention fusion, ensuring detailed garment preservation and adherence to text prompts; the garment extractor is a plug-in compatible with various finetuned LDMs and extensions like ControlNet and IP-Adapter; a robust evaluation metric, MP-LPIPS, to evaluate the consistency between the generated image and the source garment. Extensive experiments show that the proposed method achieves superior performance in generating customized characters wearing target garments with high controllability and fidelity.
strength_weakness: strength_weakness: 1. **Well-written**: This paper is crafted organized and well-written.
2. **Diverse experiments and evaluations**: This paper includes rich experimental results and evaluations that provide strong evidence of the effectiveness of the proposed method.
3. **Garment extractor**:  The garment extractor module is one of the key innovations in the proposed method, that accurately captures and preserves the complex details of the input garments.
limitations: limitations: 1. **Garments mismatched for men**: Due to VITON-HD containing only frontal views of women and top clothing image pairs, the generation results (from male faces) in the last row of Figure 6 show obvious female characteristics, such as earrings, makeup, etc. It would be better to discuss this mismatch issue.

2. **Limitations on simple posture**: The proposed method outperforms the existing works on "subject-driven image generation" and "virtual try-on". However, using the generated human images instead of real human images, while increasing the garment controllable rate to a certain extent and reducing the generation error rate, also reduces the diversity of character postures.

3. **Reference dataset**: Due to the VITON-HD dataset having limitations on gender and garment generation (only upper body), I suggest authors also make comparisons between proposed methods and existing works on the DressCode dataset [1], which contains rich garments, images, and multiple categories (upper body, lower body, and dresses). 

   [1] Morelli, Davide, et al. "Dress code: high-resolution multi-category virtual try-on." CVPR 2022
paper_summary: paper_summary: This work investigates an unexplored garment-driven image synthesis task, presenting a novel network architecture to incorporate the garment-image features into latent diffusion models while maintaining faithfulness to the text prompts.
strength_weakness: strength_weakness: +1. This paper proposes Magic Clothing, a latent diffusion model (LDM)-based network architecture for an unexplored garment-driven image synthesis task.

+2.  This paper proposes a garment extractor to incorporate garment features into the denoising process via self-attention fusion. The joint classifier-free guidance is applied to balance the control of garment features and text prompts.

+3. This paper develops a robust metric namely MP-LPIPS to evaluate the consistency of the target image to the source garment.
limitations: limitations: 1). Although the methods proposed in this paper have shown promising results, the author has failed to clearly explain the fundamental differences between this method and previous methods, and why it produces favorable outcomes. For example, what are the specific differences between this method and OOTDiffusion (OOTDiffusion: Outfitting Fusion-based Latent Diffusion for Controllable Virtual Try-on)? What quality improvements do these differences bring about?

2). This method is pretty much an image-based virtual try-on task, so why do the authors consider this an unexplored new task, the claimed garment-driven image synthesis task?

3). It is regrettable that this method has not been compared with any image-based virtual try-on baseline methods.

4). The paper lacks appearance similarity metrics for quantitative experiments, such as SSIM.

5). The ablation experiments lack depth.

6).  There is a lack of visualized ablation experiments.

7).  The absence of coat samples in VITON-HD, as claimed by the authors: "Magic Clothing may fail to generate perfect results for complicated garments such as down jackets and coats.", is only unconvincing due to dataset limitations.

8). The author's work leans more towards addressing engineering problems, and the entire network still remains within the confines of current mature virtual try-on architectures. Furthermore, comparing a general architecture with a specialized one in experiments is unfair, so it is suggested to use specialized baselines (e.g. StableVITON) to further demonstrate the effectiveness of the proposed method.