Title: Mitigating Occlusions in Virtual Try-On via A Simple-Yet-Effective Mask-Free Framework
Venue: NeurIPS 2025
Accepted: yes
Keywords: virtual try-on

de-occlusion

diffusion models
Abstract: This paper investigates the occlusion problems in virtual try-on (VTON) tasks. According to how they affect the try-on results, the occlusion issues of existing VTON methods can be grouped into two categories: (1) Inherent Occlusions, which are the ghosts of the clothing from reference input images that exist in the try-on results. (2) Acquired Occlusions, where the spatial structures of the generated human body parts are disrupted and appear unreasonable. To this end, we analyze the causes of these two types of occlusions, and propose a novel mask-free VTON framework based on our analysis to  deal with these occlusions effectively. In this framework, we develop two simple-yet-powerful operations: (1) The background pre-replacement operation prevents the model from confusing the target clothing information with the human body or image background, thereby mitigating inherent occlusions. (2) The covering-and-eliminating operation enhances the model's ability of understanding and modeling human semantic structures, leading to more realistic human body generation and thus reducing acquired occlusions. Moreover, our method is highly generalizable, which can be applied in in-the-wild scenarios, and our proposed operations can also be easily integrated into different generative network architectures (e.g., GANs and diffusion models) in a plug-and-play manner. Extensive experiments on three VTON datasets validate the effectiveness and generalization ability of our method. Both qualitative and quantitative results demonstrate that our method outperforms recently proposed VTON benchmarks.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work introduces a mask-free VTON framework that tackles occlusions through two key operations: background pre-replacement to reduce inherent confusion between clothing and background, and covering-and-eliminating to address acquired occlusions by enhancing semantic understanding. The method is generalizable and compatible with generative models like GANs and diffusion models.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1) The paper provides an effective solution to the primary occlusion issues commonly encountered in virtual try-on approaches, addressing both inherent and acquired occlusions\n2) The method is extensively compared with state-of-the-art methods, clearly demonstrating its effectiveness and generalizability.\n\n\nweaknesses:\n1) The methodological descriptions and accompanying notations in the paper are a bit hard to follow. In particular, the explanation of the background pre-replacement strategy could be improved by supplementing the mathematical formulation with a clear, step-by-step visual illustration. Such a figure would enhance clarity and accessibility, as the current presentation may hinder comprehension.\n\n2) The qualitative results are primarily demonstrated using images from fashion datasets or under controlled studio photography settings. It would be more meaningful to evaluate the proposed method on in-the-wild data to better assess its robustness and generalization to real-world scenarios.\n\nQUESTIONS:\n1) Do g and g_un both refer to upper cloth？\n2) It appears that the identity of the subject changes noticeably after applying the target clothing. What might be the underlying cause of this issue? Have the authors considered any strategies to preserve identity more effectively during the try-on process?\n3) Does the proposed method generalize well to in-the-wild images, particularly for subjects with greater variation in body shape, such as individuals with heavier or more muscular physiques?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-17"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a new training pipeline for virtual try-on (VTON) tasks. The pipeline involves multiple complicated stages that trying to find the most appropriate supervision signals for training a generative model while addressing Inherent-Occlusion and Acquired-Occlusion. In experiments, the proposed method was evaluated on 3 datasets against a total of 21 baselines.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The presentation of the paper is pretty good. The problems are clearly defined, and the figures and tables are mostly clear and straightforward.\n2. The experiments are comprehensive and solid. The model outperforms other methods across multiple benchmarks.\n3. The appendix is thorough, with Algorithm 1 being particularly clear and helpful.\n\nWeaknesses:\n1. The technical contribution is incremental. While the proposed approach for identifying a more reliable training objective is valuable, it appears to be more of an engineering effort, as the optimization target remains training or fine-tuning a generative model. Moreover, the problems of Inherent Occlusion and Acquired Occlusion have already been defined in [16], which further diminishes the novelty of this work.\n2. Although cloth/body occlusions are greatly improved based on the figures and supplementary materials, the generation quality of human faces and limbs is significantly worse than that of the baseline methods. The generated faces are often distorted and blurred. Is there a specific explanation for this?\n\nSee \"questions\" below for further doubts and minor weakness.\n\nQUESTIONS:\n1. It seems that the mask-free approach serves as an additional image refinement step built upon the inpainted results from mask-based methods. Similarly, the proposed approach appears to be an extra refinement step based on a pre-trained mask-free model. Is this understanding correct? If so, direct comparisons with mask-based methods may not be strictly fair, given that those methods involve little to no refinement.\n2. Why did the authors use SD1.5 instead of SDXL as the base generative model? BooW-VTON has shown that SDXL produces better results, so it is unclear why SDXL was not used.\n3. In Section 4.2, Equation 10, the gating parameter $\\gamma$ is determined solely by the sizes of the neck and arms. What is the rationale behind using only these two body parts?\n4. The authors claim that their method generalizes to in-the-wild scenes. However, according to Section 5.2, the model was trained on the VITON-HD dataset and tested on the DressCode dataset, which is not strictly in-the-wild evaluation. I suggest including additional experiments on real-world images of humans and clothing sourced from the internet.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-20"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a mask-free VTON framework, which includes background pre-replacement operation and covering-and-eliminating operation, to address the inherent and acquired occlusion problems in existing VTON methods. The background pre-replacement operation mitigates inherent occlusions by preventing the model from confusing target clothing information with the human body or image background. The covering-and-eliminating operation enhances the model’s ability to understand and model human semantic structures, thereby reducing acquired occlusions.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths of this paper include:\n1. The motivation of the proposed method is pretty reasonable, as it deals with the major concerns of current VTON frameworks, which are inaccurate masking that missegments the human body, background, and clothing regions, as well as inaccurate representations that leads to erroneous human structure parsing.\n2. The writing of this paper is well organized. It first introduces the two types of occlusions in detail, then proposes the method that addresses these two types of occlusions respectively.\n3. Extensive quantitaive and qualitative experiments validate the effectiveness and generalization ability of the proposed method.\n\nPlease refer to next section for questions.\n\nQUESTIONS:\n1. From the visualizations in Fig.3 and in supplementary materials, I noticed that there exists common distortion in human models' face area, especifically around the eyes. In addition, the generated image resolution of the proposed method seems to be lower than comparison baselines. Do authors notice these issues, and what do you think might be the cause?\n\n2. From the illustration of Background Pre-Replacement process in Figure 13, I notice that there still exists inherent occlusion issue, where the mask for right arm is not accurate. This leads to another question regarding the background pre-replacement process. It essentially depends on the quality of the segmentation mask, so how can it eliminate the inherent occlusion issue brought out by inaccurate masking?\n\nLIMITATIONS:\nPlease refer to previous sections.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis is a virtual try-on paper focusing on scenarios where occlusion appears in images. More specifically, the authors group the occlusion-related problems into two main categories: (1) inherent, and (2) acquired. Main contributions are: (1) the analysis and formulation of the occlusion-related problems, (2) novel pre-replacement operation to prevent inherent occlusions, (3) novel covering-and-eliminating procedure to prevent acquired occlusions, (4) comparisons against many other methods (useful as a benchmark).\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\nThe paper is intuitive and provides insights into where the current methods are failing and provides potential solutions to those problems.\n\nThe method produces most detailed results (according to the perceptual error (LPIPS)) in Table 2 and maintains high ranking on other metrics in comparison with many other methods (useful as a benchmark).\n\n\nWeaknesses:\n\nQuality of the results: In Figure 5 there are still seams/mismatches in the “GANs + Ours” column.\n\nData: only standard fashion datasets were used, limiting the variety of clothing, body shapes, and poses.\n\nThere are no examples of results with more complicated backgrounds even though the model technically should be able to handle such examples due to the background augmentations that are used. Also see Line 305: “The results in Fig. 6 show that our method can better handle person images with diverse background styles” -> Fig. 6 contains examples with mostly white backgrounds only.\n\nThe paper does not contain many examples of failure.\n\nThere are some typos, for example: Line 165 aim -> aims.\n\nQUESTIONS:\nI think that showing and discussing limitations even further in this work would be useful for the community. The paper already identifies and suggests potential solutions to two interesting limitations. Can the authors include more qualitative results that depict the failures of the proposed method (showing a larger variety of clothing, body shapes, and poses)?\n\nIt seems that in Figure 5 (GANs), there are some artifacts in the results. Can the authors explain why? Is there a significant difference in the results for GANs as compared to diffusion models?\n\nCan the authors introduce some qualitative results with more complex backgrounds?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}