Title: ShoeFit: A New Dataset and Dual-image-stream DiT Framework for Virtual Footwear Try-On
Venue: NeurIPS 2025
Accepted: yes
Keywords: diffusion model

virtual try-on

controllable generation
Abstract: Virtual footwear try-on (VFTON), a critical yet underexplored area in virtual try-on (VTON), aims to synthesize faithful try-on results given diverse footwear and model images while maintaining 3D consistency and texture authenticity. 

Unlike conventional garment-focused VTON methods, VFTON presents unique challenges due to (1) Data Scarcity, which arises from the difficulty of perfectly matching product shoes with models wearing the identical ones, (2) Viewpoint Misalignment, where the target foot pose and source shoe views are always misaligned, leading to incomplete texture information and detail distortion, and (3) Background-induced Color Distortion, where complex material of footwear interacts with environmental lighting, causing unintended color contamination. 

To address these challenges, we introduce MVShoes, a multi-view shoe try-on dataset consisting of 7305 well-annotated image triplets, covering diverse footwear categories and challenging try-on scenarios. Furthermore, we propose a dual-stream DiT architecture, ShoeFit, designed to mitigate viewpoint misalignment through Multi-View Conditioning with 3D Rotary Position Embedding, and alleviate background-induced distortion using the LayeredRefAttention which leverages background features to modulate footwear latents. The proposed framework effectively decouples shoe appearance from environmental interferences while preserving high-quality texture detail through decoupled denoising and conditioning branches. 

Extensive quantitative and qualitative experiments demonstrate that our method substantially improves rendering fidelity and robustness under challenging real-world product shoes, establishing a new benchmark in high-fidelity footwear try-on synthesis. The dataset and benchmark will be publicly available upon acceptance of the paper.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper focuses on the task of Virtual Footwear Try-On (VFTON) and introduces MVShoes, a multi-view shoe try-on dataset with 7,305 well-annotated image triplets, covering various footwear categories and challenging try-on scenarios. In addition, the paper proposes a customized dual-image-stream DiT architecture called ShoeFit for generating high-fidelity try-on results. If MVShoes will be released publicly, I believe this work will make a contribution to the field.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. Virtual Footwear Try-On (VFTON) is a promising and worthwhile direction to explore.\n2. The proposed MVShoes dataset includes human foot images, pose annotations, masks, and multi-view product shoe images, which—if released—would be a valuable contribution to the field.\n\nWeaknesses:\n1. The proposed generation method, ShoeFit, appears to lack substantial novelty. However, since the main contribution of the paper seems to be the MVShoes dataset, this may be acceptable.\n2. There are questions regarding the virtual try-on methodology, please refer to the “Questions” section.\n\nQUESTIONS:\n1. Since Virtual Footwear Try-On is a task closely tied to real-world applications, it’s important to consider actual consumer behavior. In practice, many users prefer to virtually try on different styles within the same shoe series (e.g., Nike Dunk, adidas Ultraboost). These shoes typically share similar geometry but differ in materials, textures, and fine decorative details. The scenarios explored in the paper primarily focus on trying on shoes with substantial variations in style, shape, and color, such as transitioning between sneakers, boots, or high heels, which differs from this specific use case. However, both scenarios demand high attention to detail in the virtual try-on results. I would like to see ShoeFit’s performance on such subtler, yet practically relevant, use cases.\n\n2. It is noted that in Figure 5, the sock regions in the ShoeFit results sometimes differ from those in the input images. This may have undesirable effects—for example, in the first row, the generated socks differ from those in the input, potentially affecting try-on accuracy. However, it might also lead to positive effects in certain cases—for instance, in the fourth row, after trying on high heels, the socks in the result may be replaced by sheer stockings or bare feet. Is the change in socks controllable?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 2, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces ShoeFit, a framework for virtual footwear try-on (VFTON), addressing key challenges unique to shoes: data scarcity, viewpoint misalignment, and background-induced color distortion. To tackle the lack of realistic datasets, the authors curate MVShoes, a multi-view shoe try-on dataset comprising annotated image triplets, covering various footwear categories and real-world scenes.\n\nAt the core of their solution is a dual-image-stream DiT architecture, built upon FLUX.1-Lite, that separates denoising and conditioning to preserve shoe texture fidelity. The method incorporates two key modules: Multi-View Conditioning, using Reference Concatenation and 3D Rotary Position Embeddings to align different shoe perspectives, and LayeredRefAttention, which helps disentangle shoe appearance from environmental lighting and background. Their experiments show that ShoeFit significantly outperforms existing baselines, establishing a new benchmark for future VFTON research.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths: \n\nThis paper offers a contribution to the virtual try-on space by zooming into the challenges of virtual footwear try-on (VFTON), which has been largely underserved. The introduction of the MVShoes dataset, with 7305 annotated triplets across multi-view scenes and diverse shoe categories, and the authors have been meticulous in data curation, including pose alignment, background separation, and high-resolution filtering. From a quality standpoint, this dataset sets a strong potential for future work in VFTON.\n\nThe proposed ShoeFit model, built upon a dual-image-stream DiT framework, and the architectural decisions such as decoupling conditioning from denoising, using Multi-View Conditioning with 3D RoPE, and introducing LayeredRefAttention, are well motivated and grounded in the unique difficulties of shoes (reflectivity, misalignment, subtle variations). The experiments are well executed, with comprehensive metrics (SSIM, FID, DISTS, etc.) and comparisons to both strong baselines and state-of-the-art models like CatVTON and OOTDiffusion. Ablation studies are also nicely done, showing the incremental value of each module.\n\nWeaknesses: \n\nWhile the technical content is strong, the clarity of presentation could be improved in some parts. The paper occasionally leans on heavy architectural jargon (e.g., long sequences explaining attention math) without providing more visual or intuitive explanations, which might make it harder for readers to digest. A clearer visual roadmap, especially of the dual-stream pipeline, would help.\n\nFrom a significance angle, this is a clear step forward for VFTON, but the broader impact could be more explicitly explored. The work is very tailored to footwear, which limits its generalization to other product categories. It would be good to know whether the architecture can scale or adapt to accessories, or other reflective materials like handbags or sunglasses.\n\nOriginality: \n\nThe originality is solid. While it builds on existing diffusion and DiT frameworks, the way it adapts them specifically for VFTON is novel and well justified. Concepts like LayeredRefAttention and multi-view RoPE adaptation are interesting technical contributions.\n\nQUESTIONS:\n1. Generalization Beyond Footwear: Can the authors elaborate on whether the ShoeFit framework, especially the LayeredRefAttention and multi-view modules, could generalize to other reflective or hard-structured apparel items (e.g., handbags, belts, or sunglasses)?\nSuggestion: Even a small exploratory discussion or experiment would give insight into the versatility of the architecture.\n\n2. Efficiency and Inference Time Trade-offs: Given the dual-image-stream design and multi-view input handling, could you provide more details on inference time, GPU memory usage, and how this might scale to production systems?\n\n3. Failure Cases and Robustness: You mention limitations with small logos and fine text rendering. Could you show a few concrete failure cases or patterns of breakdown (e.g., shoe types or lighting scenarios that remain challenging)?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the unique challenges of Virtual Footwear Try-On (VFTON), including data scarcity, viewpoint misalignment, and background-induced color distortion, which differentiate it from garment-based VTON. To tackle these issues, the authors introduce MVShoes, the first large-scale multi-view footwear try-on dataset with 7,305 annotated image triplets. They also propose ShoeFit, a dual-stream DiT-based architecture that incorporates Multi-View Conditioning with 3D Rotary Position Embedding and a LayeredRefAttention module to enhance texture fidelity and reduce environmental interference. The model effectively decouples shoe appearance from background noise and improves rendering quality under real-world conditions. Extensive experiments demonstrate that ShoeFit sets a new benchmark in high-fidelity footwear try-on synthesis.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths\n- The most significant strength of this paper lies in the introduction of the MVShoes dataset, which, if released publicly, could catalyze research in the underexplored domain of Virtual Footwear Try-On (VFTON). The lack of publicly available footwear datasets has been a major bottleneck in expanding VTON research beyond garments like tops and bottoms. By providing a high-quality, multi-view dataset, this work opens the door for adapting existing VTON models (e.g., CATVTON) to a new task, potentially establishing VFTON as a standard benchmark in the community.\n- The proposed ShoeFit architecture integrates multiple components—such as dual-image streams, multi-view conditioning, and background-aware attention—which collectively contribute to improved realism and robustness in try-on synthesis.\n\n### Weaknesses\n- While the dataset itself is valuable, the data processing pipeline lacks technical novelty, appearing to follow conventional VTON data collection and preprocessing practices without introducing new methodologies tailored to footwear-specific challenges.\n- The ablation study is insufficiently thorough, making it difficult to isolate the contributions of individual components. For instance, the Multi-View Image Concatenation is directly borrowed from CATVTON and involves simple concatenation without architectural innovation. Similarly, the use of 3D RoPE is somewhat misleading in terminology—it merely adds an index-based channel per image, which may not fully justify the \"3D\" label and could benefit from clearer naming.\n- The LayeredRefAttention module includes a conditional mask mechanism, but it is unclear whether the ablation study isolates the effect of the mask itself versus the attention mechanism. This ambiguity raises questions about whether performance gains stem from the attention design or simply from background suppression via masking, as seen in works like Diptych Prompting [1].\n- Additionally, the impact of 3D RoPE is not clearly quantified. A comparison with 2D RoPE (as used in CATVTON) would help clarify whether the proposed positional encoding offers meaningful improvements.\n\n[1] Large-Scale Text-to-Image Model with Inpainting is a Zero-Shot Subject-Driven Image Generator, CVPR 2025\n\n\nI believe the paper presents a promising direction for expanding the VTON domain into footwear, and the introduction of the MVShoes dataset is a valuable contribution. However, the current version lacks sufficient ablation analysis to convincingly demonstrate the effectiveness of each proposed component. If the authors can provide a more thorough and transparent ablation study—particularly isolating the effects of 3D RoPE, LayeredRefAttention, and the conditional masking mechanism—I would be inclined to raise my score. I look forward to the authors’ response and clarification on these points.\n\nQUESTIONS:\nOne notable failure case appears in the teaser image, specifically in the second row's final example, where the rear shoe is not properly changed in the try-on result. This highlights a critical challenge in VFTON: handling spatially separated footwear, such as when shoes are not symmetrically aligned or are partially occluded. These cases are inherently more difficult due to depth ambiguity, occlusion, and the lack of explicit 3D reasoning in the model. It would be valuable to understand whether such failures are isolated or occur frequently across the dataset, and whether the model is robust to varying foot poses and inter-shoe distances. (It would be better discussing such observations in the limitation part, which is not clearly discussed in the current paper.)\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper explores a new task of virtual footwear try-on (VFTON). To address the challenges of data scarcity, viewpoint misalignment, and background-induced color distortion in VFTON, this paper first establishes a new dataset called MVShoes, which comprises 7305 annotated high-resolution image triplets, each containing two product shoe images from different views and a corresponding try-on result, then proposes a new dual-image-stream DiT framework (named ShoeFit) by using Multi-View Conditioning and LayeredRefAttention modules. These modules enhance the fidelity of try-on results by managing complex lighting interactions and preserving high-quality texture details. Both quantitative and qualitative experiments validate the superiority of ShoeFit in generating high-fidelity and robust try-on results.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. This paper pioneers a new task of VFTON, creates a new shoe dataset, and proposes a somewhat novel framework to achieve good VFTON results.\n2. The qualitative results, quantitative results, and ablation study all demonstrate the effectiveness of the proposed method and dataset.\n3. This paper is well organized and easy to read.\n\nWeaknesses:\n1. Personally, I don't think the VFTON is more challenging than the VTON of clothing, especially for loose and unrigid clothing. Moreover, the dataset consisted of 7305 image triplets is not large, which may limit the performance of VFTON, degrading the contribution of this work.\n2. Some visualization results of the proposed method are distorted, such as Figure 5(3) and the second example in Figure 6(b). As its limitation says, for some special texture materials, it is difficult for this method to generate the same results as the input. I think this may be because the dataset is not large or rich enough.\n\nQUESTIONS:\nMy questions are mainly about the weaknesses listed above, i.e., \n1. Personally, I don't think the VFTON is more challenging than the VTON of clothing, especially for loose and unrigid clothing. Moreover, the dataset consisted of 7305 image triplets is not large, which may limit the performance of VFTON, degrading the contribution of this work.\n2. Some visualization results of the proposed method are distorted, such as Figure 5(3) and the second example in Figure 6(b). As its limitation says, for some special texture materials, it is difficult for this method to generate the same results as the input. I think this may be because the dataset is not large or rich enough.\n3. Can the proposed method perform well on other public datasets, such as some images from DeepFashion that are suitable for VFTON?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-02"}