Title: PhysDiff-VTON: Cross-Domain Physics Modeling and Trajectory Optimization for Virtual Try-On
Venue: NeurIPS 2025
Accepted: yes
Keywords: virtual try-on

garment deformation modeling

physics modeling

trajectory optimization
Abstract: We present PhysDiff-VTON, a diffusion-based framework for image-based virtual try-on that systematically addresses the dual challenges of garment deformation modeling and high-frequency detail preservation.  The core innovation lies in integrating physics-inspired mechanisms into the diffusion process: a pose-guided deformable warping module simulates fabric dynamics by predicting spatial offsets conditioned on human pose semantics, while wavelet-enhanced feature decomposition explicitly preserves texture fidelity through frequency-aware attention.  Further enhancing generation quality, a novel sampling strategy optimizes the denoising trajectory via least action principles, enforcing temporal coherence, spatial smoothness, and multi-scale structural consistency.  Comprehensive evaluations across multiple datasets demonstrate significant improvements in both geometric plausibility and perceptual quality compared to existing approaches.  The framework establishes a new paradigm for synthesizing photorealistic try-on images that adhere to physical constraints while maintaining intricate garment details, advancing the practical applicability of diffusion models in fashion technology.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose a diffusion-based method for virtual try-on. To enhance the realism of the generated images, they introduce a strategy that incorporates pose guidance, wavelet-enhanced features, and sampling guided by multiple constraints. Experimental results demonstrate that the method produces higher-quality outputs compared to state-of-the-art approaches.\n\nSTRENGTHS AND WEAKNESSES:\nThe contribution of this paper is clear. It addresses two key challenges: preserving garment details during warping and effectively controlling the diffusion process. To tackle the first issue, the authors design a pose-guided deformation model combined with a wavelet-enhanced attention module. For the second, they introduce a novel sampling strategy based on potential-regularized path optimization, incorporating temporal and spatial smoothness terms as well as multiscale similarity for additional regularization. The effectiveness of these components is demonstrated through comprehensive ablation studies and comparisons with existing methods. Overall, the paper is well-structured and easy to follow.\n\nDespite these strengths, several points need further clarification:\n- Section 2.2 is somewhat confusing. In Equation 5, what does $x$ ​represent? What is its relationship to $p_k$? Are the $p_k$ sampled around $x$? Additionally, the relationship between $F_g$ and $\\tilde{F}_g$ is unclear. Including an illustrative figure would significantly help comprehension. This ambiguity also makes it difficult to understand the discussion on occluded regions in Lines 139–140.\n- In Lines131-132, the authors claim that `the offset predictor $G$ learns material-dependent deformation patterns`. However, it is not clear how the model infers material properties from the input garment image. Estimating physical material characteristics from appearance alone is non-trivial, especially without additional input. This claim should be either clarified or supported with empirical evidence.\n-  In Section 3.2, the formulations for $E_{smooth}$ and $E_{MS}$ use different terms $z_{t+1}$ and $z_{t-1}$. Shouldn’t these be the same?\n- Limited qualitative results. The qualitative results presented are too few to fully support the results. More examples should be provided, particularly for the ablation study, to better illustrate the contribution of each component in an intuitive manner.\n\nQUESTIONS:\nPlease address the questions mentioned above for further evaluation.\n\nLIMITATIONS:\nThe limitation of this paper is not discussed. More qualitative results and failure cases would help for the evaluation.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nAuthors present PhysDiff-VTON, a novel diffusion based framework  for virtual try-on that aims to solve the dual problems of realistically deforming garments realistically and preserving their high-frequency texture details. In order to achieve this, authors have created a method that connects physics-inspired mechanics with spectral fidelity preservation. \n\nThe framework has three main components:\n\n1) Pose-Guided Deformable Fabric Dynamics Modeling:  \nThe geometric discrepancy between canonical garment representations and dynamically posed poses two problems: \n- Irreversible Texture Distortion: When the flat clothing image is rigidly stretched or moved to fit the body, its texture can be permanently distorted.\n- Physically Implausible Deformation: The garment's movement appears unrealistic because the simulation fails to consider the specific stress-and-strain properties of the fabric material.\n\nTo address these problems present in traditional methods, the proposed physics-aware deformation field predicts garment movement by emulating strain-dependent displacement based on the person's pose. This method simulates the dynamics of fabric by using pose embeddings to predict spatial offsets, which preserves the natural curvature of the clothing. To further enhance realism, a cross-attention mechanism between the pose maps and garment segments introduces material adaptability. This allows the model to emulate real-world drape physics, where stiffer fabrics deform less than softer ones during the same movement.\n\n2) Wavelet-Enhanced Texture Preservation.\n\nThe loss of fine texture details is a major challenge in diffusion-based virtual try-on. This \"texture erosion\" happens because the iterative downsampling process within the model's UNet architecture progressively weakens the micro-patterns and directional gradients that define a fabric's look. To solve this, the paper proposes a Haar wavelet transform module embedded within the model's skip connections. This module works by explicitly separating and reinforcing these high-frequency components throughout the image generation process. This approach is based on the insight that traditional attention mechanisms are not effective at preserving the crucial frequencies that contain the most important perceptual information about a textile's microstructure..\n\n3) Potential-Regularized Path Optimization (PRPO) \n\nThis final stage addresses errors that can build up during the diffusion process, which might otherwise create unnatural wrinkles or jerky fabric movements. PRPO treats the image generation as a physics problem, guiding the process to find the most plausible and \"energetically favorable\" result. This is achieved by handling the diffusion sampling process as a variational optimization problem. Thus, allowing the injection of physics-inspired constraints into the generative trajectory, conditioning it toward states that balance data reconstruction with physical realistic results.\n\nTogether, these three physics-inspired modules create a control loop that ensures the generated images are both geometrically accurate and rich in texture, even in complex poses.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\nThe work directly addresses weakness of previous methods: \n- preserving high-frequency texture details while accommodating complex fabric deformations\n- enforcing physical plausibility in garment dynamics under diverse human poses.\n\n- Authors present an unified frameworks that bridge data-driven feature learning with continuum mechanics principles governing fabric deformation. There are strong novel contributions: \n\n  - Pose-Guided Deformable Fabric Dynamics Modeling\n  - Wavelet-Enhanced Spectral Fidelity Preservation\n   Clean introduction of high frequency preservation in skip connections.\n  - Potential-Regularized Path Optimization for trajectory optimization:\n    Mathematical derivation and explanation of the proposed approach is clear and well motivated. \n\n- Above state of the art results in baseline datasets\n- Ablation study is clear and highlights the importance of each contribution\n- Paper is well written and structured\n\nWeakness:\n- Missing diagram figure to clarify network architectures and proposed approach\n- Missing some qualitative(a zoom in to high frequency areas) and quantitative example to support some preservation of high frequency: could we have some measurement of this, i.e analysis of the spectrum on the area of the garment?\n - More insight in the computational cost of each one of the proposed novelties(inference time analysis or some theoretical insight).\n- Out of distribution performance:  While authors train and test in VITON-HD and DressCode, it would be interesting to see the generalization capacity of the proposed approach with some results in other datasets or images on the wild.\n\nQUESTIONS:\nCheck Strengths And Weaknesses\n\nLIMITATIONS:\nNo", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose to integrate physics into diffusion-based methods for image-based virtual try-on, to improve the geometric fidelity and wrinkle realism without sacrificing texture detail. The authors introduce a pose-conditioned warping module to simulate fabric dynamics, introducing wavelet decomposition and frequency-aware attention, and introducing several regularizations into the diffusion sampling strategy (temporal consistency, total variation constraint for spatial consistency, and multi-scale self-similarity). The authors outperform several baselines on VITON-HD and DressCode test sets, both qualitatively and quantitatively, with more natural and physically accurate cloth wrinkles than prior art.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- The proposed pose-conditioned warping successfully computes warped garment features conditioned on a pose embedding extracted from the input image. The wavelet decomposition enhances high-frequency textures (as seen in Fig. 1 bottom left).\n- The results indicate more physically accurate wrinkles and cloth deformation than prior art.\n- The core innovations are systematically evaluated in an ablation study\n\nWeaknesses:\n- Similar to baselines, the method is unable to faithfully copy text from the input clothing image onto the subject (as seen in Fig. 1 top right).\n- The results in Fig. 1 are shown under rather constrained settings: all participants are in a well-lit studio environment in a standing pose. In contrast, the project page of IDM-VTON shows high-quality results on 16 diverse in-the-wild examples. Comparison to IDM-VTON and other baselines on those 16 examples would be beneficial to understand the performance in challenging in-the-wild scenarios, where physics integration would be most beneficial, such as diverse backgrounds, varying lighting, and challenging poses (e.g. hands on hips, eating with chopsticks while seated, etc).\n\nQUESTIONS:\n- How well does the model perform on in-the-wild images, similar to the qualitative examples shown in IDM-VTON's Github page (https://github.com/yisol/IDM-VTON), where the proposed physics integration (e.g. pose-conditioned fabric warping) would be most necessary?\n- Not sure whether it is possible to ablate and visualize the impact of each of the individual regularizations (temporal consistency, total variation constraint for spatial consistency, and multi-scale self-similarity) within PRPO? To visualize, could show qualitative examples of final-timestep predictions throughout the denoising process, with each of the regularizations removed. PRPO seems to have overall the least impact according to Tab. 3, and it would be informative to understand whether all three regularizations are necessary.\n\nLIMITATIONS:\nThe authors do not discuss limitations of the work in the paper.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces PhysDiff-VTON, a novel framework for virtual try-on that integrates physics-inspired mechanisms into diffusion models to address the challenges of realistic garment deformation and high-frequency texture preservation. The key contributions of this work includes:\n1. a pose-guided deformable warping module that simulates fabric dynamics by predicting spatial offsets conditioned on human pose semantics.\n2. a wavelet-based feature decomposition method which used to address the issue of texture erosion during the diffusion process\n3. Potential-Regularized Path Optimization (PRPO), a novel sampling strategy that optimizes the denoising trajectory by enforcing temporal coherence, spatial smoothness, and multi-scale structural consistency.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n1. The paper presents a well-thought-out integration of physics-inspired mechanisms into diffusion models. The proposed methods, such as the pose-guided deformable warping module and wavelet-enhanced texture preservation, are technically sound and well-supported by mathematical formulations.\n2. The integration of physics-inspired mechanisms into diffusion models could have broader implications beyond virtual try-on, such as in other areas of computer vision and graphics where realistic simulations are required.\n\nWeaknesses\n1. Some sections, particularly those involving the mathematical formulations of the diffusion process and the potential functions, may be challenging for readers without a strong background in the field. More intuitive explanations or visual aids could help make these sections more accessible.\n2. The paper presents a limited number of visual examples of the virtual try-on results, which makes it difficult to fully assess the comparative advantages of the proposed method over other approaches across a broader range of cases.\n\nQUESTIONS:\n1. It would be even more helpful for readers if there were structural diagrams.\n2. Adding more visual comparisons, especially for challenging cases (e.g., complex poses, intricate textures), would provide a clearer picture of the method's strengths and limitations.\n3. Adding a section that explores potential applications in other domains (e.g., animation, video editing) would highlight the broader impact of the work.\n\nLIMITATIONS:\n1. The authors should include a dedicated section on limitations, where they discuss the current scope and boundaries of their work.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-07-03"}