AnyFit: Controllable Virtual Try-on for Any Combination of Attire Across Any Scenario
Authors: Yuhan Li
Hao Zhou
Wenxiang Shang
Ran Lin
Xuanhong Chen
Bingbing Ni
Venue: NeurIPS 2024 poster (2024)
Keywords: Virtual try-on
Diffusion models
Multiple conditions
Primary area: diffusion_based_models

Abstract: While image-based virtual try-on has made significant strides, emerging approaches still fall short of delivering high-fidelity and robust fitting images across various scenarios, as their models suffer from issues of ill-fitted garment styles and quality degrading during the training process, not to mention the lack of support for various combinations of attire. Therefore, we first propose a lightweight, scalable, operator known as Hydra Block for attire combinations. This is achieved through a parallel attention mechanism that facilitates the feature injection of multiple garments from conditionally encoded branches into the main network. Secondly, to significantly enhance the model's robustness and expressiveness in real-world scenarios, we evolve its potential across diverse settings by synthesizing the residuals of multiple models, as well as implementing a mask region boost strategy to overcome the instability caused by information leakage in existing models. 
Equipped with the above design, AnyFit surpasses all baselines on high-resolution benchmarks and real-world data by a large gap, excelling in producing well-fitting garments replete with photorealistic and rich details. Furthermore, AnyFit’s impressive performance on high-fidelity virtual try-ons in any scenario from any image, paves a new path for future research within the fashion community.

AnyFit: Controllable Virtual Try-on for Any Combination of Attire Across Any Scenario
Yuhan Li Hao Zhou', Wenxiang Shang", Ran Lin", Xuanhong Chen', BingBing Ni
Shhnnhaai Jiao Tong University, Shanghai 200240, China
Alibaba
[melodious, mibingbing @sju.edu.cn
(a) High-fidelity virtuil try-on across any sccnario
(b) High-fidelity virtuil try-on for any combination of attire
Figure 1: AnyFit shows superior try-ons for any combination of attire across any scenario.
Abstract
While image-based virtual try-on has made significant strides, emerging approaches still fall short of delivering high-fidelity and robust fitting images across various scenarios, as their models suffer from issues of ill-fitted garment styles and quality degrading during the training process, not to mention the lack of support for various combinations of attire. Therefore, we first propose a lightweight, scalable, operator known Hydra Block for attire combinations. This is achieved through parallel attention mechanism that facilitates the feature injection of multiple garments from conditionally encoded branches into the main network. Secondly, to significantly enhance the model's robustness and expressiveness in real-world scenarios, we evolve its potential across diverse settings by synthesizing the residuals of multiple models, as well as implementing a mask region boost strategy to overcome the instability caused by information leakage in existing models. Equipped with the above design, AnyFit surpasses all baselines on high-resolution benchmarks and real-world data by large gap, excelling in producing well-fitting garments replete with photorealistic and rich details. Furthermore, AnyFit's impressive performance on high-fidelit virtual try-ons in any scenario from any image, paves new path for future research within the fashion community.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Introduction
The dramatic success of e-commerce is steadily demanding a more convenient and personalized customer shopping experience. Among them, image-basee based virtual try-on (VTON) [7] 22 has emerged as promising topic in the research community and witnesses rapid advancements [13 24
47
18] whose task is to fit the target garment to the human body with various gestures. However, current methodologies do not meet the high-fidelity and robustness required in real-world applications, often resulting in artifacts or the mismatch of clothing details. Furthermore, the support for variety of try-on combinations of attire (10) remains an area of ongoing research.
Most prior methodologies 6 [4) utilize separate warping module to align garments the human body, subsequently employing a Generative Adversarial Network (GAN) (19) for their integration. This explicit warping process typically yields overly smooth garment transformations and struggles to cope with complex poses and occlusions. (50). While some diffusion based methods (58) 50)
leverage pre-trained diffusion models (43), with structures akin to ReferenceNet (26) to preserve fine-grained garment information; However, these methods encounter difficulties in producing vivid fabric textures and photorealistic lighting and shadows. They also show artifacts in cross-category try-ons, diminishing their inherent text-image capacity when applied to specialized tasks as depicted in Fig. 4 In summary, existing methods still fall short in producing images of high fidelity that exhibit clothing styles rendered with exceptional detail and true-to-life accuracy across scenes. Moreover, these methods are designed solely for trying on individual items of clothing and do not support multi-conditions, thereby failing to facilitate the free combination of tops and bottoms.
As discussed above, believe that ideal VTON workflow should exhibit the following properties:
Scalability, The ultimate goal of the VTON model is to enable any free and customizable virtual outfit combination of multiple garments (57). It should support multi contiiinnniion injection, allowing for easy expansion to more applications, such as mixing and matching tops and bottoms, layering inner and outer garments, etc.
Robustness. Given the diverse scenarios encountered in e-commerce settings (33) the VTON model should generate authentic fabric textures and natural lighting, reproducing the details of the target clothing (e.g., logos, patterns, texts and strips) stably and accurately.
We present the following critical contributions to establish AnyFit as novel VTON paradigm, which adeptly addresses the challenge of any combination of attire across any conceivable scenario. in Fig. [ AnyFit mainly consists of two isomorphic U-Nets, namely HydraNet and MainNet. The former is tasked with extracting fine-grained clothing features, while the latter is responsible for generating try-ons. (1) Scalability: A hallmark of AnyFit is its innovative introduction of the Hydra Encoding Block that only parallelizes attention matrices within a sharing HydraNet, enabling effortless expansion to any quantity of conditions with only 8% increase in parameters for each additional branch. The proposal of parallelizing these blocks is built on the insight that only the self-attention layers are crucial for implicit warping (50), while the remaining components primarily serve as generic feature extractors. We further invent Hydra Fusion Block to seamlessly integrate the features of Hydra Encoding into MainNet, with positional embeddings to distinguish encodings from different sources. It is important note that ReferenceNet (26) 99) GarmentNet (50) could be seen as specific instances of HydraNet when limited to a single condition. (2) Robustness: Observations indicate a noticeable reduction in the robustness and quality of images generated by existing Virtual Try-On (VTON) works, in comparison to the original stable diffusion performances. Inspired by discussions in the community 2, we present the Prior Model Evolution strategy. This innovative approach involves merging parameter variations within a model family (e.g., a collection of fine-tuned versions of SDXL (41),, enabling the independent evolution of multiple capabilities of the base model. This strategy emerges as an intuitively logical and highly effective method for amplifying the model's innate potential prior to training. This particularly relevant when contending with the significant escalation in training costs associated with dual U-Nets an aspect that is overlooked in previous research. Furthermore, we introduce the Adaptive Mask Boost to further enhance the fit of the attire as a bonus. It requires length augmentation of parsing-free mask regions during the training phase, allowing the model to autonomously understand the overall shape of the elothing, which emancipates the model from previous reliance on hints of masks derived from garments. During inference, we adapt the shape of the mask area based on the aspect ratio of the target garment, thereby markedly encouraging the generation of well-fitted try-ons, particularly for long garments (e.g., windbreaker).
2

---

HydraNet
Block
"The mo wearing
Cuncalonak:
1e PiiCe y storts, parted Vootor tnlected pornny"
Uydtr
Closs Allestinn
IP-Adayter
Hydto Encoding
Poetee
AA
IP Mr:iiegee
Hydra Fusion Block
MainNet
Figure 2: Overall framework of our method.
To the best of our knowledge, AnyFit stands as pioneering VTON method fulfill scalability and robustness requirements. Our innovative HydraNet and Prior Model Evolution strategies have the potential to transform not just the domain of VTON, but to catalyze advancements across broader spectrum of conditional generation applications. Finally, we have carried out comprehensive experiments on try-on benchmarks (6) and engaged in challenging validation using in-the-wild sets. These experiments demonstrate that our model shows exceptional performance that eclipses current methods by a substantial margin, terms of garment fidelity and robustness when addressing street-captured scenarios. In addition, our method has realized a formidable capability for multi- garment try-ons, culminating in results that exhibit strikingly harmonized upper and lower styles.
Related works
GAN-based virtual try-on. The virtual try-on task is concerned with synthesizing images of person donning the designated garment with appropriate fit (23), while retaining salient characteristics of the original garment and person, given a pair of images depicting a person and a target garment. To execute this task, numerous works (17
30
34
6
22 have utilized Generative Adversarial Networks 14
(GANs) 199 with two-stage strategy (35
I8 52): (1) warping the clothing to the desired shape (7))
and (2) fusing the deformed clothing via try-on generator based on GAN. HR-VITON (35) conducts both warping and segmentation concurrently to address issues related to body occlusion and misfit of garments. GP-VTON (49) introduces localized warping along with global parsing to independently simulate the deformation of different regions of clothing, aiming to achieve a more form-fitting result. However, these existing approaches that rely supplementing the sides of the clothing and the natural lighting and shadows (50).
explicit warping module incapable of
Diffusion-based virtual try-on. As significant progress in Eext-to-Image diffusion models (25) 39
27 witnessed in recent years, some works (10) have been motivated to incorporate pre-traines diffusion models (13)4) 41) priors into virtual try-on task. LADI-VTOR (38) and DCI-VTON 20)
explicitly deform the clothing to achieve lixel-level alignment with the human body, followed by diffusion model to blend the clothing with the human body refinement. StableVITON (II) introduces an end-to-end approach that injects intermediate feature maps from a spatial encoder into the U-Net decoder via recently, OOTDiffusion (50) and IDM (13) achieve garment feature extraction with a parallel U- Net and feed them through self-attention for enhanced integration. Unfortunately, these methods intrinsically lack support for try-ons that involve multiple garments. Moreover, they exhibit artifacts and unstable garment fits for arbitrary images, which leads to degradation of performance on out-of-distrios images in complex backgrounds and poses.
cross-attention block, akin the ControlNet (55) structure. Most
3
Method
3.1
Model overview
An overview of the AnyFit is presented in Fig. The backbone of AnyFit employs the SDXL (43).
with the preliminary detailed in Appendix C.2 Given a human image 2y E # and target
garment image Zg E HxW XxHxxxx AnyFit aimed to generate an authentic try-on image Str- We
employ OpenPose
8 53) to obtain clothing-agnose mask Im and masked person image Tag
3

---

adjusting for the size of different garments, as detailed in Sec. S.c. We treat VTON as a specific case of image inpainting (5T), endeavoring to fill the masked person Eag with the cloth Tg. The main inpainting U-Net (MainNet) inputs 3 concatenated components with 9 channels: the noisy image 2t, the latent agnostic image F( and the resized agnostic mask I'm where E.()) represents VAE (32) encoding. A Pose Guider (26) with 4 convolution layers (4 x 4 kernels, 2 x 2 strides, 16, 32, 64, 128 channels) is incorporated to align the pose image E(zp) with noise St.
Scalability: To preserve the fine details of the clothing, as well as to support both single and multiple garment VTONs, we employ a HydraNet that mirrors the MainNet in encoding clothing information. It shares the same weight initialization as the MainNet and innovatively parallelizes attention metrics based on the number of conditions to create Hydra Encoding Blocks for different conditional encodings. Robustness: During training, issues such as mask information leakage and quality degradation were observed. To address these issues, we adopt Adaptive Mask Boost and Prior Model Evolution, respectively, which significantly bolster the model's 's robustness across different scenarios cost-effectively and straightforwardly.
3.2 HydraNet for multi-condition VTON
HydraNet. Inspired by successful practice in human editing (26, 99. we introduce garment encoding network isomorphic the main generative network (MainNet), that precisely preserves the details of clothing. When dealing with multi-garment VTON, a direct method might involve replicating multiple garment encoding nets to manage different conditions. This approach, however, would lead to a significant increase in the number of parameters, rendering it computationally prohibitive. Experimentally we discover that for conditions with similar content (such as different types of clothing), the self-attention module plays a vital role in the latent warping of the garments, aligning them with the locations requiring inpainting. Conversely, other network architectures, which typically tasked with general feature extraction, can be shared across different condition encoding branches without compromising the model's 's performance. In view of this, we innovatively propose HydraNet for multi-condition encoding. It operates based on a shared Unet structure, while parallelizing the attention modules according to the number of input conditions, thereby constructing Hydra Encoding Blocks. Specifically, we parallelize the self-attention matrices with identical initial weights, and feed the multi-condition key and value features < " into MainNet, which encode the fine-grained details of the clothing. It's notable that ReferenceNet (26) 9 or GarmentNet (50) could be seen as specific instances of HydraNet limited to a single condition. HydraNet requires only one forward pass (timestep 0) to encode clothing before the multiple denoising steps in MainNet, with the additional temporal and parameter overheads being minimal for each added condition, in Tab. Z
Hydra Fusion. We propose a highly efficient and easily scalable Hydra Fusion Block to replace the self-attention layers in MainNet, accomplishing the feature injection from HydraNet to MainNet for any length via concatenation. Specifically, given the key and value features { { 2 E bxixe from the HydraNet, we introduce learnable position embeddings to distinguish features from different source conditions. The superscript denotes different input conditions. Subsequently. we concatenate the key and value along the dimension to obtain the final 2 2all] E RgxxNixc as:
2hv
(I) where N denotes the total number of input conditions, PE represents positional encoding, and ① signifies concatenation. <he follows a similar formulation. Facing the key and value features {zmk: } E 1xxxxx from the MainNet and concatenated HydraNet features { { zhe J. we once again concatenate the corresponding features along the dimension into { Ree } E b which are then used in subsequent attention calculations with 2mq- It is noteworthy that, with the lightweight design leveraging parallelization and concatenation, HydraNet can be effortlessly extended to perform injections with any number of conditions, thereby possessing a more expansive application potential within the domain of generative models.
a!l
(2hk + PE' (zhx)) + (2 + (zhe))
@ (E + PE
_all
3.3 Model evolution and mask boost for robust VTON
Prior Model Evolution. A diminution image generation performance with the SDXL-inpainting model compared to the SDXL base model is noted (I. We attribute this degradation to the disruption of previously well-aligned correspondences between text and images during the inpainting pre- training phase. Drawing inspiration from the open-source community (2), we develop Prior Model

---

Garment
AnyFit(ours)
OOTDiffusion
StableVITON
LADI-VTON
HR-VTON
Figure 3: Visual comparisons VITON-HD. AnyFit displays superior details and outfit styling.
Evolution strategy, which enhances the model's strength and adaptability in generating outfit images at a very low cost, even without training. Specifically, we meticulously amalgamate the weights from three distinct, powerful models to evolve the initial weights for our model. These models comprise: SDXL-base- 1.0 (4T), IDXL-inpainting-0
4 with inpainting capabilities, and DreamshaperXL alpha2 (3), which demonstrates superior performance in generating clothing and human figures. Their weights are denoted by Wase, W imp: ds. The evolution formula as follows:
W new W + (W kpp
W Wase)
+ 3- (Wae W Whose)
(2)
Inpainting
Outfitting Incremeut
where and 3 are the balancing coefficients that account for the capability enhancements from SDXL-inpainting and DreamshaperXL. It is important to note that we directly copy the extra channels the conv layer of the SDXL-inpainting into the merged model, multiplying them by a.
However, the optimal values of a and B are not apparent. We hope to determine the optimal a and to ensure that the initial weight W new achieve the best evaluation performance, i.e.
arg min [(a + (W base + (W inp
Wbase)
3 (Was ds <Wbase)
(1,23)
where o is non-differentiable evaluation function. Empirically, we assume that exhibits monotonic or convex properties with respect to the balancing coefficients (a,, 33 in most regions. Therefore, we discretize the continuous domain [0, 2] into a grid with 5 0.1 the step size and design the discrete greedy algorithm 1, to search for the optimal (a. 8). In our algorithm, we selecte the CLIP score (42) on 20 fixed inpainting image-text pairs the evaluation function 4. The optimal solution obtained is (a, 3) (1.0, 1.1). Please refer to the Appendix C.I for more explanations.
Adaptive Mask Boost. Previous works gen- erally exhibit limited robustness in cross- category try-on scenarios, resulting in inac- curately rendered clothing styles as shown in and [0] This largely due depen- dence on agnostic masks derived from eloth- ing parsing, which tends leak the edges of the clothing shape during training. This leakage may cause the generated garments to almost entirely cover the agnostic mask re- gion. In response these limitations, we em- play an intuitive and effective approach that significantly enhances the model's robustness about cross-category try-on, i.e., the Adaptive Mask Boost strategy, which primarily com- prises mask augmentation during training and
Algorithm Discrete greedy algorithm
Require: Evaluation function f. stop size 8. Initialize (a, () < 0..5
2: while True
3: Feurrent - f(a. 3)
4: N (a 5, B), 6, 6,8), (a, (a. 8))
5: N (a' B¹) ) EENN
$2 and 5 2)
6:
FN - (f(a'')9) (aL, (ed.B') E N)
7:
if (Fin feument then
8:
break
9:
else
(a, + arg mig , f(o..8)
11:
end if
12:
end while
13:
return (a,
5

---

adaptive elongation during inference. Specifically, during training, the agnostic mask is extracted solely using OpenPose body joint detections without leveraging human parsing. We perform random elongation of the mask by a factor ~ Uniform(1.. 1.2, 1.5) with probability of P = 0.5. This training setting forces the model autonomously determine the optimal cloth length. During inference, we assess the aspect ratio of the bounding box of the laid-out garment. If a 1.2, we proportionally extend the agnostic area to match o, creating an adaptive agnostic mask that conforms to the garment's style. Experiments have validated that AnyFit Fit with Adaptive Mask Boost autonomously determines the appropriate garment length, yielding robust try-on results across different clothing categories.
(a) Visual comparisons the proprietary dataset. AnyFii(proprietarr) has capabilities for both open-garment and layered clothing. Even only trained on the VITON HDD, Any,AnyHi Anybit has demonstrated excellent preservation of the fidelity of clothing. Input AnyFiippreprietrr) AnyFiiVITTON HD)
IDM
OOTDiffusion StableVITON
LADI-VTON
(1) Openggmment rendering
(2) Layering capability
HERE
(3) Complex patterns
AnyFit demonstrates robust and high quality result in large variety of scenarios.
Figure 4: Visual results on proprietary and in-the-wild data, Best viewed when zoomed in.
4 Experiments
4.1 Experimental setup
Datasets.
Our experiments carried out publicly available datasets, VITON-HD 12 (11647 training pairs) and DressCode (16) (48392 training pairs) using the official splits for training and testing, as well an additional proprictary e-commerce dataset. This proprictary dataset contains 50602 training pairs and 2500 testing pairs of mainly upper-body person and upper garment images, featuring complex patterns, backgrounds, and postures, alongside a rich variety of styles including layered garment ensembles, which present a more challenging scenario. As for multi- garment try-on, we utilized a HumanParsing model (28 29) to extract clothing items from DressCode and constructed triplets consisting of (upper-body garment, lower- garment, model image). In these triplets, one garment is an original laid-out image, while the other is a cropped image from the person's image. Finally, we construct 24314 publicly available upper-lower triplets by garment crop from the 15363 upper-body pairs and 8951 lower-body pairs DressCode, called DressCode-multipp 1800 triplets is reserved as the test set. Please refer to the Appendix B for more details.
A subset of

---

Gamment
AnyFicoum
VTON-
IDM-2Singe
Paint exsmple
Ganment
AnyFi(ouu))
VTON-conca
IDM-2Siage
Paint by example
Figure 5: Visual comparisons on the DressCode-multiphe AnyFit exhibits an elegant integration between upper and lower garments, accurate length control, and appropriate overall styling.
Comparison performance
training
(b)
previous
image
(c) Visual results about model evolution with various a, f after initialization without training. We discover that within reasonable range, raising improvements inpointing outfitingg capabilities.
a=
A=11
8-0.5
$-009
f
0-5
Outfittng quality increasing
Impaintnn ability increusing
(d) Aspect sensitivity. Our model exhibits optimal performance approgriate Additionally, the mask aspect ratio continues incresse, the length of the try-on tends to remain constant, suggesting that our model potentially leams the correc length the garments. Gurment
Pessn
1.25
2.25
Figure 6: Visual validation about model evolution and mask boost in (a), (c), (d). We also provide visual results about mask reliance in (b) found in previous work.
Implementation details. We initialize the AnyFit with Prior Model Evolution strategy described in Sec. 3.3) and fine-tune it using an AdamW optimizer (37) with a constant learning rate of 5e - 5. We train three variants of models the VITON-HD. DressCode and proprictary dataset a resolution of 1024 x 768, independently. Subsequently, we extend the model trained on DressCode to enable multi- garment try-ons by training on the TressCode-multipee dataset. All the models are trained for 150 epochs on 8 NVIDIA A100 GPUs with DeepSpeed
(5) ZeRO-2 to reduce memory usage, at inference time, we run AnyFit on single NVIDIA RTX 3090 GPU for 30
a batch size of sampling steps with the DDIM sampler (4)). Outfitting dropout (50) is used with guidance scale
8y 1.3. The data augmentation follows the same protocol as MtableVTON (3I). We also employ
pretrained Id-Adepter (54) for SDXL. Please refer to the Appendix D for more details.
7

---

Person
Closh
Agnostic image
Full model
Network structure illustration of SDXL. Orange lines represent self connection.
Upiredds
Iadia
Only Redlictt, Ne
RecRbck-Am
connection
up
up
up connection Bel
crous actn images.
Figure 7: We separately cut off the self-attention injections between different blocks HydraNet and MainNet, as well as the image features from IP-Adapter in cross-attention layers. The results show that the self-attention layers between the up blocks are the decisive factor affecting the performance.
Table 1: Quantitative comparisons the VITON-HD 12) and DressCode (16).
Dataset
VITON-IID (2
DressCode (16)
Method
LPIPS
SSIM f
FID 1
KID
LPIPS J
SSIM f
FID L
KID
HR-VTON
35
0.097
0.878
12.31
3.86
DCI-VTON
0.92
StahleVTON
0.076
0.891
9.35
1.51
OOTDiffusion
50
0.093
0.856
9.16
0.68
GP-VTON
(4)
0.083
0.892
9.17
0.93
0.051
0.921
5.88
1.28
LADI-VTON
38
0.091
0,875
9.42
1.63
0.072
0.902
6.94
2.33
IDM
13
0.078
0.881
9.12
1.03
0.046
0.923
5.32
1.24
Fit(our))))
0.075
0.893
8.60
0.55
0.044
0.904
4.51
0.48
Evaluation protocols. We measure reconstruction accuracy by LPIPS (56) and SSIM (48) in paired setting provided ground truth images, and authenticity of unpaired synthesized images by FID (40) and KID (45) without ground truth. All evaluations conducted at resolution of 512 384.
Baselines. We compare our model on single try-on tasks VITON-HD, DressCode and our proprietary dataset with previous baselines including HR-V HR-N (35), LADI-VTON (38). DCI-
VTON (20). VITTO (31), OOTDiffusion (50), and the state of the IDM (3). We directly utilize their released pre- treirra models. As for multi- multi-garment try-on, we developed two-stage IDM model as strong baseline, referred to as IDM-2Stage, which dresses the upper and lower garments sequentially. Inspired by (10), we concatenate the upper and lower garments width spatially and feed them single-conditional HydraNet for training as another baseline, termed VTON-coneat. Finally, compared AnyFit with Paint by Example (51). IDM-2Stage, and VTON-concat.

---

Table 2: Quantitative comparisons the DressCode- multiple. The "Time" represents the inference time increase compared its single-garment try-on.
Table 3: Comparisons on proprietary dataset.
AnyFit (xxx) is trained only on XXX dataset.
Method
FID
KID 4
Method
FID 4
KID +
Time 4
LADI-VTON 33
52.24
6.51
DCI-YTON 20)
$7.96
12.35
Paint-by- bx-Expppeeeeee (S1
35.17
13.12
95%
StableVTON 31
53.80
8.13
IDM-2Stage (13)
21.47
7.85
93%
1DM 13
48.76
4.35
VTON-coneat (00
21.11
7.30
8%
AnyFit (VITON-HD)
46,95
2.73
AnyFit (ours)
20.43
7.10
9%
AnyFit (proprietary)
43,97
0.69
4.2 Qualitative results
Single-garment try-on. Fig and provide a qualitative comparison between AnyFit and the baselines on VITON-HD, the more challenging proprictary and in-the-wild data, covering open- garment and layering rendering scenarios. For fair comparison with the baselines, we include results of AnyFit trained on VITON-HD. AnyFit excels retaining intricate pattern details, owing to the effective collaboration between HydraNet and the IP-Adapter. It also maintains the correct silhouette of the clothing at a semantic level. This suggests that, through Mask Boost, AnyFit enhances the recollection of the original shape of the clothing, while other models, influenced by the mask, tend to generate incorrect appearances. The Prior Model Evolution further strengthens the texture representation of the apparel. Notably, when trained on the proprietary dataset, AnyFit automatically fills in inner garments or unzips clothing based on posture, a capability absent in the version trained on VITON-HD due the lack of such training data.
Multi-garment try-on. Fig. 5 offers a qualitative comparison for multi-garment try-ons using the compiled PressCode-multippe dataset. Firstly, AnyFit demonstrates high-fidelity cloth preserva- tion. Importantly, thanks to the distinct and individual Hydra-Bloos Blocks situated in different condition branches, AnyFit accurately depicts the demarcation between the upper and lower garments. show- casing a reasonable transition at the interconnection. In contrast, "TON-conee mishandles the relative clothing sizes after concatenation, leading to garment distortion and blurring. Meanwhile, IDM-2Stage faces artifacts at the juncture of the upper and lower garments, because it obscures parts of one garment while trying another. Remarkably, despite training with one garment presented as a flat lay image and the other as a warped cloth cropped from a person image, AnyFit remains strikingly robust when faced with both garments presented as flat lays during inference.
4.3 Quantitative results
As indicated in Tab. T2 extensive experiments conducted on VITON-HD (2). DressCode (16), the proprictary dataset, and mressCode-mulippl consistently prove that AnyFit significantly surpasses all baselines. This confirms AnyFit's capability to deliver superior tr quality in both single- garment and multi-garment tasks across various scenes. Moreover, note that AnyFit shows considerable improvement in unpaired settings in terms of the FID and KID metrics, demonstrating our model's robustness for cross- cross-yy try-ons. For more results, please refer to the Appendix
4.4 Ablation study
Hydra Blocks. To validate our proposed Hydra Blocks, we directly employ a singular conditioned HydraNet (which degenerates to ReferenceNet (26) actually) as the baseline "w/o Hydra Block" to encode both the top and bottom garment conditions concurrently, and then concatenate them into MainNet. As illustrated in Tab. 4 Fig. and [| model lacking the Hydra Block tends to produce artifacts at the junction of the top and bottom garments. Such models also frequently allow the features of one garment influence the other, leading to incorrect clothing styles. However, with the introduction of the Hydra Block, AnyFit consistently exhibits more stable results.
Prior Model Evolution. We qualitatively demonstrate the effects of the Prior Model Evolution in Fig. 13 and (a). The SDXL-evolved model reduces artifacts and enhances robustness significantly, while outputs without Prior Model Evolution typically feature oversaturated colors as well as lighting and shadows that do not harmonize with the background. The gradual enhancement of model capabilities is visualized in Fig. C(c). We also empirically and quantitatively validate the effectiveness

---

Table 4: Quantitative ablation study.
Dataset
Method
LPIPS
SSIM 1
FID
KID 4
w/o Hydra Blocks
22.48
8.02
PressCdde-mulippe
w/o Prior Model Evolution
21.35
7.58
Full AnyFit
20.43
7.10
wlo Adaptive Mask Boost
0.183
0.748
44.75
1.44
the proprietary datasel
w/o Prior Model Evolution
0.192
0.740
45,01
1.52
Full AnyFit
0.181
0.743
43,97
0.69
Visual comparisum ablotion experiments Priur Model
(h) Visual compariiso
experiments Hyydra Hydra Block
Garment
Person
wio model ovolution
Blocks
Figure 8: Visual ablation study. Without Prior Model Evolution, AnyFit suffers reduced fabric detail and less realistic textures. While Hydra Blocks improve intersections of upper and lower garments.
of the Prior Model Evolution strategy after training on the Virtual Try-On (VTON) task Fig. and Tab. 4 The Prior Model Evolution, by improving the model's initial capabilities, lessens the difficulty of learning and fosters a dramatic improvement outfitting capacity and logo fidelity.
Adaptive Mask Boost. We illustratively showcase the issues of information leakage and mask reliance found in previous methods in Fig. (b) and Fig. TO Additionally, we empirically and quantitatively validate the effectiveness of the Adaptive Mask Boost strategy in Table and Fig. TO This strategy significantly heightens the model's robustmess towards different categories of clothing. enabling the autonomous determination of appropriate garment length rather than relying on masks. Furthermore, we manually adjust the aspect ratios Fig. impact of adaptive elongation during inference. More ablation studies are detailed in the Appendix A
6 (d), which demonstrates the positive
5 Conclusion
We introduce AnyFit, a novel and robust VTON pipeline suitable for any combination of attire across any imaginable scenario, offering a revolutionary leap in realistic try-on effects. To support multi- garment try-ons, AnyFit constructs HydraNet with lightweight and scalable parallelized attention that facilitatee the feature injection of multiple garments. Observing artifacts in evolve its potential by synthesizing the residuals of multiple models, as well as implementing a mask region boost strategy. Comprehensive experiments high-resolution benchmarks and real-world data have demonstrated that AnyFit significantly surpasses all baselines by a large gap.
scenarios,
Broader impacts. With the ability synthesize images, arises the risk that AnyFit might be used for inappropriate purposes such as producing media that breaches intellectual property rights or privacy norms. Because of these risks, strongly advocate for the conscientious use of this technology.
Limitation and future work. Our approach exhibits excellent performance in single-garmet and multi-gan garment virtual try-on applications. However, it still faces some limitations. Firstly, it shares the shortcomings of large text-image models, sometimes showing instability generating hands with complex structures. Secondly, our model offers initial but not yet fully mature text control capabilities (for details, please refer to the Appendix (A). providing opportunities for future enhancements.
10

---

Acknowledgements
This work was supported by National Science Foundation of China (U20B2072, 61976137). This
work was also partly supported by SJTU Medical Engineering Cross Research Grant YG2021ZD18.
References
[1] Stable diffusion xl
shows StableDiffusion/come commantttsfooloooooooooaaaaaeaaa
worse
inpainting.
https://ww.reeditt.c
worse 15/
[2] Discussions about model http:///sg/pp///.oo..o......../a
[3] Dreamshaper xl. http::///attaaaacco/o//a///////r
[4] Sdxl-inpainting-O.1. 0-inpainting-O.1
[5] Deepspeed. httte://gghabbbccc///ccooottettp
[6] Bai, S., Zhou, H., Li, Z., Zhou, C., Yang, H.: Single stage virtual try-on via deformable attention flows. In: European Conference on Computer Vision (2022)
[7] Bookstein, FIL.: Principal warps: Thin-plate splines and the decomposition of deformations. IEEE Transactions on pattern analysis and machine intelligence (1989)
[8] Cao, Z., Simon, T., Wei, S.E., Sheikh, Y.: Realtime multi-person 2d pose estimation using part affinity fields. In: Proceedings of the IEEE conference on computer vision and pattern recognition (2017)
[9] Chang, D., Shi, Y., Gao, Q., Fu, J., Xu, H., Song, G., Yan, Q. Yang. X., Soleymani, M.: Magicdance: Realistic human dance video generation with motions & facial expressions transfer. arXiv preprint 21Xiv:2311.12000
(2023)
[10] Chen, M., Chen, X., Zhai, Z., Ju, C.,, Hong, X., Lan, J.. Xiao, S.: Wear-any any-way Manipulable virtual try-on via sparse correspondence alignment. arXiv preprint (2024)
[11] Chen, X., Huang, L., Liu, Y., Shen, Y., Zhao, D., Zhao, H.: Anydoor: Zero-shot object-level image customization. arXiv preprint arXiv:2307.0944 00941 (2023)
[12] Choi, S., Park. S., Lee, M., Choo, J.: Viton-hd: High-resolution virtual try-on via misalignment- aware normalization. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (2021)
[13] Choi, Y., Kwak, S., Lee, K., Choi, H., Shin, J.: Improving diffusion models for virtual try-on. arXiv preprint arXiv:2403.051 1119 (2024)
[14] Chopra, A., Jain, R., Hemani, M., Krishnamurthy, B.: Zflow: Gated appearance flow-hased virtual try-on with 3d priors. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (2021)
[15] Cui, A., McKee, D., Lazebnik, S.: Dressing in order: Recurrent person image generation for pose transfer, virtual try-on and outfit editing. In: Proceedings of the IEFE/CVF international conference on computer vision (2021)
[16] Davide, M., Matteo, F., Marcella, C., Federico, I.., Fabio, C., Rita, C.: Dress code: High- resolution multi-category virtual try-on. In: Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (2022)
[17] Dong, H., Liang, X., Zhang, Y., Zhang, X., Shen, X., Xie, Z., Wu, B., Yin, J.: Fashion editing with adversarial parsing learning. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 8120-8 128 (2020)

---

[18] Ge, Y., Song, Y., Zhang, R.. Ge. C.. Liu, W., Luo, P:: Parser-free free virtual try-on via distilling appearance flows. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (2021)
[19] Goodfellow, I., Pouget-Abadie, J. Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial nets. Advances neural information processing systems (2014)
[20] Gou, J., Sun, S., Zhang, J., Si, J., Qian, C., Zhang, L.: Taming the power of diffusion models for high-quality virtual try-on with appearance flow. In: Proceedings of the 31st ACM International Conference Multimedia (2023)
[21] Güler, R.A., Neverova, N., Kokkinos, I.: Densepose: Dense human pose estimation in the wild. In: Proceedings of the IEEE conference on computer vision and pattern recognition (2018)
[22] Han, X., Hu, X., Huang, W., Scott, M.R.: Clothflow: flow-based model clothed person generation. In: Proceedings of the IEEE/CVF international conference on computer vision (2019)
[23] Han, X., Wu, Z., Wu, Z., Yu, R., Davis, L.S.: Viton: An image-based virtual try-on network. In: Proceedings of the IEEE conference computer vision and pattern recognition (2018)
[24] He, S., Song, Y.Z., Xiang, T.: Style-based global appearance flow for virtual try-on. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2022)
[25] Ho, J., Jain, A., Abbeel, P.: Denoising diffusion probabilistic models. Advances in neural information processing systems (2020)
[26] Hu, L., Gao, X., Zhang, P., Sun, K., Zhang, B., Bo, L.: Animate anyone: Consistent and controllable image-to-video synthesis for character animation. arXiv preprint arXiv:2311.17117 (2023)
[27] Huang, L., Chen, D., Liu, Y., Shen, Y., Zhao, D., Zhou, J.: Composer: Creative and controllable image synthesis with composable conditions. arXiv preprint arXiv:2302.09778 (2023)
[28] Jin, 7.. Sssegmenation: An open source supervised semantic segmentation toolbox based on pytorch. arXiv preprint arXiv:2305. 17091 (2023)
[29] Jin, Z., Hu, X., Zhu, L.., Song, L., Yuan, L., Yu, L.: Idrnet: Intervention-driien relation network for semantic segmentation. Advances Neural Information Processing Systems 36 (2024)
[30] Jo, Y., Park, J.: Sc-fegan: Face editing generative adversarial network with user's sketch and color. In: Proceedings of the IEEE/CVF international conference on computer vision. pp. 1745-1753 (2019)
[31] Kim, J., Gu, G., Park, M., Park, S., Choo, J.: Stableviton: Learning semantic correspondence with latent diffusion model for virtual try-on. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2024)
[32] Kingma, D.P., Welling, M.: Auto-encoding variational bayes. arXiv preprint arXiv:1312.611 (2013)
[33] Kirillov, A., Mintun, E., Ravi, N., Mao. H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A.C., Lo, W.Y., et al.: Segment anything. arXiv preprint arXiv:2304.022 0..300000263333 (2023)
[34] Lee, C.H., Liu, Z., Wu, L.,, Luo, Maskgan: Towards diverse and interactive facial image manipulation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 5549-5558 (2020)
[35] Lee, S., Gu, G., Park, S., Choi, S., Choo, J.: High-resolution virtual try-on with misalignment and occlusion-handled conditions. In: European Conference on Computer Vision (2022)
[36] Lin, T.Y., Dollár, P., Girshick, R., He, K., Hariharan, B., Belongie, S.: Feature pyramid networks for object detection. In: Proceedings the IEEE conference on computer vision and pattern recognition (2017)
12

---

[37]
Loshchilov, I,, Hutter, F.: Fixing weight decay regularization in adam (2018)
[38] Morelli, D., Baldrati, A., Cartella, G., Cornia, M., Bertini, M., Cucchiara, R.: LaDI-VTON: Latent Diffusion Textual-Inversinn Enhanced Virtual Try-On. In: Proceedings of the ACM International Conference on Multimedia (2023)
[39] Nichol, A., Dhariwal, P., Ramesh, A., Shyam, P., Mishkin, P., McGrew, B., Sutskever, I., Chen, M.: Glide: Towards photorealistic image generation and editing with text-guided diffusion models, arXiv preprint arXiv:2112. 107.4 (2021)
[40] Parmar, G., Zhang, R., Zhu, J.: On aliased resizing and surprising subtleties in gan evaluation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2022)
[41] Podell, D., English, Z., Lacey, K., Blattmann, A., Dockhorn, T., Müller, J., Penna, J., Rombach, R.: Sdxl: Improving latent diffusson models for high-resolution image synthesis. arXiv preprint arXiv:2307.01952 (2023)
[42] Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., al.: Learning transferable visual models from natural language supervision. In: International conference on machine learning (2021)
[43] Rombach, R.. Blattmann, A., Lorenz, D., Esser, P., Ommer, B.: High-resolution image synthesis with latent diffusion models. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recogmition (2022)
[44] Song, J., Meng. C., Ermon, S.: Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502 (2020)
[45] Sutherland, J., Arbel, M., Gretton, A.: Demystifying mmd gans. In: International Conference for Learning Representations (2018)
[46] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L.., Gomez, A.N., Kaiser, E.., Polosukhin, I:: Attention is all you need. Advances in neural information processing systems (2017)
[47] Wang, B., Zheng, H., Liang, X., Chen, Y., Lin, I.., Yang, M.: Toward paaracteristic-preervvii image-based virtual try-on network. In: Proceedings of the European conference on computer vision (ECCV) (2018)
[48] Wang. Z., Bovik, A.C., Sheikh, H.R.. Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing (2004)
[49] Xie, Z., Huang, Z.., Dong, X., Zhao, F., Dong, H., Zhang, X., Zhu, F., Liang, X.: Gp-vton: Towards general purpose virtual try-on via collaborative local-flow global-parsing learning. In: Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (2023)
[50] Xu, Y., Gu, T., Chen, W., Chen, C.: Ootdiffusion: Outfitting fusion based latent diffusion for controllable virtual try-on. arXiv preprint arXiv:2403.01779 (2024)
[51] Yang, B., Gu, S., Zhang, B., Zhang, T., Chen, X., Sun, X., Chen, D.. Wen, F:: Paint by exam- ple: Exemplar-basee image editing with diffusion models. In: Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (2023)
[52] Yang, H., Zhang, R., Guo, X., Liu, W., Zuo, W., Luo, P.: Towards photo-realistic virtual try-on by adaptively generating- preeeerrreeeeggegee image content. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (2020)
[53] Yang, Z., Zeng, A., Yuan, C., Li, Y.: Effective whole-body pose estimation with two-stages distillation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 4210-4220 (2023)
[54] Ye, H., Zhang, J., Liu, S., Han, X., Yang, W.: Ip-adapter: Text compatible image prompt adapter for text-to-image diffusion models, arXiv preprint arXiv:2308. 066721 (2023)
13

---

[55] Zhang, L., Rao, A., Agrawala, M.; Adding conditional control to text-to- -iagg diffusion models. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 3836-3847 (2023)
[56] Zhang, R., Isola, P., Efros, A.A., Shechtman, E., Wang, O.: The unreasonable effectiveness of deep features as a perceptual metric. In: Proceedings of the IEEE conference on computer vision and pattern recognition (2018)
[57] Zhang, X., Lin, E., Li, X., Luo, Y., Kampffmeyer, M., Dong, X., Liang, X.: Mmtryon: Multi-modal multi-reference control for high-quality fashion generation. arXiv preprint trXiv:2405.00448 (2024)
[58] Zhu, L., Yang, D., Zhu, T., Reda, F,, Chan, W. Saharia, C., Norouzi, M., Kemelmacher- Shlizerman, I: Tryondiffusion: A tale of two unets. In: Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition (2023)
14

---

Appendix
In Section A we provide more ablation studies about the role of text prompts in AnyFit, Hydra Block, Adaptive Mask Boost strategy and Prior Model Evolution. In Section B we provide additional details about the proprietary dataset and the multi-gam garment VTON dataset. In Section [ we provide additional details about discrete greedy algorithm during Prior Model Evolution, and preliminary knowledge about stable diffusion. In Section D we provide additional details about experimental setup, including data augmentation and outfitting dropout. Lastly, in Section B of AnyFit generated images, including annle-garment visual try-ons and garfmet virtual try-ons, with additional results displayed in challenging scenarios.
we present a multitude
More ablation study A
A.1
The role of text prompt in AnyFit
In fact, we have discovered that text plays certain role in controlling the overall try-on style. As show in Fig. 9 by adjusting the prompt, AnyFit is able to achieve variations in Virtual Try-On (VTON) apparel styles. However, this form of control is unstable, leaving space for further exploration.
Garment
Original tn
Open-garment
Sleeve up
With necklace
Figure 9: By adjusting the prompt, AnyFit is able to achieve variations in VTON apparel styles.
A.2 More ablation study about Adaptive Mask Boost strategy
The Adaptive Mask Boost strategy primarily comprises mask augmentation during training and adaptive elongation during inference. The role of adaptive elongation has been thoroughly illustrated in Fig. 6 in the main body, where it addresses the challenge of try-ons for long garments. Here, we predominantly discuss the function of parsing- free mask augmentation during training.
Specific the implementation, during training, the agnostic mask extracted solely using OpenPose body joint detections to occlude the original clothing, without leveraging human parsing, and random elongation of the mask performed. Consequently, the shape of the mask is correlated only with the human pose and not with the clothing. The significance of this procedure lies in eliminating the possibility of the mask leaking clothing information, ensuring that the model cannot cheat from the shape of the mask. From Fig. TO we observe that previous models often suffer from leakage of clothing information, where the models tend to generate garments that fill the entire mask areas. In roal-world inference scenarios, it is challenging provide entirely accurate mask region.
15

---

Garment
Person
AnyFit(ours)
IDM
StableVITON
LADI-VTON
Figure 10: Visual validation of the role of parsing-free mask augmentation during training within Adaptive Mask Boost strategy. AnyFit is only trained on VITON-HD for a fair comparison. Previous methods often suffer from leakage of clothing information, where the model tends to generate garments that the entire mask area, while AnyFit autonomously determines the correct length of the garments, producing attractive try-on results.
In contrast, our model is free from this predicament. Through mask augmentation during training, AnyFit autonomously determines the correct length of the garments, producing attractive try-on results.
A.3 More ablation study about Hydra Block
In Fig. I1 we present additional visual results to validate the effectiveness of the Hydra Block. "w/ Hydra Block" represents our fully proposed AnyFit, whereas the version "w/o Hydra Block" omits the Hydra Encoding Block as well the positional embeddings within the Hydra Fusion. It directly
16

---

w/o Hydra Block
w/ Hydra Block
w/o Hydra Block
w/ Hydra Block
Garment Person
Garment Person
AR
Figure 11: Visual comparisons the PressCode-multiphe Model lacking the Hydra Block is more prone to producing artifacts at the junction the top and bottom garments.
employs a singular conditioned HydraNet (which, in this scenario, degenerates to ReferenceNet) to encode both the top and bottom garment conditions coneurrently, and then injects them into MainNet.
As illustrated in Fig. [1 model lacking the Hydra Block is more prone to producing artifacts junction of the top and bottom garments. They also tend to allow the features of the bottom garment influence the top, resulting incorrect clothing styles, such as overly long tops or erroneous stripe patterns. After equipped with the Hydra Block, AnyFit noticeably exhibits more stable results.
17

---

A.4 More ablation study about Prior Model Evolution
In the main body of the paper. we have demonstrated the effects of Prior Model Evolution. However, we also notice that the capabilities of large-scale text-to-image models are closely related to the value of their classifier-fres free guidance (CFG). For simplicity, in the main body of the paper, all images are generated using CFG value of 3.0 in Fig. G when discussing the comparisons related to Prior Model Evolution. Here, we provide more granular comparison concerning the presence or absence of the Prior Model Evolution strategy. The results presented here are all based on models directly initialized with synthetic weights, without any training.
As depicted in Fig. 13 an increase in the CFG value leads to generalized oversaturation in the images. Within a reasonable range of CFG values, models incorporating Prior Model Evolution exhibit more realistic fabric textures and more plausible impainting results. In contrast, outputs without Prior Model Evolution typically feature oversaturated colors and the absence of detailed wrinkles, as well as lighting and shadows that do not harmonize with the background. This illustrates the SDXX--inpaintingO0.. (41) model diminishes the exemplary text-to-image capabilities of the original SDXL-BASE model, resulting in more mediocre outcomes. We attribute this degradation to the disruption of the previously well-aligned correspondences between text and images during the inpainting pre-training phase. However, our model with Prior Model Evolution, significantly ameliorates this issue, enhancing its overall robustness. actually find that after a minimal amount of fine-tuning, the model's synthetic outputs become substantially more powerful.
B Datasets
In this section, we provide a detailed description of the proprietary dataset and the multi-garment VTON dataset, i.e., DressCode (16) dataset.
which is constructed based the publicly available
B.1 The proprietary dataset
Our proprictary dataset comprises 50,602 training image pairs and 2,500 testing pairs. Each pair consists flat-laid garment image and frontal upper body model image, with most model images featuring complex backgrounds. The proprietary dataset is collected from e-commerce websites, with attention paid to achieving a balanced distribution of clothing categories. We gathered apparel from 12 different categories, including both menswear and womenswear, across various seasons such as spring, summer, autumn, and winter. As illustrated in Fig. T2 the person images display diverse backgrounds. We standardized the image resolution 1024x and conducted preprocessing this basis to test different models. This preprocessing involved using Densepose (21) and OpenPose (8) extract human features, employing the SAM (33) model to extract the main garment body, and constructing the gnostic mask images, etc.
BROKAGNS
Figure 12: Examples of the proprietary dataset.
18

---

Person
Mask
CFG = 1.0
CFG 3.0
CTG =.0
CPG =7.0
CTG 10.0
SDXI -evolvvd
LF
LT
LN
LD
SDXL-INP INP-0.1
LU
SDXI -evolved
SDXL SDXL-INP-4 INNP-0J
SDXI -evolved
SDXL SDXL-INP-OL INP-0.1
Figure 13: Visual validation of the role of Prior Model Evolution in various CFG weights without any training. Outputs without Prior Model Evolution typically feature oversaturated colors and the absence of detailed wrinkles, as well as lighting and shadows that do not harmonize with the background. Best viewed when zoomed in.
19

---

(a) Training data from upper-body data Flat lay upper Cropped lower Person
(b)Training data from lower-body data Cropped upper Flat lay lower Person
(c) Testing data
Figure 14: Examples of the PressCode-multipe dataset.
B.2 The DressCode-multiple dataset
To facilitate research on multi-garment virtual try-on, we require dataset composed of image triplets, each containing an upper garment image, a lower garment image, and a model image wearing the corresponding garments. However, obtaining such data with strict alignment is challenging. Leveraging the DressCode dataset (16), which includes upper and lower garment data as well as full-body model images, we set out to construct the DressCode-muilnn mole-multplle dataset, consisting of triplets. as illustrated in Fig. 14 Assuming we start with the upper garment data, where we already have flat lay upper garment image and model image wearing the corresponding upper garment, we use human parsing techniques (28) 29) to roughly segment the lower garment portion and extract it from the model image serve as the corresponding lower garment image. this point, the triplet consists of (flat lay upper, cropped lower, model image). Similarly, triplets derived from the lower garment data result in a composition of (cropped upper, flat lay lower, model image). Using this approach, we construct 24,314 publicly available upper-lower triplets by cropping garments from the 15,363 upper-body pairs and 8,951 lower-body pairs for training. For testing, we take the flat-laid garments from the upper and lower garment test sets, shuffle them, and combine them randomly, pairing them
20

---

with the test models from the upper garment data to create (flat lay upper, flat lay lower, model image) configurations for unpaired, real-world multi-ge galtmet virtual try-on tests. The paired testing is not conducted due to the lack of ground truth triplets.
It worth mentioning that model has encountered triplets in the form of (flat lay upper garment, flat lay lower garment, model image) during training. However, it has already learned the correct way wear upper and lower garments through training on cropped images and demonstrates good robustness. We believe that if real triplets consisting of (flat lay upper garment, flat lay lower garment, model image) were available for training. the model would exhibit even better performance.
C Method supplement
C.1 Discrete greedy algorithm
Our objective is to determine the optimal balancing coefficients and & to ensure that the initial weight W new achieve the best evaluation performance, i.c.
arg min f(a, 3) (Waase base + a (W imp W buse) + B (Wds W (aase)) [),,)2
Note that the function & is non-differentiable, which precludes the use of gradient- optimization algorithms find the minimum point. Empirically, we find that the evaluation functioon f exhibits monotonic or convex properties with respect to the balancing coefficients (a, 33 in most regions. Therefore, we discretize the continuous domain [0, 2] into a grid with ó as the step size and design the following algorithm, inspired by the greedy method, to search for the optimal (a, B).
Algorithm 2 Discrete greedy algorithm
Require: Evaluation function f. step size Ó. 1: Initialize (a, B) (0.5, 0.5)
2: while True do
3: feurren f(a, 3)
4: N - {(a+ 5. 3), (a -6. (0, (a,
5:
N + (a', E NNI 5 and 5 2}
Filter out-of-bound neighbors
6:
FN - ((',(31 (a',8' E N]
b Compute f values for valid neighbors
7:
if (nin(Fx) > Seurrent then
8:
break
9: else
10:
(a. + arg (NN MNEN f(d
end
12: end while
return (a. ())
In this algorithm, we initialize (a, 3) at (0.5, 0.5). During each iteration, we calculate the evaluation value f at the current point and four adjacent points in the directions up. down, left, and right. The algorithm then updates the current point to the one with the minimum evaluation value among these adjacent points. The termination condition the algorithm met when the evaluation values of all neighbors are greater than that of the current point. This method can be viewed as a discrete version of gradient descent method. Please note that under the assumption that f is convex or monotonic function, the algorithm is guaranteed to converge to the global optimal solution within the discrete parameter space.
C.2 Preliminary
Stable Diffusion. Our AnyFit is an extension of Stable Diffusion (43), which is one of the most commonly used latent diffusion models. Stable Diffusion employs variational autoencoder (32) (VAE) that consists of an encoder & and a decoder to enable image representations in the latent space. And UNet eg is trained denoise Gaussian noise with conditioning input encoded by CLIP text encoder (42) тy- Given an image x and a text prompt y, the training of the denoising UNet Eg is performed by minimizing the following loss function:
LLDM Ee(x) N.0,,,), [[le cø(ze, (
(4)
21

---

Table 5: Full quantitative comparisons the proprietary dataset. AnyFit (xxx) represents AnyFit trained on the corresponding xxx dataset.
Method
LPIPS $
SSIM
FID 4
KID 5
LAOI-VTON 388
0.252
0.734
52.24
6.51
DCI-VTON 20
0.264
0.734
57.96
12.35
StableVTON base
0.245
0.694
54.70
8.44
StablevTON-repaintinn (31)
0.242
0.720
53.80
8.13
IDM (13)
0.247
0.701
48.76
4.35
AnyFit (VITON-HD)
0.200
0.740
46.95
2.73
AnyFit (proprietary)
0.181
0.743
43.97
0.69
where 1 E (1,..., 7 denotes the time step of the forward diffusion process, and Ze is the encoded image E(x) with the added Gaussian noise (0, (i.e., the noise latent). Note that the conditioning input To (y) ) is correlated with the denoising UNet by the cross- coss--attetioon mechanism (46).
D Experimental details
Data augmentation. We have implemented data augmentation techniques that could potentially enhance the model's generalization ability as well its color accuracy performance. Specifically, the data augmentation operations include (a) horizontal flipping of images, (b) resizing garments and human figures through padding (up to 10% of the image size), (c) randomly adjusting the image's hue within a range of -5 to +5, and (d) randomly adjusting the image's s contrast within specified range (between 0.8 and 1.2 times the original contrast). Each of these operations occurs independently with a 50% probability. Moreover, these operations are simultaneously applied to both the garment and model images.
Outfitting dropout. Following the OOTDiffusion approach, we applied Outfitting Dropout, which essentially acts form of mmage-conditioned classifier-free guidance. It enhances the contrast and sharpness the generated images. Specifically, during the training process our MainNet, we randomly drop the input garment latent as E(g) Ø. where Ø E R4xhx refers to all-zero latent. In this way, the denoising UNet is trained both conditionally and unconditionally. Then at inference guidance scale 8y
time, we simply predicted noise:
adjust the strength of conditional control over the
E0(2r, war (E(g)) = (2 Ø) + (ca(Ze, war (E(g)) (x(2,,
(5)
In practice, we empirically the outfitting dropout ratio to 10% in training, and the guidance scale sg to 1.2 as default. We exclusively employ Outfitting Dropout within the Hydra Fusion Block that bridges HydraNet and MainNet. We have observed that applying outfitting dropout elsewhere introduces pixel-level artifacts.
E
More experiment results
E.1
Full comparison on the proprietary dataset
Due to space constraints, we have not presented quantitative comparisons on the proprietary dataset in a paired setting within the main body of the paper; they are additionally reported in Tab. [ Furthermore, showcase additional results in Fig.[5. which include outcomes from state-of-the-art baselines and AnyFit trained on both the proprictary dataset and VITON-HD. Our method exhibits a considerable lead in performance.
E.2
More visual results
We provide more visual results on VITON-HD, DressCode, and proprietary dataset for inspection in Fig.
16
17
and
18
Fig.
provides more results about multi-garment try-
22

---

Input AnyFitiproprietrry) VITON-)
IDM
Stable VITON
DCI-VTON
LADI-VTON
CARRM
ReT BA
AST BAR
TOT BSYD
GRAY
PERSACE
Figure 15: More visual comparisons the proprietary dataset. AnyFit displays superior garment details and outfit styling. Best viewed when zoomed in.
23

---

FEMINICM
EXPL DR
08088
Figure 16: More visual results the VITON-HD test data by AnyFit trained on VITON-HD training data. Best viewed when zoomed in.

---

Garment
Person
Output
Garment
Person
Output
RMS
PIGG
MOËE
Figure 17: More visual results on the proprietary test data by AnyFit trained on proprietary training data. Best viewed when zoomed in.
25

---

Upper body
Lower body
Dresses
Figure 18: More visual results on the DressCode test data by AnyFit trained on DressCode training data. Best viewed when zoomed in.
26

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction clearly state the claims made. And the claims match theoretical and experimental results.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: Please refer the limitations part in the main paper.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
27

---

Justification: All theoretical results are accompanied by appropriate proofs and proper citations. Every claim is substantiated by empirical evidence or supported by referenced literature.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification; Our contribution is novel model architecture along with associated model en- hancement strategies. All parameters and operational steps are detailed in the Experimental Section and the Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken make their results reproducible verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open- open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
28

---

5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [No]
Justification: We utilized two open-source datasets in addition to one proprietary dataset. Should our paper be accepted, we plan to make efforts release a portion of the proprietary data to the public. While the code for the paper is not open-sourced, we have provided comprehensive instructions necessary for replication.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.. cc/
Possiccyy)y for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
haabll//guddde//odeebbbiiiii //nips.cc//// wwtppcpcccgguupaeeccccc wwwpc//ggi/eeeeeee Code//bmeeeeeeee SusSSuubissss bolbbmmsssins sssissossiccyyyyic for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We have provided all the training and test details. Please refer to the "Experi- mental Setup" part in the main paper and other related sections in the Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can provided either with the code, appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: We adhered to the common evaluation practices established by prior work in our field. Error bars are not reported hecause it would be too computationally expensive. We believe that the metrics reported in main paper prove the efficacy of the model we proposed.
Guidelines:
The answer NA means that the paper does not include experiments.
29

---

The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: We report the sufficient information on the computer resources for each experiment in "Experimental Setup" part.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification:
The research conducted in the paper fully conforms to the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Please refer the broader impacts part in the main paper.
30

---

Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
This paper poses no risks for misuse.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We follow the instructions by the creators of each asset. We also cite the original paper that produced the code package or dataset.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
31

---

For scraped data from a particular source (e.g., website). the copyright and terms of service of that souree should be provided.
If assets are released, the license, copyright information, and terms use in the datasets.paperssiii paperswi thcode. www.com//aaaa datasets package should be provided. For popular datasets,
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: Our paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmodel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
32

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
33


Reviews:
[{"review_id": "6al1US8EAi", "reviewer": "Reviewer_KSKd", "summary": "Current image-based virtual try-on methods struggle with achieving high-fidelity and robust fitting across diverse scenarios due to issues like ill-fitting garments and quality degradation. To address this, AnyFit is introduced, leveraging a lightweight, scalable Hydra Block that facilitates feature injection of multiple garments through parallel attention. Additionally, AnyFit enhances robustness and expressiveness by synthesizing residuals from multiple models and implementing a mask region boost strategy. AnyFit outperforms baselines in high-resolution benchmarks and real-world data, enabling high-fidelity virtual try-ons in any scenario, paving the way for future fashion research.", "questions": "1) How does AnyFit overcome issues with ill-fitted garment styles and quality degrading during the training process?\n\n2) Why are the baseline methods in Table 3 missing from Table 1?\n\n3) How does the proposed framework effectively prevent errors when the clothing-agnostic mask results in misestimations?\n\n4) What about the computational complexity of the proposed model?", "limitations": "As the authors mentioned,\n1) The proposed model shares the shortcomings of large text-image models, sometimes showing instability in generating hands with complex structures. \n\n2) The proposed model offers initial but not yet fully mature text control capabilities.", "rating": 5, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1) AnyFit introduces the innovative Hydra Block, a lightweight and scalable operator that revolutionizes attire combinations through a parallel attention mechanism. This ensures seamless feature integration of multiple garments, overcoming issues of ill-fitting styles and quality degradation.\n\n2) By synthesizing residuals from multiple models and implementing a mask region boost strategy, AnyFit significantly enhances its robustness and expressiveness in real-world scenarios. This ensures stable and high-fidelity virtual try-on experiences across diverse settings.", "weaknesses": "1) While AnyFit offers advanced features and high-fidelity results, its complex parallel attention mechanism and multi-model residual synthesis may lead to increased computational requirements.\n\n2) The performance of AnyFit, like many image-based virtual try-on systems, relies heavily on accurate garment segmentation from the input image. In cases where the segmentation is imprecise or incomplete, the resulting virtual try-on may suffer from artifacts or incorrect fitting, affecting the overall realism and user experience.\n\n3) Training on two limited general datasets may not achieve the desired level of high generalization.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer KSKd,\n\nThank you for your detailed review and constructive feedback on our paper. We appreciate the effort you have put into assessing our work. Below, we have provided detailed responses to each of your comments and concerns.\n\n\n## Weaknesses \n\n### 1. Increased Computational Requirements\n\n**Response:** We believe that users need not worry about computational burden. We acknowledge that the parallel attention mechanism can lead to a slight increase in computational requirements. However, this increase in computational overhead is minimal. This is because we carefully selected attention matrices for parallelization, while sharing as many parameters as possible across different conditioning branches. As a result, adding an additional conditioning branch only leads to an 8% increase in parameters (Line #92) and a 9% increase in inference time overhead (Tab. 2 in main paper). We contend that Hydra Blocks may appear complex on the surface, but they are, in fact, sufficiently lightweight.\n\n\n### 2. Dependence on Accurate Garment Segmentation\n\n**Response:** If I understand correctly, you believe that we need a sufficiently accurate model to segment the flat clothing from the input images. However, we do not actually employ a segmentation model in our approach. In our private dataset, many clothing items have backgrounds that are not pure white, but rather light gray, light green, and even contain some brand texts and logos. We found that AnyFit, when exposed to such training data, can easily distinguish between the clothing subject and the background, showing remarkable resilience to minimal background noise. Please refer to **Fig.5 in global Rebuttal PDF** for a comparison, where we inputted garments with irrelevant text in the background alongside garments with background text cropped through object detection. Both produced similar results, which suggests that AnyFit does not require precise clothing segmentation; it possesses the ability to recognize the main subject of the clothing. We believe that if AnyFit is trained on a sufficient quantity of complex background clothing images, or even images of models wearing the same style of clothing from different angles, the model will automatically learn to implicitly extract the clothing, thereby achieving model-to-model virtual try-on. This has already been demonstrated by TryonDiffusion and Wear-Any-Way.\n\n### 3. Limited Training Data\n\n**Response:** Our training results on two limited general datasets, VITON-HD and DressCode, significantly outperform the baselines (Fig. 5(a) and Tab. 3 in main paper), especially in complex scenarios. This should partially demonstrate our model's generalization. Additionally, training on a larger private dataset has further enhanced the model's generalization and stability to a very high level, as shown in Fig. 5 and Fig. 16 in main paper.\n\n\n## Questions\n\n### 1. Overcoming Issues with Ill-Fitted Garment Styles and Quality Degradation During Training\n\n**Response:** Regarding the quality degradation issue of the SDXL-INPAINTING baseline model, we have enhanced its performance by merging excellent t2i model parameter variations within a model family. Additionally, to address the ill-fitted phenomenon that arises during cross-category fitting, we implemented an Adaptive Mask Boost strategy. Both methods are detailed in Section 3.3 of the main paper. If you still have any questions or concerns, please feel free to reach out to us; we would be happy to assist you.\n\n### 2. Missing Baseline Methods in Table 1\n\n\n**Response:** In Tab. 1, two baselines, HR-VTON and GP-VTON, are missing from Tab. 3. The GP-VTON model relies on a specific preprocessing model that segments clothing into two sleeves and a torso. Unfortunately, the authors of GP-VTON have not made their segmentation preprocessing model publicly available, which prevents us from testing GP-VTON on the private dataset. HR-VTON was omitted because it is an earlier model that uses GAN and has not been pretrained on a large-scale dataset. Its robustness is significantly lower, as it tends to overfit on the VITON-HD dataset and fails to generate meaningful results on the private dataset. To maintain a compact format, we have excluded the results of HR-VTON in the main paper. However, we are happy to provide visual results from HR-VTON in **Fig.2 in global Rebuttal PDF**, which demonstrates that it did not produce meaningful outcomes, nor was there a necessity for evaluation metrics.\n\n\n### 3. Handling Errors from Clothing-Agnostic Masks\n\n**Response:** If I understand correctly, you are concerned that the proposed model may produce incorrect clothing-agnostic masks. In fact, the Adaptive Mask Boost strategy we adopted specifically addresses this issue. As shown in **Fig.3 in global Rebuttal PDF** , the masks generated by our model are noticeably more expansive than those of previous methods. This approach helps us to minimize the remnants of the original clothing, thereby not affecting the synthesis of new garments. Additionally, it forces the model to learn the correct clothing boundaries during training and to reconstruct large non-clothing areas that have been masked out. During inference, we provide the correct mask area length according to the aspect ratio of the clothing to be tried on (detailed in lines #182 to #204 in main paper). Through this more flexible masking strategy, we aim to avoid small leakage of the original clothing mask areas (**shown as red box regions in Fig.3 in global Rebuttal PDF**); at the same time, the larger mask areas do not lead to artifacts, which enhances the stability of the VTON results.\n\n### 4. Computational Complexity\n\n**Response:** Our proposed model takes approximately 5 seconds to perform 30-step sampling on a single NVIDIA RTX 3090 GPU, using around 15GB of memory.\n\nWe hope that our responses address your concerns adequately. Thank you again for your valuable feedback.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "VMGqEWLwxQ", "reviewer": "Reviewer_vsbp", "summary": "This paper proposes a novel approach of controllable virtual try-on for any combination of attire across any scenario. It employs a Hydra Block, a lightweight and scalable operator designed for attire combinations, facilitated by a parallel attention mechanism that enables injecting multiple garment features into the main network. The model's robustness and expressiveness are enhanced by synthesizing multiple models' residuals and a mask region boost strategy, which addresses instability issues. The proposed method outperforms benchmarks on high-resolution images and real-world data, showing promising generated results.", "questions": "Although the paper claims to combine any clothing items, the visualized results only show outfits with tops and bottoms. Can this method currently support combinations of more clothing items, such as shirts, jackets, and pants?", "limitations": "As discussed in the paper by the authors, AnyFit sometimes shows instability in generating hands with complex structures and the text control capabilities of the method are not perfect yet.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Anyfit successfully mitigates the limitations of current virtual try-on methods, i.e., handling diverse attire combinations and scenarios. \n2. The novel Hydra Block and its parallel attention mechanism are well-explained, and the mask region boost strategy is theoretically sound. \n3. The experiments are comprehensive, with detailed reporting of the training and testing procedures, improving the reproducibility of the results. \n4. Overall the paper is well written, with clear explanations and logical structuring.", "weaknesses": "The manuscript commendably provides a thorough ablation study with convincing results. However, the comparison based on SD1.5 is missing. Adding that leads to a more fair comparison with baselines that use SD1.5 backbones, such as LADI-VTON, StableVTON, and OOTDiffusion.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer vsbp,\n\nThank you for your detailed review and constructive feedback on our paper. We appreciate the effort you have put into assessing our work. Below, we have provided detailed responses to each of your comments and concerns.\n\n\n## Weaknesses \n\n### Comparison with Baselines using SD1.5 Backbones\n\n**Response:**  Your observations are insightful, and your suggestions are very valuable. We conduct a more fair comparison with the baselines using SD1.5 backbone as follows.\n\n\n| VITON-HD           |   LPIPS  |    SSIM  |    FID   |   KID    |\n|--------------------|----------|----------|----------|----------|\n| StableVTON-base    | 0.081    | 0.867    | 9.71     | 1.68     |\n| StableVTON-repaint | 0.076    | 0.891    | 9.35     | 1.51     |\n| OOTDiffusion       | 0.093    | 0.856    | 9.16     | 0.68     |\n| LADI-VTON          | 0.091    | 0.875    | 9.42     | 1.63     |\n| AnyFit (SD1.5)     | 0.080    | 0.879    | 8.90     | 0.63     |\n\nTable 1: Quantitative comparisons on the VITON-HD with SD1.5 as the backbone.\n\nIn fact, during the training process, we found that the model's performance with SD1.5 backbone did not further improve when increasing the input resolution (from 512$\\times $384 to 1024$\\times $768). We believe this is because the base model of SD1.5 was pretrained at a resolution of 512. To pursue a higher-quality fitting effect, we shifted our focus to the SDXL model in subsequent research. The SD1.5 metrics reported in Tab.1 above are from an early version of the training that we can find now and this early version has not actually undergone sufficient training.\n\n## Questions\n\n### Support More Various Combination\n\n**Response:** We believe that if we have corresponding training data, we would be able to support a wider variety of clothing combinations, such as a shirt + jacket + pants + shoes. Currently, the trained model only supports tops + bottoms because we use samples from DressCode for training, which does not provide additional combinations for training pairs. Notably, we observed that MMTRYON[1] produces multi-item combinations with a simple pipeline. We are confident that our AnyFit model, which is specialized for multi-item dressing, would perform even better when exposed to similar data.\n\n\nWe hope that our responses address your concerns adequately. Thank you again for your valuable feedback.\n\n\n*[1] Zhang, X., Lin, E., Li, X., Luo, Y., Kampffmeyer, M., Dong, X., Liang, X.: Mmtryon: Multi-modal multi-reference control for high-quality fashion generation. arXiv preprint arXiv:2405.00448 (2024)*", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "HeCD2M3sd7", "reviewer": "Reviewer_v6TS", "summary": "The paper introduces AnyFit, a solution for multi-garment try-on. Specifically, they propose HydraNet to simultaneously extract features of both upper and lower garments. Then, they present Hydra Fusion to integrate these garment features into a denoising Unet. Additionally, they propose an Adaptive Mask, combined with text, to explore initial controllable capabilities.", "questions": "1. Why is there a qualitative comparison between AnyFit and OOTDiffusion in the paper, but no quantitative comparison? Additionally, the original paper of OOTDiffusion reported their quantitative results on VITON-HD and Dress Code at the resolution of 512×384.\n\n2. OutfitAnyone (https://github.com/HumanAIGC/OutfitAnyone, https://huggingface.co/spaces/HumanAIGC/OutfitAnyone) may serve as a good qualitative baseline for the multi-garment try-on task.\n\n3. Although it doesn't affect the conclusions, I'm curious why some baseline methods reported significantly different quantitative metrics between AnyFit and OOTDiffusion? For example, on Dresscode, AnyFit reported an FID of 6.94 for LADI-VTON, whereas OOTDiffusion reported an FID of 5.66.", "limitations": "Yes.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. They explore multi-garment virtual try-on, an area that has not been extensively covered in previous literature based on diffusion models.\n\n2. The paper is clearly written, and the related experiments are sufficient.\n\n3. They explored initially controllable try-on, which has been less investigated in prior literature.", "weaknesses": "1. There are no major theoretical and technical innovations. For example, HydraNet is a simple variant of the Unet copy which has been extensively adopted as the feature extractor in previous work such as AnimateAnyone and OOTDDiffusion.\n\n2. Similar to other try-on methods based on diffusion models, AnyFit relies on a large amount of high-quality images. However, their proprietary e-commerce dataset is not open, which does not further prompt the development of the try-on task.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer v6TS,\n\nThank you for your detailed review and constructive feedback on our paper. We appreciate the effort you have put into assessing our work. \n\n## Weaknesses \n\n### 1. Major Innovations\n**Response:** Thank you for your valuable feedback. We believe that identifying a problem is more important than proposing a solution in research. Introducing a problem can stimulate a series of innovative efforts within the community. In this paper, we first identified the issue of artifacts at the junction of upper and lower garments in the multi-garment try-on approach of simple VTON-concat (used by wear-any-way[1]) and analyzed it. We find that it is due to mishandling the relative clothing sizes after concatenation, which leads to garment distortion and blurring. Secondly, we pointed out that the previous agnostic mask leaked the clothing contours, resulting in poor cross-category try-on performance. We proposed a mask boost strategy as a solution. While our approach may seem intuitive, it is effective, and the identification of these research problems paves the way for future investigations.\n\nFurthermore, we think that HydraNet is not simply a variant of U-Net. During the parallelization of the modules, we meticulously studied their functions (as shown in **Explanation 1 in global Rebuttal**) and achieved the most lightweight expansion scheme. Implementing multi-garment try-on is not particularly difficult, but designing an efficient and lightweight solution is a challenge and our area of innovation. These insights are absent in concurrent works, such as Mmtryon [2].\n\n### 2. Proprietary E-commerce Dataset \n**Response:** I apologize for the situation. Due to copyright and privacy issues associated with the proprietary dataset, we are unable to make it publicly available until these legal concerns are resolved. However, we believe that the methods proposed in our paper, including the multi-condition injection in HydraNet and the model evolution strategy that enhances baseline capabilities, can significantly advance the development of the try-on task and potentially benefit a broader range of conditional generation tasks.\n\n\n## Questions\n### 1. The Reason of Absent of OOTDiffusion Qualitative Comparison.\n**Response:** We originally planned to perform a quantitative comparison with OOTDiffusion; however, it was quite perplexing that the official code for OOTDiffusion encountered compatibility issues on our machine, resulting in out-of-memory errors. Therefore, we were unable to complete the quantitative comparison with OOTDiffusion by the time of our paper submission. As for the qualitative comparison, we used the online generation gradio provided by OOTDiffusion. To ensure the rigor and fairness of our comparisons, the results reported in our experiments were primarily generated and evaluated by ourselves using the official models. We found some discrepancies between our results and those from OOTDiffusion paper, which led us to not utilize the data directly from the original OOTDiffusion paper in our work. For the sake of rigor, we have re-run the official OOTDiffusion model and conducted quantitative evaluations in the past few days, and we report those metrics below, hoping they meet your requirements.\n| VITON-HD       |   LPIPS  |    SSIM  |    FID   |   KID    |\n|----------------|----------|----------|----------|----------|\n| HR-VTON        | 0.097    | 0.878    | 12.31    | 3.86     |\n| DCI-VTON       | 0.072    | 0.892    | 8.76     | 0.92     |\n| StableVTON     | 0.076    | 0.891    | 9.35     | 1.51     |\n| GP-VTON*       | 0.083    | 0.892    | 9.17     | 0.93     |\n| LADI-VTON      | 0.091    | 0.875    | 9.42     | 1.63     |\n| IDM            | 0.078    | 0.881    | 9.12     | 1.03     |\n| OOTDiffusion   | 0.093    | 0.856    | 9.16     | 0.68     |\n| AnyFit         | 0.075    | 0.893    | 8.60     | 0.55     |\n\nMoreover, we observed in the qualitative comparison that the performance of IDM-VTON significantly outperformed OOTDiffusion, while our model AnyFit surpassed IDM-VTON in the quantitative comparison. Therefore, we believe that the conclusion that AnyFit is superior to OOTDiffusion is evident.\n\n### 2. Comparison About OutfitAnyone\n**Response:** In **Fig.4 in global Rebuttal PDF**, we provide the generation results by AnyFit using OutfitAnyone person. Unfortunately, the open-source demo provided by OutfitAnyone is no longer available (reported Error), so we are unable to supply the corresponding generation comparisons. However, our generated results demonstrate comparable performance to the images presented in the OutfitAnyone paper. Considering that the OutfitAnyone team collected a significant amount of private multi-garment data triplets for training, while AnyFit only utilized the cropped unclean data from the open-source DressCode for training, the comparable results we achieved are sufficient to demonstrate the superiority of our method.\n\n\n### 3. The Reason of Different Quantitative Metrics Between AnyFit and OOTDiffusion.\n**Response:** Your observation is very meticulous. Since OOTDiffusion did not provide the code for evaluation, we used the metric evaluation scripts from LADI-VTON. It can be observed that the metrics we reported are closer to the results in LADI-VTON. Given the differences between our metrics and those reported in the OOTDiffusion paper, we did not directly accept the metrics reported in OOTDiffusion. Additionally, the official DressCode dataset does not provide clothing-agnostic images (masked images), and the significant difference between OOTDiffusion and AnyFit on the DressCode dataset may stem from the differences in masking strategies.\n\nWe hope that our responses address your concerns adequately. Thank you again for your valuable feedback.\n\n*[1] Wear-any-way: Manipulable virtual try-on via sparse correspondence alignment.*\n\n*[2] Mmtryon: Multi-modal multi-reference control for high-quality fashion generation.*", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "KBEdf93k9s", "reviewer": "Reviewer_Xer8", "summary": "To enhance the scalability (mix-and-match) and robustness (details clothing texture preservation) in virtual clothing try-on tasks, the paper proposed three contributions: i) design the Hydra Encoding Block to do parallel self-attention features transmition, ii) present the Prior Model Evolution to leverage groups of models' capabilities, and iii) introduce the Adaptive Mask Boost for generate garments structures without being affected by the original clothes on human.", "questions": "1. Regarding line #54, there is inconsistency between the description in the paper and the original shared from TryOnDiffusion [58]. [58] uses cross-attention instead of self-attention, which is claimed to cause pixel-wise structural bias, to do implicit warping. Please clarify. On the other hand, OOTDiffusion [50] utilizes self-attention for preserving features. Can you share the pros and cons of leveraging cross-attention vs. self-attention for preserving features? And please state clearly the reason why this paper selected self-attention.\n2. Regarding line #68, it is unclear what “parsing-free mask region” is? Considering line #197, does it mean to exclude the third image counted from the left in left-bottom of Fig. 2 when training? What is the difference between the proposed Adaptive Mask Boost without using parsing to eliminate the contour of original clothes and the method used in HR-VITON, mentioned at the beginning of page 5 (Sec. 3: pre-processing)?\n3. I suggest including the notation mentioned in Sec. 3 in Fig. 2 to provide a clearer understanding.\n4. Does the Pose Guided mentioned in line #116 mean the Pose Encoder in Fig. 2?\n5. Why does the method train at the resolution of 1024x768 but evaluate at 512x384?\n6. Regarding the ablation study of Adaptive Mask Boost (AMB) in Table 4, the only matrix that evaluates the structure is SSIM but the result is worse when applying the Adaptive Mask Boost module. Can the paper describe more to prove that the table is good evidence to prove the effectiveness of AMB?\n7. Letting the model automatically find the suitable clothing length would misinterpret the accurate garment pattern, which is not expected in commercial usage. For example, we can only know that the model does not follow the original clothing length on input human to generate try-on results but we can’t know whether it generates accurate length, e.g., Fig. 9 and the sleeve length of the fourth case in the first row in Fig. 1. How does the paper think about designs that can ensure accuracy, e.g., COTTON [A]?\n8. It is unclear how the method adjusts the size that is mentioned in line #112 after reading Sec. 3.3. Does it mean automatically learned clothing size determination?\n9. Regarding line #133, how to prove that self-attention module instead of cross-attention module plays a vital role in the latent warping of the garments?\n10. Regarding the N parameter in Hydra Fusion, is it set to 2 when trying on top and bottom clothes? Can it extend to try on scarves, shoes, etc.?\n11. Fig. 12 lacks a direct visual comparison for proving the effectiveness of the proposed Prior Model Evolution. I suggest providing the figure with i) input human, ii) target clothes, and results from the four models, including SDXL-base-1.0, SDXL-inpainting-0.1, DreamshaperXL, and the merged one.\n12. How is the inference time for top and bottom try-on?\n\n[A] C. -Y. Chen, Y. -C. Chen, H. -H. Shuai and W. -H. Cheng, \"Size Does Matter: Size-aware Virtual Try-on via Clothing-oriented Transformation Try-on Network,\" *2023 IEEE/CVF International Conference on Computer Vision (ICCV)*, Paris, France, 2023, pp. 7479-7488, doi: 10.1109/ICCV51070.2023.00691.", "limitations": "The authors have shared the limitations and potential societal impact.", "rating": 7, "confidence": 5, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- The paper designs the HydraNet to achieve multi-garment try-on with diffusion-based models. This is the first to extend the parallel UNet architecture towards multi-garment try-on.\n- The paper points to a critical challenge but few papers addressed this question, generating accurate torso length and sleeve length of target garments on humans. Also, the method shows decent performance to generate unknown textures when the target clothes are shorter than the original clothes on a human image.\n- The paper works decently on a complex background.", "weaknesses": "- The clearness of the presentation could be improved. For example, the usage of TryOnDiffusion [58] is cross-attention but the paper claims that [58] use self-attention (line #54). Also, it is unclear how to adjust clothing sizes (line #112), etc.\n- Fig. 6 (b) is not a new observation. The issue has been solved by the pre-processing method proposed by VITON-HD [12].", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "I will respond directly to save space.\n\n## Weaknesses \n\n#### 1. Misuse of Citing of TryonDIffusion\n**Response:** I apologize for the lack of precision in our wording here. In fact, the principles behind the injection of self-attention and cross-attention are the same, and we will address this issue in detail in **Explanation 1 in global Rebuttal**.\n\n### 2. How to Adjust Clothing Sizes\n**Response:** Please refer to **Explanation 2 in global Rebuttal**. I hope my response addresses your concerns. If you still have any confusion, feel free to reach out to me again.\n\n### 3. Observation of Fig.6(b)\n**Response:** This issue is not resolved by VITON-HD. The results **in Fig. 6(b) in main paper** were obtained using the DCI model, which is significantly later than VITON-HD and utilizes the pre-processing method of VITON-HD. The reason for this phenomenon is that their cloth-agnostic mask leaks the shape of the original clothing; please refer to **Explanation 2 in global Rebuttal** for more details. In fact, this is a common issue, as shown **in Fig. 9 of main paper**, where all other models fail to generate the lower hems of garments correctly—they are either too long or blurry.\n\n\n## Questions\n\n### 1. Different About Self-attention and Cross-attention.\n**Response:** Please refer to **Explanation 1 in global Rebuttal**.\n\n### 2 & 8. Parsing-free Mask Region\n**Response:** It doesn't mean to exclude the third image counted from the left in left-bottom of Fig. 2, but human parsing/segmentation map when training. Please refer to **Explanation 2 in global Rebuttal**\n\n### 3 & 4. Advise About Typing\n**Response:** Thank you for your valuable suggestions. \"Pose Guided\" refers to the Pose Encoder. If the paper is accepted, we will incorporate your suggestions in the camera-ready version.\n\n### 5. Question About Different Resolutions\n**Response:** This is because most of the metrics from previous methods were measured at a size of 512$\\times$384. To ensure comparability with past methods, we generate try-on results at a size of 1024$\\times$512, then resize them to 512$\\times$384 for metric calculations. We also provide the metrics obtained from AnyFit directly generated at a smaller version (SD1.5, 512$\\times$384 generating) here.\n\n| VITON-HD       |   LPIPS  |    SSIM  |    FID   |   KID    |\n|----------------|----------|----------|----------|----------|\n| AnyFit (SD1.5) | 0.080    | 0.879    | 8.90     | 0.63     |\n| AnyFit (SDXL)  | 0.075    | 0.893    | 8.60     | 0.55     |\n\n### 6. The reason of Worse SSIM on AMB\n**Response:** Worse SSIM on AMB is a reasonable phenomenon. The metrics that truly reflect the effectiveness of AMB should be FID and KID in an unpaired evaluation. This is because AMB performs adaptive mask extension, primarily targeting different cloth types of try-ons (the original outfit and the outfit to be tried on belong to different categories). SSIM is calculated in a paired setting, where the outfit to be tried on is the same as the one the model is wearing. In such a reconstruction scenario (paired setting), a smaller mask area is preferable. If the masked area is zero, meaning no inpainting or try-on occurs, you will attain the best LPIPS and SSIM metrics (exactly same with original one). However, this does not reflect the model's true try-on capability. Our AMB strategy may expand the mask area in certain scenarios, which could lead to slightly worse SSIM, for instance, when the imagined background differs from the original background. Nevertheless, AMB performs exceptionally well on more realistic unpaired try-on tasks, resulting in lower FID and KID.\n\n### 7. Worry About Suitable Clothing Length \n**Response:** AnyFit does not strictly follow to the original clothing length for the input human, which means that the model's generation will at least not be misled by incorrect clothing lengths. This clearly results in fewer erroneous outcomes compared to the \"follow\" version. Moreover, we are not entirely leaving it up to the model to generate; we provide hints by extending the mask region according to the length of the clothing to be tried on during inference. Additionally, the model consistently generates reasonable sleeve lengths in practice even without AMB.\n\n### 9. How to Prove That Self-attention Plays a Vital Role\n**Response:** This is a very valuable question. Please refer to **Fig.1 note in global Rebuttal**.\n\n### 10. Expensions to More Hydra Conditioning Branches.\n**Response:** Absolutely yes to both questions! The primary intention behind designing Hydra Blocks is to enable multi-condition injections of arbitrary quantities. We believe that if appropriate training data were available, we could support a wider variety of clothing combinations, such as shirts + outer suits + pants + shoes.\n\n### 11. Fig.12 Presenatation.\n**Response:** In **Fig.6(a)&(c) in main paper**, we demonstrate the effectiveness of Model Evolution and the impact of varying fusion weights on the output, which strongly showcases the benefits of Model Evolution. Regarding Fig. 12, we cannot provide 4 versions of models as you mentioned. This is because the try-on task is a specific type of inpainting, where models like SDXL-base-1.0 and DreamshaperXL, which are text-to-image models, cannot be directly utilized in this context, as shown in the far-right column of Fig. 6 (c). In this scenario, training SDXL-base-1.0 and DreamshaperXL as base models would be highly impractical and extremely costly (8 A100 for ten days for each one). Their training would require the model to first master the fundamental skills of inpainting before moving on to the refined learning of garment details. As a result, we can only present SDXL-INP results.\n\n### 12. Inference Time Cost. \n**Response:** AnyFit takes approximately 5 seconds to generate 30 steps of sampling on a single NVIDIA RTX 3090 GPU, with a memory usage of around 15GB. When injecting both upper and lower garments simultaneously, the time overhead increases by 9%.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
## Notes of the figures in the attached PDF.

**Fig.1 note: Ablation about connection from different blocks between HydraNet and MainNet.** In fact, we have questioned the connection between HydraNet and MainNet, and we aimed to understand which specific layers are contributing to this pipeline. Please refer to **Fig.1 in global Rebuttal PDF**. We attempted to sever the connection between HydraNet and MainNet by removing the injection of self-attention layers. The results showed that without self-attention layers between the up blocks, the model could not produce the correct try-on results at all. When we removed the injection of cross-attention layers (by using a zero image feature vector instead), the model only exhibited color biases with certain brightly colored garments. This demonstrates that self-attention layers, especially those between the up blocks, are the most important.

**Fig.2 note:** HR-VTON (GAN-based method) performs poorly on private data.

**Fig.3 note:** Comparison of our mask strategy and the previous mask strategy. The latter leaks the clothing outline, causing model cheating.

**Fig.4 note:** Fitting results of characters derived from OutfitAnyone.

**Fig.5 note:** The showcase of robustness against clothing background noise.

## Global explanations

### Explanation 1. Different About Self-attention and Cross-attention injection.
The self-attention layer uses features from the Unet itself to obtain Query tensors and performs attention calculation with the Key and Value tensors also from Unet features. In contrast, the cross-attention layer computes attention with Key and Value tensors derived from textual features. Their module structures are completely same; the only difference lies in the sources of K and V features and the number of channels. When we use Stable Diffusion as the base model for training, the initialized cross-attention weights tend to emphasize the computation of global features from prompts, while the self-attention weights focus more on the details of visual features. This is influenced by the pre-trained weights, and their structures are isomorphic. Therefore, many recent try-on models based on SD primarily utilize self-attention for feature injection. TryonDiffusion is trained from scratch, so it can work with either self-attention or cross-attention. As an early work, it followed the precedent of using cross-attention for feature injection. Regarding your mention of pixel-wise structural bias, the original text should read, "However, channel-wise concatenation cannot handle complex transformations such as garment warping (see Sec. 4). This is because ..., these primitives have strong pixel-wise structural bias." This indicates that the bias arises from channel-wise concatenation injection and is not related to self-attention injection.

### Explanation 2: Explanations about Adaptive Mask Boost and how it differs from HR-VITON
Firstly, the parsing-free mask refers to the fact that we do not use human parsing results (referred to as segmentation maps at the beginning of page 5 of HR-VITON) when generating mask. Such models segment the human body into semantic parts such as limbs, head, upper garments, and lower garments. In the preprocessing of works like HR-VITON, they first generate masks using the torso and limbs from the OpenPose keypoints, for instance, covering the arms and the area between the shoulders and hips with rectangular areas, forming an OpenPose mask. Next, because some clothes are long or bulky, solely using the OpenPose mask may not fully cover the original clothing. They use the clothing map from human parsing as a mask, combining it with the OpenPose mask to form the final cloth-agnostic mask, ensuring that all clothing areas are covered.

However, this reliance on the parsing map for the mask can inadvertently leak the clothing outline, as shown in **Fig.3 in global Rebuttal PDF**. For instance, if you use a short-sleeve model for inferring with long-sleeve clothing, due to the parsing map, the resulting cloth-agnostic mask may have protrusions at the shoulders (derived from the short sleeves). The model would perceive this information, as similar masks during training suggested that this was short-sleeve clothing; thus, the generated result would likely exhibit incorrect protrusions at the shoulders of the long-sleeve garment, as shown in **Fig. 6(b) in main paper**. Essentially, the cloth-agnostic mask during training leaked the correct outline of the original clothing, while the mask used during inference leaked an incorrect outline, leading to errors. The same applies to clothing length; if overly reliant on the human parsing map, when a short garment model tries on a long piece of clothing, it may produce a cloth-agnostic mask that is not long enough, resulting in generated clothing being too short.

We observed this phenomenon in our usage, abandoning the human parsing models and instead opting to inflate the openpose masks in width and randomly extend the masks in length to avoid the issue of the original openpose masks not covering the clothing hem or other areas. Furthermore, due to the inflation and random growth during training, our masks completely avoid leaking the original shape of the clothing. Therefore, the model can only determine how the clothing should appear by observing the clothing itself. Additionally, during inference, we utilize a subject detection model to detect the bounding box of the clothing and suitably extend the length of the openpose mask based on the aspect ratio $\sigma$ of the bounding box. This adaptive mask provides cues for the clothing length during inference, thus assisting in delivering the correct try-on effect. Specific parameters for our mask strategy are detailed in Sec. 3.3 Adaptive Mask Boost of our paper.

Meta-review: This paper proposes a lightweight and scalable operator to enhance the feature injection of multiple garments and address the ill-fitting garment style and quality degradation issues in the traditional methods. The authors also propose a mask region boost strategy to overcome the instability. The reviewers agree that the proposed method is novel and effective, the experiments are sufficient with decent performance, and the presentation is good. All reviewers acknowledge that they have read the author responses and do not have further concerns. The AC reads through the paper and discussions and recommends Accept for this paper.
Decision: Accept (poster)