# MT-VTON: Multilevel Transformation-Based Virtual Try-On for Enhancing Realism of Clothing
## 1 Introduction  
Virtual try-on technology, which generates images of people wearing desired clothing items, has become a key application in online shopping. Broadly, the virtual try-on problem can be divided into two sub-problems: warping the target clothing item (the \"warping process\") and integrating it onto a human body (the \"try-on process\"). The warping process adjusts the clothing item based on pose information to align it with the human figure. The try-on process then merges this warped clothing item onto an image of the person to produce the final output. Typically, pixelwise warping techniques, such as Thin-Plate Spline (TPS) warping and appearance flow, are used in the warping process [1; 2; 3; 4; 5; 6; 7; 8]. These techniques focus on pixelwise mapping to the original clothing image, making them robust for preserving details like logos. However, they often fail to maintain contextual features such as shadows or fabric folds, mainly because the original clothing images usually lack the color range to represent these features. The try-on process integrates the warped cloth onto the person image. Simultaneously, this process also refines the warped cloth to retain the contextual features. However, since this process commonly adopts conditional generative models [9; 10], we found that this generative process tends to compromise the detail of logos. Additionally, this process typically necessitates a cloth-agnostic person image, meaning that the clothing part is removed. This requirement arises due to constraints in data collection. In practical scenarios, collecting paired images of the same person in different outfits, but with exactly identical poses is difficult. Therefore, the available data often consist of paired images of a cloth and a person wearing that cloth. This limitation forces the use of cloth-agnostic person images to ensure that the model is not provided withground truth person images already wearing the clothing during the training phase. While the use of this cloth-agnostic image is highly practical, it prevents layering an outer clothing item over an inner one. In this work, we aimed to address the preservation of both logos and contextual features by reversing the traditional try-on process: we used a generative model for warping and an appearance-flow-based method to refine logos. Specifically, for the warping process, we propose to utilize the appearance-flow-based feature transformation at multiple levels for accurately aligning clothes with the person's pose. Since this generative process often blurs the logos, we complemented it by substituting the blurred logos with their counterparts from the warped clothes at the pixel level. Another important issue we focused on is enabling the layering of clothes. To achieve this, we need to exclude the use of cloth-agnostic images in the generative process. However, this exclusion might lead to a decrease in the quality of contextual features, as the cloth-agnostic image can provide a rich context. To relieve the loss of the contextual information, we propose to utilize a cloth segmentation map instead of just a cloth mask. Our cloth segmentation map involves segmentation masks for each part of a cloth (e.g., left, right arms, and torso for the upper clothing case), which offer more information for generating the contextual features. We assessed the effectiveness of our approach both qualitatively and quantitatively, comparing it to state-of-the-art methods such as HR-VITON [8] and Flow-Style VTON [6]. Our experiments indicated that our approach can produce more-realistic outcomes and achieves superior performance to other baseline methods. Furthermore, we argue that the Zalando dataset [1], frequently used in virtual try-on research, is well-curated, lacking practical applicability in real-world scenarios. Hence, we utilized the more-realistic dataset, the Korean fashion dataset from AIHub [11], thereby strengthening the practical relevance of our work. Our contributions can be organized as follows:  
1.