Cloth-aware Augmentation for Cloth-generalized Person Re-identification

Authors: reid  person
training  testing
Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-03 14:43:56

Abstract: Person re-identification (ReID) is crucial in video surveillance, aiming to match individuals across different camera views while cloth-changing person re-identification (CC-ReID) focuses on pedestrians changing attire. Many existing CC-ReID methods overlook generalization, crucial for universality across cloth-consistent and cloth-changing scenarios. This paper pioneers exploring the cloth-generalized person re-identification (CG-ReID) task and introduces the Cloth-aware Augmentation (CaAug) strategy. Comprising domain augmentation and feature augmentation, CaAug aims to learn identity-relevant features adaptable to both scenarios. Domain augmentation involves creating diverse fictitious domains, simulating various clothing scenarios. Supervising features from different cloth domains enhances robustness and generalization against clothing changes. Additionally, for feature augmentation, element exchange introduces diversity concerning clothing changes. Regularizing the model with these augmented features strengthens resilience against clothing change uncertainty. Extensive experiments on cloth-changing datasets demonstrate the efficacy of our approach, consistently outperforming state-of-the-art methods. Our codes will be publicly released soon.

Main paper: 
 introduction: 
 Person Re-identification (ReID) [67] is a traditional computer vision task, aiming to associate pedestrians with the same identity across different scenes. In recent years, the surge in popularity of deep learning has propelled ReID methods, leading to the development of robust models that learn discriminative features. However, many recent ReID methods have primarily addressed conventional challenges, such as adapting to human pose changes [76] , handling camera view variations [39] , overcoming occlusions [34, 56] , and addressing Visible Infrared ReID scenarios [63, 68] . Surprisingly, these methods often overlook a crucial aspect: the possibility of pedestrians changing their clothing.Figure 1 : Illustrating our motivation: Cloth-changing ReID tasks undergo more drastic appearance changes compared to traditional (cloth-consistent) ReID tasks. In real-world applications, both cloth-consistent and cloth-changing scenarios coexist, prompting our exploration of the cloth-generalized task. Our cloth-aware augmentation strategy, in contrast to other models, adapts features to multiple fictitious domains, enabling the learning of more generalizable and robust features.In real-world applications, individuals frequently change clothes in various situations, including long-term video surveillance or instances involving suspects fleeing the scene. When confronted with scenarios involving clothing changes, recent methods, primarily appearance-based, prove less effective. This limitation significantly compromises their performance, as highlighted in Figure 1 . Clothing changes introduce substantial alterations in appearance, making it more challenging to re-identify individuals with the same identity. Moreover, many traditional biometric techniques become impractical in these situations, where facial features may be unclear, and gait analysis may not be feasible in surveillance videos. Hence, it is imperative to address this specific challenge [73] .In response to the cloth-changing problem, researchers have introduced datasets and explored solutions to mitigate variations caused by clothing changes. Recent cloth-changing ReID methods aim to obtain identity-discriminative and cloth-invariant feature representations, broadly categorized into three groups: (1) Designing learning strategies to mitigate the influence of clothing changes: Methods like [47] and [25] attempt to disentangle clothing-related information using various strategies. This includes adversarial-based techniques [18] and [59] , as well as disentangle-based approaches [36] and [59] . (2) Leveraging auxiliary clothing-invariant modality: Methods in this category, such as [60] , [47] , [24] , [7] , [18] , [8] , [38] , and [19] , incorporate additional information like pedestrian contours, key points, and human parsing estimation results. (3) Applying augmentation strategies to enhance model robustness: Methods involve synthesizing cloth-changing images, as seen in [70] and [27] , to reinforce the feature learning process.Despite advancements, existing methods often neglect the exploration of generalization ability [82] , a crucial aspect for achieving universality in real-world applications. As depicted in Figure 1 , traditional cloth-consistent ReID overlooks clothing changes that may occur when pedestrians alter their attires. Conversely, clothchanging methods may falter in cloth-consistent cases. Moreover, when tackling ReID tasks, two critical problems arise: (1) When only cloth-consistent data is available, how do we train a cloth-changing model? (2) How do we ensure the cloth-changing model performs well on cloth-consistent cases? Therefore, as illustrated in Figure 1 , we embark on an exploration of the cloth-generalized ReID task. This task involves training a model with the versatility to adeptly handle both cloth-consistent and cloth-changing scenarios. Recognizing the potential of augmentation strategies in improving generalization ability, we introduce Cloth-aware Augmentation (CaAug) strategies specifically tailored for both types of scenarios. Traditional augmentation methods such as cutout [15] and erasing [80] often fall short in addressing domain differences. In contrast, our approach deliberately mimics various domain gaps, allowing learned features to adapt seamlessly to diverse cloth-changing scenarios. To tackle the uncertainties associated with generative networks [61] , we enhance generalization both from the latent domain and the feature aspect overhead. CaAug comprises two essential components, each contributing significantly to the model's generalization ability: Domain Augmentation (DomainAug) and Feature Augmentation (FeatAug).Domain Augmentation (DomainAug): This component involves the generation of fictitious domains, mirroring clothing domains as depicted in Figure 1 . Features learned within these domains are supervised to be both discriminative and akin to real clothing domains. It's noteworthy that we emulate actual clothing domains using four commonly employed data augmentation strategies: mixup [72] , cutmix [71] , cutout [15] , and color jittering [52] . This deliberate choice stems from the rationale that each of these augmentation techniques introduces distinct variations in the data, simulating the diverse conditions encountered in real-world clothing domains. The advantage of incorporating these diverse augmentation strategies lies in the model's exposure to a wide range of synthetic clothing variations. To bridge the gap between fictitious and actual clothing domains, we employ domain alignment. By mimicking diverse clothing domains through these fictitious domains, our model learns discriminative clues and progressively adapts features to different outfits. This adaptability enhances the model's robustness against clothing changes by capturing shared identity-discriminative clues.Feature Augmentation (FeatAug): We randomly interchange feature elements of the same sample (Inner-sample Augmentation) and the same identity (Inter-sample Augmentation), a strategy aimed at enriching feature diversity and effectively addressing variability in clothing changes. This intentional manipulation is grounded in the rationale that different elements within a feature set inherently carry distinct clues related to the identity being represented. By encouraging the random interchange of these elements, our approach stimulates the exploration of various facets of the feature space. The advantage of this strategy lies in its ability to foster a more comprehensive understanding of the underlying patterns and intrinsic complexities present in the data. As the model engages in the exploration of different feature elements, it becomes more adept at discerning subtle nuances and capturing the diverse manifestations of identity within the context of clothing changes. This results in a more robust and adaptable model, ultimately enhancing its performance in cloth-changing re-identification scenarios by capturing a broader spectrum of identity-related information.Hence, the benefits of our CaAug approach are as follows: (1) Versatility Across Tasks: This strategy can be readily applied to various tasks, including scenarios where clothing consistency is crucial. It also enhances generalization capabilities across different conditions. (2) Comprehensive Approach: In contrast to simple data augmentation, our method is more comprehensive, as it directly transforms features, enhancing both robustness and generalizability. In summary, our contributions encompass four key aspects:We initiate the first exploration of CG-ReID and introduce a Cloth-aware Augmentation (CaAug) strategy to tackle this task. The core idea is to enhance the generalization ability of learned features without the need for additional information.We create fictitious domains to simulate clothing domains, enabling our model to learn features that are generalizable across various cloth-changing scenarios and more comprehensive in nature.We extend our approach by incorporating feature augmentation, which involves random interpolation of different samples. This enhances feature diversity and equips learned features to handle the uncertainties associated with varying degrees of clothing changes, further enhancing generalization ability.We conduct extensive evaluations of our method across clothchanging, cloth-consistent, and cloth-generalized scenarios. Our method consistently outperforms state-of-the-art techniques, as demonstrated by a robust set of experimental results. 
related works 2.1 person re-identification: 
 Person re-identification [67] has attracted much attention in realworld surveillance systems, which aim to associate the same pedestrian under various scenarios in different camera environments. Note that this task only relies on flexible surveillance videos, and it is more feasible for human-relevant applications compared with other biometric techniques. For example, in many scenarios, this additional bio-information is usually unavailable in real-world applications.Cloth-consistent Person Re-identification. Person Re-identification [68, 86] has been explored much in recent years, which aims to learn robust features against multiple variations. According to the addressed challenges, traditional ReID methods can be roughly listed: 1) Methods for camera view or pose changes [40, 45, 58, 74, 76] . These methods aim to learn robust features against view/pose changes, which usually take vantage of domain adaptation methods. For example, VCFL [40] takes advantage of adversarial learning for learning view-invariant identity-discriminative features. 2) Methods for occlusion [16, 23, 35, 44, 54] . Methods in this category aim to solve the challenging task as the appearance varies substantially with various obstacles, especially in the crowd scenario. For example, PVPM [16] aims to learn discriminative features with poseguided attention and part-aware visibility. 3) Methods for modality changes [5, 11, 30, 37, [63] [64] [65] [66] . These methods aim to match the same pedestrian between the visible and infrared cameras and solve modality gaps. With the development of deep learning, more and more challenges have been widely explored. However, these methods are vulnerable to clothes changes more or less, because they rely heavily on appearance and do not explicitly consider the clothing changes. Therefore, researchers begin to explore the cloth-changing person re-identification and aim to learn robust features against cloth changes, which is important in long-term surveillance.Cloth-changing Person Re-identification. In recent years, there have been several cloth-changing datasets for paving ways: LTCC [47] , PRCC [60] , Celeb-ReID [25] , COCAS [69] , VC-Clothes [53] and CCVID [18] . Recent methods aim to learn features robust against cloth changes, which can be roughly categorized into three groups: 1) Removing cloth-variant information: For example, CESD [47] removes clothing-related information and focuses on body type information that is unrelated to view/pose. CASE [36] learns bodystructural visual representations via adversarial learning and structural disentanglement. AFDNet [59] disentangle identity-related and identity-unrelated features with intra-class reconstruction and inter-class adversary. 2) Benefiting from cloth-variant information: For example, SPT [60] uses contour sketches to learn discriminative features because the shape of pedestrians won't change much with cloth-changing. 3DSL [7] leverage SMPL [42] for learning texture-insensitive 3D features. GI-ReID [28] learns cloth-agnostic representations by leveraging personal unique and cloth-independent gait information. 3) Benefiting from augmentation: For example, PosNeg [27] designs both positive and negative augmentations for enriching diversity. CCFA [20] augments feature with different semantic directions in the feature space.Current approaches primarily focus on cloth-changing tasks, overlooking the potential for mining generalizable information. Besides, augmentation-based methods, while effective, may introduce varying degrees of noise during the augmentation process, limiting their comprehensiveness. In response, CaAug addresses these limitations:(1) Capturing Generalizable Clues: Unlike previous methods, CaAug aims to capture shared generalizable clues among multiple fictitious domains, enhancing the model's ability to generalize across diverse conditions. (2) Enhancing Features with Generalizable Information: Instead of introducing meaningless information, CaAug enhances features by introducing exclusive information within the same sample and identity through random transformations at the feature level.In the ReID field, data augmentation strategies have been widely explored. GAN-based methods [13, 57, 81] and VAE-based methods [58] aim to generate multiple samples. Cutout [15] , CutMix [71] and MixUp [72] are all beneficial for increasing the diversity of samples. However, some data augmentation may distort the discriminative clues in pedestrian images as mentioned in [61] . Besides, generative networks usually need large computational costs. We originally proposed domain augmentation, which alleviates the potential risk that the quality of augmented data ruins learning discriminative features. In [32] , the effectiveness of feature augmentation is validated. We further design a feature augmentation strategy tailored for cloth-generalized tasks, which improves the diversity of features and handles the uncertainty that the cloth-changing degree varies.Variational Auto-Encoder (VAE) [29] is an important generative neural network. It consists of an encoder that aims to reduce dimensions and a decoder that aims to decrease the differences between original and output contents. Besides, VAE is capable of disentangling the latent factors of variations from the abstract representations. There are many methods of improving VAE for better handling disentangling problems. Abdi [1] focuses on representation learning and disentanglement through developing a VAE library that consists of a kind of improved VAE model. Seitzer [49] uses VAE to disentangle latent factors of variation. This paper pioneers the use of Variational Autoencoders (VAE) for generating fictitious domains:(1) Unlike Generative Adversarial Networks (GAN) and diffusion models, VAE offers a straightforward and cost-effective approach to generating domains with distinct distributions. (2) In the context of cloth-generalized problems, VAE facilitates the disentanglement of useful information for fictitious domains, improving the model's ability to capture relevant features for domain generalization. 
our approach: 
 Under the cloth-generalized setting, pedestrian appearances may change greatly, and therefore the appearance-based ReID methods become unreliable. Our goal is to learn robust features that resist clothes changes. To achieve this, we designed the Cloth-aware Augmentation (CaAug) strategy to alleviate the variations caused by clothes changes. As illustrated in Figure 2 , the whole framework includes Domain Augmentation (DomainAug) for latent domain learning and Feature Augmentation (FeatAug) for increasing the diversity of features. We first introduce the methodology details of the proposed strategy and then demonstrate the formulation of the whole model employed in this method.Recent methods for changing clothes rely on either clothing-specific information or other information unrelated to clothing, which is often inaccessible in real-world applications. We propose the domain augmentation strategy as an alternative approach, aimed at enhancing features without the need for additional information. The core concept of domain augmentation is to adapt features to various conditions, thereby promoting their generalizability and resilience to significant variations induced by changes in clothing. Moreover, domain augmentation is theoretically grounded. It is posited that data augmentation can be approximated through first-order feature averaging and second-order variance regularization components [12] . Consequently, regularizing the distribution of latent spaces with augmented data provides a more appropriate means to capture essential cues while mitigating the introduction of excessive noise. Specifically, this strategy comprises three key steps: Domain Generation, Domain Alignment, and Domain Feature Learning.Domain Generation: For domain generation, we generate fictitious domains by transforming the original domain, so that the fictitious can maintain some discriminative clues. To achieve this, we simply use variational auto-encoder (VAE) [29] , mainly for two reasons: 1) VAE can generate multiple domains with different distributions, which meets our demands. L D Dec Enc R 1 R 2 R D R 1 R 2 R D F 1 F 2 F D F dEQUATIONThen, we can obtain 𝐷 fictitious domains using 𝐷 VAEs. In the 𝑑 -𝑡ℎ domain, the latent features 𝐿 𝑑 = {𝑙 𝑑 𝑖 } 𝑁 𝑖=1 and the reconstructive features 𝑅 = {𝑟 𝑑 𝑖 } 𝑁 𝑖=1 are included. Domain Alignment: To achieve the goal that the fictitious domains can simulate different clothes domains, three demands should be satisfied: 1) Clothes domains should be related to clothes appearance, which means features in different clothes domains can be transformed back. 2) Different clothes domains should have different distributions from each other. 3) Features in different clothes domains should share some common information such as body shape, and body contour. Therefore, we use the generated images with different clothes to guide and align the fictitious distributions. First, the intuitive change of cloth-changing is color-changing, therefore we generate the images with different colors as guidance [10, 84] .Note that color affects the appearance more compared with the effect of accessories since color occupies the main part of the pedestrian's appearance. Therefore, for each sample 𝑥 𝑖 , we also generate guiding samples with different augmentation strategies including color-jittering [52] . The feasibility of generating images with different colors has been proved in [2] , and it is a straightforward manner. Second, we can obtain the corresponding features𝐹 𝑑 = {𝑓 𝑑 𝑖 } 𝐷 𝑑=1and logits 𝐺 𝑑 = {𝑔 𝑑 𝑖 } 𝐷 𝑑=1 by passing the generated samples through the backbone network. Third, we use the features of the generated samples to guide the learning of fictitious domains, aiming to narrow the gaps. We use KL divergence for domain alignment because of its ability to measure the distribution gaps.EQUATIONwhere 𝐾𝐿(•) denotes the KL divergence operation. Compared with using the generated samples for data augmentation, our domain augmentation has three advantages: 1) The features of the generated samples (𝐹 𝑑 ) do not back-propagate gradients since they are used as a reference distribution, and therefore we can save much computational cost. 2) We consider the overall distribution of the generated samples rather than their discriminative clues, therefore the quality of them does not influence the overall performance much.3) Different kinds of augmentation methods are aggregated in this manner, benefiting from the capturing of shared generalizable clues. Domain Feature Learning: Our goal is to obtain robust features that preserve discrimination under multiple conditions. To supervise the features in fictitious domains, we introduce the commonly used triplet loss. First, we give the formulation of triplet loss:EQUATIONwhere 𝑓 𝑖 , 𝑓 𝑗 , 𝑓 𝑘 form the triplets. ℎ(•) is the commonly used Euclidean distance, and 𝑚 is the margin for triplets. Then, the triplet loss for the fictitious domains can be formulated:EQUATIONThe formulation of domain augmentation is described:EQUATIONIt has been proved that feature augmentation can improve the generalization ability [32] . To enhance the robustness of features against clothes changes, we further design the Feature Augmentation strategy, which mainly handles the uncertainty in the cloth-generalized problem. The uncertainty in the cloth-generalized problem mainly lies in two terms. 1) Under a cloth-unchanging setting, different pedestrians may look similar to the same pedestrian. This problem is more severe in a cloth-generalized setting, because the same pedestrian may wear different clothes while different pedestrians are with similar clothes.2) The degree of clothes changes varies a lot, for example, some people only change their shirts while some people change the whole suit. To cope with this problem, we design the FeatAug with random interpolation. Random Interpolation. This process is tailored for the clothgeneralized task. The synthesized features are relevant to clothes changes, thus the random interpolation is suitable for handling the uncertainty in the cloth-generalized problem. This process enlarges the feature diversity using the linear combination of two selected instance features with different suits. In the first step, two feature selection manners are included for better harnessing identity-relevant clues. 1) Inner-sample: we select the features (F) and randomly chosen refactoring codes (𝑅) of the same sample. 2) Inter-sample: we select the features of the same pedestrian into different groups from each sampled training batch. Then, we synthesize two augmented features sets {𝑓 ′ 𝑖 } 𝑁 𝑖=1 and {𝑓 ′′ 𝑖 } 𝑁 𝑖=1 . Each sample feature 𝑓 ′ 𝑖 is generated by the interpolation of selected instance pair < 𝑓 𝑖 , 𝑓 𝑗 > with length 𝑡 while 𝑓 ′′ 𝑖 is generated by the interpolation of selected instance pair < 𝑓 𝑖 , 𝑟 𝑖 >, we random set 𝑚 * 𝑡 (𝑚 ∈ [0, 1]) bits as ones and the others as zeros to form the indicator vector 𝐼 . Then the random interpolation for feature elements is formulated:EQUATION) where 1 is the all one vector and 𝐿 2 (•) is the normalization function. We use triplet loss to supervise the augmented features:EQUATIONFor the backbone network, we use the commonly used ResNet-50 network and choose CAL [18] and AGW [67] as the baseline. The baseline supervision losses L 𝐵 are triplet loss ℓ 𝑡𝑟𝑖 and classification loss ℓ 𝑐𝑙𝑠 as in CAL [18] . Domain Augmentation (Domain-Aug) is supervised by L 𝐷 -𝐴𝑢𝑔 , aiming to learn robust features that are generalizable to multiple clothes domains. For Feature Augmentation, we use triplet loss L 𝐹 -𝐴𝑢𝑔 to supervise these augmented features, aiming to increase the diversity of features and handle the uncertainty of cloth-generalized problems.EQUATION)Network Configuration. Throughout all experiments, we utilize CAL [18] and AGW [18] for different cloth-generalized cases. This choice underscores the versatility of our method as an easily integrable module for enhancing generalization capabilities. For domain generation (with 𝐷 = 4), we employ four commonly used data augmentation strategies and utilize four VAE networks to generate corresponding fictitious domains. The lengths of latent codes are set equal to the number of classes, ensuring they adequately represent the distribution. Feature augmentation comprises Random Interpolation, with 𝑚 set to 0.1 to regulate the number of exchanged bits. These parameters are carefully selected through parameter analysis (refer to the appendix for further details).Training. The learning rate and the training strategy are the same as that of CAL [18] . For Domain Alignment: we use the backbone to extract features with the frozen model since they are used as the reference distributions, and KL divergence is used for alignment. For Domain Feature Learning: we supervise the reconstructed features with triplet loss in fictitious domains, and the margin is set as 0.3. For feature augmentation, the augmented features are obtained through synthesizing features in the original domain and supervised by triplet loss with a margin set as 0.3.Inference. In the inference phase, neither domain augmentation nor feature augmentation is necessary. Consequently, the trained model is employed solely to extract features from query and gallery images, facilitating feature matching. 
experiments 4.1 datasets and evaluation protocol: 
 To cope with the cloth-changing and cloth-generalized problem in the ReID field, two datasets are used for performance evaluation, including LTCC [47] and PRCC [60] . A cloth-consistent dataset (Market1501 [77] is used to prove the effectiveness of our method for cloth-unchanged tasks.LTCC [47] contains 17,138 person images of 152 identities, which can be divided into two subsets: one cloth-changing set where 91 persons appear with 417 different sets of outfits in 14,756 images, and one cloth-consistent subset containing the remaining 61 identities with 2,382 images without outfit changes.PRCC [60] consists of 221 identities with three camera views. Each person in Cameras A and B wears the same clothes, and the Evaluation Protocol follows the same protocol as the original protocol of the dataset. For LTCC, to better analyze the results of long-term cloth-changing Re-ID in detail, we introduce two test settings: standard-setting (we use all the data for training) and clothchanging setting (we only match the cloth-changing pedestrians ) as in [47] . For PRCC, we also introduce two test settings: crossclothes (images of camera A and camera C are used as the gallery and query sets) and same-clothes (images of camera A and camera B are used as the gallery and query sets). Cumulative Matching Curves (CMC) [17] and mean Average Precision (mAP) [3] are used as the evaluation metrics.To gain more insights into CaAug, we conduct a set of ablative studies on LTCC [47] and PRCC [60] , with ResNet-50 as the backbone network. Specifically, we analyze the influence of Domain Augmentation (DomainAug) and Feature Augmentation (FeatAug). When we combine the two components, we obtain the best performance. This suggests that these modules are complementary to each other, and confirms the effectiveness of our whole design.Effectiveness of Domain Augmentation (DomainAug). First, we examine the impact of domain augmentation using L 𝐷 -𝐴𝑢𝑔 (Eq.( 5)). The results are summarized in Table 2 . For LTCC, we observe a significant improvement of 2.7% and 4.3% in Rank1 through a comparison between the baseline (CAL [18] ) with and without DomainAug. This suggests that learned features become more robust and comprehensive by adapting to multiple fictitious domains.To further validate the superiority of the proposed DomainAug, we compare it with directly applying data augmentation, as shown in Table 4 . The results indicate that our strategy effectively mitigates the potential issue of augmenting data with poor quality. Moreover, our domain augmentation approach saves computational resources by utilizing a frozen model and serves as an efficient integration strategy for various augmentation techniques.Effectiveness of Feature Augmentation (FeatAug). Second, we investigate the feature augmentation with L 𝐹 -𝐴𝑢𝑔 (Eq.( 7)). Concretely, for the LTCC dataset, we compare the results of the baseline CAL [66] with/without L 𝐹 -𝐴𝑢𝑔 to evaluate its effectiveness. As shown in Table 2 , we gain 1.4% and 6.1% on Rank1 separately. This indicates that we can improve the diversity of features and handle the uncertainty of the cloth-changing problem, demonstrating that our feature augmentation strategy is suitable for cloth-generalized tasks. Besides, we also conduct experiments to explore the performance of random noise and random interpolation.Parameter Analysis. In our experiments, we explore the effect of the loss weight parameter 𝜆 1 and 𝜆 2 by systematically varying its values from 0 to 1 in increments of 0.1. The results, as shown in Figure 3 (a) and Figure 3 (b), clearly demonstrate consistent improvements over the baselines. Therefore, we set 𝜆 1 = 1 and 𝜆 2 = 1. This indicates that finding appropriate values for loss weights plays a crucial role in optimizing the overall performance of our method in cloth-generalized ReID tasks.We compare our method with cloth-changing methods and clothconsistent methods over multiple cases (cloth-changing cases, clothconsistent cases, and cloth-generalized cases), and prove its superiority over discrimination and generalization ability.1. Cloth-Changing Cases. These cases focus on the robustness against cloth changes, and we conduct experiments on two commonly used cloth-changing datasets.LTCC [47] . We evaluate our proposed method on the LTCC dataset and compare it with several competitors. In Table We find that the feature augmentation strategy consistently improves performance. Through (a) and (b), we find that setting hyper-parameters 𝜆 1 = 1 and 𝜆 2 = 1 is better.scenarios) such as AGW [67] are significantly impacted by the cloth-changing problem. Our method notably improves upon AGW, indicating its effectiveness in mitigating the negative impact of clothing changes and enhancing feature robustness. (2) By following the same protocol of LTCC, we report results for both general and clothchanging settings, demonstrating the superiority of our method. For instance, compared to the recent cloth-changing method SCNet [19] , we achieve an additional 2% gain in Rank1. PRCC [60] . Following the protocol of PRCC, we report the Rank1 and mAP results under cross-clothes and same-clothes settings, which are shown and compared in Table 1 . We have the following observations: (1) AGW [67] fails to achieve competitive performance over the cloth-changing setting. Our method achieves a significant increase with both Rank1 and mAP evaluation. (2) Compared with other cloth-changing methods, our method reaches SOTA under mAP and Rank1 evaluation metrics, which suggests the effectiveness of our method.2. Cloth-Consistent Cases. These cases focus on the robustness against traditional ReID challenges (such as illumination, view changes, and so on). These cases are also crucial since they are also general in real-world applications, and we conduct experiments on a commonly used cloth-consistent dataset.Market1501 [77] . To validate that our method is also suitable for cloth-consistent cases, we further conduct experiments on Mar-ket1501 (Market). As shown in Table 2 , we can conclude that our 3. Cloth-Generalized Cases. These cases focus on the generalization ability, which is relevant to the scope of the application. We design three settings that can roughly represent the generalizable setting over cloth-changing and cloth-consistent datasets: Market1501 → Duke, LTCC → Market1501, LTCC → PRCC.Market1501 [77] → DukeMTMC-reID [48] . This setting addresses scenarios involving both cloth-consistent and cloth-changing cases. In Table 5 , we provide a comparison of our method with other cloth-consistent techniques, demonstrating clear improvements. Additionally, we compare the results of Market1501 [77] → LTCC [47] with CAL and our method, yielding 15.6% and 5.1% versus 35.7% and 12.3% in rank1 and mAP, respectively. These findings underscore the enhanced generalization capability of our approach, particularly highlighting its efficacy in scenarios requiring the learning of a cloth-generalized model with only cloth-consistent data available.LTCC [47] → Market1501 [77] . This setting effectively encompasses scenarios spanning both cloth-consistent and cloth-changing cases. Table 6 demonstrates the efficacy of our method, showcasing improvements. These results indicate our ability to discern discriminative cues solely from cloth-changing data, thereby benefiting scenarios where ensuring performance in the presence of unknown cloth-changing occurrences is crucial.LTCC [47] → PRCC [60] . This setting captures scenarios involving cloth-changing cases. Table 7 presents qualitative comparisons within cloth-changing cases, clearly demonstrating our ability to learn generalizable features that adapt to various cloth-changing scenarios. These findings are particularly beneficial in scenarios where pedestrians exhibit significant diversity in clothing styles. 
conclusion: 
 We seek to tackle the formidable cloth-generalized problem in person ReID, an area where existing cloth-changing methods often overlook the extraction of generalizable insights. In this paper, we introduce the Mixed Augmentation strategy, designed to guide feature learning and cultivate resilient features resistant to changes in clothing. Our approach involves generating multiple fictitious domains to emulate clothing variations and adapting features to these domains, thereby fostering the acquisition of robust and versatile features. Additionally, to address the uncertainty inherent in the cloth-generalized problem, we synthesize features by incorporating inner-sample and inter-sample random interpolation techniques, while also supervising the learning of augmented features. While our method is tailored for the cloth-generalized problem, we firmly believe that our model possesses the capability to contend with even more intricate conditions characterized by significant variations.  
 
 Appendix:  


Decision: All reviewers are satisfied with the submission and responses. I hence recommend its final acceptance.

Reviews: review: paper_summary: paper_summary: This paper proposed a novel cloth generalized person re-identification (CG-ReID) task and tackled the proposed problem with domain augmentation and feature augmentation strategies.
strength_weakness: strength_weakness: 1. The experiments and ablations in this work are sufficient and well analysed.
2. The proposed method performs well on existing large-scale Clothes-changing benchmarks (e.g. PRCC, LTCC), with a competitive performance on traditional ReID benchmarks (e.g. Market).
3. The proposed CG-ReID setting is pratical and useful as it is highly corresponding to real- world applications. Therefore, the ideas of 'train a cloth-changing model with cloth-consistent data' and 'ensure the cloth-changing model performswell on cloth-consistent cases' are well motivated. Moreover, the general motivation was well delivered in the introduction section and easy to follow. In a word, I believe the newly proposed pratical setting would benefit further research for the ReID community.
limitations: limitations: 1. From my understanding, the proposed CG-ReID should focus on tackling the generalizable across both clothes-changing and clothes consistent/ same-clothes cases. However, in the experiment part, only the LTCC → Market (clothes-changing → same-clothes) was presented in Tab.6 . Considering that the same-clothes cases also exist (cases of one pedestrain wearing the same clothing) in clothes-chaning datasets (e.g. LTCC), this is not extremely tough case,  and such missing results (same-clothes →  clothes-changing ) would weaken the contributions. As aforementioned, it is suggested to compare existing single-source domain generalizable ReID (DG-ReID) methods with the proposed CG-ReID methods.
2. For the method part, domain alignment was well explored in DG-ReID. Using feature augmentation for CC-ReID shared similar ideas with CCFA[20] and SPS[a]. Considering their roles in a newly proposed setting, It is suggested to discuss the difference between the proposed alignment method and those employed in DG-ReID and existing augmentation-based CC-ReID methods.
a: Semantic-guided Pixel Sampling for Cloth-Changing Person Re-identification 2021
paper_summary: paper_summary: The paper presents an augmentation-based method for Cloth-Generalized Person Re-Identification, in which domain augmentation and feature augmentation are performed. For domain augmentation, fictitious domains are generated using VAEs, and aligned using samples generated by popular augmentation methods. For feature augmentation, features are exchanged in inter-sample and inner-sample manners for augmented features.
strength_weakness: strength_weakness: - Ideas of augmentation have always been promising for CCRe-ID [1][2]
- For domain augmentation, the paper leveraged VAEs to obtain fictitious domains, which is a novel approach
- For feature augmentation, inter-sample and inner-sample feature exchange sounds effective
- Writing is quite clear and easy to follow

[1] Nguyen et al., CCPA: Long-term Person Re-Identification via Contrastive Clothing and Pose Augmentation
[2] Han et al., Clothing-change feature augmentation for person re-identification
limitations: limitations: - Cloth-Generalized Scenario is vague and not novel. Intuitively, if a model can perform well under cloth-changing scenario, it is likely to perform well on cloth-consistent scenario which is less challenging. 

- The necessity and novelty of Cloth-Generalized Re-ID task is not validated. Consider choosing a SOTA CCRe-ID method, running experiment on CCRe-ID datasets, then run on standard Re-ID datasets and show performance drop for example.

- Related work in CCRe-ID lacks a review of more recent works (There are dozens of papers in CCRe-ID published in 2023 and early 2024 besides the covered papers)
- Not comprehensive experimental results

        + Similarly, in all experiments, results are compared to a very small number of previous (especially recent) methods, which limits the comprehensiveness of the effectiveness of the proposed method. More recent methods in 2023 and 2024 (see [1-5] for examples) should be taken into comparison. Eg. Tables 2, 5, 6, 7 only take into comparison methods from 2022 backward.
        + Your method leverages original RGB image only for training, while many CCRe-ID methods ([1, 3, 5] and others) also capture auxiliary modalities like pose, silhouette, SMPL shape. Specific comparison with this line of methods should be performed to further validate the superiority of the proposed method. 

- Incorrect/unclear technical details

       + On lines 400-402, you stated the rationale for your used augmentation methods is to make different domains should share common body shape or contours, however, [mixup, cutout, cutmix] will distort the body shape and contour, which is a conflict.
       + Unclear where $F^d$ is obtained from. Is it from a mixed of samples generated by all 4 augmentation methods, and for each sample in a batch, a random method out of those 4 is chosen?

- In implementation details, why use 4 VAEs? Is that because you used 4 augmentation methods or a finding after trials and errors?

- The used datasets are quite small-scale. Evaluation on very large-scale datasets with diverse distributions in data like DeepChange [6] would be more convincing. This also validates generalizability of the method.

- Insufficient evaluation 
       
       + Since the proposed method is augmentation-based, results should be specifically compared with previous augmentation-based methods to validate superiority. 
       + In Table 5, 6, 7, are those results produced on the second dataset after training on the first dataset? This should be clarified. 
       + In Table 5, 6, 7, the paper lacks showing the performance on the first dataset compared to on the second dataset, which helps validate the necessity of cloth-generalized Re-ID task, and the generalizability of the proposed method (e.g. shown by a smaller gap in performance produced by the proposed method compared to previous method)
       + In Table 6, compared methods belong to CCRe-ID should be noted and specifically compared to. This helps to show the superiority of the proposed method on cloth-generalized Re-ID scenario over previous CCRe-ID methods which only dealt with clothing changes without maintaining performance on cloth-consistent scenario.
       + In Table 3 and 4, experiment with FeatAug and without DomainAug would be helpful to further validate the effectiveness of FeatAug
       + On line 727, you should refer to Table 3, not Table 2
       + Feature map visualization will be very helpful to study the effectiveness of feature augmentation.

- Critical ethical issue: Duke-MTMC has been retracted and should not be used.

[1] Liu et al. Pose-Guided Attention Learning for Cloth-Changing Person Re-Identification, In IEEE TMM, 2023

[2] Huang et al. Meta Clothing Status Calibration for Long-Term Person Re-Identification. In IEEE TIP, 2024.

[3] Nguyen et al. Contrastive viewpoint-aware shape learning for long-term person re-identification. In WACV, 2024

[4] Yang et al. Win-win by competition: Auxiliary-free cloth-changing person re-identification. In IEEE TIP, 2023.

[5] Zhang et al. Multi-biometric unified network for cloth-changing person re-identification. In IEEE TIP, 2023.

[6] Xu, Peng, and Xiatian Zhu. Deepchange: A long-term person re-identification benchmark with clothes change. In ICCV. 2023.
paper_summary: paper_summary: In summary, the paper introduces a novel task of cloth-generalized person re-identification (CG-ReID) and proposes a Cloth-aware Augmentation (CaAug) strategy that comprises domain augmentation and feature augmentation to learn identity-relevant features that are adaptable to both cloth-consistent and cloth-changing scenarios. The experiments demonstrate the effectiveness of the approach, surpassing state-of-the-art methods on cloth-changing datasets.
strength_weakness: strength_weakness: 1) The paper pioneers the cloth-generalized person re-identification (CG-ReID) task, bridging the gap between cloth-consistent and cloth-changing scenarios.
2) The Cloth-aware Augmentation (CaAug) strategy, comprising domain and feature augmentation, enables learning identity-relevant features robust to clothing changes.
3) Extensive experiments on cloth-changing datasets demonstrate the effectiveness of the proposed approach, outperforming existing methods.
4) The codes will be publicly released, ensuring reproducibility and facilitating further research.
limitations: limitations: 1) The choice of augmentation methods is not well justified and may exclude more recent and effective techniques.
2) The use of a CNN baseline as the network architecture may limit performance.
3) Win-Win by Competition: Auxiliary-Free Cloth-Changing Person Re-Identification, TIP 2023, that serves as the predecessor of AIM [62] and would provide valuable insights for inclusion in the comparison table.
4) The evaluation of Tables 5-7 seems lack comparisons with state-of-the-art methods from 2023.