Enhancing Unsupervised Visible-Infrared Person Re-Identification with Bidirectional-Consistency Gradual Matching

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-03-25 11:56:43

Abstract: Unsupervised visible-infrared person re-identification (USL-VI-ReID) is of great research and practical significance yet remains challenging due to significant modality discrepancy and lack of annotations. Many existing approaches utilize variants of bipartite graph global matching algorithms to address this issue, aiming to establish cross-modality correspondences. However, these methods may encounter mismatches due to significant modality gaps and limited model representation. To mitigate this, we propose a simple yet effective framework for USL-VI-ReID, which gradually establishes associations between different modalities. To measure the confidence whether samples from different modalities belong to the same identity, we introduce a bidirectional-consistency criterion, which not only considers direct relationships between samples from different modalities but also incorporates potential hard negative samples from the same modality. Additionally, we propose a cross-modality correlation preserving module to enhance the semantic representation of the model by maintaining consistency in correlations across modalities. Extensive experiments conducted on the public SYSU-MM01 and RegDB datasets demonstrate the superiority of our method over existing USL-VI-ReID approaches across various settings, despite its simplicity. Our code will be released.

Main paper: 
 introduction: 
 Visible-infrared person re-identification (VI-ReID) aims to retrieve the same person captured by the visible camera when the query image from infrared camera is provided, and vice versa [16, 34, 44] . It has garnered attention for its potential in night vision applications, where traditional visible camera-based methods cannot work well due to poor lighting. Recently, many methods have been proposed for VI-ReID and made impressive progress [1, 17, 25, 32, 48] . However, the requirement for extensive annotations poses a significant challenge and impedes the scalability of these approaches. In response to this limitation, unsupervised visible-infrared person re-identification (USL-VI-ReID) has been raised and become a hot research topic [36, 38] .Due to the absence of precise manual annotations, establishing a robust association between different modalities is crucial in USL-VI-ReID. Recently, many approaches [5, 6, 30, 36] have endeavored to address this challenge by employing variants of bipartite graph matching algorithms. Specifically, these methods usually treat cross-modality label assignment as a graph matching task, aiming to identify correspondences between modalities by minimizing global matching costs [36] . Then, the modality-shareable feature representations can be derived from the generated crossmodality correspondences. However, the global matching approach may lead to mismatches of unreliable samples due to significant modality disparities and the limited representational capacity of the model. This mismatching phenomenon can adversely impact the overall performance of the model. Thus, how to build the reliable association between different modalities remains an open challenging problem nowadays.In this paper, we rethink the problem of cross-modality label assignments from the perspective of curriculum learning. Regarding the initial limitations of the model in addressing significant modal discrepancies, we advocate for the gradual establishment of associations between different modalities. Initially, we focus on generating cross-modality correspondences for easier samples. As training progresses and the model is more powerful to handle modal discrepancies, we incrementally incorporate more correspondences for harder samples. To measure the confidence whether samples from different modalities belong to the same identity, we propose a simple bidirectional-consistency criteria. This criterion not only leverages the direct relationship between samples from different modalities, but also considers potential hard negative samples from the same modality. However, merely aligning positive pairs from different modalities may weaken the structure information, i.e., the correlations of samples with unmatched clusters from the other modality. To address the concern, we propose the cross-modality correlation preserving module to further enrich the semantic representation of the model by maintaining consistency in correlations across different modalities.The main contributions of our work can be summarized as follows:• Revisiting the challenge of cross-label assignments through the lens of curriculum learning, we propose a simple yet effective framework for USL-VI-ReID employing a progressive matching paradigm. • We introduce a straightforward bidirectional-consistency criterion to evaluate whether samples from different modalities share the same identity. This criterion accounts for both positive instances across different modalities and potential negative instances within the same modality.  
related work 2.1 supervised visible-infrared person reid: 
 Supervised visible-infrared person ReID (SVI-ReID) aims to retrieve the same person across visible and infrared camera views [16, 34, 44] . The key of SVI-ReID is to learn modality-shareable feature representations with accurate manual annotations. Recently, many works have been proposed for SVI-ReID by relieving the discrepancy between different modalities [1, 17, 25, 32, 48] . Among them, [25] proposes to suppress the modality-related features by aligning persons in the pixel-level based on cross-modality dense correspondences. CoAL [32] proposes two novel attention modules to learn discriminative features for each modality and collaborative features across the modality. LUPI [1] generates the intermediate domain between visible and infrared modalities, subsequently mitigating modality shift by incorporating the synthesized intermediary domain as supplementary information. FMCNet [48] aims to generate the missing modality-specific discriminative features for each modality from the information obtained from the other modality, thereby combining these generated features to achieve more precise cross-modality retrieval. [17] proposes a novel data augmentation technique, named PartMix, which can generate positive and negative samples by combing local patches from the images of the same identity and different identities. In addition, some methods aim to reduce the modality gap by employing GANs [10, 26, 33, 40] to transfer the style of the image from one modality to another. Although these methods have achieved great performance in retrieving the same person across different modalities, they all require expensive manual annotations within the modality and across modalities, which heavily hinders their applications in the real world.Unsupervised visible-infrared person ReID (USL-VI-ReID) aims to match images of the same person across visible and infrared modalities without annotations [22, 38] . Compared with unsupervised single-modality person ReID [3, 20, 28, 46, 50] , USL-VI-ReID is more challenging as the modality discrepancy is usually more serious than the inter-class variance within each modality. To build reliable association between different modalities, many works [5, 6, 30, 36] have been proposed to learn modality-shareable feature representations by generating cross-modality correspondences. Among them, PGM [36] proposes a novel paradigm, which formulates the crossmodality correspondences mining as the graph matching problem. By adopting this approach, the model can effectively incorporate global information through minimizing the global matching cost. OTLA [30] proposes the optimal-transport method to reduce the modality gap by assigning pseudo labels for images from visible modality to infrared modality. To guarantee that all clusters from infrared modality can be matched in the training process, [5] proposes a many-to-many bilateral cross-modality cluster matching method to generate cross-modality correspondences. In addition, [6] proposes a dual optimal transport label assignment method to solve the cross-modality correspondences assignment problem in a reinforcement manner.Although these methods have achieved great progress on the USL-VI-ReID task with their carefully generated cross-modality correspondences, they usually solve the problem with the global matching algorithms. In this way, some unreliable samples can be mismatched due to serious discrepancy between different modalities and limited feature representations in the early period. To tackle this and avoid label noise accumulation, we propose a novel paradigm to generate correspondences in a gradual manner. To weight the confidence whether samples from different modalities belong to the same identity, we devise a simple yet effective bidirectionalconsistency criteria.Inspired by the human learning process, curriculum learning is proposed by [2] as a training strategy by learning from easy samples to hard samples. Curriculum learning can enhance the performance of the machine learning models, and it has been widely used in many areas, including object detection, segmentation and retrieval [19, 31, 47, 49, 51] . Specifically, [47] proposes a self-paced curriculum learning framework for weakly supervised object detection by utilizing instance-level and image-level prior-knowledge. [49] proposes an adaptive alignment module for remote sensing crossmodal text-image retrieval by utilizing image-text pairs from easy to hard for better feature representations. [31] proposes a selective training framework for the task of node classification by relieving the influence of low-quality training nodes. To prevent the additional bias introduced by human interventions, [51] introduces the curriculum learning strategy in the task of pre-training GNN task by supervising the training process with different structures and feature spaces. To refine the pseudo labels for semi-supervised semantic segmentation, [19] proposes a strategy for confidence score by adopting curriculum learning. To the best of our knowledge, this is the first work to explore cross-modality correspondences generation with curriculum learning. 
method: 
 The framework of our method is illustrated in Fig. 1 . In Sec. 3.1, we first introduce the augmented dual-contrastive learning (DCL [38] ), which is also regarded as the baseline in our work by learning within each modality. Based on DCL, we further introduce our devised simple bidirectional-criteria and gradual matching strategy, which are described in Sec. 3.2 in detail. Finally, we describe our proposed cross-modality correlation preserving module in Sec. 3.3.Given a unlabeled visible and infrared training set 𝑋 = {𝑋 𝑣 , 𝑋 𝑖 }, where 𝑋 𝑣 = 𝑥 𝑣 1 , 𝑥 𝑣 2 , . . . , 𝑥 𝑣 𝑁 denotes the visible training set containing 𝑁 images, while 𝑋 𝑖 = 𝑥 𝑖 1 , 𝑥 𝑖 2 , . . . , 𝑥 𝑖 𝑀 denotes the infrared training set with 𝑀 images. It is noted that to relieve the discrepancy between different modalities, random Channel Augmentation (CA) [41] is also applied in the visible stream for assistance.To extract features from images, the two-stream encoders 𝑓 𝑣 𝜃 and 𝑓 𝑖 𝜃 are adopted, which share the same convolution backbone but modality-specific classifiers. In this way, visible feature vectors , where 𝐿 and 𝐾 represent the number of clusters for visible and infrared modalities. Then modality-specific prototypes Φ 𝑣 ∈ R 𝐿×𝑑 and Φ 𝑖 ∈ R 𝐾 ×𝑑 can be obtained by averaging the feature vectors within the same cluster, where 𝑑 is the dimension of the feature vector.𝑈 𝑣 = 𝑢 𝑣 1 , 𝑢 𝑣 2 , . . . ,During training, the PK sampling strategy [15] is adopted to construct the mini-batch. For the infrared modality, we randomly sample 𝑃 identities and each identity contains 𝐾 images. As the random channel augmentation is also applied in the visible modality, to balance the number of images from different modalities, we random sample 𝑃 identities from the visible modality and each identity contains 𝐾/2 images and 𝐾/2 corresponding augmented images. Then for the infrared modality, the ClusterNCE [8] loss can be applied as follows,EQUATIONwhere 𝑞 𝑖 represents the query sample from infrared modality, 𝜙 𝑖 𝑘 denotes the prototype of the 𝑘-th cluster from infrared modality and 𝜙 𝑖 + denotes the prototype of the cluster which 𝑞 𝑖 belongs to, 𝜏 is the temperature hyper-parameter. Similarly, the ClusterNCE loss can also be applied on the visible and random channel augmented images in the same way by comparing their images with the corresponding prototypes from visible modality, which can be denoted as 𝐿 𝑞 𝑣 . In this way, the final objective function of DCL can be obtained by combining them together as follows,𝐿 𝐷𝐶𝐿 = 1 |𝐵 𝐼 | ∑︁ 𝑞 𝑖 ∈𝐵 𝐼 𝐿 𝑞 𝑖 + 1 |𝐵 𝑉 | ∑︁ 𝑞 𝑣 ∈𝐵 𝑉 𝐿 𝑞 𝑣 .(EQUATIONwhere 𝐵 𝐼 and 𝐵 𝑉 are input batch from infrared modality and visible modality, respectively. It is noted that 𝐵 𝑉 also contains samples augmented by random channel augmentation. For simplicity we omit the notation of augmented samples as visible and augmented samples are all included in the same stream and are treated equally.During training, the modality-specific prototypes from visible and infrared modalities can be updated in the momentum manner as EQUATIONwhere 𝑞 𝑒 is the query sample from modality 𝑒 (𝑒 = {𝑖, 𝑣 }, representing the infrared modality and visible modality, respectively), 𝜙 𝑒 + is the prototype of the cluster which 𝑞 𝑒 belongs from modality 𝑒. 𝑚 is the momentum factor, which decides the updating speed of the corresponding prototypes.Although DCL can relieve the modality discrepancy between visible and infrared modalities to some extent with augmented samples serving as the intermediate connection, it is still insufficient for the model to learn modality-invariant feature representations as it only performs ClusterNCE loss within each modality independently.Due to the absence of annotations, the key of USL-VI-ReID lies in establishing the reliable association across different modalities. Different from existing works that establish correspondences by minimizing the overall graph matching cost, our approach advocates for a gradual association construction. Thus, how to design a criteria which can reflect the confidence whether image pairs from different modalities belong to the same identity is the key in our work.Compared with the unsupervised single-modality ReID, USL-VI-ReID is more challenging as the modality gap is more serious than the inter-class variance within each modality. Directly modeling the similarity score is inadequate for reflecting reliability, as nearby clusters within the same modality may also serve as potential candidates for matching corresponding clusters in other modalities. To capture the structural essence of these clusters, we introduce a simple yet effective bidirectional-consistency criteria. To streamline computational efficiency, we formulate this criterion directly on modality-specific prototypes Φ 𝑣 ∈ R 𝐿×𝑑 and Φ 𝑖 ∈ R 𝐾 ×𝑑 . Specifically, given the 𝜙 𝑣 𝑙 , which represents the prototype of the 𝑙-th cluster in the visible modality, we can find its nearest prototype 𝑁 𝑉 →𝐼 (𝜙 𝑣 𝑙 ) from infrared modality as follows, 𝑁 𝑉 →𝐼 (𝜙 𝑣 𝑙 ) = arg maxEQUATIONwhere 𝑆 (𝐼 ) denotes the set of prototypes in the infrared modality.Then the bidirectional-consistency criteria can be formed as follows,EQUATIONwhere 𝑅 𝑉 →𝐼 (𝜙 𝑣 𝑙 ) denotes the similarity between 𝜙 𝑣 𝑙 and its nearest cluster from the other modality, which can reflect the reliability whether they belong to the same identity. 𝑆 (𝑉 )/𝜙 𝑣 𝑙 denotes the set of prototypes in the visible modality except for 𝜙 𝑣 𝑙 . In this way, we can take both the positive samples from different modalities and those potential negative samples from the same modality into fully consideration. By applying Eq. ( 4), we can obtain the reliability of all 𝐿 samples in the visible modality as R 𝑉 →𝐼 = 𝑅 𝑉 →𝐼 (𝜙 𝑣 1 ), 𝑅 𝑉 →𝐼 (𝜙 𝑣 2 ), . . . , 𝑅 𝑉 →𝐼 (𝜙 𝑣 𝐿 ) . Regarding that the model is limited in dealing with the serious modality discrepancy in the early period, simply apply global alignment like existing works [5, 6, 30, 36] to generate correspondences for all samples may cause the label noise accumulation during training. To build reliable association between different modalities, we aim to generate correspondences in a gradual scheme, i.e., we initially generate correspondences for those most reliable image pairs, then involve harder samples in the matching process when the model is more powerful in the cross-modality representation. Specifically, we firstly sort the reliability set R 𝑉 →𝐼 in the descending order, then select the first 𝑇 -percent samples from it as R𝑉 →𝐼 , where the value of 𝑇 increases linearly with current epoch as follows,EQUATIONwhere 𝑇 0 is the hyper-parameter that controls the initial proportion of samples to be matched in the beginning, and it is simply set to 0.1 in our experiments. Then, we can obtain the cross-modality correspondences for a subset of samples from the visible modality, which can be represented as follows,EQUATIONwhere M 𝑉 →𝐼 is the obtained correspondence set for visible modality. Then we can learn cross-modality feature representations with it. Specifically, given the feature vector 𝑞 𝑣 from the 𝑗-th cluster in visible modality, if its corresponding prototype 𝜙 𝑣 𝑗 is in the set M 𝑉 →𝐼 , the unidirectional learning from visible to infrared can be represented as follows,EQUATIONwhere 𝑁 𝑉 →𝐼 (𝜙 𝑣 𝑗 ) is the matched prototype in the infrared modality of 𝜙 𝑣 𝑗 . By comparing visible images with prototypes in the infrared modality, the modality discrepancy can be efficiently reduced. Similarly, for 𝑞 𝑖 which is extracted from infrared modality, its unidirectional learning from infrared to visible modality can be obtained in the same way, which is denoted as 𝐿 𝐼 →𝑉 𝑞 𝑖 . Then the cross learning can be obtained by combining these two streams as follows,EQUATIONIn this way, the model can learn cross-modality feature representations in a gradual manner to avoid the label noise accumulation in the early period.As shown in Fig. 1 , for those selected image pairs from different modalities, cross-modality learning can reduce the modality gap by pulling them together. However, this process may compromising the underlying structure information, particularly the correlation between the sample and other unmatched clusters in the alternate modality by pushing them away. To address this issue, we introduce the cross-modality correlation preserving module, which aim to preserve the underlying structure information for those selected samples when aligning them with their correspondences. Specifically, considering the feature vector 𝑞 𝑣 from visible modality, if it is matched with the 𝑙-th cluster in the infrared modality, then the cross-modality correlation between 𝑞 𝑣 and all clusters in the infrared modality except for the 𝑙-th cluster can be represented as the probabilistic vector 𝑃 𝑉 →𝐼 𝑞 𝑣 . Then, the 𝑙-th element of 𝑃 𝑉 →𝐼 𝑞 𝑣 is assigned a value of 0, while the remaining elements are defined as:EQUATIONwhere 𝑃 𝑉 →𝐼 𝑞 𝑣 𝑗 denotes the 𝑗-th entry of 𝑃 𝑉 →𝐼 𝑞 𝑣 , 𝜙 𝑖 𝑙 is the correspondence of 𝑞 𝑣 , i.e., the prototype in the infrared modality which is matched with 𝑞 𝑣 . In this way, we can represent the correlation between 𝑞 𝑣 and all clusters in infrared modality except for the matched 𝑙-th cluster. Analogously, the intrinsic correlation between the 𝑙-th cluster with other clusters in the same infrared modality can be represented as 𝑃 𝐼 →𝐼 𝜙 𝑖 𝑙 . Similarly, we set the 𝑙-th entry of 𝑃 𝐼 →𝐼 𝜙 𝑖 𝑙 to 0, with the remaining elements defined as follows,EQUATION)Given that 𝑞 𝑣 and 𝜙 𝑖 𝑙 are linked in the matching procedure, they are likely to represent the same identity across different modalities. Therefore, to preserve the cross-modality correlations, we align these probabilistic vectors as follows,EQUATIONSimilarly, for 𝑞 𝑖 which is extracted from infrared modality, the cross-modality correlation preserving loss can be obtained in the same way by comparing it with visible prototypes, which can be denoted as 𝐿 𝐼 →𝑉 𝑐𝑜𝑟𝑟𝑒 (𝑞 𝑖 ). Then the final cross-modality correlation preserving module can be formed by combing these two terms as follows,EQUATIONDiscussion. For those matched clusters from different modalities by our bidirectional-consistency gradual matching module, Eq. ( 9) can align them together by pulling the sample from one modality and the matched cluster from the other modality closer. However, this alignment strategy risks compromising the underlying structure information since Eq. ( 9) also pushes the sample further away from other unmatched clusters. To address this challenge, we propose the cross-modality correlation preserving module, which aims to preserve the correlation between the sample and clusters from the other modality except for the matched cluster. As the probability vector used in Eq. ( 13) contains no information about the matched cluster in the other modality, it doesn't violate the goal of Eq. ( 9) in the training process.Following existing works [5, 6, 36, 38] , our model is also trained with two stages. In the first stage, the model is optimized with the DCL module as follows,EQUATIONAfter the first training stage, the model can relieve the modality discrepancy to some extent by leveraging the augmented visible images as intermediate associations. Then the model can be trained by combining our proposed bidirectional-consistency gradual matching module and cross-modality correlation preserving module to further boost the discrimination of its cross-modality feature representations as follows,EQUATIONwhere 𝜆 1 and 𝜆 2 are hyper-parameters which balance basic DCL module and our proposed modules. 
experiment 4.1 datasets and evaluation protocols: 
 Our experiments are conducted on two publicly available VI-ReID datasets: RegDB [24] and SYSU-MM01 [34] . Following existing works [6, 36, 38] , we adopt mean average precision (mAP) and Cumulative Matching Characteristics (CMC) as the evaluation metrics. Among the CMC evaluation metric, Rank-1, Rank-10 and Rank-20 are also reported for more exhaustive evaluations. In addition, the mean Inverse Negative Penalty (mINP) metric proposed in [43] is also presented. SYSU-MM01 is a large-scale public VI-ReID dataset collected from 4 visible cameras and 2 near-infrared cameras. The training set contains 395 identities, including 22,258 visible images and 11,909 infrared images, while the test set contains 96 identities. Following [36] , we evaluate our method on this dataset with two settings, including All-search mode and Indoor-search mode. For all-search mode, the gallery set contains images collected from all visible cameras. For indoor-search mode, only images captured by indoor visible cameras are used to constitute the gallery set.RegDB is collected from one visible camera and one infrared camera. The dataset contains 4,120 visible images and 4,120 infrared images of 412 identities. Our method is evaluated on this dataset with two settings, including visible-to-infrared search and infraredto-visible search. For fair comparison, following existing works [6, 36, 38] , we conduct experiments on this dataset for 10 times and report the average result as the final performance.Our method is implemented in the PyTorch platform. Following [36] we adopt the ImageNet-pretrained ResNet50 [14] network as the backbone in the experiment. In the mini-batch, the number of identities for each modality is set to 16 and each identity contains 16 instances. Following [36] , we resize the input image as 288 × 144. For data augmentation, random horizontal flipping, padding, random cropping, and random erasing, random grayscale and channel argumentation [41] are applied on the input image.In the training process, DBSCAN [9] clustering algorithm is applied to generate pseudo labels within each modality. For DBSCAN, the minimal number for neighbours is set to 4 for different datasets, the maximum distance is set to 0.6 for SYSU-MM01 dataset while 0.3 for RegDB dataset, which are the same as existing works [36, 38] .Table 1 : Experimental results (%) of the proposed method and SOTA methods on the SYSU-MM01 and RegDB datasets under different settings. * means the model is pre-trained on an extra labeled visible dataset, † means the model is pre-trained on AGW [43] while ‡ means the model is pre-trained on CLIP [27] . As described in Sec. 3.4, our model is trained with two stages. To train the model, Adam optimizer with weight decay 5e-4 is adopted.For each stage, we set the initial learning rate as 3.5e-4, and reduce it every 20 epochs for a total 50 epochs. In Eq. ( 15), 𝜆 1 and 𝜆 2 are set to 0.3 and 0.5, respectively. Following [36] , the alternate learning strategy is applied on the cross-modality learning with different directions.To demonstrate the effectiveness of our method, we compare our method with state-of-the-art supervised visible-infrared person ReID (VI-ReID), unsupervised single-modality person ReID (USL-ReID) and unsupervised visible-infrared person ReID (USL-VI-ReID) methods on two public datasets under different settings. The results are shown in Tab. 1.Comparison with SVI-ReID methods. We compare our method with existing SVI-ReID methods, including DDAG [42] , AGW [43] , CA [41] , MCLNet [13] , MPANet [35] , DART [39] , MAUM [23] , CTFT [21] , MUN [45] and PartMix [18] . Although these methods train their models with ground truth, our proposed method can still outperform some of them on these datasets. Specifically, on the SYSU-MM01 dataset, we can achieve competitive performance compared with some methods, such as DDAG and AGW. While on the RegDB dataset, our method can outperform most of them, including DDAG, AGW, CA, MCLNet, MPANet and DART. The reason could be that compared with RegDB dataset, SYSU-MM01 is more challenging and it is limited to train the model with generated pseudo labels.Comparison with USL-ReID methods. We also compare our method with existing USL-ReID methods, including SPCL [12] , MMT [11] , ICE [3] , IICS [37] , CCL [8] , ISE [50] and PPLR [7] . Although these methods can achieve great performance on the task of unsupervised single-modality person ReID, they are very limited on the USL-VI-ReID task compared with our method. The reason could be that compared with the USL-ReID, in USL-VI-ReID the modality discrepancy is usually more serious than the inter-class variance within each modality. Thus, it is hard to directly apply the clustering algorithms to generate cross-modality correspondences for the task of USL-Vi-ReID, which verifies the necessity in generating correspondences between different modalities.Comparison with USL-VI-ReID methods. We compare our method with existing USL-VI-ReID methods, including H2H [22] , OTLA [30] , ADCA [38] , PGM [36] , CCLNet [4] , DOTLA [6] and MBCCM [5] . Among them, H2H, OLTA and DOTLA utilize the extra labeled visible dataset while CCLNet utilize the pretrained CLIP as the encoder. Compared with these methods, our method can achieve better performance on both SYSU-MM01 and RegDB datasets under different settings. The reason could be that these methods build the association between different modality with global matching algorithms. In this way, some unreliable samples can be mismatched due to the serious modality discrepancy and limited feature representations of the model in the early period. Different from them, we generate cross-modality correspondences in a gradual manner, which can relieve the label noise accumulation when associating different modalities.In this section, we conduct experiments on SYSU-MM01 and RegDB datasets to evaluate the effectiveness of different components in our method, and the results are shown in Tab. 𝜆 1 and 𝜆 2 in Eq. ( 13) are two important hyper-parameters introduced in our work. In this section, we conduct experiments to evaluate the influence of these hyper-parameters in our work.4.5.1 Influence of 𝜆 1 . In the objective function, 𝐿 𝐷𝐶𝐿 can learn within the modality to promote more accurate clusters for association, while 𝐿 𝐶𝑟𝑜𝑠𝑠 explores the cross-modality relationships based on the learned homogeneous feature representations. Thus, how to balance them in the learning process is crucial in our work. In Fig. 2 , we conduct experiments on SYSU-MM01 dataset to determine the value of 𝜆 1 . The results show that our method is relatively robust against 𝜆 1 . When 𝜆 1 is set to 0, our method can still achieve good performance, the reason could be that our method can benefit from preserving correlations with CMCP module. When 𝜆 1 is set to 0.3, our method can achieve the best performance.4.5.2 Influence of 𝜆 2 . 𝜆 2 determines the weight of our CMCP module in the final objective function. Fig. 3 shows the results under different values of 𝜆 2 on SYSU-MM01 dataset. As illustrated in the figure, setting 𝜆 2 to 0.5 and 1.0 yields superior performance compared to the performance when 𝜆 2 is set to 0, which verifies the effective of our CMCP module. However, when 𝜆 2 is assigned with a larger value, such as 1.5, the performance of the model shows a notable decline, the reason can be that the overemphasis on 𝜆 2 weaken the influence of DCL and cross-modality learning modules. 4.6.1 Evaluating the quality of generated correspondences. In the task of USL-VI-ReID, cross-modality correspondences are necessary for associating different modalities due to the absence of annotations. Different from existing works [5, 6, 36] that typically employ variants of bipartite graph matching algorithms to establish crossmodality correspondences (i.e., determining whether samples from different modalities have the same identity), our approach advocates for a gradual correspondence discovery process to mitigate mismatches stemming from global alignment issues. In Fig. 4 , we evaluate the cross-modality matching accuracy and matching radio in PGM [36] and our method. As depicted in the figure, the matching accuracy of our method significantly surpasses the accuracy of PGM. The reason could be that the model learned simple patterns at first, enabling it to handle more complex matching rules. To further investigate the effectiveness of our method in learning discriminative feature representations, we employ t-SNE [29] to visualize features learned by PGM and our method, and results are shown in Fig. 5 . For RegDB, we randomly select 10 identities, while for another challenging SYSU-MM01 dataset, 7 identities are randomly selected. From Fig. 5 (a) and Fig. 5 (b), we can find that PGM tends to separate features of the same identity across modalities, while our method produces compact clustering of features from different modalities belonging to the same identity. On another challenging SYSU-MM01 dataset (Fig. 5 (c) and Fig. 5 (d)), both PGM and our method show a larger variance among the same identity. However, for some hard samples, e.g., samples with green and orange colors, PGM may confuse them due to incorrect association caused by global alignment during training. 
conclusion: 
 In the paper, we propose a simple yet effective framework for USL-VI-ReID by establishing the association between different modalities through a gradual matching approach. To measure the reliability that samples from distinct modalities belong to the same identity, we introduce a straightforward bidirectional-consistency criteria, which accounts for both intra-modal and inter-modal relationships. Furthermore, we introduce the cross-modality correlation preserving module to maintain the coherence of correlations across modalities. Extensive experiments on the public SYSU-MM01 and RegDB datasets demonstrate the effectiveness of our approach under various settings. 
 
 Appendix:  


Decision: This paper got 3 WA and 1 BA, and the authors have addressed most of the concerns from the reviewer.

Reviews: review: paper_summary: paper_summary: This paper proposes a bidirectional-consistency gradual matching paradigm for unsupervised visible-infrared person re-identification (USL-VI-ReID), which gradually establishes associations between visible modality and infrared modality. To measure the confidence whether samples from different modalities belong to the same identity, this paper introduces a straightforward bidirectional-consistency criterion, which not only considers positive instances across different modalities but also incorporates potential hard negative instances within the same modality. Additionally, this paper devises a cross-modality correlation preserving module to enhance the structure information and the semantic representations of the model by maintaining consistency in correlations across modalities. Extensive experiments conducted on public SYSU-MM01 and RegDB datasets demonstrate the superiority of the proposed method over existing state-of-the-art approaches across various settings in spite of its simplicity.
strength_weakness: strength_weakness: 1.This paper rethinks the problem of cross-modality label assignments from the perspective of curriculum learning and proposes a straightforward and effective strategy to gradually establish cross-modality correspondences, which significantly improves matching accuracy and relieves modality discrepancy.

2.This paper is well written and the experiments are detailed and plausible. An exhaustive ablation study and hyper-parameter analysis are provided to verify the effectiveness and robustness of the proposed method.

3.Compared with the latest USL-VI-ReID approaches in recent years, the proposed method demonstrates superior performance on public SYSUMM01 and RegDB datasets under various settings.
limitations: limitations: 1.In Fig. 4, this paper compares the cross-modality matching accuracy and matching radio (typo, which should be “ratio”) of PGM with the proposed method. In Sec. 3.2, this paper has already claimed that the matching ratio grows linearly from $T$-percent to one hundred percent as the current epoch number increases. Therefore, Fig. 4(b) and (d) are redundant. In addition, it is recommended that Fig. 4(a) and (c) be explained and discussed in further detail.

2.In Sec. 3.1, this paper claims that the two-stream modality encoders share the same convolution backbone but modality-specific classifiers. However, the modality-specific classifiers do not appear in Fig. 1.

3.Some equations in this paper are improperly formulated. For example, $\hat{\mathcal{R}}^{V\to I}$ in Eq. (7) denotes the set of the reliability of selected samples in the visible modality while $\phi_j^{v}$ denotes the prototype of the $j$-th cluster in the visible modality. Therefore, $\phi_j^{v} \in \hat{\mathcal{R}}^{V\to I}$ is incorrectly formulated.

4.There are a few typos in this paper. For example, the abbreviation “VI-ReID” for supervised visible-infrared person ReID in Sec. 4.3 should be corrected to “SVI-ReID”. In Sec. 4.5, “$\lambda_{1}$ and $\lambda_{2}$ in Eq. (13)“ should be corrected to “$\lambda_{1}$ and $\lambda_{2}$ in Eq. (15)”.
paper_summary: paper_summary: This paper introduces a USL-VI-ReID framework designed from the perspective of curriculum learning. It introduces bidirectional consistency standards to measure the confidence of whether samples from different modalities belong to the same identity. Additionally, it proposes a cross-modal correlation preservation module aimed at enhancing the semantic representation of the model by maintaining consistency in cross-modal correlations.
strength_weakness: strength_weakness: 1. They propose a simple yet effective framework for USL-VI-ReID employing a progressive matching paradigm.
2. They propose the cross-modality correlation preserving module to enhance the model performance and gain SOTA results.
limitations: limitations: 1. The author claims that merely aligning positive pairs from different modalities may weaken the structural information. How can this be validated?

2. The motivation for the proposed Augmented Dual-Contrastive learning is not clear in the Introduction.

3. What are the advantages of the proposed Augmented Dual-Contrastive Learning over other methods that use Contrastive Learning? Please provide relevant experimental comparisons and analyses.

4. How to validate that the proposed method establishes more reliable associations between different modalities? Please conduct relevant experiments and analysis.

5. The terminator for Formula (2) should be ',' instead of '.'.
paper_summary: paper_summary: This paper aims to address the USL-VI-ReID task and proposes a bidirectional-consistency criterion to measure the confidence whether samples from different modalities belong to the same identity. Besides, this paper also proposes a cross-modality correlation preserving module to enhance the semantic representation of the model by maintaining consistency in correlations across modalities. Extensive experiments conducted on the public SYSU-MM01 and RegDB datasets demonstrate the superiority of the proposed method over existing USL-VI-ReID approaches across various settings.
strength_weakness: strength_weakness: 1.The method is relatively simple and effective, and the author clearly explains the proposed method.
2.Extensive experiments have demonstrated the effectiveness of the proposed method.
limitations: limitations: 1.In Section 4.3, the author should focus on comparing and analyzing the advantages and disadvantages of USL-VI-ReID based methods.

2.In line 574, "argument" seems to be "authorization".

3. Some newer related methods need to be referenced, including but not limited to the following methods:
[1] Yang B, Chen J, Ye M. Towards grand unified presentation learning for unsupervised visible informed person re identification [C]//Proceedings of the IEEE/CVF International Conference on Computer Vision 2023: 11069-11079
[2] Yang B, Chen J, Chen C, et al. Dual Consistency Constrained Learning for Unsupervised Visible Informed Person Re Identification [J] IEEE Transactions on Information Forensics and Security, 2023.

4. It would be better if the author could provide experimental results on the LLCM dataset, which is a more realistic VI-ReID dataset.
[3] Zhang Y, Wang H. Diverse embedding expansion network and low-light cross-modality benchmark for visible-infrared person re-identification[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2023: 2153-2162.
paper_summary: paper_summary: This paper investigates the unsupervised VI-ReID task. A Bidirectional Matching algorithm with a gradual matching strategy is proposed to progressively establish reliable cross-modality associations, drawing inspiration from the concept of curriculum learning. Additionally, a Cross-modality Correlation Preserving module is introduced to maintain the structural relationships between instances and their unmatched cross-modality prototypes.
strength_weakness: strength_weakness: (1) The proposed GM strategy addresses cross-modality association through the lens of curriculum learning, distinguishing it from existing methods.\
(2) Comprehensive experiments validate the effectiveness of each component of the method.
limitations: limitations: (1) While BCR introduces a straightforward bi-directional cross-modality association algorithm, this paper specifically compares it with the uni-directional BGM. The efficacy of bi-directional strategies has been demonstrated in prior works [1]. Therefore, it is recommended that the authors include a comparison between BCR and bi-directional methods (e.g., DOTLA [1] and a bi-directional BGM [2]) to validate its superiority. \
(2) The CCP module ensures structural consistency between instances and their matched prototypes, whereas existing self-consistency-based methods [3,4] focus on maintaining instance-level consistency. It is suggested that the advantages of CCP over these methods be elucidated. \
(3) The visualizations in the paper appear hastily done, the authors are recommended to standardize the figures in the experiments.