Robust Pseudo-label Learning with Neighbor Relation for Unsupervised Visible-Infrared Person Re-Identification

Authors: xiangbo  yin
jiangming  shi
yachao  zhang
yang  lu
zhizhong  zhang
yuan  xie
yanyun  qu
Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-03 13:55:06

Abstract: Unsupervised Visible-Infrared Person Re-identification (USVI-ReID) presents a formidable challenge, which aims to match pedestrian images across visible and infrared modalities without any annotations. Recently, clustered pseudo-label methods have become predominant in USVI-ReID, although the inherent noise in pseudo-labels presents a significant obstacle. Most existing works primarily focus on shielding the model from the harmful effects of noise, neglecting to calibrate noisy pseudo-labels usually associated with hard samples, which will compromise the robustness of the model. To address this issue, we design a Robust Pseudo-label Learning with Neighbor Relation (RPNR) framework for USVI-ReID. To be specific, we first introduce a straightforward yet potent Noisy Pseudo-label Calibration module to correct noisy pseudo-labels. Due to the high intra-class variations, noisy pseudo-labels are difficult to calibrate completely. Therefore, we introduce a Neighbor Relation Learning module to reduce high intra-class variations by modeling potential interactions between all samples. Subsequently, we devise an Optimal Transport Prototype Matching module to establish reliable cross-modality correspondences. On that basis, we design a Memory Hybrid Learning module to jointly learn modality-specific and modality-invariant information. Comprehensive experiments conducted on two widely recognized benchmarks, SYSU-MM01 and RegDB,  demonstrate that RPNR outperforms the current state-of-the-art GUR with an average Rank-1 improvement of 10.3%. The source codes will be released soon.

Main paper: 
 introduction: 
 With the increasing demand for intelligent security, smart monitoring sensor devices for 24-hour surveillance are becoming more prevalent [21, 31, 34, 52, 58] . Due to the different imaging principles of sensor devices during the daytime and nighttime, the data exhibits multi-modality characteristics, sparking interest in research on visible-infrared person re-identification (VI-ReID). VI-ReID aims to accurately search the special visible/infrared pedestrian images when given a query pedestrian image from another modality, however, the significant gap between the two modalities presents a considerable challenge for this task. Recently, a series of VI-ReID methods [12, 13, 18, 48, 53, 60, 61] have aimed to reduce cross-modality discrepancies by aligning visible and infrared images at both the image and feature levels, resulting in significant performance improvements. However, these methods rely on well-annotated cross-modality data, which is time-consuming and labor-intensive in practical scenarios. Therefore, increasing attention is being drawn to unsupervised visible-infrared person re-identification (USVI-ReID).The key challenges of the USVI-ReID are obtaining robust pseudolabels and establishing reliable cross-modality correspondences. Existing USVI-ReID methods [4, 29, 45, 49] mostly follow the DCL [51] framework, which generates pseudo-labels using DBSCAN and establishes cross-modality correspondences based on pseudo-labels. Since pseudo-labels are generated by clustering, they inevitably contain noise. The noisy pseudo-labels may cause the model to incorrectly learn the data distributions and feature representations. To mitigate the effects of the noisy pseudo-labels, DPIS [32] computes the confidence scores of pseudo-labels by analyzing their classifier loss, then uses confidence scores to mitigate the impact of noisy pseudo-labels. PGM [45] reduces the impact of noisy labels by alternately using two unidirectional metric losses, preventing the rapid formation of noisy pseudo-labels. However, these methods don't calibrate noisy pseudo-labels to clear ones, which makes it difficult for the model to exploit hard-to-discriminate features. In order to establish cross-modality correspondences, OTLA [39] utilizes unsupervised domain adaptation to generate pseudo-labels for the infrared images. With the aid of richly annotated visible images, it proposes an optimal transport strategy to allocate pseudo-labels from the visible modality to the infrared modality. However, OTLA adopts the strategy of independently assigning pseudo-labels to each infrared image, which is a massive task with many distractors, leading to unreliable cross-modality correspondences.In this paper, we propose the Robust Pseudo-Label Learning with Neighbor Relation (RPNR) framework, a unified approach aimed at addressing noisy pseudo-labels and cross-modality correspondences for USVI-ReID. To be specific, to calibrate noisy pseudolabels, we design two critical modules: Noisy Pseudo-label Calibration (NPC) and Neighbor Relation Learning (NRL). Unlike previous methods that only reduce the effect of noisy pseudo-labels, NPC directly calibrates them. NPC obtains robust prototypes through reliable neighbor samples and calibrates pseudo-labels based on similarity to these prototypes. The significant intra-class variations will hinder the noisy pseudo-label calibration. NRL is proposed to reduce intra-class variations by interacting across all images. NRL promotes the model to cluster closely with neighbor samples, as neighbor samples are often related. In order to establish reliable cross-modality correspondences, we also design two critical modules: Optimal Transport Prototype Matching (OTPM) and Memory Hybrid Learning (MHL). Unlike OTLA, which overlooks the intraclass information and treats all images as separate instances for establishing cross-modality correspondences, OTPM employs intraclass information to build cross-modality correspondences. In brief, OTPM obtains the prototype by clustering and establishes crossmodality correspondences based on these prototypes instead of all instances. Furthermore, the significant cross-modality gaps will hinder the establishment of cross-modality correspondences. MHL is designed to learn both modality-specific and modality-invariant information by blending two modality-specific memories, effectively bridging the substantial gaps between different modalities.In conclusion, the main contributions of our method can be summarized as follows:• We propose the Robust Pseudo-Label Learning with Neighbor Relation (RPNR) framework to address both noisy pseudolabels and noisy cross-modality correspondence problems in USVI-ReID.  
related work 2.1 unsupervised single-modality person reid: 
 Unsupervised single-modality person ReID aims to learn discriminative identity features from unlabeled person ReID datasets. Existing mainstream purely unsupervised methods primarily rely on pseudo-labels, which involve an iterative process alternating between pseudo-label generation and representation learning [6, 9, 10, 14, 24, 36, 37, 40, 41, 63, 67] . Cluster-Contrst [8, 20] presents a cluster contrast framework, which stores unique centroid representations and performs contrastive learning at the cluster level. Additionally, the momentum update strategy is introduced to reinforce the cluster-level feature consistency in the embedding space. However, a uni-proxy for a cluster may introduce bias. To address this issue, multi-proxies methods [44, 66] have been proposed to compensate for the shortcomings of uni-proxy approaches. Pseudolabels inherently contain a portion of noise. To address this issue, label refinement methods [5, 6, 62] are proposed to collect more reliable pseudo-labels. While the aforementioned methods have shown promising results in unsupervised ReID tasks, applying them directly to unsupervised VI-ReID scenarios poses a significant challenge due to the substantial cross-modality gap.There has been an escalating interest in unsupervised visible-infrared person re-identification (USVI-ReID) owing to its potential to learn modality-specific and modality-invariant information without necessitating cross-modality annotations. Existing mainstream USVI-ReID methods [4, 17, 28, 30, 45, 46, 49, 50] predominantly adhere to the DCL [51] learning framework, which involves two key steps:(1) generating pseudo-labels using a clustering algorithm, and (2) establishing cross-modality correspondences based on these pseudolabels. PGM [45] and MBCCM [17] perform multi-stage graph After that, RPNR calibrates noisy pseudo-labels (grey dots) to obtain robust pseudo-labels (color dots) at stage (b), while modeling potential interactions between all samples (the strength is indicated by thickness) to reduce intra-class variation at stage (c). Additionally, based on the robust pseudo-labels, RPNR employs the optimal transport to establish cross-modality correspondences at stage (d). Finally, with the recast-aligned pseudo-labels, RPNR mixes two modality-specific memories as a new hybrid memory to learn both modality-specific and modality-invariant information through contrastive loss at stage (e).matching via building bipartite graphs. OTLA [39] and DOTLA [4] employ the Optimal Transport strategy to assign pseudo-labels from one modality to another modality at the instance level. However, pseudo-labels inevitably contain noise, which may lead to unreliable cross-modality correspondences under the supervision of noisy pseudo-labels. Therefore, there is a need to seek more reliable pseudo-labels for the USVI-ReID task.The presence of label noise has been demonstrated to adversely affect the training of deep neural networks [35, 54, 55] . Existing methods devised for handling noisy labels can be primarily categorized into the following two classes: label correction and sample selection. Label correction methods [2, 33, 38, 65] endeavor to utilize the model predictions to rectify the noisy labels. [16] proposes an iterative learning framework SMP to relabel the noisy samples and train the network on the real noisy dataset without using extra clean supervision. [59] utilizes back-propagation to probabilistically update and correct image labels beyond updating the network parameters. Different from label correction methods, sample selection methods [15, 23, 42] aim to select clean samples while discarding noisy samples during the training stage. NCE [22] filters the clean samples according to the neighbor information. CBS [25] proposes to employ confidence-based sample augmentation to enhance the reliability of selected clean samples. For the USVI-ReID task, pseudo-labels generated by the clustering algorithm inevitably contain noise. Therefore, calibrating these noisy pseudo-labels is crucial for improving the performance of USVI-ReID. 
methodology 3.1 notation definition: 
 Given an unlabeled visible-infrared person re-identification dataset D = {D V , D R }, where D V = {𝑥 𝑣 𝑖 | 𝑖 = 1,The overall framework of the proposed method is shown in Fig. 1 .We first employ the DBSCAN [11] algorithm to cluster visible and infrared features. After clustering, we can obtain pseudo-labels 𝑦 𝑡 𝑖 ∈ {1, 2, . . . , 𝑌 𝑡 } of 𝑖-th images from modality 𝑡, where 𝑌 𝑡 is the number of clusters and 𝑡 = {𝑣, 𝑟 }. Since pseudo-labels inevitably contain noise, we first propose a Noisy Pseudo-label Calibration (NPC) module to calibrate noisy labels to obtain more robust pseudolabels. Afterward, we assign these calibrated pseudo-labels ŷ𝑡 𝑖 for each sample to obtain the "labeled" dataset D𝑉 = {(𝑥 𝑣 𝑖 , ŷ𝑣 𝑖 )} 𝑁 𝑣and D𝑅 = {(𝑥 𝑟 𝑖 , ŷ𝑟 𝑖 )} 𝑁 𝑟 𝑖=1 . However, NPC overlooks the possible interactions between all samples. To make up for this deficiency, we propose a Neighbor Relation Learning (NRL) module, which is designed to model the intricate interactions spanning across all samples. Furthermore, the pseudo-labels generated by two separate clustering for visible and infrared samples reveal a misalignment. To align correspondences between visible and infrared samples, we design an Optimal Transport Prototype Matching (OTPM) module, which considers cross-modality correspondences as alignment between visible and infrared prototypes by optimal transport. Learning modality-invariant features is crucial in cross-modality correspondences. To better mine the modality-invariant information and alleviate significant cross-modality gaps, we propose a Memory Hybrid Learning (MHL) module, which mixes aligned visible and infrared prototypes as new modality-hybrid prototypes for contrastive learning.Since pseudo-labels are generated by clustering, they inevitably contain noise. We introduce the Noisy Pseudo-label Calibration (NPC) module to correct noisy pseudo-labels. Specifically, given the 𝑐-th cluster from modality 𝑡, corresponding to a set of 𝑑-dimensional features {𝑓 𝑡 𝑐,𝑖 } 𝑛 𝑐 𝑖=1 , where 𝑛 𝑐 denotes the number of features belonging to 𝑐-th cluster and 𝑡 ∈ {𝑣, 𝑟 }. We employ the Jaccard Similarity to model the affinity matrix S of intra-class samples as follows:EQUATIONwhere 𝑆 𝑡 𝑖 𝑗 is the affinity between 𝑓 𝑡 𝑐,𝑖 and 𝑓 𝑡 𝑐,𝑗 , and R 𝑓 𝑡 𝑐,𝑖 , 𝜅 is the 𝜅-reciprocal nearest neighbors of 𝑓 𝑡 𝑐,𝑖 . The larger 𝑆 𝑡 𝑖 𝑗 , the higher similarity between 𝑓 𝑡 𝑐,𝑖 and 𝑓 𝑡 𝑐,𝑗 . For a specific 𝑓 𝑡 𝑐,𝑖 , if it is surrounded by more similar samples, the sample is more likely to be reliable. To select reliable samples for a cluster, we design a Similarity Counter 𝐺 𝑡 𝑐 for each sample:EQUATIONwhere 𝑠𝑖𝑔𝑛(•) is a 𝑠𝑖𝑔𝑛 function and 𝜌 is a threshold fixed to 0.5. We can find that correctly categorized samples should have higher similarity counts, so we regard the samples with the top-𝐾 similarity counts as reliable samples:EQUATIONwhere 𝑖𝑑 denotes the indexes of top-𝐾 similarity counts.Then we can obtain a robust prototype with these reliable samples for the 𝑐-th cluster:EQUATIONAfter that, we can own a prototype set 𝑝 𝑡 = {𝑝 𝑡 1 , 𝑝 𝑡 2 , . . . , 𝑝 𝑡 𝑌 𝑡 }. For a given sample 𝑥 𝑡 𝑖 from D 𝑡 , the similarity score 𝛿 𝑡 𝑐,𝑖 between the extracted feature 𝑓 𝑡 𝑖 and the 𝑐-th cluster is calculated as follows:EQUATIONwhere 𝛿 𝑡 𝑐,𝑖 denotes the cosine similarity between the extracted feature 𝑓 𝑡 𝑖 and the 𝑐-th cluster. Larger 𝛿 𝑡 𝑐,𝑖 indicates the sample 𝑥 𝑡 𝑖 is more likely to belong to the 𝑐-th cluster. Then, we can obtain the corrected pseudo-label by:EQUATIONAfterward, we assign these corrected labels for each sample for network training.Considering high intra-class variability profoundly hampers the NPC module, we propose the Neighbor Relation Learning (NRL) module, which is designed to reduce intra-class variability through the intricate interactions spanning across all pair-wise samples. Following [19] , we employ Relaxed Contrastive Loss for learning semantic embedding of pair-wise samples. For convenience, we only explain the process of visible samples. Given a pair of samples (𝑓 𝑣 𝑖 , 𝑓 𝑣 𝑗 ), we compute the Euclidean distance between them by:EQUATIONThen, the visible loss of the NRL module can be formulated:EQUATIONwhere 𝑁 𝐵 denotes the number of samples in each iteration and 𝛾 is a margin hyper-parameter.[𝑥] + denotes max(0, 𝑥), which is a hinge function. Moreover, 𝜔 𝑣 𝑖 𝑗 is the weight term, formulated by a Gaussian kernel function based on the Euclidean distance:EQUATIONwhere 𝜎 represents the kernel bandwidth and 𝜔 𝑣 𝑖 𝑗 ∈ (0, 1]. Obviously, it can be used to measure the similarity relation of paired samples in the embedding space.As shown in Eq. ( 8), the NRL loss contains an attracting term and a repelling term. The positive pairs will approach each other with the help of the attracting term and the repelling term encourages the negative pairs to push away beyond the margin 𝛾.Similarly, the NRL loss of the infrared modality is defined as:𝐿 𝑟 𝑁 𝑅𝐿 = 1 𝑁 𝐵 𝑁 𝐵 ∑︁ 𝑖=1 𝑁 𝐵 ∑︁ 𝑗=1 𝜔 𝑟 𝑖 𝑗 𝑑 𝑟 𝑖 𝑗 2 attracting + 1 𝑁 𝐵 𝑁 𝐵 ∑︁ 𝑖=1 𝑁 𝐵 ∑︁ 𝑗=1 (1 -𝜔 𝑟 𝑖 𝑗 ) [𝛾 -𝑑 𝑟 𝑖 𝑗 ] 2 + repelling .(10) Therefore, the total loss of the NRL module is:EQUATIONThe two modules mentioned above primarily concentrate on intramodality information while overlooking inter-modality connections, which are pivotal in the USVI-ReID task. To this end, following PGM [45] and OTLA [39] , we present the Optimal Transport Prototype Matching (OTPM) module to establish reliable crossmodality correspondences at the cluster level. Given the visible prototype set 𝑝 𝑣 = {𝑝 𝑣 1 , 𝑝 𝑣 2 , . . . , 𝑝 𝑣 𝑌 𝑣 } and the infrared prototype set 𝑝 𝑟 = {𝑝 𝑟 1 , 𝑝 𝑟 2 , . . . , 𝑝 𝑟 𝑌 𝑟 }, where 𝑌 𝑣 and 𝑌 𝑟 represent the number of visible clusters and infrared clusters, respectively. PGM revealed 𝑌 𝑣 > 𝑌 𝑟 , that is, the number of clusters is inconsistent. In that case, the essence of cross-modality correspondences is the many-tomany matching of inter-modality prototypes, which can be solved by Optimal Transport:EQUATIONwhere 𝑄 ∈ R 𝑌 𝑣 ×𝑌 𝑟 represents the transport plan for cross-modality matching. 𝐶 ∈ R 𝑌 𝑣 ×𝑌 𝑟 is the cost matrix of inter-modality prototypes, i.e., 𝐶 𝑖 𝑗 = 1/exp cos 𝑝 𝑣 𝑖 , 𝑝 𝑟 𝑗 , where cos (•) indicates the cosine similarity. ⟨•⟩ denotes the Frobenius dot-product, and 1 is an all-one vector. H (𝑄) denotes the Entropic Regularization and 𝜆 is a regularization parameter. The objective function can be solved by the Sinkhorn-Knopp algorithm [7] and derive the optimal transport plan 𝑄 * ∈ R 𝑌 𝑣 ×𝑌 𝑟 . Then we can obtain two matched pseudo-label sets 𝑌 𝑣→𝑟 and 𝑌 𝑟 →𝑣 for network training according to 𝑄 * :EQUATIONWe initialize two modality-specific memory banks M 𝑣 ∈ R 𝑌 𝑣 ×𝑑 and M 𝑟 ∈ R 𝑌 𝑟 ×𝑑 by clustering centroids. However, the two modalityspecific memories only store the modality-specific information, which can't mine modality-invariant information and reduce the cross-modality discrepancy. To this end, with the cross-modality correspondences derived from the OTPM, we propose a Memory Hybrid Learning (MHL) module to jointly learn modality-specific information and modality-invariant information. Firstly, we create a modality-hybrid memory M ℎ ∈ R 𝑌 𝑟 ×𝑑 to store modality-shared information via mixing matched visible and infrared prototypes by:EQUATIONwhere 𝑖 ∈ {1, 2, . . . , 𝑌 𝑟 } and 𝑝 𝑟 →𝑣 𝑖 denotes the visible prototype which matches with the infrared prototype 𝑝 𝑟 𝑖 . 𝛼 is a balancing hyper-parameter that balances the fusion information of the visible and infrared prototypes.Afterward, during the representation learning stage, we follow the popular memory-based methods [4, 17, 45, 49, 51] , which mainly alternate two stages: (1) performing contrastive learning during forward-propagation (FP) and ( 2) updating the memory during backward-propagation (BP). To better learn the representations, we perform multi-memory joint contrastive learning, which consists of modality-specific contrastive learning and modality-invariant contrastive learning. Modality-Specific Contrastive Learning. Based on the modalityspecific memory M 𝑣 and M 𝑟 , the ClusterNCE [8] loss is applied to learn modality-specific information for network optimization by:EQUATION)EQUATIONwhere 𝑁EQUATION)During the backward-propagation stage, the two modality-specific memories are updated by a momentum update strategy:EQUATIONEQUATION)where 𝜇 is the momentum updating factor. Modality-Invariant Contrastive Learning. Different from two modality-specific memories, we perform modality-invariant contrastive learning on modality-shared memory M ℎ to learn modalityinvariant information while reducing the cross-modality discrepancy. Following PGM [45] , we employ the alternate contrastive learning scheme on M ℎ :EQUATIONEQUATIONwhere ŷ𝑣→𝑟 𝑖 denotes the visible pseudo-label ŷ𝑣 𝑖 matched with the infrared pseudo-label ŷ𝑟 𝑖 . Then, the modality-shared memory is updated jointly by query features 𝑓 𝑣 𝑖 and 𝑓 𝑟 𝑖 : The total loss of the MHL module is:EQUATION)EQUATIONThe total training loss of the network can be formulated as follows:EQUATIONwhere 𝛽 1 , 𝛽 2 are balancing coefficients, which are set to 0.5 and 10.0, respectively. 
experiment 4.1 experiment setting: 
 Datasets. The proposed method is evaluated on two popular visibleinfrared person re-identification datasets: SYSU-MM01 [43] and RegDB [26] . More detailed explanations are presented in Supplementary Materials. Evaluation Metrics. The experiment of our method was carried out following the evaluation metrics in DDAG [57] , i.e., Cumulative Matching Characteristic (CMC) and Mean Average Precision (mAP). In the evaluation of our proposed method on the SYSU-MM01 dataset, we consider two distinct search modes: the All Search mode and the Indoor Search mode. Similarly, for the RegDB dataset, our method is evaluated across two testing modes: Visi-ble2Thermal and Thermal2Visible.Implementation Details. The proposed method is implemented on two TITAN RTX GPUs with PyTorch. During the training stage, all the input images are resized to 288×144, and data augmentations described in [56] are adopted for image augmentation. Following [58] , we employ a two-stream feature extractor pre-trained on ImageNet to extract 2048-dimensional features. The number of training epochs is set to 100. The detailed settings are presented in Supplementary Materials.Comparison with SVI-ReID Methods. Compared to SVI-ReID methods that rely on high-quality cross-modality annotations, the results of our RPNR are promising. As we can see, our method achieves comparable performance to some supervised methods (e.g., DDAG [57] , AGW [58] , and CAJ [56] ), which is attributed to the fact that our proposed method can provide reliable pseudolabels for unsupervised tasks.Comparison with SSVI-ReID Methods. Several SSVI-ReID methods have been proposed to mitigate the issue of the high cost of cross-modality annotations. These methods utilize partial annotations to accomplish the VI-ReID task. It is noteworthy that our approach, without any cross-modality annotations, achieves a 6.8% improvement in Rank-1 and a 4.4% improvement in mAP on the SYSU-MM01 dataset compared to the SOTA DPIS method. To validate the effectiveness of each module in RPNR, we conduct ablation experiments on SYSU-MM01, as shown in Tab. 2. We employ the DCL framework with multiple proxies as the baseline.Effectiveness of the NPC Module. The NPC module is proposed to explicitly rectify noisy pseudo-labels to obtain more reliable pseudo-labels. As shown in Order 5 and Order 6 in Tab. 2, the performance of Order 6 with NPC improves by about 2% compared to Order 5. To more clearly demonstrate the effectiveness of the NPC module, we utilize the Adjusted Rand Index (ARI) metric to evaluate the accuracy of visible and infrared pseudo-labels on SYSU-MM01 at each epoch. A higher ARI value indicates more accurate pseudo-labels. As depicted in Fig. 2 , the introduction of the NPC module results in improved accuracies for both visible and infrared pseudo-labels, thereby providing more reliable pseudo-labels for network training. Effectiveness of the NRL Module. The NRL module is introduced as complementary information to make up for the shortcomings of rigid pseudo-labels. After adding the NRL module, the performance can gain improvement by 2%-4% in Rank-1 on SYSU-MM01. It shows that the NRL module can explore meaningful intricate interactions spanning across all pair-wise samples to provide complementary supervision information for the network. Effectiveness of the OTPM Module. As shown in Fig. 3 , we compared the cross-modality matching accuracy of OTPM with that of PGM on four clustering evaluation metrics to show the effectiveness of OTPM. As we can see, OTPM significantly outperforms the PGM on all four metrics, indicating its superior ability to establish reliable cross-modality correspondences at the cluster level. Effectiveness of the MHL Module. We present the MHL module to jointly learn modality-specific and modality-invariant information while reducing cross-modality discrepancies. Note that the MHL module cannot be executed on its own, as it is built on top of the OTPM module. Compared to the Baseline, the combination of MHL with OTPM leads to a significant performance improvement, with a large margin of 19.8% in Rank-1 accuracy and 16.8% in mAP (See Order 1 & Order 5). This highlights the efficiency of MHL in leveraging modality-specific and modality-invariant information, effectively mitigating cross-modality discrepancies.Hyper-parameter Analysis. There are three key hyper-parameters in our method, and we give the quantitative results to evaluate their influence with different values in Fig. 4 . As we can see, the best performance is achieved when 𝛼 is set to 0.5, 𝛽 1 is set to 0.5, and 𝛽 2 is set to 10.0, respectively. Accuracy of Pseudo-labels. As shown in Fig. 5 , we compared our method with several SOTA USVI-ReID methods on four common clustering evaluation metrics to show the effectiveness of the proposed RPNR. The results show that the visible and infrared pseudo-labels generated by RPNR significantly outperform existing methods on all four clustering metrics, indicating that our method provides more reliable pseudo-labels for network training, thereby boosting performance improvement. Visualization Analysis. We visualize the visible and infrared feature distribution with t-SNE in the 2-D embedding space, which contains 10 randomly selected identities. As shown in Fig. 6 , compared to the Baseline, in our approach, the feature distributions of the same identities from the same modality are more compact (see orange and purple circles), and the feature distributions of the same identities from different modalities are also closer (see red and blue circles). This indicates that RPNR significantly reduces cross-modality disparities and establishes a solid foundation for reliable cross-modality correspondences. 
conclusion: 
 In this paper, we introduce an effective approach for addressing the USVI-ReID task, termed Robust Pseudo-label Learning with Neighbor Relation (RPNR). Our goal is to explore more reliable pseudo-labels and establish more dependable cross-modality correspondences for the USVI-ReID task. To this end, we first employ the Noisy Pseudo-label Calibration module to rectify noisy pseudolabels, thereby obtaining more reliable pseudo-labels. Subsequently, we present the Neighbor Relation Learning module to model the potential interactions between different samples. In addition, we introduce the Optimal Transport Prototype Matching module to establish dependable cross-modality correspondences at the cluster level. Finally, we propose the Memory Hybrid Learning module to mine modality-specific and modality-invariant information while mitigating significant cross-modality disparities. Comprehensive experimental results on two popular benchmarks demonstrate the effectiveness of the proposed method. 
 
 Appendix:  


Decision: All reviewers agree on the superior and comprehensive performance of the proposed in this paper. Other merits include satisfying writing and idea novelty. In addition, the rebuttal has well addressed concerns from reviewers. After rebuttal, all reviewers have made positive recommendation of this paper towards acceptance.

Reviews: review: paper_summary: paper_summary: This work is aimed at learning robust features for unsupervised visible-infrared person re-identification (USVI-ReID). To this end, the authors design robust pseudo-label generation, intr- and inter-modality feature learning methods, improving the overall learning procedure of USVI-ReID.
strength_weakness: strength_weakness: The proposed method mainly improves the training procedure without designing extra trainable modules and adding no extra cost to the inference. Both the generation of pseudo-labels and the learning of robust multi-modal person features are carefully designed. The overall USVI-ReID performance is shown to be superior. Analytical experiments are conducted to help understand the effectiveness of the proposed method.
limitations: limitations: 1. While the issue to be solved in this work is claimed to be "calibrate noisy pseudo-labels usually associated with hard samples" (as in lines 15-17), the proposed Noisy Pseudo-label Calibration method in fact re-generates the integral pseudo-labels rather than specifically refining those of the hard samples. Neither specific experiments nor in-depth discussions are presented to demonstrate the effectiveness of the proposed NPC on those hard samples.

2. Why is the Jaccard Similarity necessary in NPC? Is it practical to employ other similarity metrics, e.g. cosine similarity?

3. Basically, Neighbor Relation Learning is to employ a previously proposed metric loss for intra-modality feature learning. How about other simpler losses, e.g. triplet loss? Any discussion about the innovation/difference of NRL compared with previous works?

4. OTPM is said to be "following PGM and OTLA", while this module is claimed to be a contribution, the differences between OTPM and PGM/OTLA are not discussed. Although Figure 3 shows the superior accuracy of cross-modality correspondence compared with PGM, more technical details should be given to see if the improvement comes from the better-aligned features rather than the cross-modal matching algorithm.
paper_summary: paper_summary: This paper proposes an unsupervised RGB-IR person ReID. It introduces neighbor relation-based pseudo-label calibration to get robust pseudo-labels and reduce intra-class variations, then applies optimal transport matching to build cross-modal correspondence. Moreover, it proposes a memory hybrid module to learn modality-specific/invariant information. Experiment results indicates the superior of the proposed method.
strength_weakness: strength_weakness: 1. The paper overall is easy to follow, and the proposed method is presented in detail.
2. This paper considers both the noisy pseudo-labels and cross-modal association problems presented in the USVI-ReID task, which is interesting and realistic.
3. Comprehensive experimental results show that the proposed method surpasses the state-of-the-art methods and gains promising performances.
limitations: limitations: 1. The experiment on LLCM [R1] should be included.
2. The whole method looks similar to PGM, OTLA, and SPCL.  What is the key difference between the proposed method and the aforementioned methods?

[R1] Diverse Embedding Expansion Network and Low-Light Cross-Modality Benchmark for Visible-Infrared Person Re-identification， CVPR 2023
paper_summary: paper_summary: This paper centers on addressing the unsupervised VI-ReID problem. It introduces a Noisy Pseudo-label Calibration (NPC) method aimed at rectifying noisy intra-modality labels based on the relationships between instances and prototypes. Additionally, a Neighbor Relation Learning (NRL) module leverages pairwise instance-level relationships to enhance intra-modality learning. An Optimal Transport Prototype Matching (OTPM) module is proposed to establish cluster-level associations by formulating an optimal transport problem. Furthermore, a Memory Hybrid Learning (MHL) module is introduced to perform both modality-specific and modality-agnostic contrastive learning, facilitating the acquisition of modality-invariant features.
strength_weakness: strength_weakness: (1) The proposed method achieves remarkable performance on two public datasets, surpassing the state-of-the-art approaches by a considerable margin.\
(2) The sufficient experiments demonstrate the effectiveness of each component of the method.
limitations: limitations: (1) The OTPM module primarily addresses the inconsistency in the number of clusters across different modalities in PGM [1]. While [2,3] propose bi-directional strategies and [4, 5] directly transfer infrared labels to visible instances to tackle this issue, the superiority of OTPM over these methods is not clearly elucidated and demonstrated compared to the above methods.\
(2) The hybrid memory strategy is widely employed in existing methods [1,2,4,5], and the loss terms in Eq. 22 appear to stem from the MSMA framework [2]. The authors are suggested to analyze the differences between MHL and MSMA.\
(3) The NPC and NRL modules focus on addressing intra-modality noisy labels, offering limited insights into cross-modality learning.\
(4) The proposed method integrates existing technology (ACCL [1]); however, the performance enhancement attributed to ACCL is not delineated in the paper.

[1] Wu Z, Ye M. Unsupervised visible-infrared person re-identification via progressive graph matching and alternate learning[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023: 9548-9558.\
[2] Cheng D, He L, Wang N, et al. Efficient bilateral cross-modality cluster matching for unsupervised visible-infrared person reid[C]//Proceedings of the 31st ACM International Conference on Multimedia. 2023: 1325-1333.\
[3] Cheng D, Huang X, Wang N, et al. Unsupervised visible-infrared person reid by collaborative learning with neighbor-guided label refinement[C]//Proceedings of the 31st ACM International Conference on Multimedia. 2023: 7085-7093.\
[4] Yang B, Chen J, Ye M. Towards grand unified representation learning for unsupervised visible-infrared person re-identification[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023: 11069-11079.\
[5] Yang B, Chen J, Chen C, et al. Dual Consistency-Constrained Learning for Unsupervised Visible-Infrared Person Re-Identification[J]. IEEE Transactions on Information Forensics and Security, 2023.