Learning Commonality, Divergence and Variety for Unsupervised Visible-Infrared Person Re-identification
Authors: Jiangming Shi
Xiangbo Yin
Yachao Zhang
zhizhong zhang
Yuan Xie
Yanyun Qu
Venue: NeurIPS 2024 poster (2024)
Keywords: VI-ReID
Multi-Prototype
Progressive Contrastive Learning
Primary area: machine_vision

Abstract: Unsupervised visible-infrared person re-identification (USVI-ReID) aims to match specified persons in infrared images to visible images without annotations, and vice versa. USVI-ReID is a challenging yet underexplored task. Most existing methods address the USVI-ReID through cluster-based contrastive learning, which simply employs the cluster center to represent an individual. However, the cluster center primarily focuses on commonality, overlooking divergence and variety. To address the problem, we propose a Progressive Contrastive Learning with Hard and Dynamic Prototypes for USVI-ReID. In brief, we generate the hard prototype by selecting the sample with the maximum distance from the cluster center. We reveal that the inclusion of the hard prototype in contrastive loss helps to emphasize divergence. Additionally, instead of rigidly aligning query images to a specific prototype, we generate the dynamic prototype by randomly picking samples within a cluster. The dynamic prototype is used to encourage variety. Finally, we introduce a progressive learning strategy to gradually shift the model's attention towards divergence and variety, avoiding cluster deterioration. Extensive experiments conducted on the publicly available SYSU-MM01 and RegDB datasets validate the effectiveness of the proposed method.

Learning Commonality, Divergence and Variety for
Unsupervised Visible-Infrared Person Re-identification
Jiangming 3, Xiangbo Yin Yachao Zhang?, Zhizhong Zhang4.5
Yuan X4e Yanyun Qu 1,27
Institute of Artificial Intelligence, Xiamen University
School of Informatics, Xiamen University
Shanghai Innovation Institute
East China Normal University
Shanghai Key Laboratory of Computer Software Evaluating and Testing
j jiangming shi shi@outlook. com yxie@cs ecnu. edu, yyquεxmu. edu, cn Code: https: : com/ shtp//siiiiie haag://g/iiggm
Abstract
Unsupervised visible-infrared person re-identification (USVI-ReID) aims to match specified persons in infrared images to visible images without annotations, and vice versa. USVI-ReID methods address the USVI-ReID through cluster-based contrastive learning, which simply employs the cluster center to represent an individual. However, cluster center primarily focuses commonality, overlooking divergence and variety. To address the problem, we propose a Progressive Contrastive Learning with Hard and Dynamic Prototypes for USVI-ReID. In brief, we generate the hard prototype by selecting the sample with the maximum distance from the cluster center. We reveal that the inclusion of the hard prototype in contrastive loss helps emphasize divergence. Additionally, instead of rigidly aligning query images to a specific prototype, generate the dynamic prototype by randomly picking samples within a cluster. The dynamic prototype is used to encourage variety. Finally, we introduce progressive learning strategy to gradually shift the model's attention towards divergence and variety, avoiding cluster deterioration. Extensive experiments conducted on the publicly available SYSU-MM0I and RegDB datasets validate the effectiveness of the proposed method.
challenging yet underexplored task. Most existing
Introduction
person re-identification (VI-ReID) aims at matching the same person captured in
one modality with their counterparts in another modality [T3 It has recently gained attention in computer vision applications like video surveillance 4 and image retrieval [547]. With the
development of deep learning [T VI-RelI has achieved remarkable advancements [4]
However, the development of existing VI-ReID methods is still limited due to the requirement for expensive-anmen andoottte training data [5 16] To mitigate the problem of annotating large-scale scale cross-modality data, some semi-supervised VI-ReID methods [17119 proposed to learn modality- invariant and identity-related discriminative representations by utilizing both labeled and unlabeled data. For this purpose, OTLA [17] proposed an optimal transport label assignment mechanism to assign pscudo-labels for unlabeled infrared images while ignoring how to calibrate noise pscudo- labels. DPIS [18 integrates two pseudo- labels generated by distinct models into hybrid pseudo-label
"Equal contribution. Corresponding author.
38th Conference on Neural Information Processing Systems (NcurIPS 2024), Vancouver.

---

for unlabeled infrared data, but it makes the training process more complex, Although these methods have gained promising performances, they still rely on a certain number of manual-labeled data.
Several USVI-ReID methods [2H23] have proposed to tackle the issues of expensive visible-infrared annotation through contrastive learning. These methods create two modality-specifie memories, one for visible features and the other for infrared features. During training, these methods consider the memory center as a prototype and minimize the contrastive loss across the features of query images and prototype. Then, these methods aggregate the corresponding prototypes based on similarity. However, the centroid prototype only stores the commonality of each person, neglecting the divergence [24 26], which causes the pseudo-labels generated by the cluster to be unreliable. Just like a normal distribution, to better reflect the data distribution of dataset, we need not only the mean but also the variance.
In this paper, argue that important aspect of contrastive learning for USVI-ReID, i.e. the design of the prototype, has so far been neglected, and propose progressive contrastive learning with hard and dynamic prototype (PCLHD) method for the USVI-ReID. Firstly, we design a Hard Prototype Contrastive Learning (HPCL) to mine divergent yet meaningful information. In contrast to traditional contrastive learning methods, we choose the hard samples to serve as the hard prototype. In other words, the hard prototype is the one that is farthest from the memory center. The hard prototype encompasses distinctive information. Furthermore, we introduce the concept of Dynamic Prototype Contrastive Learning (DPCL), we randomly select samples from each cluster to serve as the dynamic prototype. DPCL effectively accounts for the intrinsic variety within clusters, enhancing the model's adaptability to varying data distributions. Early clustering results are unreliable, and utilizing hard and dynamic prototype at this stage may lead cluster degradation. Therefore, we introduce progressive contrastive learning to gradually focus on divergence and variety.
The main contributions are summarized as follows:
We propose progressive contrastive learning with hard and dynamic prototype method for the USVI-ReID. We reconsider the design of prototypes in contrastive learning to ensure that the model stably captures commonality, divergence, and variety.
We propose Hard Prototype Contrastive Learning for mining divergent yet significant information, and Dynamic Prototype Contrastive Learning for preserving the intrinsic variety in sample features.
Experiments SYSU-MM01 and RegDB datasets demonstrate the superiority of our method compared to existing USVI-ReID methods, and PCLHD generates higher-quality pseudo-labels than other methods.
2 Related Work
2.1 Supervised Visible-Infrared Person RelD
Visible-infrared person re-identification (VI-RelD) has drawn much attention in recent years 22-32] Many VI-RelD methods focused on mitigating huge semantic gaps across modalities have made advanced progress, which can be classified into two primary classes based on their different aligning ways: image-level alignment and feature-level alignment. The image-level alignment methods focus on reducing cross-modality gaps by modality translation. Some GAN-based methods [33 34]
are proposed to perform style transformation for aligning cross- modality images. However, the generated images unavoidably contain noise. Therefore. X- X-modality [35] and its promotions [36 (37
align cross- modaaity images by introducing a middle modality. Mainstream feature-level alignment methods 40) focus minimizing cross-modality gaps by finding modality-shared feature space. However, the advanced performances of the above methods build on large-scale human- labeled cross-modality data, which are quite time-consuming and expensive, thus hindering the fast application of these methods in real-scenes.
2.2 Unsupervised Single-Modality Person ReID
The existing unsupervised single-modality person ReID methods can be roughly divided into two classes: Unsupervised domain adaption (UDA) methods, which try to leverage the knowledge transferred from labeled source domain to improve performance 4144,, and fully unsupervised

---

Memory Sumples >
Query Sample <>> Pull <->> Push + Centriod Prototype X Hard Prototype E Dynamic Protolype
Online Backbon:
Clustering
Stige-
Sage
OL
Castnod Protetpe
Har Proceppe
Clustering
Dackbone
Unlabeled Dataset
Memory
Progressive Contrastive Learning
Figure I: Framework of our PCLHD. The framework consists of two stages: the first stage employs contrastive learning with centroid prototypes to learn well-discriminative representation, and the second stage introduces contrastive learning with hard and dynamic prototypes to further focus on divergence and variety.
methods (USL), which directly train a USL-ReID model on the unlabeled target domain [21] 45 Compared with the UDA methods, the USL methods are more challenging. Recently, cluster- contrast learning [46 has achieved impressive performance by performing contrastive learning at the cluster level. However, cluster-contrast with uni-proxy can be biased and confusing, which fails to accurately describe the information of a cluster. To this end, the methods [47 48] proposed maintaining multi-proxies for a cluster to adaptively capture different information within the cluster. The above methods are mainly proposed to solve the single-modality ReID task, but they are limited to solving the USL-VI-ReID task due to large cross-modality gaps.
2.3 Unsupervised Visible-Infrared Person RelD
Unsupervised visible-infrared person ReID (USVI-ReID) has attracted much attention due the advantage not relying on any data annotation. Some UDA methods (49 17 use a well-annotated labeled source domain for pre-training solve the USVI-ReID task. Some fully unsupervised adopt contrastive learning to boost performance, which mainly follow two-step loop paradigm: generating pseudo-labels using the DBSCAN algorithm [50] to create memory banks with clustering centers and establishing cross-modality correspondences based on these memory banks. However, pseudo-labels are often inaccurate and rigid. CCLNet [51] leverages the text information from CLIP to afford greater semantic monitoring insights to compensate for the rigidity of pseudo-labels. Moreover, reliable cross-modality correspondences are vital to USVI-RelD), thus PGM (33 proposes a progressive graph matching framework to establish more reliable cross- modality correspondences. However, cluster centers mainly present common information while lacking distinctive information, which results in ambiguous cross-modality correspondences when meeting hard samples (52 53).
methods [23 [2)
3 Method
3.1 Problem Formulation and Overview
Given a USVI-ReID dataset D (V, (V,R), where V - (V:), represents the visible images and R = 1,y denotes the infrared images. V, and Ry represent the set of images corresponding the i-th and j-th class. N. and N, denote the number of visible and infrared clusters, respectively. In the USVI-ReID task, the purpose is to train a deep neural network to obtain modality-invariant and identity-relatee features for matching pedestrian images with the same identity.
We propose a Progressive Contrastive Learning with Hard and Dynamic Prototype (PCLHD) method for USVI-ReID, which mainly contains online encoder, momentum encoder, and progressive con- trastive learning strategy with centroid prototype, hard prototype, and dynamic prototype, as shown in The online encoder is standard network, updated through back cekk-roppgation. The momentum
Fig. I

---

encoder mirrors the structure of the online encoder, updated through the weights of the online encoder. The clustering is used generate pseudo labels for creating cluster-aware memory, and we employ DBSCAN for clustering. PCLHD primarily focuses on representation learning, and we use PGM
23 to aggregate cross-modality memory.
3.2
Centroid Prototype Contrastive Learning
we use centroid prototype contrastive learning to optimize the online encoder in the first state, which includes memory initialization and optimization.
Following the USVI-ReID methods 22 54
Memory Initialization. Let @ be the online encoder that transforms the imput image to an embedding vector. At the beginning of each training epoch, all image features clustered by DBSCAN (50) and then each cluster's 's representations are stored in visible memory MRGB = =[emj, cm cm and infrared memory Mı cmly ",, cmly, as follows:
cm
(v());
V
VEV.
cmlj
do(r)
R2
FER,
where . denotes the number of instances belonging to specific cluster.
Optimization. During training, we update the two -ppeait memories by a momentum updating strategy [6]. We treat the memory center centroid prototype and optimize the feature extractor do using contrastive learning with the centroid prototype, computed as:
Σ COPOL
1 VeN.
log
exp (do(v) cm./ /T)
(3)
Nu
Σ exp (Øu(v) cm//77) JEN.
VEV.
1 Nr
exp (øn(r) cm, /T)
CepeI.
log
(4)
Rd
Σ exp (do(r) cm3/TT) JEN,
VeN,
re::
CCPCL = Copor. + CCPOL.
(5)
where cm, u(r) is the positive centroid prototype, denoting query and the prototype shares the same identity. The is a temperature hyper- prpeeeprrameter..
3.3 Hard Prototype Contrastive Learning
To ensure that the prototype effectively captures divergence within identity, devise novel hard prototype for contrastive learning, which is referred to as Hard Prototype Contrastive Learning. HPCL designed to provide comprehensive understanding of personal characteristics, which benefits its handling of hard samples [47]. We use the online encoder do to extract feature representations, and select samples that farthest from the memory center the hard prototype:
hm! arg max (v))(v) cm]] VvEV;
(6)
hm? arg max [o((T) cml)
VIER,
Theorem 1. The information entropy of hard sample prototypes is greater than the information entropy of centroid prototypes, thereby preserving greater divergence within the hard memory.
Given set of features (fi, f2,.... ..: JN.) for class The entropy H(em,) can be approximated by the entropy of the distribution of the sample means. Considering that em, a convex combination of the sample features ft. we have:

---

1 Σ
H (eme)) H
fi
(8)
No
By the convexity of entropy, we have:
N N
N. N
H
H(I)).
1=1
This inequality implies that the entropy of the centroid prototype generally lower due to the averaging effect, which reduces the divergence among the samples, leading to lower entropy. Given that hm,e is the sample with the maximum individual entropy among the set {ff. f2....... fN. follows that:
N₂ H(f)) Hcme)).
(hme)) Z
(10)
No
Then, we construct contrastive loss with the hard prototype to minimize the distance between the query and the positive hard prototype while maximizing their discrepancy all other cluster hard prototypes, as follows:
exp(volv) /T) exp (do(v) -tmm//)) JEN.
CHPCL No
log
(11)
Vi
sEN.
VEV.
exp (do(t) hmmf/T)
CHrCL =
log
(12)
Nr
R
Σ exp (do -mm,/77
sEN,
rER,
JEN,
CHPCL = Cireer. + CiPPLL
(13)
where hm, v(r) is the positive hard prototype representation and the r is temperature hyper- parameter.
Finally, update the two modality-specific memories with momentum-updating strategy: hm ahm", + -1- (v) e Vi hm,,, = admm", 1- + (1 - (r), Vr E R
(14)
(15)
where a is momentum coefficient that controls the update speed the memories. and 1 refer to the current and last iteration, respectively.
The hard prototype contrastive learning has two main advantages: For intra-class feature learning, it ensures that the learning process does not just focus on the shared characteristics within a cluster but also considers the diverse elements, which are often more informative. For inter-class feature learning, it is also beneficial for increasing the distances between different persons. In contrast, centroid prototypes tend average features, lacking diversity, which can affect the network's 's ability to extract discriminative features.
3.4 Dynamic Prototype Contrastive Learning
Inspired by MoCo 155 and DPM [56], we design dynamic prototype contrastive learning in order to preserve the intrinsic variety in sample features. DPCL comprises an online encoder Φu and a momentum encoder ém. The momentum encoder mirrors the structure of the online encoder, which updated by the accumulated weights the online encoder:
(16) where 3 is momentum coefficient that controls the update speed of the momentum encoder. / and I refer to the current and last iteration, respectively. The momentum encoder thm updated by the moving averaged weights, which are resistant to sudden fluctuations or noisy updates [55).
dim 3
4-1 + (1 -)))
5

---

We use the momentum encoder Om to extract feature representation and store them in visible memory RGMB dmzz. ... dm N, and infrared memory (= dmzy .... dm We randomly select M visible/infrared samples from each cluster, denoted as X and Xj.as aas follows:
F. = 2m (X;',,
(17)
Fj 6m (X)).
(18)
We select visible dynamic prototype dma from DMRGB- In the same cluster, we select the sample farthest from the query image the prototype. In different clusters, we choose the sample closest to the query image as the prototype:
arg max (m (v,) - f if リ yi
dm
(19)
arg min (V,) ffll if Øo + yı
VSFEFY
where y,, and y: represent the pseudo label of the query image and the dynamic prototype, respectively, ||-II denotes Euclidean norm. We obtain infrared prototype dmp through the same method.
The overall optimization goal of DPCL is as follows:
DPPCI.
1
Σ
log
exp (0m(v) dmm./ /T)
(20)
No ieN,
Σ exp (0m(v) dmz/51)
vEV.
JEN.
DPOL N,
1
Σ
log
exp (0m(r) dm///T) Σ exp (dm /T) JEN,
(21)
R
Ve
IER.
CDccl = DDCL + Crrcl
(22)
where dm, dm)(r) the positive dynamic prototype representation, ie.. the query image and dynamic prototype have the same identity.
v(r)
DPCL promotes flexible and adaptable learning process, aiming minimize discrepancies between samples and their respective dynamic prototypes, rather than rigidly aligning query images with fixed prototype.
3.5 Progressive Contrastive Learning
In the initial training phases, representations are generally of lower quality. Introducing hard samples this period could be counterproductive, potentially leading the model optimization in incorrect direction right from the start [47 (57]. To address this issue, we introduce the Progressive Contrastive Learning, which forms the overall loss function:
{{
LePCL if epoch & ECPCL
CPOLHD
(23)
XLHreL + (1))),rrCt, else
where À is the loss weight, ECPCI is hyper-parameter.
4 Experiment
We conduct extensive experiments to validate the superiority of our proposed method. First, we provide the detailed experiment setting, which contains datasets, evaluation protocols, and imple- mentation details. Then, we compare our method with many state-of-the-art VI-RelD methods and conduct ablation studies. addition, to better illustrate our method, we also exhibit further analysis. If not specified, we conduct analysis experiments on SYSU-MM01 in the all-search mode.

---

Table I: Comparisons with fate-of-the-asl th-thh-art methods SYSU-MM01 and RegDB, including SVI- ReID, SSVI-ReID and USVI-ReID methods. All methods are measured by Rank-1 (%) and mAP (%). GUR* denotes the results without camera information.
MMO1
RegDB
Settings
All Search
Indoor Search
Visible2
Thermal
Thermal Visible
Type
Method
Venue
Rank-1
mAP
Rank-1
mAP
Rank-1
mAP
Rank-1
mAP
DDAG
39
ECCV'20 20
54.8
53.0
61.0
68.0
69.4
63.5
68.1
61.8
AGW
TPAMI
54.2
70.5
65.9
NFS 59
CVPR 21
56.9
55.5
62.8
69.8
80.5
72.1
78.0
69.8
LbA 60
ICCV'21
55.4
54.1
58.5
66.3
74.2
67.6
72.4
65.5
CAJ 1
ICCV'21
66.9
76,3
80.4
79.1
84.8
77.8
MPANet 41
CVPR
70.6
68.2
76.7
81.0
80.9
82.8
80.7
DART 27
CVPR 22
68.7
66.3
72.5
78.2
83.6
75.7
82.0
73.8
FMCNet 38
CVPR 22
66.3
62.5
68.2
74.1
89.1
84.4
88.4
83.9
MID
AAA1'22
59.4
64.9
70.1
84.3
81.4
SVI-ReID
LUPI
ECCV
67.6
82.4
82.7
88.0
82.7
86.8
81.3
DEEN 63
CVPR 23
74.7
71.8
80.3
83.3
91.1
85.1
89.5
83.4
SGIEL
CVPR'25
77.1
72.3
82.1
83.0
92.2
86.6
91.1
85.2
PartMix
CVPR'25
81.5
82.3
84,9
82.5
CAL 65
ICCV'23
74.7
71.7
79.7
83.7
94.5
88.7
93.6
87.6
MUN 66
ICCV'23
76.2
73.8
79.4
82.1
95.2
87.2
91.9
85.0
SAAI
ICCV'23
77.0
83.2
88.0
91.1
91.5
92.1
92.0
FDNM 07
arXiv'24
87.3
89.1
95.5
94.0
88.7
LCNL 68
IJCV"24
70.2
68.0
76,2
80.3
85.6
78.7
84.0
76,9
QTLA 17
BCCV
48.2
43.9
47.4
56.8
49.9
41.8
49.6
42.8
SSVI-ReID
TAA 19
TIP'23
48.8
42.3
50.1
56.0
62.2
56.0
63.8
56.5
DPIS
ICCV'23
52.7
H2H 49
TIP
30.2
29.4
23388
18.9
OTLA 77
ECCV'22
29.9
27.1
29.8
38.8
32.9
29.7
32.1
28.6
ADCA
MM'22
45.5
42.7
50.6
59.1
67.2
64.1
68.5
63.8
NGLR 69
MM
47.4
53.5
61.7
85.6
76.7
82.9
75.0
USVI-ReID
MBCCM 70
MM
53.1
48.2
55.2
62.0
83.8
77.9
82.8
76.7
CC 51
MM'
54.0
$0.2
56.7
65.1
69.9
65.5
70.2
66.7
PGM
CVPR
51.8
56.2
62.7
69.5
65.4
69.9
65.2
GUR 2P
ICCV'23
61.0
57.0
64.2
69.5
70.2
75.0
69.9
MMM S54
BCCV224
61.6
S7.9
64.4
70.4
89.7
80.5
85.8
77.0
POLHD
Ours
64.4
58.7
69.5
74.4
84.3
80.7
82.7
78.4
PCLHD+MMM
Enhanced
65.9
61.8
70.3
74.9
89.6
83.7
87.0
80.9
Table 2: Ablation studies on SYSU-MM01 all search mode and indoor search mode. "Baseline"
means the model trained following PGM
23 Rank-R accuracy(%) and mAP(%) ) are reported.
Component
All Search
Indoor Search
Index
Bascline
HPCL
DPCL
PCL
Rank-1
mAP
Rank-1
mAP
56.3
60.5
2
59.1
54.4
63.6
68.8
3
62.1
56.8
65.2
69.8
4
63.7
57.8
67.0
72.6
d
64.4
58.7
69.5
74.4
4.1 Experiment Setting
Dataset. We evaluate method two common benchmarks in VI-ReID: SYSU-MM01 [71] and RegDB [72). SYSU-MM01 is a large-scale public benchmark for the VI-ReID task, which contains 491 identities captured by four RGB cameras and IR cameras in both outdoor and indoor environments. In this dataset, 22,258 RGB images and 11, 1,909 IR images with 395 identities are collected for training. In the inference stage, the query set consists of 3,803 IR images with 96 identities and the galley set contains 301 randomly selected RGB images. RegDB is collected by an RGB camera and an IR camera, which contains 120 RGB images and 4,120 IR images with 412 identities. To be specific, the dataset is randomly divided into two non-overlgpping sets: one set is used for training and the other is for testing.
Evaluation Protocols. The experiment follows the standard evaluation settings in VI-ReID, i.e., Cumulative Matching Characteristics (CMC) 773 and mean Average Precision (mAP).
Implementation Details. We adopt the feature extractor AGW [88, which initialized with MingeNet-pretrainee weights to extract 3048-dimensional features. During the training stage, the
7

---

input images are resized to 288x 144. We follow augmentations in CAJ M for data augmentation. In one batch, we randomly sample 16 pseudo identities, and each pseudo identity samples 16 instances. We set M to be 16 for computational convenience. The number of epochs is 100, in which the first 50 epochs are trained by contrastive loss with the centroid prototype. For the last 50 epochs, we train the model by contrastive loss with both the hard and dynamic prototypes. ECPOL is 50. At the beginning of each epoch, we utilize the DBSCAN (50) algorithm to generate pseudo labels. During the inference stage, we use the momentum encoder @m to extract features and take the features of the global average pooling layer to calculate cosine similarity for retrieval. The momentum value a and B is set to 0.1 and 0.999, respectively. The temperature hyper-parameee is set 0.05 and the weighting syper-parameter À in Eq. 23 is 0.5.
4.2 Results and Analysis
To comprehensively evaluate our method, we compare our method with 18 supervised VI-RelD meth- ods, 3 semi-supervised VI-RelD methods, and 9 unsupervised VI-RelD methods. The comparison results on the SYSU-MM01 and RegDB are reported in Tab. [T
Comparison with USVI-ReID Methods. As shown in Tab. 1 our method achieves superior performance compared with state-of-the-art USVI-ReID methods. MMM [54] is proposed establish reliable mooss-mo correspondences and is also the current best best-performing method. Our method with MMM can achieve 65. .9% in Rank- I and 61.8% in mAP, which surpasses that of MMM by large margin of 4.3% and 3.9%. Notably, our method even without MMM gains the best performance with 64.4% in Rank-1 I and 58.7% in mAP. Although existing USVI-ReID methods mentioned in
Tab. | have made great progress in the USVI-RelD task, the neglects of divergence and variety hinder their further improvement. They overlook divergence and variety, which often constitutes hard samples. Thus, we propose progressive contrastive learning with hard and dynamic prototypes to mine hard samples, which can guide the model to learn more robust and discriminative features.
Comparison with SSVI-ReID Methods. There are three SSVI-ReID methods proposed to alleviate the problem of labeling cost by using a part of annotations. Remarkably, our method achieves superior performance without any annotations, outperforming all existing SSVI-ReID methods that utilize partial annotations. Moreover, the results suggest that our method can significantly reduce the dependency on manual annotations.
Comparison with SVI-ReID Methods. Surprisingly, method without annotation outperforms several SVI-ReID methods, e.g., DDAG (39. AGW [58), NES [59]. LbA [60]. This shows the immense competitiveness of PCLHD compared SVI-ReID methods that rely on complete data annotations. The superior performance of PCLHD mainly benefits from the hard prototype and dynamic prototype contrastive learning. Additionally, we have to acknowledge that a significant disparity still exists between PCLHD and the state-of-the-art fully-supervised results.
4.3 Ablation Study
We conduct ablation studies on the SYSU-MM01 dataset in both all-search and indoor-search modes to show the effectiveness of each component in our method. The results shown in Tab. 2
Baseline Settings. We use PGM 23 as our baseline. Although PGM has achieved promising performance on the USVI-ReID task, the neglect of hard samples hinders its further improvement.
Effectiveness of HPCL. The HPCL is proposed to mine divergence. As shown in Tab. 2 When adding the HPCL on Baseline, the performance improves a large margin of 5.8% in Rank-1 and 5.1 %% in mAP, respectively. It shows that divergence can be effectively mined using hard prototype contrast learning, facilitating the model to learn more discriminative features.
Effectiveness of DPCL. The DPCL is proposed to mine variety. The results show that Rank-1 1 accuracy can be improved by 2.8% in Rank-1 and 2.7% in mAP when adding the DPCL on Baseline, which confirms that contrastive learning with dynamic prototype can learn variety.
Effectiveness of PCL. PCL is introduced to smoothly shift the model's 's attention from commonality to divergence and variety. The results show that Rank-1 accuracy can be improved by about 1% in Rank-1 and mAP compared to adding simultaneously the HPCL. and DPCL. on the Baseline. This confirms that progressive contrastive learning plays valuable role in assisting HPCL and DPCL.
8

---

075
10
PSX
103M
(3) Hyge parrcete
C9 Roper parandia
(0) ARI Eeemic
Figure 2: (a) The effect of hhyer-parameter À with different values. (b) The effect of hyper-parameter with different values. (c) Comparisons with ARI values of different methods.
Baxdine
Figure The t-SNE visualization of 10 randomly selected identities. Different color indicates different IDs. Circle means visible features and the pentagram means infrared features.
Surprisingly, contrastive learning with both hard and dynamic prototypes significantly exceeds the baseline by large margin of 8.1 % in Rank-1 and 7.0% mAP. The HPCL. and DPCL can complement each other to learn divergence and variety, which effectively guides the network to learn more robust and discriminative features.
4.4 Further Analysis
Hyper-parameter analysis. Hyper- srper-parameter is a weighting parameter to trade-off L and LnPeL. Fig. (a) presents the results under different values A. can observe that when is small, i.e., LDPeI. contributes more to the model, the performance degrades. However, when À is large, i.e., L contributes heavily to the model, model both achieves superior performance. Note that when À 1, i.c., the proposed method is trained without DPCL, the performance drops significantly. is finally set 0.5 and our method achieves the best performance of 64.4% in Rank-I. Moreover, also analyze the effect of the number of hard samples at hard prototype. As shown in Fig. 2 (b), we vary the l from to 3 and keep the other that PCLHD achieves the best performance when & = Hard samples distributed in multiple directions, so multiple hard samples cannot be represented by single prototype. This is why using more hard samples as prototypes leads to decline in overall performance
fixed, which shows
we utilize the Adjusted Rand Index (ARI) metric for clustering evaluation. The larger the ART value, the higher the clustering quality. In Fig. (c), "RGB" and "IR" denote the ARI values of visible and infrared clusterings, which can measure the quality of visible and infrared pseudo-labela "ALL" means the ARI values of overall clusterings, which can evaluate the reliability of mosdaltoy correspondences. PCLHD surpasses other methods significantly all of the mentioned ARI values, which demonstrates PCLHD can effectively mine divergence and variety to improve clustering quality.
The ART metric. Following MMM 55
Visualization. As shown in Fig. we visualize the t-SNE map of 10 randomly chosen identities from SYSU-MMO1. Compared modality is more compact and the distance of the same identity from different modalities is closer together. Moreover, some hard samples the baseline are incorrectly clustered, while these hard samples well clustered in PCLHD, which shows the effectiveness of the proposed PCLHD.
baseline, the distribution of the same identity from the same

---

Conclusion and Limitation
In this paper, we propose a novel method for USVI-ReID called Progressive Contrastive Learning with hard and dynamic prototype (PCLHD), which learns commonality, divergence and variety. To be specific, we design Hard Prototype Contrastive Learning to mine divergent yet significant information and Dynamic Prototype Contrastive Learning to preserve intrinsic variety features. Furthermore, we introduce a progressive learning strategy to incorporate both HPCL and DPCL into the model. Extensive experiments demonstrate that PCLHD outperforms state-of-the-ar USVI-ReID methods.
This work relies on DBSCAN generate pseudo-labels, However, for extremely large-scale datasets, DBSCAN's performance may be limited, which could affect the overall effectiveness of our approach. To address the limitation, we plan to explore hierarchical clustering in future research to better handle large-scale datasets.
Broader Impacts
This work was developed using publicly available datasets and aims to enhance the capabilities of VI-ReID, which plays vital role in scenarios where traditional ReID systems fail, such as in low-light or nighttime conditions. VI-ReID offers significant benefits in improving security and surveillance by enabling more reliable identification across varying environmental conditions. Importantly, this work raises no ethical, safety, or environmental concerns, and no harm was inflicted on living beings during the research. However, we acknowledge the risk of misuse, particularly privacy invasion if used to track individuals in public spaces without appropriate regulation. While VI-ReID does not directly identify specific individuals, its unauthorized deployment could still result in significant privacy violations. Therefore, public surveillance systems using VI-RelD should be controlled by authorized entities, ensuring proper regulatory frameworks and adherence ethical standards.
Acknowledgements
This work was supported by the National Natural Science Foundation of China (No. 62176224, 62176092, 62222602, 62306 165, 62106075, 62476090), Natural Science Foundation of Shanghai 223ZR1204000 Natural Science Foundation of Chongqing Computer Federation (CCF) Lenovo Blue Ocean Research Fund, China Academy of Railway Sciences No. 2023YJ357.
CO0022SSSNC-JJX0000 China
References
[1] Mang Ye, Weijian Ruan, Bo Du, and Mike Zheng Shou. Channel augmented joint learning for visible- infrared recognition. In ICCV, pages 13547- 13556,, 2021.
[2] Yunpeng Gong, Zhun Zhong, Zhiming Luo, Yansong Qu. Rongrong Ji. and Min Jiang. Cross-modality perturbation synergy attack for person re-identification. arXiv preprint arXiv:2401. 10..24400..00,,,, 2024.
[3] Jiangming Shi, Xiangbo Yin, Demao Zhang, and Yanyun Qu. Visible embraces infrared: Cross- modality person re-identification with Linge-modallie supervision. In China Automation Congress, pages 4781- 4787, 2023.
[4] Jiaxu Leng, Zhanjic Wu, Mingpi Tan, Yiran Liu, Ji Gan, Haosheng Chen, and Xinbo Gao. Beyond cuclidean: Dual-space representation learning for weakly supervised video violence detection. arXiv preprint 4rX9::2409. 19252, 2024.
[5] Yuanmin Tang, Jing Yu, Keke Gai, Jiamin Zhuang, Gang Xiong, Yue Hu, and Qi Wu. Context-i2w: Mapping images context-dependent words for accurate zero-shot composed image retrieval. In AAAI, pages 11880-1185 2024.
[6] Yuanmin Tang, Jing Keke Gai, Jiamin Zhuang, Gaopeng Gou, Gang Xiong, and Qi Wu. Denoise-i2w: Mapping images to denoising words for accurate zero-shot composed image retrieval. arXiv preprint arXiv:2410. 17393, 2024.
[7] Junjie Zhou, Zheng Liu, Shitao Xiao, Bo Zhao, and Yongping Xiong. Vista: Visualized text embedding for universal multi- modal retrieval. In ACL, pages
2024.
10

---

[8] Yachao Zhang, Runze Hu, Ronghui Li, Yanyun Qu, Yuan Xie, and Xiu Li. Cross-modal match for language conditioned 3d object grounding. In AAAI, volume 38, pages 7359-7367, 2024.
[9] Yachao Zhang. Yuan Xie, Cuihua Li, Zongze Wu, and Yanyun Qu. Learning all-in collaborative multiview binary representation for clustering. IEEE TNNLS, 35(3)4260-4273,, 2022.
[10] Ling Lin, Hao Liu, Jinqiao Liang, Zhendong Li, Jiao Feng, and Hu Han. Consensus-agent deep reinforce- ment learning for face aging. IEEE Transactions Image Processing, 2024.
[11] Ling Lin, Tao Wang, Hao Liu, Congcong Zhu, and Jingrun Chen. Toward quantifiable face age transforma- tion under attribute unbias. IEEE TCSVT, 2024.
[12] Jiawei Feng, Ancong Wu, and Wei-Shi Zheng. Shape- Shape-crased feature learning for visible-infrared person re-identification. CVPR, pages 22752 22761, 2023.
[13] Xingye Fang. Yang Yang. and Ying Fu. Visible-infrared person re-identification via semantic alignment and affinity inference. ICCV. pages 11270-11277 11799, 2023.
[14] Rui Sun, Long Chen, Lei Zhang, Ruirui Xie, and Jun Gao. Robust visible-infrared person re-identification based on polymorphic mask and wavelet graph convolutional network. IEEE TIFS, 2024.
[15] Xiangbo Yin, Jiangming Shi, Yachao Zhang, Yang Lu, Zhizhong Zhang, Yuan Xic, and Yanyun Qu Robust pscudo-label learning with neighbor relation for unsupervised visible-infrared person re-identification. In ACM MM, 2024.
[16] Bo Pang, Deming Zhai, Junjun Jiang, and Xianming Liu. Fully unsupervised person re-identification via selective contrastive learning. ACM TOMM, 18(2):1-15, 2022.
[17] Jiangming Wang. Zhizhong Zhang, Mingang Chen, Yi Zhang, Cong Wang, Bin Sheng. Yanyun Qu, and Yuan Xie. Optimal transport for label-efficient 93-109, 2022.
person re- ideenifficaai.n In ECCV, pages
[18] Jiangming Shi. Yachao Zhang, Xiangbo Yin, Yuan Xie, Zhizhong Zhang, Jianping Fan, Zhongchao Shi, and Yanyun Qu. Dual lacudo-labels interactive self-training for semi-supervised visible-infrared person re-identification. In ICCV, pages 11218 1 11228, 2023.
[19] Bin Yang, Jun Chen, Xianzheng Ma, and Mang Ye. Translation, association and augmentation: Learning cross-modality re-identification from mingle-modality annotation. IEEE TIP, 32:5099-511 5113, 2023.
[20] Bin Yang, Mang Ye, Jun Chen, and Zesen Wu. Augmented dual-contrastive aggregation learning for unsupervised visible-infrared person re-identification. ACM MM, pages 2843-2851, 2022.
[21] Guoqing Zhang, Hongwei Zhang, Weisi Lin, Arun Kumar Chandran, and Xuan Jing. Camera contrast learning for unsupervised person re-identification. IEEE TCSVT, 3366) 2023.
[22] Bin Yang, Jun Chen, and Mang Ye. Towards grand unified representation learning for unsupervised visible-infrared person re-identification. In ICCV, pages 1069--1079, 2023.
[23] Zesen Wu and Mang Yc. Unsupervised visible-infrared person re-identification via progressive graph matching and alternate learning. In CVPR, pages 9548 9958,,9555888 2023.
[24] Lei Tan, Jiaer Xia, Wenfeng Liu, Pingyang Dai, Yongjian Wu, and Liujuan Cao. Occluded person re-identification via guiiddd patch transfer. In AAAI, volume 38, pages 5070-5078, 2024.
[25] Lei Tan, Pingyang Dai, Jie Chen, Liujuan Cao, Yongjian Wu, and Rongrong Ji. Partformer: Awak- ening latent diverse representation from vision transformer for object re-identification. arXiv preprint ar0iv:2408. 16684, 2024.
[26] Kunlun Xu. Xu Zou, Yuxin Peng. and Jiahuan Zhou Distribution-aware knowledge prototyping for non-exemplar lifelong person re-identification. In CYPR, pages 16604-16613, 2024.
[27] Mouxing Yang. Zhenyu Huang, Peng Hu, Taihao Li, Jiancheng Lv, and Xi Peng. Learning with twin noisy labels for visible-infrared person re-identification. In CVPR, pages 14288-14297, 2022.
[28] Pingping Zhang, Yuhao Wang, Yang Liu, Zhengzheng Tu, and Huchuan Lu. Magic tokens: Select diverse tokens for multi-modal object re-identification. In CVPR, pages 17117-17126, 2024.
[29] Yuhao Wang. Xuehu Liu, Pingping Zhang, Hu Lu, Zhengzheng Tu, and Huchuan Lu. Top-reid: Multi- spectral object re-iientificatinn with token permutation. In AAAI, pages
2024.
11

---

[30] Jialong Zuo, Hanyu Zhou, Ying Nie, Feng Zhang, Tianyu Guo, Nong Sang, Yunhe Wang, and Changxin Gao. Ufinebench: Towards text-based person retrieval with ultra-fine granularity. In CVPR, pages 22010-22019, 2024.
[31] Chenyang Yu, Xuehu Liu, Yingquan Wang, Pingping Zhang, and Huchuan Lu. Tf-clip: Learning text-free clip for video-based person re-identifiicatin.. In AAAI, pages 6764-6772, 2024.
[32] Yukang Zhang, Yan Yan, Jie Li, and Hanzi Wang. Mren: A novel modality restitution and compensation network for visible-infrared person re-identification. In AAAI, volume 37, pages 3498-3506, 2023.
[33] Guan's an Wang, Yang Yang. Tianzhu Zhang, Jian Cheng, Zengguang Hou, Prayag Tiwari, and Hari Mo- han Pandey. Cross-modality satred-imaees generation and augmentation for rgb-infrared person re- identification. Neural Newworks, 128:294-300 1044444442,2 2020.
[34] Guan' an Wang, Tianzhu Zhang, Jian Cheng, Si Liu, Yang Yang, and Zengguang Hou. Rgb-infrared cross-modality person re-identification via joint pixel and feature alignment. In ICCV, pages 3622-3631, 2019.
[35] Diangang Li, Xing Wei, Xiaopeng Hong, and Yihong Gong. Infrared-visiin cross-modal person re- identification with an X modality. In AAAI, pages 4610-4617, 2020.
[36] Yukang Zhang, Yan Yan, Yang Lu, and Hanzi Wang. Towards unified middle modality learning for visible-infrared person re-identiiication. In ACM MM, pages 788-796, 2021.
[37] Ziyu Wei, Xi Yang. Nannan Wang, and Xinbo Gao. Syncretic modality collaborative learning for visible infrared person re-identification. In /CCV, pages 225-234, 2021.
[38] Qiang Zhang, Changzhou Lai, Jianan Liu, Nianchang Huang, and Jungong Han. Fmenet: Feature-level modality compensation for visible-infrared person re-identification. In CVPR, pages 7339-7348, 2022.
[39] Mang Ye, Jianbing Shen, David J. Crandall, Ling Shao, and Jiebo Luo. Dynamic dual-attentive aggregation learning for visible-infrared person re- re-identification. In ECCV, pages 229 247, 2020.
[40] Qiong Wu, Pingyang Dai, Jic Chen, Chia-Wor Lin, Yongjian Wu, Feiyue Huang, Bineng Zhong, and Rongrong Ji. Discover cross-modality nuances for visiblc-infrared person re-identification. In CVPR, pages 4330-4339, 2021.
[41] Yixiao Ge. Dapeng Chen, and Hongsheng Li. Mutual mean-teaching: Pseudo label refinery for unsuper- vised domain adaptation on person re- re-identification. In ICLR, 2020.
[42] Yixiao Ge, Feng Zhu, Dapeng Chen, Rui Zhao, and Hongsheng Li. Self Self-paced contrastive learning with hybrid memory for domain adaptive object re-id. In NeurlPS, 2020.
[43] Neng Dong, Liyan Zhang, Shuanglin Yan, Hao Tang, and Jinhui Tang, Erasing, transforming, and noising defense network for occluded person re-identification. JEEE TCSVT, pages 1-1, 2023.
[44] Neng Dong. Shuanglin Yan, Hao Tang, Jinhui Tang, and Liyan Zhang. Multi-view information integration and propagation for occluded person re-identification. Information Fusion, 104: 102201, 2024.
[45] Yoonki Cho, Wao Jae Kim, Seunghoon Hong, and Sung-Fui Yoon. Part-based pseudo label refinement for unsupervised person re-identification. In CVPR, pages 7298-7308, 2022.
[46] Zuozhuo Dai, Guangyuan Wang, Weihao Yuan, Siyu Zhu, and Ping Tan. Cluster contrast for unsupervised person re-identification. In ACCV, pages 319 137-337,222 2022.
[47] Chang Zou, Zeqi Chen, Zhichao Cui, Yuehu Liu, and Chi Zhang. Discrepant and multi-instance proxies for unsupervised person re-identification. In ICCV, pages 11058-11068 2023.
[48] Yuhang Wu, Tengteng Huang, Haotian Yao, Chi Zhang, Yuanjie Shao, Chuchu Han, Changxin Gao, and Nong Sang. Multi-centroid representation network for domain adaptive person re-id. In AAAI, pages 2750- 2758, 2022.
[49] Wenqi Liang, Guangcong Wang, Jianhuang Lai, and Xiaohua Xic. Homogencous-to hoctooogoooo:ctrrooooooouuu Unsupervised learning for rgb-infrared person re-identification. JEEE TIP, 30:6392 6407, 2021.
[50] Martin Ester. Hans-Peter Kriegel, Jörg Sander, and Xiaowei Xu. A density-based algorithm for discovering clusters in large spatial databases with noise. In KDD. pages 226-231. 1996.
[51] Zhong Chen. Zhizhong Zhang, Xin Tan, Yanyun Qu. and Yuan Xie. Unveiling the power of clip in unsupervised visible-infrared person re- re-identification. In ACM MM, pages 3667-3675 2023.
12

---

[52] Mouxing Yang. Yunfan Li. Zhenyu Huang, Zitao Liu, Peng Hu, and Xi Peng. Partially view-aligned representation learning with noise-robust contrastive loss. In CVPR, pages 1134-1143, 2021.
[53] Mouxing Yang. Yunfan Li, Peng Hu, Jinfeng Bai, Jiancheng Lv, and Xi Peng. Robust multi-view clustering with incomplete information. IEEE TPAMI, 45(1):1055- 1069, 2023.
[54] Jiangming Shi, Xiangbo Yin, Yeyun Chen, Yachao Zhang. Zhizhong Zhang. Yuan Xie, and Yanyun Qu. Multi-memory matching for unsupervised visible-infrared person re-identification. In ECCV, page 456 474, 2024.
[55] Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In ICCV, pages 9729-9738, 2020.
[56] Lei Tan, Pingyang Dai, Rongrong Ji, and Yongjian Wu. Dynamic prototype mask for occluded person re-identification. In ACM MM. pages 531-540, 2022.
[57] Yunpeng Gong. Liqing Huang, and Lifei Chen. Person re-identification method based on color attack and joint defence. In CVPRW, pages 4312-4321, 2022.
[58] Mang Ye, Jianbing Shen, Gaojie Lin, Tao Xiang. Ling Shao, and Steven C. H. Hoi. Deep learning for person re-identification: survey and outlook. IEEE TPAMI. pages 2872-2893, 2022.
[59] Yehansen Chen, Lin Wan, Zhihang Li, Qianyan Jing, and Zongyuan Sun. Neural feature search for rgh-infrared person re-identification. CVPR, pages 587-597, 2021.
[60] Hyunjong Park, Sanghoon Lec, Junghyup Lee, and Bumsub Ham. Learning by aligning: Visible-infrared person re-identiiication using modal correspondences. In ICCV, pages 12026-12035, 2021.
[61] Zhipeng Huang, Jiawei Liu, Liang Li, Kecheng Zheng, and Zheng-Jun Zha. Modality-ddaptive mixup and invariant decomposition for rgh-infrared person re-identification. In AAAI, pages 1034-1042, 2022.
[62] Mahdi Alchdaghi, Arthur Josi, Rafael M. O. Cruz, and Eric Granger. Visible-infrared person re- identification using privileged intermediate information. In ECCV, pages 720-737, 2022.
[63] Yukang Zhang and Hanzi Wang. Diverse embedding expansion network and low-light cross-modality benchmark for visible-infrared person re-identification. In CVPR, pages 2153-2162, 2023.
[64] Minsu Kim, Seungryong Kim, Jungin Park, Scongheon Park, and Kwanghoon Sohn. Partmix: Regu- larization strategy to learn part discovery for visible-infrared person re-identification. In CVPR, pages 18621-18632, 2023.
[65] Jianbing Wu, Hong Liu, Yuxin Su, Wei Shi, and Hao Tang. Learning concordant attention via target-aware alignment for visible-infrared person re-identification. In ICCV, pages 11 11122-11131, 2023.
[66] Hao Yu, Xu Cheng, Wei Peng, Weihao Liu, and Guoying Zhao. Modality unifying network for visible- infrared person re-identification. In ICCV, pages 11 11185-11 195,, 2023.
[67] Yukang Zhang, Yang Lu, Yan Yan, Hanzi Wang, and Xuelong Li. Frequency domain nuances mining for vistble-infrared person re-identification. arXiv preprint arXiv:2401 022122 2024.
[68] Mouxing Yang, Zhenyu Huang, and Xi Peng. Robust object re-identification with coupled noisy labels.
JCV, pages 1-19, 2024.
[69] De Cheng, Xiaojian Huang. Nannan Wang, Lingfeng He, Zhihui Li, and Xinbo Gao. Unsupervised visible-infrared person reid by collahorative learning with neighhor-guided label refinement. In ACM MM, pages 7085-7093, 2023.
[70] Lingfeng He, Nannan Wang, Shizhou Zhang, Zhen Wang, Xinbo Gao, et al. Efficient bilateral cross- modality cluster matching for unsupervised visible-infrared person reid. In ACM MM, pages 1325-1333. 2023.
[71] Ancong Wa, Wei-Shi Zheng, Hong-Xing Yu, Shaogang Gong, and Jianhuang Lai. Rgb-infrared cross- modality person re-identification. In ICCV, pages 5390-5399, 2017.
[72] Dat Tien Nguyen, Hyung Gil Hong, Ki-Wan Kim, and Kang Ryoung Park. Person recognition system based combination of body images from visible light and thermal cameras. Sensors, 17(3):605, 2017.
[73] Mang Ye, Zheng Wang, Xiangyuan Lan, and Pong C. Yuen. Visible thermal person re-identification via dual-constrained top-ranking. In ISCAI, pages 1092-1099, 2018
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The main claims made in the abstract and introduction accurately reflect the paper's contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: This work relies on DBSCAN to generate pseudo-labels. However, for extremely large-scale datasets, DBSCAN's performance may be limited, which could affect the overall effectiveness of our approach. To address the limitation, we plan to explore hierarchical clustering in future research to better handle large-scale datasets.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged
the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
14

---

Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: For each theoretical result, the paper provide the full set of assumptions and complete (and correct) proof.
Guidelines:
The answer NA means that the paper does not include theoretical results. All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof' can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: The paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims. Our code will be released after the acceptance of our paper.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to model. In general. releasing code and data often one good way to accomplish this, but reproducibilityy can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that are appropriate to the research performed.
specific model and empirical evaluation, may
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily the architecture clearly and fully.
model architecture, the paper should describe
(c) the contribution is a new model (c.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open- open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibiliity In the case of closed-source models, it may be that access to the model is limited in
15

---

some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: The paper provide open access to the code, with sufficient instructions to faithfully reproduce the main experimental results.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (nttps://nnis.cc)
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
7 //nips. /tipss.c/pubbi.. httpccppccccccceeeeeeeeeeeeeeeee wwllcc/guiiss//CC httpc/ggguudeee/ooouuubbibiiiiie
for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The paper specify all the training and test details.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: Error bars reported because it would be too computationally expensive.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
16

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: The paper provide sufficient information on the computer resources.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https::/neurips. httpprppp....ppppiiicccccceaaeee
Answer: [Yes]
Justification: The research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: The paper was developed using publicly available infrared-visible ReID datasets and aims to enhance capabilities of visible-infrared ReID, which plays a vital role in scenarios where traditional ReID systems fail, such in low-light or nighttime conditions. This technology offers significant benefits in improving security and surveillance by enabling more reliable identification across varying environmental conditions. Importantly, our
17

---

research raises no ethical, safety, environmental concerns, and no harm was inflicted on living beings during the research. However, we acknowledge the risk of misuse, particularly privacy invasion if used to track individuals in public spaces without appropriate regulation. While RelD technology does not directly identify specific individuals, its unauthorized deployment could still result in significant privacy violations. Therefore, public surveillance systems using ReID should be controlled by authorized entities, ensuring proper regulatory frameworks, transparency, and adherence to ethical standards.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied partieular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question:
Are the creators or original owners of assets (c.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: They are properly credited and respected.
18

---

Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [No]
Justification: The datasets used in this paper, SYSU-MM01 and RegDB, are publicly available and widely used in research. These datasets were collected by their original creators and made accessible for research purposes.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurlPS Code Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
19

---

Answer: [No]
Justification: Since we are using datasets that are already publicly available and have beenextensively used in previous research, and given that the content does involve sensitivepersonal information, this study did not undergo an independent IRB review.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
20


Reviews:
[{"review_id": "iNqrbaeOjV", "reviewer": "Reviewer_edeS", "summary": "The paper proposes a method for unsupervised visible-infrared person ReID. Towards this goal, it extends the PGM method with a progressive contrastive learning with hard and dynamic prototypes. Initially a typical centroid prototype contrastive learning approach is used. A mixed batch of images is encoded through an online encoder and contrastive learning is used to update this encoder using contrastive learning with the centroid prototypes computed by DBSCAN clustering the training data and updating it with a momentum-updating strategy throughout the training. The paper hypothesizes that additionally hard samples should be used as prototypes to find divergent samples and dynamic prototypes should be used to preserve variability during training. Hard prototypes are selected as those samples within a cluster that are farthest away from the centroid prototypes and dynamic prototypes are random samples from a cluster and encodes them with an encoder that keeps a momentum version of the online weights. Since the hard and dynamic prototypes might not be well-defined at the start of training, the proposed method initially uses the standard centroid prototype contrastive learning and after a certain period of training, changes to a loss function based on hard and dynamic prototype contrastive learning. This setup shows significant improvements of the PGM baseline, but also over the current state of the art.", "questions": "- Why are the \"Baseline\" results in table 2 so different from the PGM results? I get the fluctuation of ~1% for All Search, but why is Indoor Search so much better in your training setup?\n- Why did you duplicate almost every single equation just for visible and infrared? This is a major waste of space and could have been used for actually relevant discussions.\n- In figure 2c, it very much confused me to see a line between the data points since the different methods are in no way connected.\n- Is figure 3 generated based on training of test images?", "limitations": "The authors claim \"We have not found the limitations of the work.\" but then again the paper is far from solving the problem. Given that this indicates there are obviously limitations, I find it somewhat lazy to not put some effort into coming up with a meaningful discussion of what still needs to be improved.\n\nAnd then this is actually outrageous: \"There is no societal impact of the work performed.\" The complete paper is about person re-identification, now even beyond the visible spectrum. Performing this kind of research can have major effects on how effective surveillance is and it's ridiculous to state that there is no impact! I will not judge someone for doing this research out of scientific curiosity, but thinking it has no impact and propagating that belief is not acceptable.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- The paper is fairly easy to follow.\n- The results look strong.\n- Code is promised to be released upon acceptance.\n- Overall the idea makes sense and the ablations do show that the different parts of the method are relevant.", "weaknesses": "- L215 states that each batch contains 16 person IDs and 16 samples per person. Is this a confusion and it should be based on the assignments from the clustering? Or are these actually person IDs from the annotations? This is kind of a big point, so please make sure to answer this in the rebuttal. I checked some other papers and there similar things are mentioned, but if this is truly based on annotations, the approach CANNOT be called unsupervised, unless you show you can do without annotations here!\n\n- The novelty isn't that great, the base method PGM covers most of the approach and the real novelty lies in a slight modification to the contrastive learning setup for a part of the training. The dynamic prototype learning, as the paper states, is inspired by MoCo and when searching a bit for hard examples in contrastive learning, there is also quite some literature available. Nevertheless, in this specific setting, there is a contribution of showing the improvements are possible.\n- I find the related work very weak! The paper is completely focused on the contrastive learning aspect of the PGM method, however, the related work section is purely focused on Visible-Infrared and unsupervised person ReID. Why is there no discussion about unsupervised learning in general, specifically with a focus on contrastive learning and its modifications?!\n- While the performance boost in general is quite good, figure 2a does raise the question how stable it is w.r.t to some of the hyperparameters. Then again a fixed value of 0.5 isn't too unexpected and I guess it might have even been an initial default value. Nevertheless an actual validation set for these kinds of ablations would have been valuable, but I guess also uncommon in the community.\n- Overall the discussion of the results is often a bit meaningless and just describes the values in the tables, partially even in a bit of a weird fashion. For example L238-240 points out that the proposed method \"remarkably\" is able to outperform even some semi-supervised methods, but quite some other unsupervised methods in the table also do so. Then in L241-246 it's pointed out that even some supervised methods from 2021 are outperformed, but almost every other supervised method outperforms every single unsupervised method in the table.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### Weaknesses\n**W1: L215 states that each batch contains 16 person IDs and 16 samples per person. Is this a confusion and it should be based on the assignments from the clustering? Or are these actually person IDs from the annotations?**\n\nA1: We apologize for the confusion. The statement in L215 indeed refers to the selection of samples based on the assignments **from the clustering, not annotations.** We followed the same experimental setup as existing unsupervised VI-ReID methods such as ADCA, PGM, and GUR. We will clarify this in the revised version.\n\n**W2: The novelty isn't that great, the base method PGM covers most of the approach and the real novelty lies in a slight modification to the contrastive learning setup for a part of the training.**\n\nA2: PGM solves the USL-VI-ReID by creating modality-specific memories and minimizing contrastive loss across the query images and centroid prototypes. The limitation of PGM is that it only focuses on the commonality among features,  which causes the pseudo-labels generated by the clusters to be unreliable, as shown in Figure 2c. Just like a normal distribution, which requires both the mean and the variance to accurately reflect the data distribution, HPCL and DPCL are designed to focus both divergence and variety. Importantly, we address issues not only present in PGM but also in most unsupervised VI-ReID methods, which overlook aspects of divergence and variety. As shown in the following table, our method significantly improves the performance of other methods. Specifically, when combined with PCLHD, MMM's Rank-1 improves from 61.6% to 65.9%. Additionally, we need to correct a misunderstanding: our momentum update is inspired by MoCo, not dynamic prototype learning. MoCo focuses on contrastive learning using a queue and momentum encoder. It does not involve any dynamic prototype learning.\n|  Methods   | Rank-1@All | mAP@All | Rank-1@Indoor | mAP@Indoor |\n| :--------: | :--------: | :--------: | :-----------: | :-----------: |\n|    ADCA    |    45.5    |    42.7    |     50.6      |     59.1      |\n| ADCA+PCLHD |    52.5    |    48.5    |     56.6      |     62.3      |\n|    MMM     |    61.6    |    57.9    |     64.4      |     70.4      |\n| ADCA+PCLHD |    65.9    |    61.8    |     70.3      |     74.9      |\n\n**W3: Why is there no discussion about unsupervised learning in general, specifically with a focus on contrastive learning and its modifications?**\n\nA3: We agree that the related work can be strengthened to include more discussion on unsupervised learning in general. We will revise the related work in the future version.\n\n**W4: While the performance boost in general is quite good, figure 2a does raise the question how stable it is w.r.t to some of the hyperparameters.**\n\nA4: Hyper-parameter $\\lambda$ is a weighting parameter to trade-off $L_{HPCL}$ and $L_{DPCL}$. We conducted extensive experiments with various values of $\\lambda$ (e.g., 0.1, 0.25, etc.) and found that $\\lambda = 0.5$ provided the best results, as demonstrated in Figure 2a.\n\n**W5: Overall the discussion of the results is often a bit meaningless and just describes the values in the tables, partially even in a bit of a weird fashion.**\n\nA5: We acknowledge that the discussion of the results is insufficient. We admit there is still a large gap between our method and SOTA fully-supervised methods on SYSU-MM01. Nonetheless, fully supervised methods have drawbacks, such as high annotation costs. Additionally, our method can enhance the performance of popular methods like ADCA, PGM, and MMM. For instance, combining PCLHD with MMM improves its Rank-1 from 61.6% to 65.9% and mAP from 57.9% to 61.8%, and the gain is remarkable by 4.3% and 3.9%. The improvement indicates the benefits of focusing on divergence and variety in this field, and demonstrates that our approach is not only effective on its own but also beneficial when integrated with other methods.\n\n### Questions\n**Q1: Why are the \"Baseline\" results in table 2 so different from the PGM results?**\n\nA1: The official paper may have contained typos, as we obtain better results on indoor search using the official code. The better performance on indoor search is normal because indoor environments are relatively simple, stable lighting and fewer background distractions, making matching easier. Most methods in Table 1 also show better results for indoor search compared to all search.\n\n**Q2: Why did you duplicate almost every single equation just for visible and infrared?**\n\nA2: Thanks for your valuable comments, we will revise this in the future version.\n\n**Q3: In figure 2c, it very much confused me to see a line between the data points since the different methods are in no way connected.**\n\nA3: Thank you for your suggestion, we will replace it with a bar chart to improve clarity.\n\n**Q4: Is figure 3 generated based on training of test images?**\n\nA4: The t-SNE map in Fig.3 is plotted by 10 randomly chosen identities of test images on SYSU-MM01.\n\n### Limitations\n**L1: I find it somewhat lazy to not put some effort into coming up with a meaningful discussion of what still needs to be improved.**\n\nA1: Our method relies on the DBSCAN to generate pseudo-labels. For extremely large-scale datasets, DBSCAN might not perform well, thereby affecting the overall method's performance. Besides, if the data has a large intra-class discrepancy, it might impact the results, and we plan to address this in future research by hierarchical clustering.\n\n**L2: This is actually outrageous: \"There is no societal impact of the work performed.\"**\n\nA2: Thank you for your thorough review and detailed feedback. We apologize for the misunderstanding. As you rightly pointed out, performing this kind of research can significantly impact the effectiveness of surveillance. We mistakenly interpreted the Broader Impacts section as referring only to negative societal impacts, hence our response of [NA]. We will review and correct the Paper Checklist accordingly.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "v4nHSNtuiB", "reviewer": "Reviewer_mgXA", "summary": "The paper is presented in the field of unsupervised visible-infrared person re-identification (USVI-ReID) which aims to match individuals in infrared images to those in visible images without annotations. Existing methods often use cluster-based contrastive learning, which fails to account for divergencies by focusing solely on commonalities. To overcome this, the authors propose a Progressive Contrastive Learning method with Hard and Dynamic Prototypes, which emphasizes divergencies and feature variety through the use of hard prototypes and a progressive learning strategy.", "questions": "Did the authors explore alternative methods for representing diversity aside from using the farthest sample? If such exploration was undertaken, it would be beneficial to discuss the findings. Conversely, if no alternative methods were considered, does this represent a limitation of the proposed solution? Understanding the rationale behind the chosen approach or the constraints of the methodology would add valuable context to the study.\n\nHow does the performance of the proposed solution vary with different values of the E_CPCL constant? Was any ablation study conducted to investigate this aspect? An examination of the impact of varying E_CPCL  on the solution's effectiveness would provide deeper insights into the robustness and adaptability of the proposed method. Such analysis would be crucial for validating the consistency and reliability of the results.", "limitations": "The limitations of the proposed approach are not adequately addressed. The weaknesses related to assumptions (e.g. diversity) and constant values definitions, should have brought the authors to a deeper ablation study and thus comprehension of the limitations of the proposed solution.", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The motivation behind the proposed methodology is exceptionally clear, as is the well-structured solution scheme. The logic of the presentation is sound, and the progression of the proposed solution is easy to follow, making the entire approach both coherent and engaging.\n\nThe innovative idea of adopting contrastive learning to enhance discrimination by incorporating diversity information is particularly noteworthy. This approach stands out as it not only addresses existing challenges but also brings a fresh perspective to the field.\n\nThe results, which have been demonstrated on two benchmark datasets, are truly impressive. They outperform recent state-of-the-art solutions that have been presented in top-tier venues in the computer vision (CV) field over the past few years. This achievement highlights the effectiveness and robustness of the proposed methodology.", "weaknesses": "The concept of farthest equal diversity in the HPCL (equations 6 and 7) is based on a strong assumption. Distance is a weak measure and a poor indicator of how the samples within a cluster are actually spread out. While variance is a simple metric, it captures the statistical distribution of the samples within the cluster more effectively.\n\nThe DPCL section is not clearly articulated. Specifically, the concept of the query and its application in equation 19 are obscure and require further clarification.\n\nThe loss function L_PCLHD  relies on the constant E_CPCL , which is arbitrarily set to 50 without any justification or ablation study to support this choice. From the results presented, it appears that all the ablation studies commence with 50 epochs of CPCL.\n\nAdditionally, although it is common practice to integrate the proposed methodology into unsupervised solutions, I believe that knowing the number of classes in advance and setting the number of clusters accordingly constitutes a form of supervision. While it is less intensive than labeling every sample with its respective class, it is still a form of supervision. In my opinion, these methods should be categorized under \"unlabeled samples\" rather than \"unsupervised\" learning.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Q1: The use of the farthest sample as a method for representing diversity in the HPCL (as described in Equations 6 and 7) relies on the assumption that distance is an adequate measure of sample spread within a cluster. Given that distance can be a weak indicator and variance might offer a better representation of statistical distribution, did the authors explore alternative methods for representing diversity, such as variance or other metrics? If so, please discuss the findings.**\n\nA1: We chose the farthest sample distance measure to represent diversity due to its simplicity and effectiveness. Distance measure is a common approach in machine learning and person re-identification, particularly in subspace learning. Thank you for your suggestion, we also explored using variance as an alternative measure for diversity. Our experiments indicate that using variance resulted in 2.8% reduction in Rank-1 compared to the our HPCL, as shown in following table. The reason for this is that our HPCL focuses on the extreme points in the feature space, which helps the model focus on the divergent samples, rather than simply minimizing intra-class variance.\n|  Metrics   | Rank-1@All | mAP@All | Rank-1@Indoor | mAP@Indoor |\n| :--------: | :--------: | :--------: | :-----------: | :-----------: |\n|    Variance     |    61.6   |    55.8    |     65.2      |     70.9      |\n| HPCL |    64.4    |    58.7    |     65.9      |     74.4      |\n\n\n**Q2: The DPCL section is not clearly articulated. Specifically, the concept of the query and its application in equation 19 are obscure and require further clarification.**\n\nA2: We apologize for the lack of clarity in the DPCL section. The query in this context refers to the feature representation of the input image that is being compared to the dynamic prototype. We followed the definition of the query as used in MoCo, PGM, and GUR. We will revise the manuscript to make this explanation clearer.\n\n**Q3: How does the performance of the proposed solution vary with different values of the E_CPCL constant? Was any ablation study conducted to investigate this aspect? An examination of the impact of varying E_CPCL on the solution's effectiveness would provide deeper insights into the robustness and adaptability of the proposed method. Such analysis would be crucial for validating the consistency and reliability of the results.**\n\nA3: We understand the concern regarding the constant $E_{CPCL}$ being set to 50. We have now conducted ablation studies to justify this choice. Our experiments tested various values for $E_{CPCL}$, and we found that 50 is not the best choice. The results showed that 40 epochs provided the best performance. We will include these ablation study results in the revised manuscript to support our choice of $E_{CPCL}$.\n\n| $E_{CPCL}$ | Rank-1@All | mAP@All | Rank-1@Indoor | mAP@Indoor |\n| :----: | :--------: | :-----: | :-----------: | :--------: |\n|   10   |    60.3    |  56.2   |     65.1      |    70.1    |\n|   20   |    62.0    |  57.5   |     66.7      |    71.6    |\n|   30   |    64.2    |  59.4   |     68.4      |    72.7    |\n|   40   |    64.6    |  59.8   |     70.0      |    75.0    |\n|   50   |    64.4    |  58.7   |     69.5      |    74.4    |\n|   60   |    64.0    |  59.5   |     68.7      |    73.2    |\n|   70   |    62.8    |  58.5   |     68.8      |    73.0    |\n\n**Q4: Although it is common practice to integrate the proposed methodology into unsupervised solutions, I believe that knowing the number of classes in advance and setting the number of clusters accordingly constitutes a form of supervision. While it is less intensive than labeling every sample with its respective class, it is still a form of supervision. In my opinion, these methods should be categorized under \"unlabeled samples\" rather than \"unsupervised\" learning.**\n\nA4: We appreciate your insight regarding the classification of our method. However, our pseudo-labels relies on the DBSCAN clustering algorithm, which does not require prior knowledge of the number of classes. For instance, while the actual number of classes in the dataset is 395, our clustering results in 384 clusters. This slight variation demonstrates that our method can effectively approximate the true number of classes without supervision.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "cVD7sYSIN1", "reviewer": "Reviewer_i5Cg", "summary": "This paper proposes a method for unsupervised visible-infrared person re-identification called Progressive Contrastive Learning with Hard and Dynamic Prototype. This method focuses on capturing commonality, divergence, and variety. By introducing hard prototype contrastive learning to emphasize divergence and dynamic prototype contrastive learning to preserve intrinsic feature variety, the approach addresses limitations of existing methods. The proposed progressive learning strategy further enhances model robustness and prevents cluster deterioration. Experimental results on two benchmark datasets, SYSU-MM01 and RegDB, show significant improvements in performance, with notable increases in mAP and Rank-1 accuracy compared to the current state-of-the-art methods.", "questions": "1.Is L_CPCL involved in the optimization process during the second stage?\n2.Is the comparison fair with GUR without camera information?", "limitations": "The method relies on the DBSCAN clustering algorithm to generate pseudo-labels. For large-scale datasets, DBSCAN might not perform well, thereby affecting the overall method's performance.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1.The discussion on the limitations of existing methods and how the proposed method addresses these limitations is compelling. The motivation for focusing on commonality, divergence, and variety is well-articulated.\n2.The methodology is detailed and well-structured. The explanation of hard prototype contrastive learning, and dynamic prototype contrastive learning is clear.\n3.The comparison with state-of-the-art methods is comprehensive. The proposed method’s superior performance. \n4.The further analysis, including ARI metric, provides additional insights into the robustness and effectiveness of the method. The analysis of the results is insightful, explaining why the proposed method outperforms others.", "weaknesses": "1.While introducing hard prototype contrastive learning can capture divergences, selecting hard samples may lead to model instability during the early training stages. \n2.Although a progressive contrastive learning strategy is introduced to mitigate the above problem, balancing the impact of hard samples remains a challenge.\n3.The strategy for selecting dynamic prototypes might require more complex implementation and higher computational resources on larger datasets.\n4.Although the paper mentions that the code will be released after acceptance, it currently lacks detailed instructions for reproducibility. \n5.The accuracy of pseudo-labels directly impacts the model training. Some parameters in DBSCAN are very important and should be analyzed further.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### Weaknesses\n\n**W1: While introducing hard prototype contrastive learning can capture divergences, selecting hard samples may lead to model instability during the early training stages. Although progressive contrastive learning strategy is introduced to mitigate the above problem, balancing the impact of hard samples remains a challenge.**\n\nA1: Our ablation experiments show that even starting with HPCL and DPCL without using PCL, the method achieves the Rank-1 accuracy of 63.7%, only with the 0.7% decrease. This is because CPCL is involved in training, helping the model maintain stability when learning divergence and variety. Additionally, our experiments demonstrate that method is not highly sensitive to the $E_{CPCL}$ hyperparameter. For more details, please refer to Q3 for Reviewer mgXA.\n\n**W2: The strategy for selecting dynamic prototypes might require more complex implementation and higher computational resources on larger datasets.**\n\nA2: We acknowledge that dynamic prototype learning requires more complex implementation and higher computational resources during training. However, it is important to note that during the inference stage, this process is not needed. The complexity and computational requirements are the same as other unsupervised methods, with no additional overhead introduced.\n\n**W3: Although the paper mentions that the code will be released after acceptance, it currently lacks detailed instructions for reproducibility. The accuracy of pseudo-labels directly impacts the model training. Some parameters in DBSCAN are very important and should be analyzed further.**\n\nA3: For fair comparisons, we follow the experimental details provided by ADCA and PGM for all other aspects. The detailed parameters are as follows: We use Adam optimizer to train the model with weight decay 5e-4. The initial learning rate is 3.5e-3 and decays 10 times every 20 epochs. The maximum distance for DBSCAN is set to 0.6 on SYSU-MM01 and 0.3 on RegDB. The minimal number for two datasets is set to 4 during clustering. We will add these details to the supplementary materials to enhance reproducibility.\n\n### Questions\n\n**Q1: Is $L_{CPCL}$ involved in the optimization process during the second stage?**\n\nA1: $L_{CPCL}$ does participate in the optimization of the second stage. As indicated by the background color in Figure 2, $L_{CPCL}$ covers both stages.\n\n**Q2: Is the comparison fair with GUR without camera information?**\n\nA2: GUR shows two results, with and without camera information, with the latter being lower. All the unsupervised methods in Table 1, including ours, do not use camera information. To ensure a fair comparison with all methods, we present the results of GUR without camera information, not with the intention of diminishing GUR’s results. GUR vs. our method achieves 63.5% vs. 65.9% Rank-1 accuracy with camera information, and our method outperforms GUR’s best result by 2.4%. Moreover, GUR vs. our method achieves 73.9% vs. 89.6% Rank-1 accuracy on RegDB (V2T), resulting in a remarkable gain of 15.7%.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "qvAfyMYaCl", "reviewer": "Reviewer_n3oM", "summary": "This paper proposed a Progressive Contrastive Learning with Multi-Prototype method for USVI-ReID, which learns both shared and diverse information. The Hard Prototype Contrastive Learning method aims to mine distinctive yet significant information, while the Dynamic Prototype Contrastive Learning method is designed to preserve intrinsic diversity features. The experimental results significantly outperform existing methods, validating the effectiveness of the proposed method.", "questions": "What insights does this method offer for other multimodal tasks? How can the approach be adapted to enhance performance in different multimodal scenarios?", "limitations": "Although the method shows significant performance improvement over other unsupervised methods, there is still a considerable gap when compared to fully supervised methods.", "rating": 7, "confidence": 5, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "1) The paper proposes a straightforward yet effective method, and the experimental results are promising for the USVI-ReID task.\n2) The paper is well-written and easy to follow.\n3) The proposed method is well-motivated. Its prototype designs enhance the learning of commonality, divergence, and variety. It makes up for the limits of traditional center prototype-based approaches.\n4) Figure 2(c) demonstrates that the proposed method generates higher-quality pseudo labels compared to other state-of-the-art methods.", "weaknesses": "1) From Table 2, it is evident that the method is based on a relatively strong baseline. Does the paper use different baselines, like ADCA, still achieve good results?\n2) The paper introduces a momentum backbone in the PCLHD method but does not fully explain its advantages. \n3) Cross-modality associations are especially important for USVI-ReID, what is the association strategy used in this paper? Some details need to be supplemented for clarity.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### Weaknesses\n\n**W1: From Table 2, it is evident that the method is based on a relatively strong baseline. Does the paper use different baselines, like ADCA, still achieve good results?**\n\nA1: We acknowledge the importance of evaluating our method with different baselines. As shown in following Table, we have included results with two additional baselines, ADCA and MMM. Our method, when combined with these baselines, still achieves significant improvements. For instance, ADCA's Rank-1 accuracy improves from 45.5% to 52.5% and mAP from 42.7% to 48.5% when integrated with our PCLHD. Similarly, MMM's Rank-1 accuracy improves from 61.6% to 65.9% and mAP from 57.9% to 61.8%, and the gain is remarkable by 4.3% and 3.9%. These results demonstrate the effectiveness of our approach across different baselines.\n|  Methods   | Rank-1@All | mAP@All | Rank-1@Indoor | mAP@Indoor |\n| :--------: | :--------: | :--------: | :-----------: | :-----------: |\n|    ADCA    |    45.5    |    42.7    |     50.6      |     59.1      |\n| ADCA+PCLHD |    52.5    |    48.5    |     56.6      |     62.3      |\n|    MMM     |    61.6    |    57.9    |     64.4      |     70.4      |\n| ADCA+PCLHD |    65.9    |    61.8    |     70.3      |     74.9      |\n\n**W2: The paper introduces a momentum backbone in the PCLHD method but does not fully explain its advantages.**\n\nA2: In unsupervised learning, noise is inevitable, and dynamic prototypes can introduce a certain degree of instability. The momentum encoder mirrors the structure of the online encoder and is updated by the accumulated weights of the online encoder. Our method is resistant to sudden fluctuations or noisy updates, leading to more consistent and robust feature representations.\n\n**W3: Cross-modality associations are especially important for USVI-ReID, what is the association strategy used in this paper? Some details need to be supplemented for clarity.**\n\nA3: In USVI-ReID methods, the process is typically divided into two stages. The first stage involves intra-modality identity alignment learning, while the second stage focuses on cross-modality associations. Our strategy emphasizes obtaining accurate intra-modality alignment suitable for the second stage. As shown in Figure 3c, our cross-modality association accuracy is significantly higher than existing unsupervised methods. We utilize progressive contrastive learning, including centroid and hard prototype contrastive learning, to achieve this. For inter-modality association, we build on the well-aligned intra-modality identities and use existing methods like PGM and MMM to handle the association. This ensures effective cross-modality matching by leveraging the strengths of both intra- and inter-modality learning.\n\n### Questions\n\n**Q1: What insights does this method offer for other multimodal tasks? How can the approach be adapted to enhance performance in different multimodal scenarios?**\n\nA1: Our method offers valuable insights for multimodal tasks by demonstrating the effectiveness of progressive contrastive learning with hard and dynamic prototypes. When dealing with multimodal feature alignment and representation, it may be beneficial to make the intra-modality  feature alignment and then make the inter-modality feature alignment. By mining divergence and variety through these prototypes, our method enhances clustering quality and feature discrimination, which is crucial for aligning cross-modal features. For instance, in image-text matching tasks, hard and dynamic prototypes help capture divergence and variety in visual and textual data, leading to better alignment between the two modalities.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We thank all the reviewers for their careful reading of our paper and help with improving our manuscript. We sincerely appreciate that you find our work:

- sense idea (edeS)

- well-writting (mgXA, i5Cg, n3oM)

- coherent and engaging (mgXA, i5Cg)

- innovative idea and fresh perspective (mgXA)

- clear motivation (mgXA, i5Cg, n3oM)

- higher-quality pseudo labels (i5Cg, n3oM)

- strong performance (edeS, mgXA, i5Cg)

	

We list the replies to several main concerns and suggestions raised by reviewers:

- **Question on the Unsupervised Setting of VI-ReID (Reviewer edeS and mgXA).**  We would like to clarify that we did not use the ground-truth labels and the number of classes. We followed the same experimental setup as existing unsupervised VI-ReID methods such as ADCA, PGM, and GUR. We used labels and the number of clusters generated by the DBSCAN clustering algorithm. It is a general setting and does not violate the unsupervised setting. We will clarify this in the final version.

- **Missing Discussion on Limitations (Reviewer edeS and mgXA).** Our method relies on the DBSCAN clustering algorithm to generate pseudo-labels. For extremely large-scale datasets, DBSCAN might not perform well, thereby affecting the overall method's performance. Besides, if the data has a large intra-class discrepancy, it might impact the results, and we plan to address this in future research by hierarchical clustering.

- **Missing Discussion on Different Baselines (Reviewer edeS and n3oM).** Most unsupervised VI-ReID methods overlook the aspects of divergence and variety. Our method significantly focuses on these aspects. Specifically, when combined with our PCLHD, ADCA's Rank-1 improves from 45.5% to 52.5%, and MMM's Rank-1 improves from 61.6% to 65.9%. This demonstrates the importance of divergence and variety.

We have conducted additional experiments. If not specified, we conduct analysis experiments on SYSU-MM01 in the single-shot &all-search mode.

**Additional experiments on ablation study of metrics for representing diversity (Reviewer mgXA).** For more details, please refer to Q1 for Reviewer mgXA.

**Additional experiments on ablation study of hyper-parameter $E_{CPCL}$ (Reviewer mgXA).** For more details, please refer to Q3 for Reviewer mgXA.

**Additional experiments in different baseline (Reviewer n3oM).** For more details, please refer to W1 for Reviewer n3oM.

Meta-review: This paper proposes an unsupervised approach to Infrared-Visible person re-identification. The authors propose to use progressive contrastive learning with hard and dynamic prototypes. Hard prototypes are used to find divergent samples, while dynamic ones are used to conserve variability. Reviewers raised a number of technical issues, most of which the authors responded to convincingly (or at least partially) during the rebuttal period. On the positive side, reviewers praise the clarity of technical presentation and motivation, the excellent experimental results, and the innovative use of contrastive learning to enhance discrimination.

The two Ethics Reviewers and *all four reviewers* note that there is absolutely no discussion of limitations of the work and potential societal impacts. During the rebuttal period the authors demonstrated very little willingness to engage at all with the broader implications of surveillance technologies. Although the reviewers (and one Ethics Reviewer) note that the work stands on its own technical merits, this lack of engagement to the point of defensiveness is unacceptable. The authors are encouraged to seriously reflect on these issues in any final version of this work.
Decision: Accept (poster)