Cross-video Identity Correlating for Person Re-identification Pre-training
Authors: Jialong Zuo
Ying Nie
Hanyu Zhou
Huaxin Zhang
Haoyu Wang
Tianyu Guo
Nong Sang
Changxin Gao
Venue: NeurIPS 2024 poster (2024)
Keywords: Person Re-identification
Pre-training
Self-supervised Representation Learning
Primary area: machine_vision

Abstract: Recent researches have proven that pre-training on large-scale person images extracted from internet videos is an effective way in learning better representations for person re-identification. However, these researches are mostly confined to pre-training at the instance-level or single-video tracklet-level. They ignore the identity-invariance in images of the same person across different videos, which is a key focus in person re-identification. To address this issue, we propose a Cross-video Identity-cOrrelating pre-traiNing (CION) framework. Defining a noise concept that comprehensively considers both intra-identity consistency and inter-identity discrimination, CION seeks the identity correlation from cross-video images by modeling it as a progressive multi-level denoising problem. Furthermore, an identity-guided self-distillation loss is proposed to implement better large-scale pre-training by mining the identity-invariance within person images. We conduct extensive experiments to verify the superiority of our CION in terms of efficiency and performance. CION achieves significantly leading performance with even fewer training samples. For example, compared with the previous state-of-the-art ISR, CION with the same ResNet50-IBN achieves higher mAP of 93.3% and 74.3% on Market1501 and MSMT17, while only utilizing 8% training samples. Finally, with CION demonstrating superior model-agnostic ability, we contribute a model zoo named ReIDZoo to meet diverse research and application needs in this field. It contains a series of CION pre-trained models with spanning structures and parameters, totaling 32 models with 10 different structures, including GhostNet, ConvNext, RepViT, FastViT and so on. The code and models will be open-sourced.

Cross-video Identity Correlating for Person Re-identification Pre-training
Jialong Zuo
Ying Nie Tianyu Guo
Hanyu Zhou Nong Sang"
Huaxin Zhang Changxin Gao 1*
Haoyu Wang
National Key Laboratory of Multispectral Information Intelligent Processing Technology,
School of Artificial Intelligence and Automation, Huazhong University of Science and Technology. Huawei Noah's Ark Lab.
(llonguu,, @gao]@hust.eduu
Abstract
Recent researches have proven that pre-training on large-scale person images extracted from internet videos is an effective way in learning better representations for person re-identification. However, these researches are mostly confined pre-training at the instance-level or single-video tracklet-level. They ignore the in images of the same person across different videos, which is a key focus in person re-identification. Too address this issue, we propose a pre-traiNing (CION) framework. Defining a noise concept that comprchensively considers both intra-identity consistency and inter-identity discrimination, CION seeks the identity correlation from croso-video images by modeling it as progressive multi-level denoising problem. Furthermore, an identity-guidee guided self-distillation loss is proposed to implement better large-scale pre- pre-training by mining the -eentity-invariane within person images. We conduct extensive experiments to verify the superiority of our CION in terms of efficiency and performance. CION achieves significantly leading performance with fewer training samples. For example, compared with the previous state-of-the- art [9], CION with the same ResNet50-IBN achieves higher mAP of 93.3% and 74.3% on Market1501 and MSMT17, while only utilizing 8% training samples. Finally, with CION demonstrating superior model-agnostic ability, we contribute a model zoo named RelDZoo to meet diverse research and application needs in this field. contains series of CION pre-trained models with spanning structures and parameters, totaling 32 models with 10 different structures, including GhostNet, ConvNext, ViT, VistViT and so on.
Cross-video Identity-cOrrelan
Introduction
Person re-identification (RelD), which aims to identify and match target person across different camera views, becoming increasingly influential in widespread applications, such as criminal tracking and missing individual searching. Although current RelD methods [38, 2, 21, 22, 39] have achieved significant progress for the past few years, has become evident through recent research advancements that the mere development of sophisticated specialist algorithms has already encountered a performance bottleneck.
Meanwhile, pre- trainininn on large-scale images [1, 5, 17, 29] has shown great success to further improve model performance in the field of general vision. However, due to the significant domain gap, these pre-trained models can only provide very limited improvement for person re- re-identification. Therefore, with the vast amount of person-containing videos available on the internet, some re- person images
searches [10, 11, 53, 9, 44, 27] turn to exploring the potential of pre-training extracted from such videos and demonstrate encouraging results. However, due to the unavailability of identity labels for such extracted person images, these pre-training methods are confined to learning representations at the instance-level or single-video tracklet-level, as shown Fig.
"Corresponding Author. Project Link: https: //ops::/eeh /ttp//gttttttttt/////e/eteee.. www com//pplusdragon/COO0 ReIDZoo
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

FRAME
VIDEOA
cunlassiive learzing
(al Instaane- level method
leariin
VIDED
VIDEO
contradivee learzing
lncc
Frame
Frome
[c) Cruss viden idantity comreating pm training (CION)
(b) Single video tracklet levd methud
Figure 1: Comparisons between our proposed CION with other pre-training methods. In (a), the instance-level method mines instance-ivariancee by contrastive learning on augmented views of each image, completely ignoring the invariance within different images of the same person. In (b), the single-video tracklet-level method mines fncclle-invariance by contrastive learning on images of each tracklet in single video, significantly ignoring the invariance in images of the same person across different videos. In (c), our CION learns identity-ivvaiiun by correlating the images of the same person across different videos, thus leading to better representation learning.
On the one hand, the instance-level methods [10, 53, 44, 27] completely overlook the identity- invariance in different images of the same person. They incorporate some person-related prior knowledge into general pre-training frameworks, neglecting the concept of person identity. For example, LUP [10] improves MoCoV2 [4] by systematically studying the data augmentation and contrastive loss, while PASS [53] improves DINO [1] by generating part-level features to offer fine- grained information. However, there exists notable concordance within different images of the same person. Directly ignoring such a concordance easily leads to learning weaker person representations.
On the other hand, the single-video tracklet-level methods [11, 9] only exploit the short-range tracklet- invariance while significantly neglecting the identity-invarian in images of the same person across different videos. They simply regard one tracklet single video two substantial problems. 1) Insufficient intra-identity consistency. Duc to the unavoidable inaccurate tracking, there quite few images within tracklet that do not belong the same person. Roughly considering these images as coming from the same person will lead to insufficient intra-identity consistency. 2) Limited inter-identity discrimination. It is quite evident that there are considerable instances of the same person appearing in different videos, thus being treated as different tracklets (identities). Roughly considering these images as coming from different persons will lead limited inter-identity discrimination.
person, which will result in
To address these limitations mentioned above, as shown in Fig. propose Cross-video Identity- cOrrelating pre-traiNing (CION) framework, by which the images of the same person across different videos explicitly correlated to deeply mine the identity-invarien correlation from long-range cross-video images by modeling it as a progressive multi-level denoising problem, where noise concept that comprehensively considers both intra-identity consistency and inter-identig discrimination is defined. Subsequently, with the sought identity correlation, CION introduces eeeenttty-guidee self-distillation loss to implement better pre- pre-training by learning the invariance within augmented views in multiple images of the same person.
CION first seeks the identity
We conduct extensive experiments to verify the superiority of our CION in terms of efficiency and performance. CION achieves the best performance with significantly fewer training samples. With the ResNet50-IBN backbone, compared with the previous state-of-the oftec----he-aarr method ISR [9], our CION achieves higher mAP of 93.3% and 74.3% on the M5r and MSMT17 datasets, while only utilizing 8% training samples. Noting that ISR is pre-training at the single-video tracklet-level, this experimental result fully validates the necessity of learning ddentity-invariane across different videos.
Finally, we contribute fully open-sourced model zoo named RelDZoo to meet diverse research and application needs within this field. With CION demonstrating superior performance improvements to diverse model structures (model-agnostic ability), we pre-trained series of models with spanning structures and parameters, totaling 32 models with 10 different structures, including GhostNet [16], EdgeNext [28]. ConvNext [26], ReT [36], FistViT [35] and so

---

In conclusion, the highlights of our work can be summarized into three points:
(I) We propose Cross-video Identity-COrrelating pre-traiNing (CION) framework. It explicitly mines the identity-invariance in person images extracted from internet videos by progressive multi- level denoising and identity-guided seff-distillation, leading to better representation learning.
(2) Extensive experiments verify the superiority of CION in terms efficiency and performance. achieves the best performance with much fewer training samples compared with previous methods, while also demonstrating adaptability to diverse model structures with spanning parameters.
(3) We contribute a model zoo named RelDZoo. It contains a series of CION pre-trained models with spanning structures and parameters, totaling 32 models with 10 different structures. It conveniently meets various research and application needs, and greatly promotes the development of this field.
2 Related work
Self-supervieed pre-training for general vision. In recent years, self-supervised pre-training methods have shown promising results in learning representation from large-scale data. MoCo [4] facilitates contrastive learning by building a dynamic dictionary with a queue and a encoder. BYOL [15] relies two networks, referred to online and target networks, that interact and learn from each other with a slow-moving average strategy. DINO [1] proposes a self-distillation framework by underlying the importance of momentum encoder, multi-crop training and on.
MAE [17] adopts the VIT backbone and learns representations by predicting the masked patches from the remaining visible ones. However, due to significant domain gap and neglect of person-related characteristics, these general methods show poor transferability to person re-identification.
self-supe pre-training for person re-identification. Existing pre-training methods can be classified into two categories, the first being the instance-level methods [10, 53, 44, 27, 3] and the second being single-vides tracklet-level method [9, 11]. The instance-level methods strive merge the person-related prior knowledge into existing self-supervised learning methods. For example, LUP [10] improves MoCov2 [4] through systematical studying of data augmentation and contrastive loss. PASS [53] improves DINO [1] by generating part-level features to offer fine mme-grained information. SOLIDER [3] turns to utilize pseudo semantic label and import some semantic information into the learned representations. However, these instance-level methods are incapable of harnessing the identity-invariance in different images of the same person. Meanwhile, the single-video tracklet-level methods strive to mine some invariance by regarding one tracklet in single video as one person. For example, LUP-NL [10] utilizes supervised ReID learning, label-guided contrastive learning and prototype- based contrastive learning to rectify the noisy tracklet-level labels in person images extracted from raw internet videos. ISR [9] leverages massive data to construct the positive pairs from inter-frame frame images in a single videos. The identity-invarience of the same person across different videos significantly neglected by these single-video tracklet-level methods.
3 CION: Cross-video Identity-cOrrelating pre-traiNing
We present a cross-video identity-corelating pre-training (CION) framework, which explicitly learns adentity-invariaan in long-range cross-video person images. First, we define a noise concept that comprehensively considers both intra-identity consistency and inter-identity discrimination (Sec. 3.1). Next, we model the identity correlation seeking process from cross-video images as a progressive multi-level denoising problem (Sec. 3.2). Finally, we propose an identity-guided self-distillation loss to mine identity-invariance from person images with the sought identity correlation (Sec. 3.3).
3.1 Noise definition
We demonstrate that sample set with identity labels should have the following characteristics. 1) intra-identity consistency. Samples belonging to the same identity should aggregate around a centroid in a nigh-dimensional space, i.e., they should be enveloped by hypersphere $/ (Ck, r&).. where C₁ represents the centroid of the sphere and Fı represents its radius. smaller radius of the sphere indicates denser degree of aggregation, which signifies higher intra-identity consistency. 2) Inter-identity discrimination. Samples belonging to two different identities should be enveloped by two distinct hyperspheres s, and sy respectively, with certain distance dij maintained between the
3

---

centroids of c, and Cj. A larger distance indicates a sparser distribution of hyperspheres, implying higher intra-identity discrimination. In summary, the smaller radii of the hyperspheres and the greater distances between the hypersphere centroids mean more desirable intra-identity consistency and inter-identity discrimination.
Considering the above demonstration, for a sample set S = {x L. Ji=1, where the identity label
y: E (1.2.3.... M we can further regard S as a hypersphere set {s where x; E sq yı k. Then for single hypersphere sg, the intra-consistency noise is formulated as:
" = ju(k- - ccst) 8.1. Fk max(disl(x;; og)) Vx, E 8k:
(1)
where uu()) is an unit step function, (-st(-) is any distance metric such as cosine distance, and the centroid сţ calculated by averaging all samples in st- We can observe that this formula determines the presence of noise by explicitly constraining the maximum radius of the hypersphere, indicating whether the aggregation degree of samples with the same identity meets the required criteria cst-
Meanwhile, for two distinctive hyperspheres 8; and 8y. the inter-discrimination noise hetween them formulated as:
(2) We can observe that this formula explicitly constrains the minimum distance between the centroids of the hyperspheres to determine the presence of noise, indicating whether the margin between samples of different identities meets the required criteria derm.
drm p(Darm - di.;) s.t. d,, dist(,, cÿ).
Finally, comprehensively considering both intra-identity consistency and inter-identity discrimination, can be formulated as:
the overall noise for sample set S {sk k=1
M
M
M
cst n2
Σ
Σ
drm
ns
M
1=
We consider that a sample set with favorable level of identity correlation should exhibit the smallest possible amount of ng in order to achieve superior consistency and discrimination.
3.2 Progressive multi-level denoising
Given crawled internet videos, straightforward approach [11, 9] is utilize tracking algorithms to extract person tracklets from each video and treat each tracklet as an unique identity, thereby forming sample set with initial identity correlation. However, due inevitable false positives in tracking and the neglect of the same person across different videos, the identity correlation in this sample is significantly noisy, lacking satisfactory intra-identity consisitency and inter-identity discrimination. Therefore, we propose progressive multi-level denoising strategy to seek superior identity correlation by minimizing the overall ny of the sample set as much as possible.
Single-tracklet denosing. First, we denoise each single tracklet to guarantee the intra-identity consistency. For initial single-tracklet sample with m extracted image features to = {x; } we obtain the denoised sample set by constraining the maximum hypersphere radius through multiple iterations. That is, for the sample sample and the centroid of the remaining samples to identify the sample with the maximum deviation. the deviation exceeds Jest, we remove the sample from t to an exclusion set texe and proceed to the next iteration: otherwise, we terminate the iteration and regard e as the denoised sample set with good intra-identity consistency. The iterative process can be formulated as:
12 of the j-th iteration,
compute the distances between each
x den = arg maxx, dist(x;;, Cer))x)) Vx, E tt / w-o= Di dev: Leee lewe Ux s.l.
dist(x) Co corx) ) Tests
We observe that by iteratively constraining the maximum hypersphere radius of the tracklet, achieve the exclusion of anomalous samples. In essence, this iterative process aims to eliminate the noise cst defined in Eq. thereby ensuring favorable intra-identity consistency for each tracklet.
Short-range single-video denosing. Then, we denoise each single video with multiple tracklets to guarantee both intra-idenn intra-identity consistency and inter-identity discrimination. For an initial single-video

---

Sliding Range
Linking
Liniigg
pomn potbont
begiining
CuTTEE postIo
Figure 2: A toy example for Sliding Range and Linking Relation.
sample set with n denoised tracklets e = [== respective exclusion sets (lere, / we first reallocate the anomalous samples of each exclusion set to ensure their proper tracklet assignment. The reallocation process for an anomalous sample x E Llzz, can be formulated as:
Emin arg min,, dist(x;, ct,) Vt, E Euitk
(5)
Imin Iii Uxf s.1. disl(x;, Ctmme)) < <.estt
Through the aforementioned reallocation process, we can assign each anomalous sample to its correct tracklet, or discard those extreme anomalous samples that do not belong to any tracklet, such as samples devoid of person presence. Subsequently, we merge tracklets with closely located hypersphere centroids to ensure robust inter-identity discrimination, which can be formulated as:
mergeltp. fg) = pl di,t Ct,)) s.t. Vtp. tg v.
(6)
where ja(-) is an unit step function to indicate whether the merge operation is applied to lp and lg-
We can observe that through the aforementioned two-stage process, we achieve the reallocation of anomalous samples and the merging of similar tracklets. In essence, this two-stage process aims to eliminate the overall noise n, as defined in Eq. 3, thereby ensuring both favorable intra-identity consistency and inter-identity discrimination for each video.
Long-range cross-video demosing. Finally, we denoise multiple videos to seek identity correlation across videos. straightforward method for denoising a large-scat scage-scaleaae sample set containing many videos is to concatenate the videos together and treat them as a single ultra-long video, then apply the single-video denoising strategy mentioned above. However, when a video contains N tracklets, the computational complexity of Eq. 6 is O( For an ultra-long video with an overwhelming number of tracklets, this undoubtedly introduces unmanageable computational overhead. To address this issue, we design Sliding Range and Linking Relation to implement ultra-long video denoising.
As shown in Fig. 2, for an ultra-long concatenated video composed of N sub-video tracklets V [ta).N we construct a Sliding Range SR. = {c, r. Pe:--N to slide from the beginning to the end of V. where e represents the index of the current position, and rg represents the half-width of RR. Then, for all tracklets covered by SRe, we obtain the Linking Relation between each tracklet and t. by utilizing the merging function defined in Eq. 6, which formulated as:
LR(t;, to) merge(ti, to) s.t. Vt, E SRaNte,
where LR(-) indicates whether a bidirectional Linking Relation will be established between the two tracklets. Subsequently, after the Sliding Range traverses the entire video, we perform closure operation based on the obtained Linking Relations to derive the closures. Finally, the tracklets within each closure are merged as one single identity.
We can observe that the designing of Sliding Range significantly reduces the computational complexity from O(N2) to O(N), which makes it possible to denoise massive-scale crawled videos. Meanwhile, the closure operation based on Linking Relations ensures long-range identity correlation for different tracklets of the same person across different videos. In essence, this long-range cross-video denosing strategy aims to climinate the inter-discrimination noise n drm as defined in Eq. 2 at the multi-video level, thereby further ensuring favorable inter-identity discrimination of all videos.
Identity correlation seeking. We summarize the whole process of seeking the identity correlation from large-scale crawled
videos as follows. Firstly, we should utilize an off-the- shelf person tracking algorithm to extract person tracklets from each video and assign each tracklet with class label, forming noisy sample set with initial identity correlation. Then, the noisy sample set should undergo single-tracklet denoising, short-range single-video denoising and long-range cross-video denoising in sequence to seek satisfactory identity correlation. Finally, we consider all images within a single denoised tracklet as belonging to the same person, and this denoised sample set can serve as a well-prepared dataset with good identity correlation for subsequent pre-trane pre-training
5

---

3.3 Identity-guided self-distillation
With the sought identity correlation, we propose to uti- lize simple but effective ceentity-guide self- diffddistilationn to pre-train our models, leading to better identity- invariance learning. The whole framework shares similar overall structure as the popular self-distillation pre-training paradigm DINO for general vision [1]. as illustrated Fig. Differently, we introduce the con- cept of identity and conduct contrastive learning on the augmented views from images the same person. We train a student network feg to match the output prob- ability distribution of teacher network fe,, in which the two share the same architecture and are parameter- ized by 0g and 0r, respectively. Given an input image x, both networks predict probability distributions over K dimensions denoted by Ps and Pr, which are ob- tained by normalizing the output of each network with a softmax function and formulated as:
IXI
IXI
softmax
softmax
centering
Teacherfer
EMA
Student fes
Xt
X5
exp (fos (xa)))
Ps(x)
(8) Figure 3: Self-distillation with identity guidance. The overall structure shares a similarity with [1], while the concept of identity is introduced. We illustrate case of N.a 2 and pairs of views (x,, x,) for simplicity. T. and T, represent different random transformations. All transformed views from images of the same person will engage contrastive learning.
Σx= exp (f))(x)(h) (x) /T.)"
where (x(x) is the predicted distribution of the image x for fog and Tg > 0 is hyperparameter controlling the output distribution sharpness. The similar formular holds for Pr with T5.
the
In the following, we detail how we introduce the iden- tity concept into self-distillation. First, with the sought identity correlation, we construct an identity set con- taining Nia images of the same person X = {x Then, given an image X5. we construct a set V2 of different augmented views. This set contains two global views x and x and several local views of smaller resolution. We pass the whole set V& through the student while only pass the global views through the teacher. Finally, the student learns match the output probability distribution of teacher by minimizing the cross-entropy:
ΣΣ Σ x,EX X,EX XE J'EE
Σ HPPr()) XEE, x'yx
min 0s
where Ha, a log and the parameters 0g are optimized with stochastic gradient descent. Following [1]. the teacher network is frozen over epoch and updated with an exponential moving average (EMA) on the student weights. Meanwhile, the output of the teacher network is centered with mean calculated over the batch.
4 Experiments
4.1 Experimental setup
Pre-training and fine-tuning. During pre-training, we directly regard the images of SYNTH- PEDES [55] as a noisy sample set with initial identity correlation. Then, we utilize our proposed progressive multi-level denoising strategy to seek the identity correlation in it. Mcanwhile, to ensure fair comparison with previous methods, we adopt random selection to align the overall sample size with that of the commonly used pre- traiiiinn dataset LUPerson [10]. We name the whole sample set as CION-AL, and it contains 3,898,086 images of 246,904 identities totally. Finally, we utilize the CION-AL and our proposed pre-trained models of 10 architectures in total. During fine-tuning, as a common practice, we directly utilize TransRelD [20] to fine-tune our pre-trained models with input size as 256 x 128 for the supervised person ReID setting, if not specified. Also, the commonly used C-Contrast [6] adopted for the settings of unsupervised domain adaptation (UDA) and unsupervised learning (USL).
guided self-distillation loss to pre-train our models. We

---

Datasets and evaluation metries. Following common practices, we conduct experiments on two datasets, i.e. Market1501 [50] and MSMT17 [40], for evaluating the performance on RelD tasks. Market1 501 and MSMT17 are widely used person RelD datasets, which contain 32,688 images of 1,501 persons and 126,411 1 images of 4,101 persons, respectively. We use the cumulative matching characteristics at top-1 (Rank-1) and mean average precision (mAP) as the evaluation metrics.
Implementation details. To implement the progressive multi-level denoising, we utilize a ViT-Base model [27] trained on the images of UFine6926 [56] as a normalized feature extractor. The intra- consisitency criteria cest and ditermiicriminntiinn criteria d drm are set as 0.2 and 0.18, respectively, We utilize cosine distance to calculate the distance between two samples. The half-width r. of the sliding range set as 1000. For pre- mre-trainnng the global views and local views are resized to 256 x 128 and 128 x 64, respectively. We pre-train our models on 8x V V100 GPUs for 100 epochs. We adopt curriculum learning strategy for setting the images per identity, where the initial Nid is set 2, and at epochs 40, 60, and 80, Nia increases to 4, 6, and 8, respectively. To optimize the utilization of GPU memory, the batch size per GPU and the number of cropped local views are adjusted according to the varying parameter counts of the models. All other settings for pre-training, such learning rate, are consistent with those used in DINO [1]. For fine-tuning, we employ the commonly used methods [20, 38, 6] and make only slight adjustments to settings such as learning rate and batch size. Specific settings and more details can be found in Sec. A.5 of the appendix.
4.2 Comparison with state-of-the-art methods
Supervised person ReID. We compare our CION with several outstanding state- of-the-art methods supervised person RelD in Tab. 1. We divide previous meth- ods by model initialization, i.e., utiliz- ing supervised ImageNet pre-training [7] and self-supervised large-scale person im- ages pre-training as backbone initializa- tion, respectively. Compared with ex- isting best methods pre-trained on Ima- geNet, our method significantly outper- forms them without any extra complex de- signs. For example, our ViT-B achieves remarkable performance gains over the previous best method DCAL [52] Mar- ket1501 and MSMTI7 by 5.6% and 8.6% mAP, respectively. It should be noted that DCAL utilizes much more images for pre- training than ours (14M vs 3.9M). On the other hand, compared with existing best self-supervised methods pre-trained on large-scale person images, our method still shows significant superiority over them. It is worth mentioning that the single-video tracklet-level method ISR [9] utilizes a substantially larger number of person im- ages for pre-training than ours (47.8M vs 3.9M). However, both fine-tuning with MGN [38], our ResNet50-IBN obtains 93.3% and 74.3% mAP on Market1501 and MSMTI7, surpassing ISR by 1.0% and 2.8%, respectively. This effectively demonstrates the necessity of learning identity invariance across different videos.
Table I: Comparison with SoTA methods of supervised person ReID, means the result with the input size as 384 x 128. represents fine-tuning with MGN.
M5r MSMT17
Method
Backhome mAP Rankl RankI
Pre Pre-training ImageNer/K-I.3M
MGN" [38]
R50
87.5
95.1
63.7
85.1
ABDNet [2]
R50
88.3
95.6
60.8
82.3
SAN [22]
R50
88.0
96.1
55.7
79.2
NFormer [39]
R50
91.1
94.7
55.8
77.3
Pre-traitiin imageNei21K
[4M
TransRelD [20]
ViT-B
87.4
94.6
63.6
82.5
DCAL [52]
ViT-B
87.5
94.7
64.0
83.1
Pre -re-training with Large-scale Person Images
LUP (4.2M) [10]
R50
91.0
96.4
65.7
85.5
UPReID (4.2M) [44]
R50
91.1
97.1
63.3
84.3
LUP-NL"* (10.7M) [11]
R50
91.9
96.6
68.0
86.0
PLIP (4.8M) [55]
R50
91.3
96.7
66.2
85.3
ISR (47.8M) [9]
R50-IBN
92.3
96.9
71.5
88.4
Ourst (3.9M)
R50
92.3
97.1
70.1
87.8
Ours th (3.9M)
R50-IBN
93.3
97.3
74.3
89.8
MoCov3 (4.2M) [5]
ViT-S
82.2
92.1
47.4
70.3
TranSSL. (4.2M) [27]
ViT-S
91.1
95.9
66.8
85.5
PASS (4.2M) [53]
VIT-S
92.2
96.3
69.1
86.5
PASS (4.2M) [53]
ViT-B
93.0
96.8
71.8
88.2
PATH (1IM) [33]
VIT-B
89.5
95.8
69.1
84.3
Ours (3.9M)
VIT-S
92.8
96.5
70.3
86.9
Ours (3.9M)
ViT-B
93.1
96.9
72.6
88.5
SOLIDER" (4.2M) [3]
Swin-T 91.6
96.1
67.4
85.9
SOLIDER* (4.2M) [3]
Swin-S
93.3
96.6
76.9
90,8
Ours" (3.9M)
Swin-T
92.2
96.8
71.1
88.1
Ours* (3.9M)
Swin-S 93.4
96.6
77.0
90.4
Unsupervised person ReID. We compare our CION with some SoTA methods of unsupervised
person ReID Tab. 2, which contains two settings, i.e., unsupervised domain adaptation (UDA) and unsupervised learning (USL). All the person-centric pre-trained models are fine-tuned with C- Contrast [6]. For both settings, our method achieves new SoTA performance, significantly surpassing all previous methods. For example, with ResNet50-IBN backbone, our method outperforms the
7

---

Table 2: Comparison with state-of-the-an methods of unsupervised person ReID under two different settings. "Mar" and "MS" denote M50 and MSMT17 respectively. * denotes adding CFS [27].
(a) Comparison on UDA person ReID.
(b) Comparison on USL person ReID.
MS Mar Mar- Ranka mAP RankT
Mar
MS
Method
Backbone
Method
Backbone
MAP Rankl mAP RankI
Pre-training on ImageNet/K-I..
Pre-training on ImageNet/
K-1.3M
DG-Net++ [54]
R50
64.6
83.1
22.1
48.4
MMCL [37]
R50
45.5
80.3
11.2
35.4
MMT [12]
R550
75.6
83.9
24.0
50.1
HCT [49]
R550
56.4
80.0
SpCL [13]
R50
77.5
89.7
26.8
53.7
IICS [43]
R50
72.9
89.5
26.9
52.4
C-Contrast [6]
R50
82.4
92.5
33.4
60.5
C-Contrast [6]
R50
82.6
93.0
33.1
63.3
MCRN [41]
R50
32.8
64.4
MCRN [41]
R50
80.8
92.5
31.2
63.6
Pre-training on Large-scale Person Imagge
Pre-training on Large-scale Person Images
LUP [10]
R50
85.1
94.4
28.3
53.8
LUP [10]
R50
84.0
93.4
31.4
58.8
LUP [10]
R50-IBN
86.9
84.6
42.6
69.1
LUP [10]
R50-IBN
86.4
94.2
39.8
66.1
TranSSL [27]
ViT-S
89.6
95.6
55.0
77.9
TranSSL [27]
ViT-S
89.3
94.8
48.8
74.4
PASS [53]
VIT-S
90.2
95.8
49.1
72.7
PASS [53]
VIT-S
88.5
94.9
41.0
67.0
TranSSL" [27]
VIT-S
89.9
95.5
57.8
79.5
TranSSL" [27]
VIT-S
89.6
95.3
50.6
75.0
Ours
R50
90.6
96.1
57.4
81.3
Ours
R50
90.6
95.7
52.5
78.3
Ours
R50-TBN
92.1
96.5
62.7
84.3
Ours
R50-IBN
91.4
96.1
59.4
82.4
Ours
ViT-S
90.4
95.6
58.0
79.9
Ours
ViT-S
90.6
95.6
55.0
78.4
previous SoTA LUP [27] by large margin, i.e., 20.1% 5% mAP on Market 501 - MSMT17 UDA setting and 19.6% mAP on MSMT17 USL setting, respectively. Noting that LUP is an instance-level method, these results verify the superiority of our identity-level method in person ReID pre-training.
4.3 Generalizing to different model structures
During pre-training, CION im- poses minimal constraints on the model structure, allowing for its generalization across a variety of structures. To verify this, we conduct extensive experiments investigate whether CION can improve the models with differ- ent structures and parameters, en- compassing 32 models with 10 structures, including CNNs, ViTs, and other specialized architec- tures. The baselines are the mod- els with supervised ImageNet [7]
Table 3: CION significantly improves different models. FLOPs are computed based on the input size. means the result with the input size as 384 x 128. 1 represents fine-tuning with MGN. + in- dicates the improvement brought by our CION over the baseline.
Model
Parama.
(M)
FLOPs
(G)
mAP
Market1501
Ranki
MSMT17
ResNecso " 1181
23.51
6.14
92.3114.7)
92.1122.1
20.1/16.6
87.8/12.8)
ResNertor 1* [18]
42,50
9,80
93.1 (1.491
17.111
71.7(16.9)
88.8(12.9)
ResNeti52 [18]
58.14
(3.1(14.4)
91.3111.))
72.4/16.5)
89.2/13.1)
ReSNecS0-BBN [42]
23.51
6.14
93.3(114.1)
(7.3(11.5)
74.3(18.9)
89.8(13.1)
ResNer 181- [47]
42.50
9.80
(113)
97.2(11.3)
76.0(18.4)
90.7(13.1)
ResNe1122-BBN [42]
58.14
94.3114.55
91.31111.)
76.8:1866)
90.8/13.3)
GhossNar 055x [16]
1.31
0.03
0X 10.7)
92.0(15.80
39.6(1 10.6)
65.1
GbosiNet 100x [16]
3.90
0.10
$7.70
95 95.1(16.1)
53.7(112.0)
76.9
GhosdNe 1.3x [16]
6,08
0.16
S5 12.4)
95.2(15.9)
(1.14 15.4(111.7)
EdyoNest-XS [25]
2,14
94.7(16.5)
17.9(118.1)
13.7)
EdgeNest S [28]
5,28
0.63
91 14.5)
96 l(15.9)
63.2(119.8)
83: 63.2(1 13.7)
pre-training. As common prac-
EdgeeNexi-B [28]
17.93
1.92
92. 14.7)
96 8(16.5)
68.6(
86 661 14.7)
tice, we utilize TransReID [20]
RepVIT-MB-9 [16]
4,72
0.56
92. 12.88
06 9/5.8)
68.4(119.8)
86, SQT 12,10
R1p [26]
11.4(111.4) ((2.6)
69.5(147.55
43.4)
to fine-tune the models on ReID datasets with input size as 256 x
RepVTTMMI.5 [16]
13.62
1.54
97 02.8(123.2
6(1 10.3)
69.5(135.1)
87.5 26.2)
Fas [35]
8.45
0.93
91.2(1
96. 96.011
59.0(
11.4(11366)
FasNIT-SA12 [35]
10.55
1.00
90.91 14.7)
95 8(1.9)
59.0(117.D)
81 11.6)
128, if not specified. As the re- sults listed in Tab. 3, our CION consistently yields significant per- formance improvements across all models. When applying TransRelD, the average perfor- mance improvements of all the 32 models over the baselines are 11.7% and 18.8% mAP on Mar- ket1501 and MSMT17, respec- tivelyy With CION demonstrat- ing such model- we construct a model zoo named RelDZoo, which contains all the CION pre-trained models with spanning structures and parameters, to meet diverse research and application needs in this field.
FudNiT-SA24 [35]
20.53
69.5(132.6)
23.8)
ResNer18 [18]
11.18
2.00
89.31 10.9
95.6114.5)
55.6(114.8)
79.6/112.0)
Re<Ne.00 [16]
23.51
4.09
92.5 19.9)
96 13399
67.7(115.5)
86 66.45)
Ro [18]
42,50
6.54
94 04.01 (1.88
97. 4(16.)
73.2(117.3)
ResNed15
58,14
97.101300) (4.2
14.5(118.2)
89.3(110.4
ReNNel18-IBN [42]
11.18
2.00
90.00 10.2)
96 44.44
79.5(1 16.1(1
10.4
Ro-IBe50-IBN [42]
23.51
4.09
7(
96 (131)
72.3(1 18.6)
88
11.0)
ResNec101: IBN (42)
42,50
6.54
94.3
97.3(13.))
15.8(118.0)
89,8
152-11N [42]
94.6(17.2)
97.6(13.0)
76.7(117.8)
18.9)
Com-NestT [26]
27.82
2.92
92 (1.0(
96.9116.0)
68.9(122.7)
14.8
Com-Nest- S [26]
49.45
5.68
92 12.55
96. ((1.2)
88.0 69.9(121.8)
Com-NestBB [26]
87,57
10.04
13.2)
96 (5.9))
72.7(124.1)
89.00 14.5)
VOLO
25.84
93.00(7.88
96.9(13.3)
73.9(115.8)
18.7)
VOLO-D2 [47]
57.62
9.15
92.1(17.1)
96. 96.4(12.9)
70.9(111.6)
16.6)
VOLO-D3 (47]
85.26
13.26
50.5(15.6)
95 95.7(11.8)
83.0K 62.8(12.1)
VIFT
6.43
1.55
91.(110.55
96.3115.1)
65.0(118.9)
14.9
23.39
92.8(11.0)
96.5(12.6)
70.3(116.7)
$6.9(11116)
ability,
VITB [S
89.15
12:37
93.1(16.4)
96.9112.3)
72.6(111.4)
88.5/76.6)
Swin [
27.55
3.13
92.5(198))
97.0(14.3)
(1.1(1996))
87.6(112.6)
Swin S [25]
48.85
3.93
93.6(19.1)
96 8(136)
19.7)
89, 59.6K1 10.9)
Swin [25]
86.76
94.0(19.6)
96.9113.0)
76.7(120.7)
90.3/712.11
Average Impr.
111.7
15.0
118.8
112.9
8

---

4.4 Ablation studies and analyses
Identity-level correlating holds significance. We validate the significance of identity-level cor- relating in learning better person representations through both feature distribution visualization and performance comparison. For visualization, collect samples of 15 persons randomly se- lected from the gallery of Market 551 [50], and
LUP-NI (ww.aaaaeeeaaaall......
CIDNioun
visualize their features via I-SNE [34] in Fig. 4.
Figure 4: The I-SNE [34] visualization of extracted image features, Our CION enjoys good identity consistency and discrimination, while the instance- level method LUP [10] and tracklet-level method LUP-NL [11] do not (marked in red).
The instance-level method LUP [10] and the
tracklet-level method LUP-NN [11], which are respectively pre-trained on 4.2M and 10.7M per- son images, are employed as a comparison. As we can see, the features of LUP are almost ran- domly distributed, while the features of LUP-NL remain challenging to aggregate at the identity level. However, our CIÓN consistently enjoys good intra-consistency and inter-discrimination. For performance comparison, we compare the fine-tuning results of our CION with two popular identity-ignored pre- pre-training methods LUP [10] and DINO [i] in Tab. 4. For a fair comparison, the models are all ResNet50-
Table 4: Our identity-level method achieves sig- nificant leading performance compared with two
identity-ignored pre-training methods [10, 1J.
M5rket1501
MSMT17
Method
Pre-train mAP Ranki
mAP
RankT
LUP+MGN CION-AL 91.4
96.6
68.2
86.4
DINO+MGN CION-AL
90.9
96.3
66.7
85.8
Our++MGN CION-AL 93.3
97.3
74.3
89.8
IBN [42] pre-trained on our CION-AL and the fine-tuning algorithm is MGN [38]. We can see that the model pre-trained with CION achieves significant leading performance. These results demonstrate the significance of implementing identity-level correlating for person ReID pre-training.
Effectiveness of progressive multi-level de- noising. Fig. shows the ablation study of each denoising strategy. The number of training sam- ples for all groups set to 3.9M for a fair com-
95
99
77
32
D1
22
03
8/
010
71.
12111
18%
53-
71
18
parison. The backbone is ResNet50-IBN [42] and the fine-tuning algorithm is MGN [38]. We can observe that each strategy significantly im- proves the fine-tuning performance of the pre- trained model on downstream datasets. Partic- ularly, the cross-video denoising results in the most substantial improvement, further increas- ing the mAP by 0.6% and 1.9% on M5r and MSMT17, respectively. These results verify the effectiveness of our progressive multi-level denoising strategy. which guarantees satisfactory identity correlation seeking.
92-
.66
68-
96
65
Mariettal1
M1SM117
Figure 5: Results with different denoising strate-
gies: (I) without denoising, (2) add single-tracklet denoising, (3) further add single-video denoising, (4) further add cross-video denoising.
Scalability to large-scale training data. We also study the scalability of our method to large- scale training data. Aside from the training data size, the rest settings are same with the exper- iment mentioned above. As shown in Fig 6, with training data size ratio varying from 10% to 100%, the fine-tuning performance on down- stream datasets consistently keeps improving, especially on the more challenging MSMT17.
80-
52-
91
67
20%
Thaning
Figure 6: The fine-tuning performance consistently keeps improving as the training data increases. This indicates our method's potential for further improved performance with increasing training data.
Conclusion
We present a novel framework, CION, learn person re- re-identifictton pre-training. In particular, we model the identity correlation seeking process as a progressive multi-level denoising problem, with a novel noise concept defined. Also, we propose an identity-guided self-distillation loss to implement better large-scale pre-training. Compared to existing pre-training methods, we achieve significantly leading performance with even fewer training samples. Mcanwhile, contribute model zoo to promote the development of this field.
invariance from cross-video person images for

---

Acknowledgements. This work was supported by the National Natural Science Foundation of China No.62176097, and the Hubei Provincial Natural Science Foundation of China No.2022CFA055. We gratefully acknowledge the support of MindSpore, CANN (Compute Architecture for Neural Networks) and Ascend Al Processor used for this research.
References
[1] Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In ICCV, pages 9650-9660, 2021.
[2] Tianlong Chen, Shaojin Ding, Jingyi Xie, Ye Yuan, Wuyang Chen, Yang Yang, Zhou Ren, and Zhangyang Wang. Abd-net: Attentive but diverse person re-identification. In ICCV, pages 1551-8361, 2019.
[3] Weihua Chen, Xianzhe Xu, Jian Jia, Hao Luo. Yaohua Wang. Fan Wang, Rong Jin, and Xiuyu Sun. Beyond appearance: a semantic controllable self-supervised learning framework for human-centric visual tasks. In CVPR, pages 15050-15061, 2023.
[4] Xinlei Chen, Haoqi Fan, Ross Girshick, and Kaiming He. Improved baselines with momentum contrastive learning, arXiv preprint arXiv:2003. 04297, 2020.
[5] Xinlei Chen, Saining Xie, and Kaiming He. An empirical study of training self-supervised vision transformers. ICCV, pages 9640-9649, 2021.
[6] Zuozhuo Dai, Guangyuan Wang, Weihao Yuan, Siyu Zhu, and Ping Tan. Cluster contrast for unsupervised person re-identification. ACCV, pages 1142-1 1160, 2022.
[7] Jia Deng, Wei Dong. Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large- scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248-255. leee, 2009.
[8] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. ICLR, 2021.
[9] Zhaopeng Dou, Zhongdao Wang. Yali Li, and Shengjin Wang. Identity-seeking self-supervised representation learning for generalizable person re-identification. In ICCV, pages 15847-15858, 2023.
[10) Dengpan Fu, Dongdong Chen, Jianmin Bao, Hao Yang, Lu Yuan, Lei Zhang, Houqiang Li, and Dong Chen. Unsupervised pre-training for person re-identification. In CVPR, pages 14750-14759, 2021.
[11] Dengpan Fu, Dongdong Chen, Hao Yang, Jianmin Bao, Lu Yuan, Lei Zhang, Houqiang Li, Fang Wen, and Dong Chen. Large-scale pre-training for person re-identification with noisy labels. In CVPR, pages 2476-2486, 2022.
[12) Yixiao Ge, Dapeng Chen, and Hongsheng Li. Mutual mean-teaching: Pseudo label refinery for unsupervised domain adaptation on person re-identification. arXiv preprint 2rXiv:2001.0/526, 2020.
[13] Yixiao Ge, Feng Zhu, Dapeng Chen, Rui Zhao, al. Self-paced contrastive learning with hybrid memory for domain adaptive object re-id. NeurlPS, 33:1 1309-11321, 2020.
[14] Yunpeng Gong, Liging Huang, and Lifei Chen. Person re-identification method based on color attack and joint defence. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages
2022.
[15] Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Guo, Mohammad Gheshlaghi Azar, al. Bootstrap your own latent-a new approach to self-supervised learning. NeurIPS, 33:21271- 21284, 2020.
[16] Kai Han, Yunhe Wang. Qi Tian, Jianyuan Guo, Chunjing Xu, and Chang Xu. Ghostnet: More features from cheap operations. In CVPR, pages 1580-1589, 2020.
[17] Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. In CVPR, pages 16000-16009, 2022.
10

---

[18] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image
recognition. In CVPR, pages 770-778, 2016.
[19]
Lingxiao He, Xingyu Liao, Wu Liu, Xinchen Liu, Peng Cheng, and Tao Mei. Fastreid: A pytorch toolbox for general instance re-identification. arXiv preprint arXiv:2006.02631 2020.
[20] Shuting He, Hao Luo, Pichao Wang, Fan Wang, Hao Li, and Wei Jiang. Transreid: Transformer-
based object re identification. In ICCV, pages 15013-15022, 2021.
[21] Jiahao Hong. Jialong Zuo, Chuchu Han, Ruochen Zheng, Ming Tian, Changxin Gao, and Nong Sang. Spatial cascaded clustering and weighted memory for unsupervised person re- identification. arXiv preprini wrXiv:2403.00261. 2024.
[22] Xin Jin, Cuiling Lan, Wenjun Zeng. Guoqiang Wei, and Zhibo Chen. Semantics-aligned representation learning for person re-identification. In AAAI, volume 34, pages 11173-11180, 2020.
[23] He Li, Mang Ye, and Bo Du. Weperson: Learning generalized re-identification model from all- weather virtual data. In Proceedings of the 29th ACM international conference on multimedia, pages 3115-3123, 2021.
[24] He Li, Mang Ye, Ming Zhang, and Bo Du. Anl one framework for multimodal re-identification in the wild. In Proceedings of the IEEE/CVE Conference on Computer Vision and Puttern Recognition, pages 17459-17469, 2024.
[25] Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In ICCV, 2021.
[26] Zhuang Liu, Hanzi Mao, Chao- Wu, Christoph Feichtenhofer. Trevor Darrell, and Saining Xie. A convnet for the 2020s. CVPR, 2022.
[27] Hao Luo, Pichao Wang. Yi Xu, Feng Ding, Yanxin Zhou, Fan Wang, Hao Li, and Rong Jin. Self-supervised pre-training for transformer-based person re-identification. arXiv preprint arXiv:2111.12084, 2021.
[28] Muhammad Maaz, Abdelrahman Shaker, Hisham Cholakkal, Salman Khan, Syed Waqas Zamir, Rao Muhammad Anwer, and Fahad Shahbaz Khan. Edgenext: efficiently amalgamated cnn- transformer architecture for mobile vision applications. In ECCV, pages 3-20. Springer, 2022.
[29] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In /CML, pages 8748-877 8763. PMLR, 2021.
[30] Jiangming Shi, Xiangbo Yin, Yeyun Chen, Yachao Zhang. Zhizhong Zhang, Yuan Xie, and Yanyun Qu Multi-memon matching for unsupervised disible-infrared person re-identification. In Computer Eision-ECCI 2024: 18th European Conference, page 456-474, 2024.
[31] Jiangming Shi, Xiangbo Yin, Yachao Zhang, Zhizhong Zhang, Yuan Xie, and Yanyun Qu. Learning commonality, divergence and variety for unsupervised visible-infrared person re- identification. arXiv preprint arXiv:2402. 19026v2, 2024.
[32] Jiangming Shi, Yachao Zhang, Xiangbo Yin, Yuan Xie, Zhizhong Zhang, Jianping Fan, Zhongchao Shi, and Yanyun Qu. Dual pseudo-labels interactive self-training for semi-supervised visible-infrared person re-identification. In Proceedings of the IEEE/CVF International Confer- ence on Computer Vision, pages 11218-12222 11228, 2023.
[33] Shixiang Tang, Cheng Chen, Qingsong Xie, Meilin Chen, Yizhou Wang, Yuanzheng Ci, Lei Bai, Feng Zhu, Haiyang Yang, Li Yi, et al. Humanbench: Towards general human-centric perception with projector assisted pretraining. CVPR, pages 21970-21982, 2023.
[34] Laurens Van Der Maaten. Accelerating t-sne using tree-based algorithms. The journal of machine learning research, 15(1):3211-3255 2014.
[35] Pavan Kumar Anasosalu Vasu, James Gabriel, Jeff Zhu, Oncel Tuzel, and Anurag Ranjan. Fastvit: A fast hybrid vision transformer using structural reparameterization. In ICCV, pages 5785-5795, 2023.
[36] Ao Wang, Hui Chen, Zijia Lin, Hengjun Pu, and Guiguang Ding. Repvit: Revisiting mobile cnn from vit perspective. CVPR, 2024.
[37] Dongkai Wang and Shiliang Zhang. Unsupervised person re-identification via multi-label classification. In CVPR, pages 10981-10990, 2020.

---

[38] Guanshuo Wang. Yufeng Yuan, Xiong Chen, Jiwei Li, and Xi Zhou. Learning discriminative features with multiple granularities for person re-identification. In ACM MM, pages 274-282, 2018.
[39] Haochen Wang, Jiayi Shen, Yongtuo Liu, Yan Gao, and Efstratios Gavves. Nformer: Robust
person re-identification with neighbor transformer. In CVPR, pages 7297-7307, 2022.
[40]
Longhui Wei, Shiliang Zhang, Wen Gao, and Qi Tian. Person transfer gan to bridge domain gap for person re-identification. In CVPR, pages 79-88, 2018.
[41] Yuhang Wu, Tengteng Huang, Haotian Yao, Chi Zhang, Yuanjie Shao, Chuchu Han, Changxin Gao, and Nong Sang. Multi-centroid representation network for domain adaptive person re-id. In AAAI, volume 36, pages 2750-2758, 2022.
[42] Pan Xingang, Luo Ping, Shi Jianping, and Tang Xiaoou. Two at once: Enhancing learning and generalization capacities via ibn-net. In ECCV, 2018.
[43] Shiyu Xuan and Shiliang Zhang. Intra-inter camera similarity for unsupervised person re- identification. In CVPR, pages 11926-11935 2021.
[44] Zizheng Yang, Xin Jin, Kecheng Zheng. and Feng Zhao. Unleashing potential of unsupervised pre-training with intra-identity regularization for person re-identification. CVPR, pages 14298-14307, 2022.
[45] Mang Ye, He Li, Bo Du, Jianbing Shen, Ling Shao, and Steven CH Hoi. Collaborative refining for person re-identification with label noise. IEEE Transactions on Image Processing. 31:379-391, 2021.
[46] Xiangho Yin, Jiangming Shi, Yachao Zhang, Yang Lu, Zhizhong Zhang, Yuan Xie, and Yanyun Qu. Robust pseudo-label learning with neighbor relation for unsupervised visible-infrared person re-identification. arXiv preprint arXiv:2405.0566
2024.
[47] Li Yuan, Qibin Hou, Zihang Jiang, Jiashi Feng, and Shuicheng Yan. Volo: Vision outlooker for visual recognition. IEEE TPAMI, (555)66575-6 2022.
[48] Gong Yunpeng. Zhong Zhun, Luo Zhiming, Qu Yansong, Ji Rongrong, and Jiang Min. Cross-modality perturbation synergy attack for person re-identification. arXiv preprint arXiv:2401.10090 2024.
[49] Kaiwei Zeng, Munan Ning, Yaohua Wang, and Yang Guo. Hierarchical clustering with hard- batch triplet loss for person re-identification. In CVPR, pages 13657-13665, 2020.
[50] Liang Zheng. Liyue Shen, Lu Tian, Shengjin Wang, Jingdong Wang. and Qi Tian. Scalable person re-identification: benchmark. In ICCV, pages 1116-112, 2015.
[51] Zhedong Zheng, Liang Zheng, and Yang. Unlabeled samples generated by gan improve the person re-identification baseline in vitro. In ICCV, pages 3754-3762, 2017.
[52] Haowei Zhu, Wenjing Ke, Dong Li, Ji Liu, Lu Tian, and Yi Shan. Dual cross-attention learning for fine-grained visual categorization and object re-identification. In CVPR, pages 4692-4702, 2022.
[53] Kuan Zhu, Haiyun Guo, Tianyi Yan, Yousong Zhu, Jinqiao Wang, and Ming Tang. Pass: Part-aware supervied pre-training for person re- re-identification. In ECCV, pages 198 214. Springer, 2022.
[54] Yang Zou, Xiaodong Yang, Zhiding Yu, BVK Vijaya Kumar, and Jan Kautz. Joint disentangling and adaptation for cross-domain person re-identification. In ECCV, pages 87-104. Springer, 2020.
[55] Jialong Zuo, Jiahao Hong, Feng Zhang, Changqian Yu, Hanyu Zhou, Changxin Gao, Nong Sang, and Jingdong Wang. Plip: Language-imal pre-training for person representation learning, 2024.
[56] Jialong Zuo, Hanyu Zhou, Ying Nie, Feng Zhang, Tianyu Guo, Nong Sang, Yunhe Wang, and Changxin Gao. Ufinebench: Towards text-based person retrieval with ultra-fine granularity. CVPR, 2024.
12

---

A Appendix
A.1 Broader impact
This paper proposes cross-video Seentitt--oorllating pre-training framework for person re- identification, and contributes a pre-trained model zoo with spanning model structures and parameters. Our framework and models can help identity and track pedestrians across different cameras, thus boosting the development of smart retail, smart transportation, smart security systems and so on in the future metropolises. In addition, our proposed pre-training framework is quite general and not limited to the specific research field of person ReID. It can be well extended to broader research areas with identity concept such as vehicle ReID.
Nevertheless, the application of person ReID, such as for identifying and tracking pedestrians in surveillance systems [14, 48, 31, 32, 30], might raise privacy concerns. It typically depends the utilization of surveillance data for training without the explicit consent of the individuals being recorded. Therefore, governments and officials need to carefully establish strict regulations and laws to control the usage of ReID technologies. Otherwise, person RelD technologies [46, 24, 23, 45] can potentially equip malicious actors with the ability surveil pedestrians through multiple cameras without their consent. Meanwhile, the research community should also avoid using any dataset with ethics issues. For example, DukeMTMC [51] has been taken down due to the violation of data collection terms and should no longer be used. We would not evaluate our models on DukeMTMC related benchmarks as well. Furthermore, we should be cautious of the misidentification of the ReID systems to avoid possible disturbance. Also, note that the demographic makeup of the datasets used not representative of the broader population.
At the same time, we have utilized a substantial amount of containnn video data from the internet for pre- pre-training purposes. Consequently, it is inevitable that the resulting models may inherently contain information about these persons. Researchers should adhere to relevant laws and regulations, and strive to avoid using our models for any improper invasion of privacy. We will have a gated release of our models and training data to avoid any misuse. We will require that users adhere to usage guidelines and restrictions to access our models and training data. Meanwhile, all our open-sourced assets can only be used for research purpose and are forbidden for any commercial use.
A.2 Limitations
CION presents a preliminary attempt to correlate person images across different internet videos to implement person re-identification pre-training. Despite its effectiveness existing public datasets, CION may still be difficult to learn good fine-grained person representations for does not explicitly achieve further fine-grained information mining. Meanwhile, due to the limitations of the feature extractor's 's representational capacity, it is inevitable that some extremely difficult noise will be hard to remove during the denoising process. As result, the automatically sought identity correlation may still have certain discrepancy in accuracy when compared to manually annotated correlation, which may introduce a certain degree of ambiguity during the training process. Additionally. the Sliding Range and Linking Relation strategy we proposed to conserve computational resources may result in difficulty seeking correlations between tracklets of the same person across different videos that are spaced extremely far apart. Also, as we have followed the conventional practice of previous works by utilizing large amount of person-containnng internet video data to implement pre-training, there is a potential for privacy and security issues to some extent. Therefore, in our subsequent work, we will focus on addressing fine-grained issues and improving the accuracy of identity correlations. We will take every possible measure prevent the misuse of our models and dataset as well.
A.3 Safeguards for dataset and models
To address the privacy concerns associated with crawling videos from the internet and the application of person RelD technology, we will implement a controlled release of our models and dataset, thereby preventing privacy violations and ensuring information security. We require that the crawled images are sourced from legitimate video websites to avoid the inclusion of harmful content. We will also require that users adhere to usage guidelines and restrictions to access our models and dataset. For instance, we have drafted the following regulations that must be adhered to, which will be refined and elaborated in subsequent releases:
13

---

1. Privacy: All individuals using the models in RelDZoo and the CION-AL dataset should agree to protect the privacy of all the subjects in The users should bear all responsibilities and consequences for any loss caused by the misuse.
2. Redistribution: The models in Rell>Zoo and the CION-AL dataset, either entirely partly, should not be further distributed, published. copied, or disseminated in any way or form without prior approval from the creators, matter for profitable use or not.
The models RelDZoo and the CION-AL dataset, entirely, partly, or any format derived thereof, is not allowed for commercial use.
Commercial Use:
The models in RelDZoo and the CION-AL dataset, either entirely or partly, is not allowed to be modified.
Modification:
In parallel, will require users provide relevant information and will rigorously screen the submitted details to restrict access to our dataset and models by institutions or individuals with a history of privacy violations.
A.4 Licenses for existing assets
We utilize some existing assets conduct our research, which can be categorized three types: code, data and models. All assets used in our paper are properly credited and all original paper are cited. We list the license and URL for each asset below.
Code
1. Emerging Properties in Self-Supervised Vision Transformers [1]. (DINO)
Apache 2.0 License, URL: http:///t///ggggttoo/.oooooooooa
TransReID:
Transformer-basedd Object Re-Identiffication [20]. (TransReID)
MIT License, URL:
3. Cluster Contrast for Unsupervised Person Re-Ieentification [6]. (C-Contrast)
MIT License, URL: http://gggg/gtthhh..o/.o/aaa/aa/ cont
4. Unsupervised Pre-training for Person Re-identification [10]. (LUU)
MIT License, URL: https: httppp//ggtttuo..coo///aaaa////s
Self-Supervised Pre-Training for Transformer-Based Person Re-Identification [27]. (TranSSL)
MIT License, URL: tttpp://github. httpg/ata...cccaaooooo//raaaaaaa
6. FastRelD: A Pytorch Toolbox for General Instance Re-identification [19]. (FastRelD)
Apache 2.0 License, URL: htt.:/ttgge/gggha.....c/////////
Data
1. PLIP: Language-Image Pre-training for Person Representation Learning [55]. (SYNTH-PEDES)
MIT License, URL:
UFineBench: Towards Text-based Person Retrieval with Ultra-fne Granularity [56]. (UFine6926)
UFineBench License, URL:
3. Person Transfer GAN to Bridge Domain Gap for Person Re-Identifiiation [40]. (MSMT17)
MSMT17 Release Agreement, URL: https://vvee wwwppppuwcccooopp http://amcccommmmaetteeeeee html
4. Scalable Person Re-Identification: A Benchmark [50]. (Market1501)
MIT License, URL: http::/zhengg lab. cecs. anu, edu,
Models
Deep Residual Learning for Image Recognition [18]. (ResNet)
BSD-3-Clause license, URL: http:p:://ggtat....o////ttrrrrde
2. Sltance-Batch Normalization Network [42]. (ResNet-IBN)
14

---

MIT License, URL: https://github.. hatpa//cccooiiigggggga///lt/
GhostNet: More Features from Cheap Operations [16]. (GhostNet)
Apache 2.0 License, URL: Ukt.hhrtpps//gghhhbb....mmaaaaae http://gg.hab.cccoomaaaaeeie-eee
4. EdgeNeXt: Efficiently Amalgamated CNN-Transiormer Architecture for Mobile Vision Applica-
tions [28]. (EdgeNext)
MIT License: URL: http://:thhu..oomm/mmaa/2/////g/
5. RepVIT: Revisiting Mobile CNN From ViT Perspective [36]. (RepViT)
Apache 2.0 License, URL: UBt.::atttpp//ggaaabbbo//////l/0
FastViT: A Fast Hybrid Vision Transformer using Structural Reparameterization [35]. (FastViT)
FastVIT License, URL:
7. A ConvNet for the 2020s [26]. (ConvNext)
MIT License, URL:
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. (ViT)
Apache 2.0 License, URL: http:: httbtpps//ggttbb...o//gggg.esss/ httpm//ogggeeererrcccooo/oc trresioonnttaaasfffmeeeem
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows [25]. (Swin)
MIT License, URL:
Transformer
10. VOLO: Vision Outlooker for Visual Recognition [47]. (VOLO)
Apache 2.0 License, URL: https: URL:httts::/giitubbbco wttp:hhhww..oommalaaaaaaaaaaaaae
Table 5; The batch sizes, numbers of local cropped
A.5
Pre-training setting for each model
views and rough training time for each model.
Model
batch size local views time(das)
Due to the varying memory requirements of different models during the pre-training pro- cess, we optimize the utilization of computa- tional resources by setting different batch sizes and numbers of local cropped views for each model. We report the batch sizes, numbers of local cropped views and rough training time for each model in Tab. 5, Each model is trained on 8 x V100 GPUs. The pre-training dataset is the whole CION-AL, which has 3,898,086 images of 246.904 persons.
GhostNet 0.5x [16]
216
4.8
GhostNet 1.0x [16]
192
8
5.7
GhostNet 1.3x [16]
168
8
7.2
FegeNext-XS [28]
192
8
5.1
Edge [28]
144
8
6.5
EdgeNNext- [28]
7.3
RepViT-M0.9 [36]
144
8
5.1
RepViT-M1.0 [36]
144
8
6.2
RepViT-MI.5 [36]
96
8.1
FastViT-S12 [35]
8
5.3
FastViT-SA12 [35]
120
8
6.8
FastViT-SA24 [35]
96
8
9,3
ResNet18 [18]
192
10
3.6
A.6 Samples used in t-SNE visualization
ResNec50 [18]
120
8
5.1
ResNet101 [18]
6.6
ResNet152 [18]
72
8
9.7
For reproducing the t-SNE visualization results shown in Fig. 4, we display the samples we ran- domly selected from Market1501 [50] in Fig. There are totally 15 persons with 20 cropped images each. We obscure the faces of each per- son to avoid the privacy violations. Researchers can use these samples and official models of LUP [10] and LUP-NL[[1] to reproduce the visualization results.
ResNet18-IBN [42]
192
10
3,6
ResNet50-IBN [42]
120
8
5.1
ResNet101-IBN [42]
96
8
6.6
52- [42]
8
9.7
ConyNext-TT [26]
120
7
6.2
ConvNext-S [26]
96
6
7.3
Conv-NettB [26]
72
6
9.5
VIT-T [8]
5.3
ViT-S [8]
96
8
6.7
ViT-B [8]
72
8
8.6
Swin-T [25]
96
7
5.8
A.7 Linear training time increasing
Swin-S [2.5]
72
6
8.3
Swin-B [25]]
11.6
We have assessed how the training time of CION scale with pre-training dataset size. We use ResNet50 to conduct the experiment. We record
VOLO-DI [47]
120
4
6.2
VOLO-D2 [47]
72
4
9.4
VOLO-D3 [47]
48
4
13.3
15

---

the training time required by CION as the pre- training data size increases from 10% 100%. As shown in Fig. 7, the training time of our CION increases almost linearly with the growth in the size of the pre-trained dataset. This lin- ear increase in training time ensures that our CION can be effectively applied to large-scale pre-training. This indicates that we can achieve better performance by adding more training data without incurring an unbearable training cost.
[dae] 12 5.
306
0-
0075
70%
Tostzing Date Size
Figure 7: The time consumption for pre-training on CION-AL with different data size ratio.
A.8 More results for unsupervised person RelD
To further assess the model structure adaptability of our CION, we evaluate the performance of more CION pre- trained models for unsupervised person RelD, inclucing ResNet [18], ResNet- IBN [42] and ViT [8]. We report the results in Tab. 6. As we can see, all the models achieve superior performance for different unsupervised settings on Mar- ket1501 [50] and MSMT17 [40]. Mean- while, as the number of parameters in- creases, the performance of each model consistently improves. These results further indicate that our CION can be effectively used to pre-train various models, enabling them to achieve significant performance improvements for person ReID.
Table 6:
The performance of different CION pre-trained models for unsupervised person ReID. "Mar" and "MS" denote 550 and MSMT17 respectively.
Model
mAP
MS
MS-MMar
Rank1
mAP
Mar-MS
Rankl
mAP
Markett501
Rankl
MAP
MSMT17
Rankl
ResNe150
95.7
52.5
ResNet10: [18]
93.1
96.8
67.5
36.9
93.2
97.0
64.2
ResNet152 [18]
93.6
97.0
70.1
58.1
93.3
97.1
68.8
$7.6
ResNe150-1BN [+2]
92.1
96.5
62.7
84.3
91.4
96.1
59.4
82.4
ResNett0: [42]
ResNet152-10 [12]
94.0
97.1
73.7
39.2
96.3
97.1
71.7
89.0
VIFT 181
88.0
94.7
44.3
09.3
88.3
94.5
43.1
69.6
VITS
79.9
90.5
95.6
55.0
73.4
Figure 8: The samples used in t-SNE feature distribution visualization.
16

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The main claims made in the abstract and introduction our paper accurately reflect our paper's contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: We conduct an in-depth discussion of the limitations of our work, which can found Sec. A.2 of the appendix.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-speciiiatiinn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
17

---

Answer: [NA]
Justification:
Our work focuses on computer vision applications, not including theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Question:
Answer: [Yes]
We have fully provided all the details of our proposed methods in the main text and appendix, to guarantee the reproducibility of our paper's main experimental results. Meanwhile, we will open-source all of our code, data, and models after our paper accepted. These details and open-source initiatives will ensure the reproducibility of our work and make significant contribution to the entire community.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution be necessary to either make it possible for others to replicate the model with the same dataset, provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
specific model and empirical evaluation, may
While NeurlPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm.
(b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (c.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the partieular way they provide for reproducibiliity In the case of closed-source models, it may be that access to the model is limited in
18

---

some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [No]
Justification: To address privacy concerns associated with dataset containing persons and the application of person RelD technology, we will implement a controlled release of our code, data, and models. Therefore, ensure private information security, would not provide open access to our data and code during the paper submission process, We provide training and testing logs in the supplementary materials to ensure the authenticity and validity of our results. Notably, we will publicly share the application link for all our sources after our paper is accepted. Applicants will be required to adhere to relevant guidelines and regulations to access our sources, and we will conduct strict review of their qualifications to prevent any privacy violations.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.. cc/
public/guided http://odSummsssoonn.eee
for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: / wwtpcc/cpppcceeeeec...eececceeee wwwiii//g/////eeeeeeeeeeeeeieeee Sttp://ooeeSSSeeeeeeeeeee issiopolicy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification: Our paper specifies all the training and test details necessary to understand the results.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
19

---

Answer: [No]
Justification: Due to pre-training technology is comparably computational expensive, the error bars are not reported in the paper like nearly all related work. We provide training and testing logs in the supplementary materials to ensure the authenticity and validity of our results.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by enror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report I-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (c.g. negative error rates).
If error bars reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Our paper provides sufficient information on the computer resources needed to reproduce the experiments, which can be found in Sec. 4. and Sec. A.5.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https:
csCuideliness
Answer: [Yes]
Justification: The research conducted in the paper complies with the NeurIPS Code of Ethics in every respect.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
20

---

The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Our paper discusses both potential positive societal impacts and negative societal impacts of the work performed Sec. A.1 of the appendix.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g.. disinformation, generating fake profiles, surveillance), fairness considerations (e.g.. deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied partieular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [Yes]
Justification: Our paper describes safeguards that have been put in place for responsible release of data and models in Sec. A.3 of the appendix.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
21

---

Answer: [Yes]
The creators and original owners of assets used in the paper are properly credited, and the license and terms of use are explicitly mentioned and properly respected, which can be found in Sec. A.4 of the appendix.
Justification:
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, papersvithcode, thcode. com/ datasets
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: To address privacy concerns associated with our assets including code, data, and models, we will implement controlled release of our assets. Therefore, ensure private information security, we would not release our assets at submission time. We will publicly share the application link for our assets after our paper is accepted. Applicants will be required to adhere to relevant guidelines and regulations to access our assets, and we will conduct strict review of their qualifications prevent any privacy violations.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: Our paper does not involve crowdsourcing research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
22

---

According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/teview based on the requirements of your country institution) were obtained?
Answer: [NA]
Justification: Our paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
23


Reviews:
[{"review_id": "SonN6geus3", "reviewer": "Reviewer_hfuf", "summary": "This paper present a novel framework CION, to learn identity-invariance from cross-video person images for person re-identification pre-training. In particular, they model the identity correlation seeking process as a progressive multi-level denoising problem, with a novel noise concept defined. Also, they propose an identity-guided self-distillation loss to implement better large-scale pre-training.", "questions": "1.It is significantly valuable to eliminate the problem of false positives of identity tags caused by tracking errors, How does it work in this paper?\n2.It is significantly valuable for the model to learn better identity representations without knowing the identity tags of the same person across different videos. How does it work in this paper?", "limitations": "The authors have given the limitations of the proposed model, the authors mentioned that the automatically sought identity correlation may still have a certain discrepancy in accuracy when compared to manually annotated correlation, which may introduce a certain degree of ambiguity during the training process. In their subsequent work, they will focus on addressing fine-grained issues and improving the accuracy of identity correlations.", "rating": 5, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The identity-invarience of the same person across different videos is significantly neglected by these single-video tracklet-level methods. This paper present a cross-video identity-correlating pre-training framework, which explicitly learns identity-invariance in long-range cross-video person images.", "weaknesses": "This paper compare the fine-tuning results with two popular identity-ignored pre-training methods LUP and DINO in Tab. 4, how about with other identity-ignored pre-training methods？", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Sincerely thanks for your constructive comments. The concerns are answered as follows.\n\n**(i) Comparison with other identity-ignored pre-training methods.**\n\nThanks for your nice concern. Firstly, the experiment in Tab. 4 has yielded the following results: \n\nwhen pre-trained on exactly the same dataset (CION-AL) and fine-tuned with exactly the same downstream algorithm (MGN) and datasets (Market1501 and MSMT17), our identity-guided pre-training method demonstrates a significant performance lead compared to other two representative identity-ignored pre-training methods.(DINO and LUP)\n\nMeanwhile, the results in Tab. 1 also validate the superiority of our method over other identity-ignored methods. For your easy reference, we list the comparison results with these methods below. The results of these methods are all derived from official papers or repositories.\n\n| Method | Pre-training Dataset |Fine-tuning Algorithm| Backbone | Mar mAP |Mar R@1| MS mAP| MS R@1|\n| --- | --- | --- | --- | --- | --- | --- |  --- |\n| LUP | LUPerson(4.2M) |MGN| ResNet50 | 91.0 |96.4  | 65.7 | 85.5 |\n| UPReID | LUPerson(4.2M) | MGN | ResNet50 | 91.1 |97.1 | 63.3 |84.3 |\n| **Ours** | CION-AL(3.9M) | MGN | ResNet50 | **92.3** |**97.1** | **70.1** |**87.8** |\n| MoCov3 |LUPerson(4.2M) |TransReID | ViT-S | 82.2 |92.1 | 47.4 | 70.3|\n| TranSSL | LUPerson(4.2M)| TransReID |ViT-S  | 91.1 |95.9| 66.8 |85.5 |\n|PASS | LUPerson(4.2M)| TransReID|ViT-S |92.2 |96.3 |69.1 |86.5 |\n| **Ours**|CION-AL(3.9M) |TransReID |ViT-S |**92.8** |**96.5** |**70.3** |**86.9** |\n\nWe can observe that, compared to these identity-ignored methods, using the same backbone and the same downstream fine-tuning algorithms, our identity-guided method has achieved a significantly leading performance even with fewer pre-training images (3.9M vs 4.2M).\n\nTherefore, analyzing the two representative experimental results above, it can be concluded that the guidance of identity is crucial for the person Re-ID pre-training.\n\nIn fact, comparing with more identity-ignored pre-training methods in Tab. 4 could make the conclusion more convincing. However, due to the limitations of computational resources and the lengthy time required for pre-training (e.g. 8×A100 and 120 hours for PASS ViT-B), as well as some methods such as UPReID not publicly releasing their code, we are unable to conduct pre-training from scratch for these methods within such a short rebuttal period. However, our commitment to make comparisons with these methods remains, and we intend to include comparative results in our final version.\n\n**(ii) How to eliminate the false positives of identity tags.**\n\nIt is an interesting question. In our task, the false positives of identity tags mean that images that should not belong to a certain person have been incorrectly assigned that person's identity tag. The single-tracklet denoising strategy is specifically proposed to eliminate these false positives of identity tags caused by tracking errors. The specifics of this strategy can be found in lines 152 to 162 of the paper. For your easy understanding, we have demonstrated the core process of this strategy in the following Python pseudocode:\n```\ndef denoise_tracklet(initial_samples, threshold):\n    # initial_samples represents the initial collection of images for a certain identity.\n    denoised_samples = initial_samples.copy()\n    while True:\n        distances = []\n        for sample in denoised_samples:\n            other_samples = denoised_samples.copy().remove(sample)\n            centroid = calculate_centroid(denoised_samples)\n            distances.append(calculate_distance(sample,centroid))\n        max_deviation = max(distances)\n        if max_deviation > threshold:\n            outlier_index = distances.index(max_deviation)\n            denoised_samples.pop(outlier_index)\n        else:\n            break\n    return denoised_samples\n```\n\n**(iii) How to learn better identity representations across videos.**\n\nIn this paper, the model learns better identity representations through the following two stages without knowing the identity tags of the same person across different videos.\n\nIn the first stage, by defining a noise concept that takes into account both intra-identity consistency and inter-identity discrimination comprehensively, we aim to seek identity correlation from cross-video images by modeling it as a progressive multi-level denoising problem. Specifically, we utilize a model that has been preliminarily trained on a small manually annotated dataset as a noise discriminator, and discover the identity tags across different videos using the proposed multi-level denoising strategy.\n\nIn the second stage, we propose an identity-guided self-distillation loss to constrain the model to learn identity representations across different videos. Specifically, by leveraging the identity tags discovered in the first stage, we design a training task that aligns both the global and local features of the same person, thereby enabling the model to learn identity-invariant representations.\n\nThe specific contents of these two stages can be found in Section 3.2 and Section 3.3 of the paper. It is hoped that the above brief method overview can help you better understand the process of learning identity representations in our work.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "QR4429nKCP", "reviewer": "Reviewer_uzbJ", "summary": "The authors propose an unsupervised method for mining identity relations in large-scale video data, incorporating self-supervised learning to pre-train the general models (like CNN, transformer). This approach demonstrates superior performance in person ReID tasks compared to other pre-training models.", "questions": "1. Is the contribution of this paper to ReID general pre-training models primarily a new large-scale dataset, or the proposal of a method to utilize identity correlation?\n2. How can you demonstrate that the proposed method effectively utilizes identity correlation information of pedestrians?\n3. When using the same dataset, How can you experimentally prove the superiority of the proposed identity correlation seeking method?", "limitations": "N/A", "rating": 5, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "Large-scale video datasets are effective training source of ReID task. The authors propose a straightforward method for pre-labeling instance identity from video streams, taking into account the appearance of pedestrians across multiple videos and the necessary computational complexity. They constructed the CION-AL dataset based on the SYNTHPEDES dataset, which contains rich pedestrian information that effectively aids model pre-training.\n\nUnder the DINO self-supervised learning framework, the authors differentiate the data source between the teacher and student models, enhancing the student model's representation capabilities through the focus on local pedestrian information.\n\nThe authors use various CNN and Transformer networks pre-trained on ImageNet as baselines, conducting extensive experiments on supervised and unsupervised domain adaptation datasets. The results demonstrate that the pre-trained models built using CION-AL are more effective for pedestrian re-identification tasks compared to those built using ImageNet.", "weaknesses": "The primary contribution of this paper lies in proposing a method for unsupervised data preprocessing of video streams, utilizing self-supervised learning to pre-train more powerful pedestrian re-identification models. However, the data preprocessing method essentially employs an unsupervised clustering strategy, which is a straightforward pre-labeling approach. The establishment of sample relationships in this manner is often suboptimal. Therefore, it is crucial to demonstrate in Section 4.4 through ablation studies whether it is the identity correlation seeking (i.e., denoising or pre-labeling) or the large scale of the dataset that enhances the model's performance, and quantify their respective contributions.\n\nAdditionally, the proof of computational complexity for cross-video denoising in Section 3.2 is insufficient, making the significance of the Sliding Range and Linking Relation unclear. \n\nIn Section 3.3, the training process of identity-guided self-distillation primarily differs from DINO in the way local information is provided to the teacher and student data streams. So, the “identity-guided” is still reflected in data preprocessing rather than in the self-distillation process.\n\nIn Section 4.2, the MGN fine-tuning has a certain impact on model performance, but the essence of this fine-tuning lacks necessary explanation. \n\nIn summary, although the proposed CION-AL dataset, in terms of volume and denoising processing (pre-labeling), enhances general pre-training models, the lack of qualitative analysis experiments on denoising significantly diminishes the significance of the proposed method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Sincerely thanks for your valuable comments. The concerns are answered as follows.\n\n**(i) Repective contribution analyses.**\n\nThanks for your concern. In Sec. 4.4, the 2nd and 3rd experiments inherently quantify the respective contributions of the identity correlation seeking and the large scale of the dataset.\n\nIn the 2nd experiment, we investigate the performance improvements brought by the three stages in identity correlation seeking. The quantitative results are shown in Fig. 5 of the paper. For your easy reference, we summarize them below. \n\n|Seeking Process|Mar mAP|Mar R@1|MSMT mAP|MSMT R@1|\n|---|---|---|---|---|\n|no seeking|92.1|96.3|69.9|87.6|\n|stage 1|92.3|96.6|71.3|88.6|\n|stage 1+stage 2|92.7|96.9|72.4|89.1|\n|stage 1+stage 2+stage 3|**93.3**|**97.3**|**74.3**|**89.8**|\n\nIt can be observed that each stage brings about a significant improvement over the previous stage. In particular, the complete seeking process (stage 1+2+3) achieves the best mAP scores of 93.3% and 74.3% on Market1501 and MSMT17, respectively, which is an improvement of 1.2 % and 4.4 % compared to the no-seeking group. These results **quantify the contribution of the identity correlation seeking** in enhancing the model's performance.\n\nMeanwhile, in the 3rd experiment, we examine the impact of pre-training dataset scale on model performance improvement. The results in Fig. 6 of the paper reveal the following:\n\n(a) With an increase in dataset scale, there's a significant performance gain. A 10% subset yields mAP of 91.1% on Market1501 and 67.5% on MSMT17, while a 100% set achieves 93.3% and 74.3% respectively.\n\n(b) As dataset scale grows, the rate of performance improvement tapers off. A 40% increase from 10% to 50% improves mAP by 5.3% and 1.5%, while a 40% increase from 50% to 90% only adds 1.2% and 0.6%.\n\nThese results  **quantify the contribution of the large scale of the dataset** in enhancing the model's performance.\n\n**(ii) Analyses of computational complexity.**\n\nWithout our proposed cross-video denoising, for N tracklets, we need to calculate the distance between the current tracklet and all other tracklets. That is, from the 1st tracklet to the Nth tracklet, we need to sequentially calculate N-1, N-2, …, 2, 1 distances, resulting in N(N-1)/2 calculations totally. In theory, when N is very large, its computational complexity is O(N^2).\n\nHowever, with our proposed cross-video denosing, we only need to calculate the distance between the current tracklet and other tracklets within the sliding range. Assuming the radius of the range is r, from the 1st tracklet to the Nth tracklet, we only need to calculate 2r distances for each, resulting in 2Nr calculations totally. When N is large, the computational complexity is O(N).\n\nThe above analyses demonstrate that our proposed strategy slashes computational complexity from O(N^2) to O(N), substantially cutting costs and proving especially crucial for large-scale cross-video denoising.\n\n**(iii) Analyses of identity-guided self-distillation.**\n\nOur CION operates in two stages: the first seeks identity correlations across videos, while the second conducts self-distillation using the sought identity correlations.\n\nIndeed, with identity supervision, identity-guided self-distillation has improved DINO's training strategy for person Re-ID pre-training. The core idea is to align global and local features of images from the same person identity, guiding the model to effectively mine identity invariance and learn stronger representations for person Re-ID.\n\nTherefore, it can be concluded that the \"identity-guided\" is also reflected in the self-distillation process.\n\n**(iv) Essence of MGN fine-tuning.**\n\nMGN is a feature learning strategy that integrates discriminative information at various granularities for person Re-ID. As an excellent open-source algorithm, using MGN as the fine-tuning algorithm is one of the standard paradigms for validating the performance of Re-ID pre-trained models. \n\nFollowing standard practice, we pre-train the model with our proposed method before fine-tuning it with the widely-used MGN for downstream tasks. As Tab. 1 shows, our pre-training method brings much better performance to the model.\n\n**(v) Primary contribution to Re-ID pre-trained models.**\n\nIt is an interesting question. We hold the view that the primary contribution of this paper to Re-ID pre-trained models is the proposal of a method utilizing identity correlation. We supplement some experiments to verify this.\n\n|#|Method|Pre-train|Fine-tune|Mar mAP|Mar R@1|MSMT mAP|MSMT R@1|\n|---|---|---|---|---|---|---|---|\n|1|DINO|LUPerson(4.2M)|MGN|91.1|96.4| 66.3 |86.1|\n|2|DINO|CION-AL(3.9M)|MGN|90.9|96.3|66.7|85.8|\n|3|CION|CION-AL(3.9M)|MGN|**93.3**|**97.3**|**74.3**|**89.8**|\n\nThe comparison between 1 and 2 shows that for DINO (identity-ignored), dataset changes have negligible effect on performance. Conversely, comparing 2 and 3, CION demonstrates a substantial performance boost, suggesting that pre-training with identity correlation significantly enhances model performance for the same dataset.\n\n**(vi) Identity correlation utilization demonstration.**\n\nWe enable the model to effectively utilize identity correlation information by using our proposed identity correlation seeking and identity-guided self-distillation. As demonstration, Fig. 4 in the paper visualizes the feature distributions of 15 randomly chosen persons, showing that our CION outperforms both the instance-level LUP and tracklet-level LUP-NL methods in identity discrimination, confirming its effective utilization of identity correlation information.\n\n**(vii) Experimental proof of proposed method's superiority**\n\nIn the 2nd experiment of Sec. 4.4, we investigate the performance improvements brought by the identity correlation seeking method when using the same dataset. The results are also shown in the 1st table of this response. The significant performance improvement experimentally proves the superiority of the proposed method.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "y17mluqMQF", "reviewer": "Reviewer_3WDg", "summary": "The paper introduces a Cross-video Identity-cOrrelating pre-traiNing (CION) framework for person re-identification. CION addresses the limitations of existing instance-level and single-video tracklet-level pre-training methods by leveraging identity-invariance across different videos. The framework uses a progressive multi-level denoising strategy and an identity-guided self-distillation loss to improve representation learning. Extensive experiments demonstrate that CION significantly enhances performance with fewer training samples compared to previous state-of-the-art methods. The authors also contribute a model zoo, ReIDZoo, which includes various pre-trained models.", "questions": "see weakness.", "limitations": "1. Fine-grained Representation Learning: The current CION framework does not explicitly target fine-grained information extraction, which may limit its ability to learn highly detailed person representations necessary for distinguishing between very similar identities.\n\n2. Noise Removal Challenges: Due to limitations in the feature extractor’s representational capacity, some complex noise may remain unresolved during the denoising process, leading to potential inaccuracies in the automatically derived identity correlations.\n\n3. Correlation Issues with Distant Tracklets: The strategy used to conserve computational resources may struggle to establish correlations between tracklets of the same person that are widely separated across different videos, potentially leading to incomplete identity mapping.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "Originality: The paper introduces a novel approach to pre-training for person re-identification by explicitly modeling identity invariance across videos, which is a significant departure from existing methods.\n\nClarity: The paper is clearly written, with detailed explanations and effective use of visual aids.\n\nSignificance: The proposed method achieves state-of-the-art performance with fewer training samples and demonstrates model-agnostic ability, making it highly valuable for both research and practical applications.", "weaknesses": "1. Pre-training models should aim for broader generalizability. The current models tend to be human-centered, covering tasks such as re-identification (reid) and human parsing, while others are more narrowly focused on ReID alone, akin to SemReID [1] or PersonMAE [2].\nThe authors' experiments have concentrated primarily on two datasets, Market-1501 and MSMT17. This limited scope may not sufficiently demonstrate the models' generalizability across varied conditions. To better assess and enhance the generalizability of the models, it is suggested that the authors include results from datasets that introduce challenges such as occlusion (Occldued-duke) and clothing change (LTCC or PRCC). This expansion would provide a more comprehensive understanding of the model's performance across different real-world scenarios.\n\n2. The proposed method relies on the quality of the initial tracking and identity correlation, which may be affected by the accuracy of the tracking algorithms used.\n\n3. Some of the methods that need to be compared are ignored. For example, HAP [3] with ViT-Base achieves better performance in MSMT17. \n\nReferences\n\n[1] Huang, Siyuan, Yifan Zhou, Ram Prabhakar Kathirvel, Rama Chellappa, and Chun Pong Lau. \"Self-Supervised Learning of Whole and Component-Based Semantic Representations for Person Re-Identification.\" arXiv preprint arXiv:2311.17074 (2023).\n\n[2] Hu, Hezhen, Xiaoyi Dong, Jianmin Bao, Dongdong Chen, Lu Yuan, Dong Chen, and Houqiang Li. \"PersonMAE: Person Re-Identification Pre-Training With Masked AutoEncoders.\" IEEE Transactions on Multimedia (2024).\n\n[3] Yuan, Junkun, Xinyu Zhang, Hao Zhou, Jian Wang, Zhongwei Qiu, Zhiyin Shao, Shaofeng Zhang et al. \"HAP: Structure-aware masked image modeling for human-centric perception.\" Advances in Neural Information Processing Systems 36 (2023).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Sincerely thanks for your constructive and valuable comments. The concerns are answered as follows.\n\n**(i) Include results from other datasets.**\n\nThanks for your suggestion. \n\nOur work focuses on proposing a superior pre-training framework to enable the model to achieve exceptional performance on the task of person Re-ID. \n\nDue to the fact that most representative works in the field of person Re-ID pre-training, such as LUP [1], LUP-NL [2], PASS [3], ISR [4], PersonMAE [5], etc., primarily use the Market1501 and MSMT17 datasets to validate the effectiveness of their methods. In order to conduct a more equitable comparison with these works, we have also mainly utilized these two datasets as downstream datasets in this paper.\n\nAt the same time, there are also some works, such as SemReID [6], that have used more datasets introducing challenges to validate the models' generalizability across varied Re-ID conditions. This exploration indeed allows for a better validation of the general capabilities of the pre-trained models. \n\nFollowing your suggestion, we include the results from Occluded-duke and PRCC datasets in the following table. The compared models all use ViT-S as the backbone. The results are derived from our implementation based on the official open-sourced model and code.\n\n| Method |Occ-duke mAP  |Occ-duke R@1  |PRCC mAP  |PRCC R@1  |\n| --- | --- | --- | --- | --- |\n| TranSSL | 58.1 | 67.6 | 46.6 | 47.9 |\n| PASS |59.4  |68.9  | 47.3 | 49.2 |\n|CION |63.2 |71.8 |52.4|54.7 |\n\nFrom the table, it can be observed that our CION pre-trained model outperforms the other two pre-trained models significantly on these two challenging datasets. This experimental result demonstrates that **our pre-trained model is indeed capable of generalizing well to other more challenging person Re-ID datasets.**\n\nNotably, despite efforts to contact the authors for access to the LTCC dataset, no response was received within the rebuttal period. Meanwhile, the comparison results of PersonMAE [5] and SemReID [6] are not included since its model and source code are not publicly available. However, our commitment to include the results of more datasets and works remains.\n\n**(ii) Analysis of the affect of tracking algorithms.**\n\nAs a common practice, similar to LUP-NL [2] and ISR [4], CION also employs a tracking algorithm to obtain the initial tracklet-level correlations. Theoretically, the more robust the tracking algorithm, the higher the quality of the initial tracklet-level correlations, and these correlations inevitably contain some noise. \n\nHowever, different from most methods that directly ignore the noise in these correlations, we propose a multi-level denoising strategy to progressively improve the quality of these correlations. The results in Fig. 5 of the paper empirically demonstrate the effectiveness of our proposed strategy. **Our strategy minimizes the impact of noise in the initial tracklet-level correlations as much as possible**, thereby significantly improving the model's performance.\n\n**(iii) Not compared related work.**\n\nThank you for pointing out such an outstanding work. When using ViT-Base, our CION outperforms HAP [7] on Market1501, but falls short on MSMT17. Overall, the two achieved comparable performance. Upon analysis, we believe that our CION holds the following advantages over HAP.\n\n(a) **CION does not require extensive human priors during pre-training**. HAP explicitly identifies human body regions in images during pre-training by utilizing a robust pose estimation model, which can somewhat affect the efficiency of pre-training. In contrast, our CION does not require such priors.\n\n(b) **CION is model-agnostic**. CION imposes minimal restrictions on model architecture, and has achieved excellent performance across a range of models, while HAP is only applicable to models like Vision Transformers and cannot be effectively utilized in other model architectures, such as CNNs.\n\nCertainly, HAP is a very meaningful related work, and we will include the results of HAP in our final version.\n\n**(iv) Limitations**\n\nThank you for summarizing the deeply discussed Limitations section in our appendix. In our subsequent work, we will focus on addressing fine-grained issues and improving the accuracy of identity correlations.\n\nReferences\n\n[1] Dengpan Fu, Dongdong Chen, Jianmin Bao, Hao Yang, Lu Yuan, Lei Zhang, Houqiang Li, and Dong Chen. Unsupervised pre-training for person re-identification. In CVPR, pages 4750–14759 (2021).\n\n[2] Dengpan Fu, Dongdong Chen, Hao Yang, Jianmin Bao, Lu Yuan, Lei Zhang, Houqiang Li, Fang Wen, and Dong Chen. Large-scale pre-training for person re-identification with noisy labels. In CVPR, pages 2476–2486 (2022).\n\n[3] Kuan Zhu, Haiyun Guo, Tianyi Yan, Yousong Zhu, Jinqiao Wang, and Ming Tang. Pass: Part-aware self-supervised pre-training for person re-identification. In ECCV, pages 198–214. Springer (2022).\n\n[4] Zhaopeng Dou, Zhongdao Wang, Yali Li, and Shengjin Wang. \"Identity-seeking self-supervised representation learning for generalizable person re-identification\". In ICCV, pages 15847–15858 (2023).\n\n[5] Hu, Hezhen, Xiaoyi Dong, Jianmin Bao, Dongdong Chen, Lu Yuan, Dong Chen, and Houqiang Li. \"PersonMAE: Person Re-Identification Pre-Training With Masked AutoEncoders.\" IEEE Transactions on Multimedia (2024).\n\n[6] Huang, Siyuan, Yifan Zhou, Ram Prabhakar Kathirvel, Rama Chellappa, and Chun Pong Lau. \"Self-Supervised Learning of Whole and Component-Based Semantic Representations for Person Re-Identification.\" arXiv preprint arXiv:2311.17074 (2023).\n\n[7] Yuan, Junkun, Xinyu Zhang, Hao Zhou, Jian Wang, Zhongwei Qiu, Zhiyin Shao, Shaofeng Zhang et al. \"HAP: Structure-aware masked image modeling for human-centric perception.\" Advances in Neural Information Processing Systems 36 (2023).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "fdpshjJ97Y", "reviewer": "Reviewer_5xRM", "summary": "This work explores learning the ReID model from a large number of unlabeled videos. The identity associations within the video and across the videos are mined. Extensive experiments under different backbones and downstream ReID settings are conducted.", "questions": "Please refer to weakness.", "limitations": "The limitations as well as the safeguards for dataset and models  are discussed in the paper.", "rating": 7, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "1. Learning from unlabeled videos makes sense for improving ReID model.\n2. Extensive experiments are conducted.\n3. The performance is impressive.", "weaknesses": "1. The technical contribution is not significant. Associating different images is the core of the work. However, the main technique is to associate based on the feature similarity following a spirit of progressive learning. Such a pipeline is not novel and widely applied in many previous works. For example, some works mine the associations between images in different tracklets based on feature similarity.[R1, R2, R3, R4]. Although some of these works learn the model from trimmed videos, but share a similar sprit of progressively associating different tracklets/videos based on feature similarity.\nBesides, such a pipeline could be sensitive to the hyperparameter, i.e., $\\sigma_{cst}, \\sigma_{drm}$, but the paper does not report the ablation study in the hyperparameter.\n\n2. Supervised learning can be employed with the identity association. Why employing self-supervised learning in Sec. 3.3.\n\n3. The proposed method achieves similar performance with SOLIDER in Table 1.\n\n[R1] Tracklet Self-Supervised Learning for Unsupervised Person Re-Identification. AAAI2020.\n[R2] CycAs: Self-supervised Cycle Association for Learning Re-identifiable Descriptions. ECCV2020.\n[R3] Unsupervised Person Re-identification by Deep Learning Tracklet. ECCV2018\n[R4] Progressive Unsupervised Person Re-Identification by Tracklet Association With Spatio-Temporal Regularization. TMM 2020", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Sincerely thanks for your constructive and valuable comments. The concerns are answered as follows.\n\n**(i) Analyses of technical contribution.**\n\nThanks for your nice concern.\n\nPerson Re-ID, a fine-grained visual retrieval task, centers on learning a feature distribution that robustly captures the nuances between person images via feature similarity. As feature similarity is essential to Re-ID, our work, like these referenced, prioritizes it. Technically, however, our work has the following contributions compared to these works:\n\n(a) By defining a noise concept, we propose a progressive multi-level denoising strategy to effectively seek the identity correlation across large-scale videos. Previous unsupervised person Re-ID methods have indeed proposed similar ideas of associating tracklets from different videos, but these methods are designed for small-scale video datasets. Their proposed methods are mostly complex and inefficient, making them ill-suited for application to video collections of an ultra-large scale. However, we have made it possible to seek identity correlations in large-scale videos by proposing a three-stage progressive denoising strategy and a sliding range & linking relation design.\n\n(b) Meanwhile, we propose a identity-guided self-distillation loss to implement large-scale pre-training. Existing researches in the field of person Re-ID pre-training ignore the identity-invariance across different videos, leading to learning comparably poor representations for the person Re-ID task. However, we have made it possible for the model to learn cross-video identity invariance in large-scale videos through the identity correlation seeking and identity-guided self-distillation technics we propose.\n\n**(ii) Ablation study in the hyperparameter.**\n\nTheoretically, the quality of the sought identity correlations is indeed influenced to some extent by these two hyperparameters. However, we do not conduct ablation studies on these two parameters for the following two reasons.\n\n(a) A series of previous works have shown that in large-scale pre-training, subtle differences in pre-training data quality do not have a significant impact on model performance. \n\n(b) Conducting ablation studies on these two parameters would mean that for each set of values, we would need to start from scratch with the process of identity correlation seeking and pre-training for the entire large-scale video collection. Given that this process is extremely resource and time-consuming, such ablation studies are beyond our capacity to undertake, and they are difficult to accomplish within such a short rebuttal period.\n\nFollowing your suggestion, we will attempt to explore the specific impacts of different hyperparameters in our future work. I hope the above response can address your concerns.\n\n**(iii) The reason for employing self-supervised learning.**\n\nA series of self-supervised learning works in general domains, such as MoCo, DINO, MAE, etc., conclude that self-supervised learning mitigates model bias from label bias compared to supervised learning, enabling deeper data understanding and more robust, generalizable representations. Hence, our paper prioritizes self-supervised learning with identity association over supervised approaches.\n\nImportantly, through our analysis and some additional experiments, we can conclude the two following representative advantages of self-supervised learning over supervised learning in our work.\n\n(a) Self-supervised learning leads to **better generalization capabilities**. To validate this, we conduct a comparative experiment using our proposed self-supervised learning method and a representative supervised person Re-ID method BoT [1]. We use 1,000,000 images from CION-AL as the pre-training dataset, with ResNet50 as the backbone. After pre-training, we directly test the zero-shot performance of each model on Market1501 and MSMT17. The table below demonstrates that the self-supervised pre-trained model surpasses the supervised counterpart in zero-shot performance, suggesting improved generalization from self-supervision.\n\n|Method|Mar mAP|Mar R@1|MSMT mAP|MSMT R@1|\n|---|---|---|---|---|\n|BoT (supervised)| 41.4 |72.1  |14.9  | 42.1 |\n|Ours (self-supervised)|46.7 |74.2  |18.2  |49.6  |\n\n(b) When training on large-scale data, our proposed self-supervised learning method incurs significantly **lower computational costs** than some representative supervised learning methods. For example, in almost all supervised learning methods for person Re-ID, ID loss is used for auxiliary training. For a pre-training dataset with a large number of identity labels, the ID loss functions as an ultra-large-scale multi-classifier. The training efficiency of such a multi-classifier is theoretically more resource-intensive and less efficient. Meanwhile, some supervised methods also design complex modules and losses. Nevertheless, we achieve better representation learning by employing a more efficient training method (a single self-supervised learning loss).\n\nIn summary, for the person Re-ID pre-training, we believe that self-supervised learning is faster and superior compared to supervised learning.\n\n**(iv) Similar performance with SOLIDER.**\n\nEmploying Swin-T, our CION outperforms SOLIDER by 0.6% on Market1501 and 3.7% on MSMT17 in mAP. With Swin-S, CION matches SOLIDER's performance, both edging up 0.1% on both datasets. However, our CION has a significant advantage over SOLIDER in that it is model-agnostic and can be applied to almost all model architectures, including the 32 models with 10 different architectures shown in this paper. In contrast, SOLIDER is only applicable to structures like vision transformers, and their work includes only 3 models. Meanwhile, our CION demonstrates exceptionally superior performance in unsupervised person Re-ID in Tab. 2.\n\nReference\n\n[1] A Strong Baseline and Batch Normalization Neck for Deep Person Re-identification. IEEE Transactions on Multimedia (2019).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We sincerely appreciate all reviewers' time and efforts in reviewing our paper. 

We are glad to find that reviewers generally recognized our contributions including the novelty and significance of our method (Reviewer hfuf, uzbJ, 3WDg, 5xRM), the comprehensive experiments (Reviewer uzbJ, 3WDg, 5xRM) and our well-written paper (Reviewer 3WDg, 5xRM).

We will **open-source all the code, models and dataset**. We believe that our work will make great and timely contributions to this community. 

1) We have pioneered to learn identity-invariance from cross-video person images for person ReID pre-training and verified its superiority through extensive experiments. This learning paradigm offers **a solid path to more powerful ReID models**.
2) Our **model zoo**, containing 32 models with 10 architectures, will enable researchers to quickly access high-quality model resources, supporting various ReID task applications and promoting the sustainable development of the technology.
3) Our **CION-AL dataset** is currently the largest in the field with high-quality identity correlations, offering abundant data resources for more researches on person-related domains.

We have replied to each reviewer individually to address any concerns. I noticed that we can continue discussions on the OpenReview system for some time to come. If the reviewers has any questions, we would be happy to continue the discussion.

Once again we thank all reviewers and area chairs!

Meta-review: The paper received all accept recommendations after rebuttal. All reviewers acknowledged the good results obtained and weighed the dataset contribution a lot. While the data and the research are about person re-identification, ethics reviewers found some particular concerns and shared some useful suggestions. Authors took this seriously and provided responses and solutions. Overall, AC agrees with the reviewers and recommends acceptance of this paper for publication.
Decision: Accept (poster)