Title: Cross-video Identity Correlating for Person Re-identification Pre-training
Venue: NeurIPS 2024
Accepted: yes
Keywords: Person Re-identification

Pre-training

Self-supervised Representation Learning
Abstract: Recent researches have proven that pre-training on large-scale person images extracted from internet videos is an effective way in learning better representations for person re-identification. However, these researches are mostly confined to pre-training at the instance-level or single-video tracklet-level. They ignore the identity-invariance in images of the same person across different videos, which is a key focus in person re-identification. To address this issue, we propose a Cross-video Identity-cOrrelating pre-traiNing (CION) framework. Defining a noise concept that comprehensively considers both intra-identity consistency and inter-identity discrimination, CION seeks the identity correlation from cross-video images by modeling it as a progressive multi-level denoising problem. Furthermore, an identity-guided self-distillation loss is proposed to implement better large-scale pre-training by mining the identity-invariance within person images. We conduct extensive experiments to verify the superiority of our CION in terms of efficiency and performance. CION achieves significantly leading performance with even fewer training samples. For example, compared with the previous state-of-the-art ISR, CION with the same ResNet50-IBN achieves higher mAP of 93.3% and 74.3% on Market1501 and MSMT17, while only utilizing 8% training samples. Finally, with CION demonstrating superior model-agnostic ability, we contribute a model zoo named ReIDZoo to meet diverse research and application needs in this field. It contains a series of CION pre-trained models with spanning structures and parameters, totaling 32 models with 10 different structures, including GhostNet, ConvNext, RepViT, FastViT and so on. The code and models will be open-sourced.

Cross-video Identity Correlating for Person Re-identification Pre-training

Abstract

Recent researches have proven that pre-training on large-scale person images extracted from internet videos is an effective way in learning better representations for person re-identification. However, these researches are mostly confined to pre-training at the instance-level or single-video tracklet-level. They ignore the identity-invariance in images of the same person across different videos, which is a key focus in person re-identification. To address this issue, we propose a Cross-video Identity-cOrrelating pre-traiNing (CION) framework. Defining a noise concept that comprehensively considers both intra-identity consistency and inter-identity discrimination, CION seeks the identity correlation from cross-video images by modeling it as a progressive multi-level denoising problem. Furthermore, an identity-guided self-distillation loss is proposed to implement better large-scale pre-training by mining the identity-invariance within person images. We conduct extensive experiments to verify the superiority of our CION in terms of efficiency and performance. CION achieves significantly leading performance with even fewer training samples. For example, compared with the previous state-of-theart [9], CION with the same ResNet50-IBN achieves higher mAP of 93.3% and 74.3% on Market1501 and MSMT17, while only utilizing 8% training samples. Finally, with CION demonstrating superior model-agnostic ability, we contribute a model zoo named ReIDZoo to meet diverse research and application needs in this field. It contains a series of CION pre-trained models with spanning structures and parameters, totaling 32 models with 10 different structures, including GhostNet, ConvNext, RepViT, FastViT and so on.

1 Introduction

Person re-identification (ReID), which aims to identify and match a target person across different camera views, is becoming increasingly influential in widespread applications, such as criminal tracking and missing individual searching. Although current ReID methods [38, 2, 21, 22, 39] have achieved significant progress for the past few years, it has become evident through recent research advancements that the mere development of sophisticated specialist algorithms has already encountered a performance bottleneck.

Meanwhile, pre-training on large-scale images [1, 5, 17, 29] has shown great success to further improve model performance in the field of general vision. However, due to the significant domain gap, these pre-trained models can only provide very limited improvement for person re-identification. Therefore, with the vast amount of person-containing videos available on the internet, some researches [10, 11, 53, 9, 44, 27] turn to exploring the potential of pre-training on person images extracted from such videos and demonstrate encouraging results. However, due to the unavailability of identity labels for such extracted person images, these pre-training methods are confined to learning representations at the instance-level or single-video tracklet-level, as shown in Fig. 1

contrastive

learning

(a) Instance-level method

(b) Single-video tracklet-level method

(c) Cross-video identity-correlating pre-training (CION)

detect

crop

…

Frame t

Frame t +

e

VIDEO

track

crop

t

t +

e

contrastive

learning

VIDEO B

…

VIDEO A

…

contrastive

learning

track

crop

track

crop

FRAME

…

an

On the one hand, the instance-level methods [10, 53, 44, 27] completely overlook the identityinvariance in different images of the same person. They incorporate some person-related prior knowledge into general pre-training frameworks, neglecting the concept of person identity. For example, LUP [10] improves MoCoV2 [4] by systematically studying the data augmentation and contrastive loss, while PASS [53] improves DINO [1] by generating part-level features to offer finegrained information. However, there exists a notable concordance within different images of the same person. Directly ignoring such a concordance easily leads to learning weaker person representations.

On the other hand, the single-video tracklet-level methods [11, 9] only exploit the short-range trackletinvariance while significantly neglecting the identity-invariance in images of the same person across different videos. They simply regard one tracklet in a single video as one person, which will result in two substantial problems. 1) Insufficient intra-identity consistency. Due to the unavoidable inaccurate tracking, there are quite a few images within a tracklet that do not belong to the same person. Roughly considering these images as coming from the same person will lead to insufficient intra-identity consistency. 2) Limited inter-identity discrimination. It is quite evident that there are considerable instances of the same person appearing in different videos, thus being treated as different tracklets (identities). Roughly considering these images as coming from different persons will lead to limited inter-identity discrimination.

To address these limitations mentioned above, as shown in Fig. 1, we propose a Cross-video IdentitycOrrelating pre-traiNing (CION) framework, by which the images of the same person across different videos are explicitly correlated to deeply mine the identity-invariance. CION first seeks the identity correlation from long-range cross-video images by modeling it as a progressive multi-level denoising problem, where a noise concept that comprehensively considers both intra-identity consistency and inter-identity discrimination is defined. Subsequently, with the sought identity correlation, CION introduces an identity-guided self-distillation loss to implement better pre-training by learning the invariance within augmented views in multiple images of the same person.

We conduct extensive experiments to verify the superiority of our CION in terms of efficiency and performance. CION achieves the best performance with significantly fewer training samples. With the ResNet50-IBN backbone, compared with the previous state-of-the-art method ISR [9], our CION achieves higher mAP of 93.3% and 74.3% on the Market1501 and MSMT17 datasets, while only utilizing 8% training samples. Noting that ISR is pre-training at the single-video tracklet-level, this experimental result fully validates the necessity of learning identity-invariance across different videos.

Finally, we contribute a fully open-sourced model zoo named ReIDZoo to meet diverse research and application needs within this field. With CION demonstrating superior performance improvements to diverse model structures (model-agnostic ability), we pre-trained a series of models with spanning structures and parameters, totaling 32 models with 10 different structures, including GhostNet [16], EdgeNext [28], ConvNext [26], RepViT [36], FastViT [35] and so on.

In conclusion, the highlights of our work can be summarized into three points:

(1) We propose a Cross-video Identity-cOrrelating pre-traiNing (CION) framework. It explicitly mines the identity-invariance in person images extracted from internet videos by progressive multilevel denoising and identity-guided self-distillation, leading to better representation learning.

(2) Extensive experiments verify the superiority of CION in terms of efficiency and performance. It achieves the best performance with much fewer training samples compared with previous methods, while also demonstrating adaptability to diverse model structures with spanning parameters.

(3) We contribute a model zoo named ReIDZoo. It contains a series of CION pre-trained models with spanning structures and parameters, totaling 32 models with 10 different structures. It conveniently meets various research and application needs, and greatly promotes the development of this field.

2 Related work

Self-supervised pre-training for general vision. In recent years, self-supervised pre-training methods have shown promising results in learning representation from large-scale data. MoCo [4] facilitates contrastive learning by building a dynamic dictionary with a queue and a moving-averaged encoder. BYOL [15] relies on two networks, referred to as online and target networks, that interact and learn from each other with a slow-moving average strategy. DINO [1] proposes a self-distillation framework by underlying the importance of momentum encoder, multi-crop training and so on. MAE[17] adopts the ViT backbone and learns representations by predicting the masked patches from the remaining visible ones. However, due to significant domain gap and neglect of person-related characteristics, these general methods show poor transferability to person re-identification.

Self-supervised pre-training for person re-identification. Existing pre-training methods can be classified into two categories, the first being the instance-level methods [10, 53, 44, 27, 3] and the second being single-video tracklet-level method [9, 11]. The instance-level methods strive to merge the person-related prior knowledge into existing self-supervised learning methods. For example, LUP [10] improves MoCov2 [4] through systematical studying of data augmentation and contrastive loss. PASS [53] improves DINO [1] by generating part-level features to offer fine-grained information. SOLIDER [3] turns to utilize pseudo semantic label and import some semantic information into the learned representations. However, these instance-level methods are incapable of harnessing the identity-invariance in different images of the same person. Meanwhile, the single-video tracklet-level methods strive to mine some invariance by regarding one tracklet in a single video as one person. For example, LUP-NL [10] utilizes supervised ReID learning, label-guided contrastive learning and prototype-based contrastive learning to rectify the noisy tracklet-level labels in person images extracted from raw internet videos. ISR [9] leverages massive data to construct the positive pairs from inter-frame images in a single videos. The identity-invarience of the same person across different videos is significantly neglected by these single-video tracklet-level methods.

3 CION: Cross-video Identity-cOrrelating pre-traiNing

We present a cross-video identity-correlating pre-training (CION) framework, which explicitly learns identity-invariance in long-range cross-video person images. First, we define a noise concept that comprehensively considers both intra-identity consistency and inter-identity discrimination (Sec. 3.1). Next, we model the identity correlation seeking process from cross-video images as a progressive multi-level denoising problem (Sec. 3.2). Finally, we propose an identity-guided self-distillation loss to mine identity-invariance from person images with the sought identity correlation (Sec. 3.3).

3.1 Noise definition

We demonstrate that a sample set with identity labels should have the following characteristics. 1) Intra-identity consistency. Samples belonging to the same identity should aggregate around a centroid in a high-dimensional space, i.e., they should be enveloped by a hypersphere s k = ( c k , r k ) , where c k represents the centroid of the sphere and r k represents its radius. A smaller radius of the sphere indicates a denser degree of aggregation, which signifies higher intra-identity consistency. 2) Inter-identity discrimination. Samples belonging to two different identities should be enveloped by two distinct hyperspheres s i and s j respectively, with a certain distance d i,j maintained between the

centroids of c i and c j . A larger distance indicates a sparser distribution of hyperspheres, implying higher intra-identity discrimination. In summary, the smaller radii of the hyperspheres and the greater distances between the hypersphere centroids mean more desirable intra-identity consistency and inter-identity discrimination.

Considering the above demonstration, for a sample set S = { x y i i } N i =1 , where the identity label y i ∈ { 1 , 2 , 3 , . . . , M } , we can further regard S as a hypersphere set { s k } M k =1 , where x i ∈ s k if y i = k . Then for a single hypersphere s k , the intra-consistency noise is formulated as:

where µ ( · ) is an unit step function, dist ( · ) is any distance metric such as cosine distance, and the centroid c k is calculated by averaging all samples in s k . We can observe that this formula determines the presence of noise by explicitly constraining the maximum radius of the hypersphere, indicating whether the aggregation degree of samples with the same identity meets the required criteria σ cst .

Meanwhile, for two distinctive hyperspheres s i and s j , the inter-discrimination noise between them is formulated as:

We can observe that this formula explicitly constrains the minimum distance between the centroids of the hyperspheres to determine the presence of noise, indicating whether the margin between samples of different identities meets the required criteria σ drm .

Finally, comprehensively considering both intra-identity consistency and inter-identity discrimination, the overall noise for a sample set S = { s k } M k =1 can be formulated as:

̸

We consider that a sample set with a favorable level of identity correlation should exhibit the smallest possible amount of n S in order to achieve superior consistency and discrimination.

3.2 Progressive multi-level denoising

Given crawled internet videos, a straightforward approach [11, 9] is to utilize tracking algorithms to extract person tracklets from each video and treat each tracklet as an unique identity, thereby forming a sample set with initial identity correlation. However, due to inevitable false positives in tracking and the neglect of the same person across different videos, the identity correlation in this sample set is significantly noisy, lacking satisfactory intra-identity consisitency and inter-identity discrimination. Therefore, we propose a progressive multi-level denoising strategy to seek superior identity correlation by minimizing the overall n S of the sample set as much as possible.

Single-tracklet denosing. First, we denoise each single tracklet to guarantee the intra-identity consistency. For an initial single-tracklet sample set with m extracted image features t 0 = { x i } i m =1 , we obtain the denoised sample set by constraining the maximum hypersphere radius through multiple iterations. That is, for the sample set t j of the j -th iteration, we compute the distances between each sample and the centroid of the remaining samples to identify the sample with the maximum deviation. If the deviation exceeds σ cst , we remove the sample from t j to an exclusion set t exc and proceed to the next iteration; otherwise, we terminate the iteration and regard t j as the denoised sample set with good intra-identity consistency. The iterative process can be formulated as:

We can observe that by iteratively constraining the maximum hypersphere radius of the tracklet, we achieve the exclusion of anomalous samples. In essence, this iterative process aims to eliminate the noise n cst as defined in Eq. 1, thereby ensuring favorable intra-identity consistency for each tracklet.

Short-range single-video denosing. Then, we denoise each single video with multiple tracklets to guarantee both intra-identity consistency and inter-identity discrimination. For an initial single-video

Sliding Range

Slide from beginning to end

Linking

Linking

current position

𝑡 4

current position

𝑡 8

sample set with n denoised tracklets v = { t k } n k =1 and respective exclusion sets { t exc,k } n k =1 , we first reallocate the anomalous samples of each exclusion set to ensure their proper tracklet assignment. The reallocation process for an anomalous sample x k i ∈ t exc,k can be formulated as:

Through the aforementioned reallocation process, we can assign each anomalous sample to its correct tracklet, or discard those extreme anomalous samples that do not belong to any tracklet, such as samples devoid of person presence. Subsequently, we merge tracklets with closely located hypersphere centroids to ensure robust inter-identity discrimination, which can be formulated as:

where µ ( · ) is an unit step function to indicate whether the merge operation is applied to t p and t q .

We can observe that through the aforementioned two-stage process, we achieve the reallocation of anomalous samples and the merging of similar tracklets. In essence, this two-stage process aims to eliminate the overall noise n S as defined in Eq. 3, thereby ensuring both favorable intra-identity consistency and inter-identity discrimination for each video.

Long-range cross-video denosing. Finally, we denoise multiple videos to seek identity correlation across videos. A straightforward method for denoising a large-scale sample set containing many videos is to concatenate the videos together and treat them as a single ultra-long video, then apply the single-video denoising strategy mentioned above. However, when a video contains N tracklets, the computational complexity of Eq. 6 is O ( N 2 ) . For an ultra-long video with an overwhelming number of tracklets, this undoubtedly introduces unmanageable computational overhead. To address this issue, we design Sliding Range and Linking Relation to implement ultra-long video denoising.

As shown in Fig. 2, for an ultra-long concatenated video composed of N sub-video tracklets V = { t i } N i =1 , we construct a Sliding Range SR c = { c, r s } c :1 → N to slide from the beginning to the end of V , where c represents the index of the current position, and r s represents the half-width of SR c . Then, for all tracklets covered by SR c , we obtain the Linking Relation between each tracklet and t c by utilizing the merging function defined in Eq. 6, which is formulated as:

where LR ( · ) indicates whether a bidirectional Linking Relation will be established between the two tracklets. Subsequently, after the Sliding Range traverses the entire video, we perform a closure operation based on the obtained Linking Relations to derive the closures. Finally, the tracklets within each closure are merged as one single identity.

Wecan observe that the designing of Sliding Range significantly reduces the computational complexity from O ( N 2 ) to O ( N ) , which makes it possible to denoise massive-scale crawled videos. Meanwhile, the closure operation based on Linking Relations ensures long-range identity correlation for different tracklets of the same person across different videos. In essence, this long-range cross-video denosing strategy aims to eliminate the inter-discrimination noise n drm as defined in Eq. 2 at the multi-video level, thereby further ensuring favorable inter-identity discrimination of all videos.

Identity correlation seeking. We summarize the whole process of seeking the identity correlation from large-scale crawled person-containing videos as follows. Firstly, we should utilize an off-theshelf person tracking algorithm to extract person tracklets from each video and assign each tracklet with a class label, forming a noisy sample set with initial identity correlation. Then, the noisy sample set should undergo single-tracklet denoising, short-range single-video denoising and long-range cross-video denoising in sequence to seek satisfactory identity correlation. Finally, we consider all images within a single denoised tracklet as belonging to the same person, and this denoised sample set can serve as a well-prepared dataset with good identity correlation for subsequent pre-training.

3.3 Identity-guided self-distillation

With the sought identity correlation, we propose to utilize simple but effective identity-guided self-distillation to pre-train our models, leading to better identityinvariance learning. The whole framework shares a similar overall structure as the popular self-distillation pre-training paradigm DINO for general vision [1], as illustrated in Fig. 3. Differently, we introduce the concept of identity and conduct contrastive learning on the augmented views from images of the same person. We train a student network f θ S to match the output probability distribution of a teacher network f θ T , in which the two share the same architecture and are parameterized by θ S and θ T , respectively. Given an input image x , both networks predict probability distributions over K dimensions denoted by P S and P T , which are obtained by normalizing the output of each network with a softmax function and formulated as:

where P S ( x ) is the predicted distribution of the image x for f θ S and τ s > 0 is a hyperparameter controlling the output distribution sharpness. The similar formular holds for P T with τ t .

In the following, we detail how we introduce the iden- tity concept into self-distillation. First, with the sought

identity correlation, we construct an identity set con-

…

…

…

Teacher

𝑓 𝜃 𝑇

Student

𝑓 𝜃 𝑆

EMA

centering

softmax

softmax

𝑥𝑡

𝑥𝑠

𝑇𝑡

𝑇 𝑠

contrastive learning

stop gradient

taining N id images of the same person X = { x k } N id k =1 . Then, given an image x k , we construct a set V k of different augmented views. This set contains two global views x g, 1 k and x g, 2 k and several local views of smaller resolution. We pass the whole set V k through the student while only pass the global views through the teacher. Finally, the student learns to match the output probability distribution of teacher by minimizing the cross-entropy:

̸

where H ( a, b ) = -a log b and the parameters θ S are optimized with stochastic gradient descent. Following [1], the teacher network is frozen over an epoch and updated with an exponential moving average (EMA) on the student weights. Meanwhile, the output of the teacher network is centered with a mean calculated over the batch.

4 Experiments

4.1 Experimental setup

Pre-training and fine-tuning. During pre-training, we directly regard the images of SYNTHPEDES [55] as a noisy sample set with initial identity correlation. Then, we utilize our proposed progressive multi-level denoising strategy to seek the identity correlation in it. Meanwhile, to ensure a fair comparison with previous methods, we adopt random selection to align the overall sample size with that of the commonly used pre-training dataset LUPerson [10]. We name the whole sample set as CION-AL, and it contains 3,898,086 images of 246,904 identities totally. Finally, we utilize the CION-AL and our proposed identity-guided self-distillation loss to pre-train our models. We pre-trained 32 models of 10 architectures in total. During fine-tuning, as a common practice, we directly utilize TransReID [20] to fine-tune our pre-trained models with input size as 256 × 128 for the supervised person ReID setting, if not specified. Also, the commonly used C-Contrast [6] is adopted for the settings of unsupervised domain adaptation (UDA) and unsupervised learning (USL).

Datasets and evaluation metrics. Following common practices, we conduct experiments on two datasets, i.e. , Market1501 [50] and MSMT17 [40], for evaluating the performance on ReID tasks. Market1501 and MSMT17 are widely used person ReID datasets, which contain 32,688 images of 1,501 persons and 126,411 images of 4,101 persons, respectively. We use the cumulative matching characteristics at top-1 (Rank-1) and mean average precision (mAP) as the evaluation metrics.

Implementation details. To implement the progressive multi-level denoising, we utilize a ViT-Base model [27] trained on the images of UFine6926 [56] as a normalized feature extractor. The intraconsisitency criteria σ cst and inter-discrimination criteria σ drm are set as 0.2 and 0.18, respectively. We utilize cosine distance to calculate the distance between two samples. The half-width r s of the sliding range is set as 1000. For pre-training, the global views and local views are resized to 256 × 128 and 128 × 64 , respectively. We pre-train our models on 8 × V100 GPUs for 100 epochs. We adopt a curriculum learning strategy for setting the images per identity, where the initial N id is set to 2, and at epochs 40, 60, and 80, N id increases to 4, 6, and 8, respectively. To optimize the utilization of GPU memory, the batch size per GPU and the number of cropped local views are adjusted according to the varying parameter counts of the models. All other settings for pre-training, such as learning rate, are consistent with those used in DINO [1]. For fine-tuning, we employ the commonly used methods [20, 38, 6] and make only slight adjustments to settings such as learning rate and batch size. Specific settings and more details can be found in Sec. A.5 of the appendix.

4.2 Comparison with state-of-the-art methods

Supervised person ReID. We compare our CION with several outstanding stateof-the-art methods on supervised person ReID in Tab. 1. We divide previous methods by model initialization, i.e. , utilizing supervised ImageNet pre-training [7] and self-supervised large-scale person images pre-training as backbone initialization, respectively. Compared with existing best methods pre-trained on ImageNet, our method significantly outperforms them without any extra complex designs. For example, our ViT-B achieves remarkable performance gains over the previous best method DCAL [52] on Market1501 and MSMT17 by 5.6% and 8.6% mAP, respectively. It should be noted that DCAL utilizes much more images for pretraining than ours (14M vs 3.9M). On the other hand, compared with existing best self-supervised methods pre-trained on large-scale person images, our method still shows significant superiority over them. It is worth mentioning that the single-video tracklet-level method ISR [9] utilizes a substantially larger number of person images for pre-training than ours (47.8M vs 3.9M). However, both fine-tuning with MGN [38], our ResNet50-IBN obtains 93.3% and 74.3% mAP on Market1501 and MSMT17, surpassing ISR by 1.0% and 2.8%, respectively. This effectively

demonstrates the necessity of learning identity invariance across different videos.

Unsupervised person ReID. We compare our CION with some SoTA methods of unsupervised person ReID in Tab. 2, which contains two settings, i.e. , unsupervised domain adaptation (UDA) and unsupervised learning (USL). All the person-centric pre-trained models are fine-tuned with CContrast [6]. For both settings, our method achieves new SoTA performance, significantly surpassing all previous methods. For example, with ResNet50-IBN as backbone, our method outperforms the

previous SoTA LUP [27] by a large margin, i.e. , 20.1% mAP on Market1501 → MSMT17 UDA setting and 19.6% mAP on MSMT17 USL setting, respectively. Noting that LUP is an instance-level method, these results verify the superiority of our identity-level method in person ReID pre-training.

4.3 Generalizing to different model structures

During pre-training, CION imposes minimal constraints on the model structure, allowing for its generalization across a variety of structures. To verify this, we conduct extensive experiments to investigate whether CION can improve the models with different structures and parameters, encompassing 32 models with 10 structures, including CNNs, ViTs, and other specialized architectures. The baselines are the models with supervised ImageNet [7] pre-training. As a common practice, we utilize TransReID [20] to fine-tune the models on ReID datasets with input size as 256 × 128 , if not specified. As the results listed in Tab. 3, our CION consistently yields significant performance improvements across all models. When applying to TransReID, the average performance improvements of all the 32 models over the baselines are 11.7% and 18.8% mAP on Market1501 and MSMT17, respectively. With CION demonstrating such model-agnostic ability, we construct a model zoo named ReIDZoo, which contains all the CION pre-trained models with

Table 3: CION significantly improves different models. FLOPs are computed based on the input size. * means the result with the input size as 384 × 128 . † represents fine-tuning with MGN. ↑ indicates the improvement brought by our CION over the baseline.

spanning structures and parameters, to meet diverse research and application needs in this field.

4.4 Ablation studies and analyses

Identity-level correlating holds significance.

Wevalidate the significance of identity-level correlating in learning better person representations through both feature distribution visualization and performance comparison. For visualization, we collect samples of 15 persons randomly selected from the gallery of Market1501 [50], and visualize their features via t-SNE [34] in Fig. 4. The instance-level method LUP [10] and the tracklet-level method LUP-NL [11], which are respectively pre-trained on 4.2M and 10.7M person images, are employed as a comparison. As we can see, the features of LUP are almost randomly distributed, while the features of LUP-NL remain challenging to aggregate at the identity level. However, our CION consistently enjoys good intra-consistency and inter-discrimination. For performance comparison, we compare the fine-tuning results of our CION with two popular identity-ignored pre-training methods LUP [10] and DINO [1] in Tab. 4. For a fair comparison, the models are all ResNet50-

(a) LUP(instance-level)

(b) LUP-NL(tracklet-level)

(c) CION(ours)

too far

too far

too close

Random

distribution

IBN [42] pre-trained on our CION-AL and the fine-tuning algorithm is MGN [38]. We can see that the model pre-trained with CION achieves significant leading performance. These results demonstrate the significance of implementing identity-level correlating for person ReID pre-training.

Effectiveness of progressive multi-level denoising. Fig. 5 shows the ablation study of each denoising strategy. The number of training samples for all groups is set to 3.9M for a fair comparison. The backbone is ResNet50-IBN [42] and the fine-tuning algorithm is MGN [38]. We can observe that each strategy significantly improves the fine-tuning performance of the pretrained model on downstream datasets. Particularly, the cross-video denoising results in the most substantial improvement, further increasing the mAP by 0.6% and 1.9% on Market1501

mAP[%]

Market1501

(1)

(2)

(3)

(4)

91

92

93

94

95

92.1

92.3

92.7

93.3

96.3

96.6

96.9

97.3

95

96

97

98

99

Rank1 [%]

mAP[%]

(1)

(2)

(3)

(4)

65

68

71

74

77

69.9

71.3

72.4

74.3

87.6

88.6

89.1

89.8

84

86

88

90

92

Rank1 [%]

and MSMT17, respectively. These results verify the effectiveness of our progressive multi-level denoising strategy, which guarantees satisfactory identity correlation seeking.

Scalability to large-scale training data. We also study the scalability of our method to largescale training data. Aside from the training data size, the rest settings are same with the experiment mentioned above. As shown in Fig 6, with training data size ratio varying from 10% to 100%, the fine-tuning performance on downstream datasets consistently keeps improving, especially on the more challenging MSMT17.

20%

10%

30%

40%

91

92

93

94

Market1501 mAP [%]

Training Data Size

91.1

91.8

92.3

92.5

92.6

100%

50%

60%

70%

80%

90%

92.9

93.0

93.1

93.2

93.3

67

69

73

75

67.5

69.9

71.1

71.6

72.8

73.1

73.5

73.7

74.0

74.3

71

Market1501

This indicates our method's potential for further improved performance with increasing training data.

5 Conclusion

We present a novel framework, CION, to learn identity-invariance from cross-video person images for person re-identification pre-training. In particular, we model the identity correlation seeking process as a progressive multi-level denoising problem, with a novel noise concept defined. Also, we propose an identity-guided self-distillation loss to implement better large-scale pre-training. Compared to existing pre-training methods, we achieve significantly leading performance with even fewer training samples. Meanwhile , we contribute a model zoo to promote the development of this field.

Acknowledgements. This work was supported by the National Natural Science Foundation of China No.62176097, and the Hubei Provincial Natural Science Foundation of China No.2022CFA055. We gratefully acknowledge the support of MindSpore, CANN (Compute Architecture for Neural Networks) and Ascend AI Processor used for this research.

References

Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In ICCV , pages 9650-9660, 2021.

Tianlong Chen, Shaojin Ding, Jingyi Xie, Ye Yuan, Wuyang Chen, Yang Yang, Zhou Ren, and Zhangyang Wang. Abd-net: Attentive but diverse person re-identification. In ICCV , pages 8351-8361, 2019.

Weihua Chen, Xianzhe Xu, Jian Jia, Hao Luo, Yaohua Wang, Fan Wang, Rong Jin, and Xiuyu Sun. Beyond appearance: a semantic controllable self-supervised learning framework for human-centric visual tasks. In CVPR , pages 15050-15061, 2023.

Xinlei Chen, Haoqi Fan, Ross Girshick, and Kaiming He. Improved baselines with momentum contrastive learning. arXiv preprint arXiv:2003.04297 , 2020.

Xinlei Chen, Saining Xie, and Kaiming He. An empirical study of training self-supervised vision transformers. In ICCV , pages 9640-9649, 2021.

Zuozhuo Dai, Guangyuan Wang, Weihao Yuan, Siyu Zhu, and Ping Tan. Cluster contrast for unsupervised person re-identification. In ACCV , pages 1142-1160, 2022.

Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A largescale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition , pages 248-255. Ieee, 2009.

Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. ICLR , 2021.

Zhaopeng Dou, Zhongdao Wang, Yali Li, and Shengjin Wang. Identity-seeking self-supervised representation learning for generalizable person re-identification. In ICCV , pages 15847-15858, 2023.

Dengpan Fu, Dongdong Chen, Jianmin Bao, Hao Yang, Lu Yuan, Lei Zhang, Houqiang Li, and Dong Chen. Unsupervised pre-training for person re-identification. In CVPR , pages 14750-14759, 2021.

Dengpan Fu, Dongdong Chen, Hao Yang, Jianmin Bao, Lu Yuan, Lei Zhang, Houqiang Li, Fang Wen, and Dong Chen. Large-scale pre-training for person re-identification with noisy labels. In CVPR , pages 2476-2486, 2022.

Yixiao Ge, Dapeng Chen, and Hongsheng Li. Mutual mean-teaching: Pseudo label refinery for unsupervised domain adaptation on person re-identification. arXiv preprint arXiv:2001.01526 , 2020.

Yixiao Ge, Feng Zhu, Dapeng Chen, Rui Zhao, et al. Self-paced contrastive learning with hybrid memory for domain adaptive object re-id. NeurIPS , 33:11309-11321, 2020.

Yunpeng Gong, Liqing Huang, and Lifei Chen. Person re-identification method based on color attack and joint defence. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 4313-4322, 2022.

Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Guo, Mohammad Gheshlaghi Azar, et al. Bootstrap your own latent-a new approach to self-supervised learning. NeurIPS , 33:2127121284, 2020.

Kai Han, Yunhe Wang, Qi Tian, Jianyuan Guo, Chunjing Xu, and Chang Xu. Ghostnet: More features from cheap operations. In CVPR , pages 1580-1589, 2020.

Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. In CVPR , pages 16000-16009, 2022.

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR , pages 770-778, 2016.

Lingxiao He, Xingyu Liao, Wu Liu, Xinchen Liu, Peng Cheng, and Tao Mei. Fastreid: A pytorch toolbox for general instance re-identification. arXiv preprint arXiv:2006.02631 , 2020.

Shuting He, Hao Luo, Pichao Wang, Fan Wang, Hao Li, and Wei Jiang. Transreid: Transformerbased object re-identification. In ICCV , pages 15013-15022, 2021.

Jiahao Hong, Jialong Zuo, Chuchu Han, Ruochen Zheng, Ming Tian, Changxin Gao, and Nong Sang. Spatial cascaded clustering and weighted memory for unsupervised person reidentification. arXiv preprint arXiv:2403.00261 , 2024.

Xin Jin, Cuiling Lan, Wenjun Zeng, Guoqiang Wei, and Zhibo Chen. Semantics-aligned representation learning for person re-identification. In AAAI , volume 34, pages 11173-11180, 2020.

He Li, Mang Ye, and Bo Du. Weperson: Learning a generalized re-identification model from allweather virtual data. In Proceedings of the 29th ACM international conference on multimedia , pages 3115-3123, 2021.

He Li, Mang Ye, Ming Zhang, and Bo Du. All in one framework for multimodal re-identification in the wild. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 17459-17469, 2024.

Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In ICCV , 2021.

Zhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie. A convnet for the 2020s. CVPR , 2022.

Hao Luo, Pichao Wang, Yi Xu, Feng Ding, Yanxin Zhou, Fan Wang, Hao Li, and Rong Jin. Self-supervised pre-training for transformer-based person re-identification. arXiv preprint arXiv:2111.12084 , 2021.

Muhammad Maaz, Abdelrahman Shaker, Hisham Cholakkal, Salman Khan, Syed Waqas Zamir, Rao Muhammad Anwer, and Fahad Shahbaz Khan. Edgenext: efficiently amalgamated cnntransformer architecture for mobile vision applications. In ECCV , pages 3-20. Springer, 2022.

Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In ICML , pages 8748-8763. PMLR, 2021.

Jiangming Shi, Xiangbo Yin, Yeyun Chen, Yachao Zhang, Zhizhong Zhang, Yuan Xie, and Yanyun Qu. Multi-memory matching for unsupervised visible-infrared person re-identification. In Computer Vision-ECCV 2024: 18th European Conference , page 456-474, 2024.

Jiangming Shi, Xiangbo Yin, Yachao Zhang, Zhizhong Zhang, Yuan Xie, and Yanyun Qu. Learning commonality, divergence and variety for unsupervised visible-infrared person reidentification. arXiv preprint arXiv:2402.19026v2 , 2024.

Jiangming Shi, Yachao Zhang, Xiangbo Yin, Yuan Xie, Zhizhong Zhang, Jianping Fan, Zhongchao Shi, and Yanyun Qu. Dual pseudo-labels interactive self-training for semi-supervised visible-infrared person re-identification. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 11218-11228, 2023.

Shixiang Tang, Cheng Chen, Qingsong Xie, Meilin Chen, Yizhou Wang, Yuanzheng Ci, Lei Bai, Feng Zhu, Haiyang Yang, Li Yi, et al. Humanbench: Towards general human-centric perception with projector assisted pretraining. In CVPR , pages 21970-21982, 2023.

Laurens Van Der Maaten. Accelerating t-sne using tree-based algorithms. The journal of machine learning research , 15(1):3221-3245, 2014.

Pavan Kumar Anasosalu Vasu, James Gabriel, Jeff Zhu, Oncel Tuzel, and Anurag Ranjan. Fastvit: A fast hybrid vision transformer using structural reparameterization. In ICCV , pages 5785-5795, 2023.

Ao Wang, Hui Chen, Zijia Lin, Hengjun Pu, and Guiguang Ding. Repvit: Revisiting mobile cnn from vit perspective. CVPR , 2024.

Dongkai Wang and Shiliang Zhang. Unsupervised person re-identification via multi-label classification. In CVPR , pages 10981-10990, 2020.

Guanshuo Wang, Yufeng Yuan, Xiong Chen, Jiwei Li, and Xi Zhou. Learning discriminative features with multiple granularities for person re-identification. In ACM MM , pages 274-282, 2018.

Haochen Wang, Jiayi Shen, Yongtuo Liu, Yan Gao, and Efstratios Gavves. Nformer: Robust person re-identification with neighbor transformer. In CVPR , pages 7297-7307, 2022.

Longhui Wei, Shiliang Zhang, Wen Gao, and Qi Tian. Person transfer gan to bridge domain gap for person re-identification. In CVPR , pages 79-88, 2018.

Yuhang Wu, Tengteng Huang, Haotian Yao, Chi Zhang, Yuanjie Shao, Chuchu Han, Changxin Gao, and Nong Sang. Multi-centroid representation network for domain adaptive person re-id. In AAAI , volume 36, pages 2750-2758, 2022.

Pan Xingang, Luo Ping, Shi Jianping, and Tang Xiaoou. Two at once: Enhancing learning and generalization capacities via ibn-net. In ECCV , 2018.

Shiyu Xuan and Shiliang Zhang. Intra-inter camera similarity for unsupervised person reidentification. In CVPR , pages 11926-11935, 2021.

Zizheng Yang, Xin Jin, Kecheng Zheng, and Feng Zhao. Unleashing potential of unsupervised pre-training with intra-identity regularization for person re-identification. In CVPR , pages 14298-14307, 2022.

Mang Ye, He Li, Bo Du, Jianbing Shen, Ling Shao, and Steven CH Hoi. Collaborative refining for person re-identification with label noise. IEEE Transactions on Image Processing , 31:379-391, 2021.

Xiangbo Yin, Jiangming Shi, Yachao Zhang, Yang Lu, Zhizhong Zhang, Yuan Xie, and Yanyun Qu. Robust pseudo-label learning with neighbor relation for unsupervised visible-infrared person re-identification. arXiv preprint arXiv:2405.05613 , 2024.

Li Yuan, Qibin Hou, Zihang Jiang, Jiashi Feng, and Shuicheng Yan. Volo: Vision outlooker for visual recognition. IEEE TPAMI , 45(5):6575-6586, 2022.

Gong Yunpeng, Zhong Zhun, Luo Zhiming, Qu Yansong, Ji Rongrong, and Jiang Min. Cross-modality perturbation synergy attack for person re-identification. arXiv preprint arXiv:2401.10090 , 2024.

Kaiwei Zeng, Munan Ning, Yaohua Wang, and Yang Guo. Hierarchical clustering with hardbatch triplet loss for person re-identification. In CVPR , pages 13657-13665, 2020.

Liang Zheng, Liyue Shen, Lu Tian, Shengjin Wang, Jingdong Wang, and Qi Tian. Scalable person re-identification: A benchmark. In ICCV , pages 1116-1124, 2015.

Zhedong Zheng, Liang Zheng, and Yi Yang. Unlabeled samples generated by gan improve the person re-identification baseline in vitro. In ICCV , pages 3754-3762, 2017.

Haowei Zhu, Wenjing Ke, Dong Li, Ji Liu, Lu Tian, and Yi Shan. Dual cross-attention learning for fine-grained visual categorization and object re-identification. In CVPR , pages 4692-4702, 2022.

Kuan Zhu, Haiyun Guo, Tianyi Yan, Yousong Zhu, Jinqiao Wang, and Ming Tang. Pass: Part-aware self-supervised pre-training for person re-identification. In ECCV , pages 198-214. Springer, 2022.

Yang Zou, Xiaodong Yang, Zhiding Yu, BVK Vijaya Kumar, and Jan Kautz. Joint disentangling and adaptation for cross-domain person re-identification. In ECCV , pages 87-104. Springer, 2020.

Jialong Zuo, Jiahao Hong, Feng Zhang, Changqian Yu, Hanyu Zhou, Changxin Gao, Nong Sang, and Jingdong Wang. Plip: Language-image pre-training for person representation learning, 2024.

Jialong Zuo, Hanyu Zhou, Ying Nie, Feng Zhang, Tianyu Guo, Nong Sang, Yunhe Wang, and Changxin Gao. Ufinebench: Towards text-based person retrieval with ultra-fine granularity. CVPR , 2024.

A Appendix

A.1 Broader impact

This paper proposes a cross-video identity-correlating pre-training framework for person reidentification, and contributes a pre-trained model zoo with spanning model structures and parameters. Our framework and models can help to identity and track pedestrians across different cameras, thus boosting the development of smart retail, smart transportation, smart security systems and so on in the future metropolises. In addition, our proposed pre-training framework is quite general and not limited to the specific research field of person ReID. It can be well extended to broader research areas with identity concept such as vehicle ReID.

Nevertheless, the application of person ReID, such as for identifying and tracking pedestrians in surveillance systems [14, 48, 31, 32, 30], might raise privacy concerns. It typically depends on the utilization of surveillance data for training without the explicit consent of the individuals being recorded. Therefore, governments and officials need to carefully establish strict regulations and laws to control the usage of ReID technologies. Otherwise, person ReID technologies [46, 24, 23, 45] can potentially equip malicious actors with the ability to surveil pedestrians through multiple cameras without their consent. Meanwhile, the research community should also avoid using any dataset with ethics issues. For example, DukeMTMC [51] has been taken down due to the violation of data collection terms and should no longer be used. We would not evaluate our models on DukeMTMC related benchmarks as well. Furthermore, we should be cautious of the misidentification of the ReID systems to avoid possible disturbance. Also, note that the demographic makeup of the datasets used is not representative of the broader population.

At the same time, we have utilized a substantial amount of person-containing video data from the internet for pre-training purposes. Consequently, it is inevitable that the resulting models may inherently contain information about these persons. Researchers should adhere to relevant laws and regulations, and strive to avoid using our models for any improper invasion of privacy. We will have a gated release of our models and training data to avoid any misuse. We will require that users adhere to usage guidelines and restrictions to access our models and training data. Meanwhile, all our open-sourced assets can only be used for research purpose and are forbidden for any commercial use.

A.2 Limitations

CION presents a preliminary attempt to correlate person images across different internet videos to implement person re-identification pre-training. Despite its effectiveness on existing public datasets, CION may still be difficult to learn good fine-grained person representations for it does not explicitly achieve further fine-grained information mining. Meanwhile, due to the limitations of the feature extractor's representational capacity, it is inevitable that some extremely difficult noise will be hard to remove during the denoising process. As a result, the automatically sought identity correlation may still have a certain discrepancy in accuracy when compared to manually annotated correlation, which may introduce a certain degree of ambiguity during the training process. Additionally, the Sliding Range and Linking Relation strategy we proposed to conserve computational resources may result in difficulty seeking correlations between tracklets of the same person across different videos that are spaced extremely far apart. Also, as we have followed the conventional practice of previous works by utilizing a large amount of person-containing internet video data to implement pre-training, there is a potential for privacy and security issues to some extent. Therefore, in our subsequent work, we will focus on addressing fine-grained issues and improving the accuracy of identity correlations. We will take every possible measure to prevent the misuse of our models and dataset as well.

A.3 Safeguards for dataset and models

To address the privacy concerns associated with crawling videos from the internet and the application of person ReID technology, we will implement a controlled release of our models and dataset, thereby preventing privacy violations and ensuring information security. We require that the crawled images are sourced from legitimate video websites to avoid the inclusion of harmful content. We will also require that users adhere to usage guidelines and restrictions to access our models and dataset. For instance, we have drafted the following regulations that must be adhered to, which will be refined and elaborated in subsequent releases:

1. Privacy: All individuals using the models in ReIDZoo and the CION-AL dataset should agree to protect the privacy of all the subjects in it. The users should bear all responsibilities and consequences for any loss caused by the misuse.

2. Redistribution: The models in ReIDZoo and the CION-AL dataset, either entirely or partly, should not be further distributed, published, copied, or disseminated in any way or form without a prior approval from the creators, no matter for profitable use or not.

3. Commercial Use: The models in ReIDZoo and the CION-AL dataset, entirely, partly, or in any format derived thereof, is not allowed for commercial use.

4. Modification: The models in ReIDZoo and the CION-AL dataset, either entirely or partly, is not allowed to be modified.

In parallel, we will require users to provide relevant information and will rigorously screen the submitted details to restrict access to our dataset and models by institutions or individuals with a history of privacy violations.

A.4 Licenses for existing assets

We utilize some existing assets to conduct our research, which can be categorized to three types: code, data and models. All assets used in our paper are properly credited and all original paper are cited. We list the license and URL for each asset below.

Code

1. Emerging Properties in Self-Supervised Vision Transformers [1]. (DINO)

Apache 2.0 License, URL: https://github.com/facebookresearch/dino

2. TransReID: Transformer-based Object Re-Identification [20]. (TransReID)

MIT License, URL: https://github.com/damo-cv/TransReID

3. Cluster Contrast for Unsupervised Person Re-Identification [6]. (C-Contrast)

MIT License, URL: https://github.com/alibaba/cluster-contrast-reid

4. Unsupervised Pre-training for Person Re-identification [10]. (LUP)

MIT License, URL: https://github.com/DengpanFu/LUPerson

5. Self-Supervised Pre-Training for Transformer-Based Person Re-Identification [27]. (TranSSL)

MIT License, URL: https://github.com/damo-cv/TransReID-SSL

6. FastReID: A Pytorch Toolbox for General Instance Re-identification [19]. (FastReID)

Apache 2.0 License, URL: https://github.com/JDAI-CV/fast-reid

Data

1. PLIP: Language-Image Pre-training for Person Representation Learning [55]. (SYNTH-PEDES)

MIT License, URL: https://github.com/Zplusdragon/PLIP

2. UFineBench: Towards Text-based Person Retrieval with Ultra-fine Granularity [56]. (UFine6926)

UFineBench License, URL: https://github.com/Zplusdragon/UFineBench

3. Person Transfer GAN to Bridge Domain Gap for Person Re-Identification [40]. (MSMT17)

4. Scalable Person Re-Identification: A Benchmark [50]. (Market1501)

MIT License, URL: http://zheng-lab.cecs.anu.edu.au/Project/project_reid.html

Models

1. Deep Residual Learning for Image Recognition [18]. (ResNet)

BSD-3-Clause license, URL: https://github.com/pytorch/vision

2. Instance-Batch Normalization Network [42]. (ResNet-IBN)

MIT License, URL: https://github.com/XingangPan/IBN-Net

3. GhostNet: More Features from Cheap Operations [16]. (GhostNet)

Apache 2.0 License, URL: https://github.com/huawei-noah/Efficient-AI-Backbones/

EdgeNeXt: Efficiently Amalgamated CNN-Transformer Architecture for Mobile Vision Applications [28]. (EdgeNext)

MIT License: URL: https://github.com/mmaaz60/EdgeNeXt

5. RepViT: Revisiting Mobile CNN From ViT Perspective [36]. (RepViT)

Apache 2.0 License, URL: https://github.com/THU-MIG/RepViT

6. FastViT: A Fast Hybrid Vision Transformer using Structural Reparameterization [35]. (FastViT)

7. A ConvNet for the 2020s [26]. (ConvNext)

MIT License, URL: https://github.com/facebookresearch/ConvNeXt

8. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. (ViT)

Apache 2.0 License, URL: https://github.com/google-research/vision_transformer

9. Swin Transformer: Hierarchical Vision Transformer using Shifted Windows [25]. (Swin)

MIT License, URL: https://github.com/microsoft/Swin-Transformer

10. VOLO: Vision Outlooker for Visual Recognition [47]. (VOLO)

Apache 2.0 License, URL: https://github.com/sail-sg/volo

A.5 Pre-training setting for each model

Due to the varying memory requirements of different models during the pre-training process, we optimize the utilization of computational resources by setting different batch sizes and numbers of local cropped views for each model. We report the batch sizes, numbers of local cropped views and rough training time for each model in Tab. 5. Each model is trained on 8 × V100 GPUs. The pre-training dataset is the whole CION-AL, which has 3,898,086 images of 246,904 persons.

A.6 Samples used in t-SNE visualization

For reproducing the t-SNE visualization results shown in Fig. 4, we display the samples we randomly selected from Market1501 [50] in Fig. 8. There are totally 15 persons with 20 cropped images each. We obscure the faces of each person to avoid the privacy violations. Researchers can use these samples and the official models of LUP [10] and LUP-NL [11] to reproduce the visualization results.

A.7 Linear training time increasing

Wehave assessed how the training time of CION scale with pre-training dataset size. We use ResNet50 to conduct the experiment. We record

the training time required by CION as the pretraining data size increases from 10% to 100%. As shown in Fig. 7, the training time of our CION increases almost linearly with the growth in the size of the pre-trained dataset. This linear increase in training time ensures that our CION can be effectively applied to large-scale pre-training. This indicates that we can achieve better performance by adding more training data without incurring an unbearable training cost.

20%

10%

30%

40%

0

Training time [day]

Training Data Size

100%

50%

60%

70%

80%

90%

0.52

1.03

1.54

2.08

2.61

3.13

3.66

4.16

4.65

5.13

1

2

3

4

5

6

A.8 More results for unsupervised person ReID

To further assess the model structure adaptability of our CION, we evaluate the performance of more CION pretrained models for unsupervised person ReID, inclucing ResNet [18], ResNetIBN [42] and ViT [8]. We report the results in Tab. 6. As we can see, all the models achieve superior performance for different unsupervised settings on Market1501 [50] and MSMT17 [40]. Meanwhile, as the number of parameters increases, the performance of each model

Table 6: The performance of different CION pre-trained models for unsupervised person ReID. 'Mar' and 'MS' denote Market1501 and MSMT17 respectively.

consistently improves. These results further indicate that our CION can be effectively used to pre-train various models, enabling them to achieve significant performance improvements for person ReID.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: The main claims made in the abstract and introduction of our paper accurately reflect our paper's contributions and scope.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We conduct an in-depth discussion of the limitations of our work, which can be found in Sec. A.2 of the appendix.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [NA]

Justification: Our work focuses on computer vision applications, not including theoretical results.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: We have fully provided all the details of our proposed methods in the main text and appendix, to guarantee the reproducibility of our paper's main experimental results. Meanwhile, we will open-source all of our code, data, and models after our paper is accepted. These details and open-source initiatives will ensure the reproducibility of our work and make a significant contribution to the entire community.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in

some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [No]

Justification: To address privacy concerns associated with our dataset containing persons and the application of person ReID technology, we will implement a controlled release of our code, data, and models. Therefore, to ensure private information security, we would not provide open access to our data and code during the paper submission process. We provide training and testing logs in the supplementary materials to ensure the authenticity and validity of our results. Notably, we will publicly share the application link for all our sources after our paper is accepted. Applicants will be required to adhere to relevant guidelines and regulations to access our sources, and we will conduct a strict review of their qualifications to prevent any privacy violations.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: Our paper specifies all the training and test details necessary to understand the results.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [No]

Justification: Due to pre-training technology is comparably computational expensive, the error bars are not reported in the paper like nearly all related work. We provide training and testing logs in the supplementary materials to ensure the authenticity and validity of our results.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: Our paper provides sufficient information on the computer resources needed to reproduce the experiments, which can be found in Sec. 4.1 and Sec. A.5.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: The research conducted in the paper complies with the NeurIPS Code of Ethics in every respect.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: Our paper discusses both potential positive societal impacts and negative societal impacts of the work performed in Sec. A.1 of the appendix.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [Yes]

Justification: Our paper describes safeguards that have been put in place for responsible release of data and models in Sec. A.3 of the appendix.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: The creators and original owners of assets used in the paper are properly credited, and the license and terms of use are explicitly mentioned and properly respected, which can be found in Sec. A.4 of the appendix.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: To address privacy concerns associated with our assets including code, data, and models, we will implement a controlled release of our assets. Therefore, to ensure private information security, we would not release our assets at submission time. We will publicly share the application link for our assets after our paper is accepted. Applicants will be required to adhere to relevant guidelines and regulations to access our assets, and we will conduct a strict review of their qualifications to prevent any privacy violations.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: Our paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: Our paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work explores learning the ReID model from a large number of unlabeled videos. The identity associations within the video and across the videos are mined. Extensive experiments under different backbones and downstream ReID settings are conducted.\n\nSTRENGTHS:\n1. Learning from unlabeled videos makes sense for improving ReID model.\n2. Extensive experiments are conducted.\n3. The performance is impressive.\n\nWEAKNESSES:\n1. The technical contribution is not significant. Associating different images is the core of the work. However, the main technique is to associate based on the feature similarity following a spirit of progressive learning. Such a pipeline is not novel and widely applied in many previous works. For example, some works mine the associations between images in different tracklets based on feature similarity.[R1, R2, R3, R4]. Although some of these works learn the model from trimmed videos, but share a similar sprit of progressively associating different tracklets/videos based on feature similarity.\nBesides, such a pipeline could be sensitive to the hyperparameter, i.e., $\\sigma_{cst}, \\sigma_{drm}$, but the paper does not report the ablation study in the hyperparameter.\n\n2. Supervised learning can be employed with the identity association. Why employing self-supervised learning in Sec. 3.3.\n\n3. The proposed method achieves similar performance with SOLIDER in Table 1.\n\n[R1] Tracklet Self-Supervised Learning for Unsupervised Person Re-Identification. AAAI2020.\n[R2] CycAs: Self-supervised Cycle Association for Learning Re-identifiable Descriptions. ECCV2020.\n[R3] Unsupervised Person Re-identification by Deep Learning Tracklet. ECCV2018\n[R4] Progressive Unsupervised Person Re-Identification by Tracklet Association With Spatio-Temporal Regularization. TMM 2020\n\nQUESTIONS:\nPlease refer to weakness.\n\nLIMITATIONS:\nThe limitations as well as the safeguards for dataset and models  are discussed in the paper.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2024-07-07"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces a Cross-video Identity-cOrrelating pre-traiNing (CION) framework for person re-identification. CION addresses the limitations of existing instance-level and single-video tracklet-level pre-training methods by leveraging identity-invariance across different videos. The framework uses a progressive multi-level denoising strategy and an identity-guided self-distillation loss to improve representation learning. Extensive experiments demonstrate that CION significantly enhances performance with fewer training samples compared to previous state-of-the-art methods. The authors also contribute a model zoo, ReIDZoo, which includes various pre-trained models.\n\nSTRENGTHS:\nOriginality: The paper introduces a novel approach to pre-training for person re-identification by explicitly modeling identity invariance across videos, which is a significant departure from existing methods.\n\nClarity: The paper is clearly written, with detailed explanations and effective use of visual aids.\n\nSignificance: The proposed method achieves state-of-the-art performance with fewer training samples and demonstrates model-agnostic ability, making it highly valuable for both research and practical applications.\n\nWEAKNESSES:\n1. Pre-training models should aim for broader generalizability. The current models tend to be human-centered, covering tasks such as re-identification (reid) and human parsing, while others are more narrowly focused on ReID alone, akin to SemReID [1] or PersonMAE [2].\nThe authors' experiments have concentrated primarily on two datasets, Market-1501 and MSMT17. This limited scope may not sufficiently demonstrate the models' generalizability across varied conditions. To better assess and enhance the generalizability of the models, it is suggested that the authors include results from datasets that introduce challenges such as occlusion (Occldued-duke) and clothing change (LTCC or PRCC). This expansion would provide a more comprehensive understanding of the model's performance across different real-world scenarios.\n\n2. The proposed method relies on the quality of the initial tracking and identity correlation, which may be affected by the accuracy of the tracking algorithms used.\n\n3. Some of the methods that need to be compared are ignored. For example, HAP [3] with ViT-Base achieves better performance in MSMT17. \n\nReferences\n\n[1] Huang, Siyuan, Yifan Zhou, Ram Prabhakar Kathirvel, Rama Chellappa, and Chun Pong Lau. \"Self-Supervised Learning of Whole and Component-Based Semantic Representations for Person Re-Identification.\" arXiv preprint arXiv:2311.17074 (2023).\n\n[2] Hu, Hezhen, Xiaoyi Dong, Jianmin Bao, Dongdong Chen, Lu Yuan, Dong Chen, and Houqiang Li. \"PersonMAE: Person Re-Identification Pre-Training With Masked AutoEncoders.\" IEEE Transactions on Multimedia (2024).\n\n[3] Yuan, Junkun, Xinyu Zhang, Hao Zhou, Jian Wang, Zhongwei Qiu, Zhiyin Shao, Shaofeng Zhang et al. \"HAP: Structure-aware masked image modeling for human-centric perception.\" Advances in Neural Information Processing Systems 36 (2023).\n\nQUESTIONS:\nsee weakness.\n\nLIMITATIONS:\n1. Fine-grained Representation Learning: The current CION framework does not explicitly target fine-grained information extraction, which may limit its ability to learn highly detailed person representations necessary for distinguishing between very similar identities.\n\n2. Noise Removal Challenges: Due to limitations in the feature extractor’s representational capacity, some complex noise may remain unresolved during the denoising process, leading to potential inaccuracies in the automatically derived identity correlations.\n\n3. Correlation Issues with Distant Tracklets: The strategy used to conserve computational resources may struggle to establish correlations between tracklets of the same person that are widely separated across different videos, potentially leading to incomplete identity mapping.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-09"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose an unsupervised method for mining identity relations in large-scale video data, incorporating self-supervised learning to pre-train the general models (like CNN, transformer). This approach demonstrates superior performance in person ReID tasks compared to other pre-training models.\n\nSTRENGTHS:\nLarge-scale video datasets are effective training source of ReID task. The authors propose a straightforward method for pre-labeling instance identity from video streams, taking into account the appearance of pedestrians across multiple videos and the necessary computational complexity. They constructed the CION-AL dataset based on the SYNTHPEDES dataset, which contains rich pedestrian information that effectively aids model pre-training.\n\nUnder the DINO self-supervised learning framework, the authors differentiate the data source between the teacher and student models, enhancing the student model's representation capabilities through the focus on local pedestrian information.\n\nThe authors use various CNN and Transformer networks pre-trained on ImageNet as baselines, conducting extensive experiments on supervised and unsupervised domain adaptation datasets. The results demonstrate that the pre-trained models built using CION-AL are more effective for pedestrian re-identification tasks compared to those built using ImageNet.\n\nWEAKNESSES:\nThe primary contribution of this paper lies in proposing a method for unsupervised data preprocessing of video streams, utilizing self-supervised learning to pre-train more powerful pedestrian re-identification models. However, the data preprocessing method essentially employs an unsupervised clustering strategy, which is a straightforward pre-labeling approach. The establishment of sample relationships in this manner is often suboptimal. Therefore, it is crucial to demonstrate in Section 4.4 through ablation studies whether it is the identity correlation seeking (i.e., denoising or pre-labeling) or the large scale of the dataset that enhances the model's performance, and quantify their respective contributions.\n\nAdditionally, the proof of computational complexity for cross-video denoising in Section 3.2 is insufficient, making the significance of the Sliding Range and Linking Relation unclear. \n\nIn Section 3.3, the training process of identity-guided self-distillation primarily differs from DINO in the way local information is provided to the teacher and student data streams. So, the “identity-guided” is still reflected in data preprocessing rather than in the self-distillation process.\n\nIn Section 4.2, the MGN fine-tuning has a certain impact on model performance, but the essence of this fine-tuning lacks necessary explanation. \n\nIn summary, although the proposed CION-AL dataset, in terms of volume and denoising processing (pre-labeling), enhances general pre-training models, the lack of qualitative analysis experiments on denoising significantly diminishes the significance of the proposed method.\n\nQUESTIONS:\n1. Is the contribution of this paper to ReID general pre-training models primarily a new large-scale dataset, or the proposal of a method to utilize identity correlation?\n2. How can you demonstrate that the proposed method effectively utilizes identity correlation information of pedestrians?\n3. When using the same dataset, How can you experimentally prove the superiority of the proposed identity correlation seeking method?\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-10"}

comments: SUMMARY:
This paper present a novel framework CION, to learn identity-invariance from cross-video person images for person re-identification pre-training. In particular, they model the identity correlation seeking process as a progressive multi-level denoising problem, with a novel noise concept defined. Also, they propose an identity-guided self-distillation loss to implement better large-scale pre-training.

STRENGTHS:
The identity-invarience of the same person across different videos is significantly neglected by these single-video tracklet-level methods. This paper present a cross-video identity-correlating pre-training framework, which explicitly learns identity-invariance in long-range cross-video person images.

WEAKNESSES:
This paper compare the fine-tuning results with two popular identity-ignored pre-training methods LUP and DINO in Tab. 4, how about with other identity-ignored pre-training methods？

QUESTIONS:
1.It is significantly valuable to eliminate the problem of false positives of identity tags caused by tracking errors, How does it work in this paper?
2.It is significantly valuable for the model to learn better identity representations without knowing the identity tags of the same person across different videos. How does it work in this paper?

LIMITATIONS:
The authors have given the limitations of the proposed model, the authors mentioned that the automatically sought identity correlation may still have a certain discrepancy in accuracy when compared to manually annotated correlation, which may introduce a certain degree of ambiguity during the training process. In their subsequent work, they will focus on addressing fine-grained issues and improving the accuracy of identity correlations.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-12