Dual-Resolution Fusion Modeling for Unsupervised Cross-Resolution Person Re-Identification

Authors: zhiqi  pang
lingling  zhao
chunyu  wang
Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-05 15:52:08

Abstract: Cross-resolution person re-identification (CR-ReID) aims to match images of the same person with different resolutions in different scenarios. Existing CR-ReID methods achieve promising performance by relying on large-scale manually annotated identity labels. However, acquiring manual labels requires considerable human effort, greatly limiting the flexibility of existing CR-ReID methods. To address this issue, we propose a dual-resolution fusion modeling (DRFM) framework to tackle the CR-ReID problem in an unsupervised manner. Firstly, we design a cross-resolution pseudo-label generation (CPG) method, which initially clusters high-resolution images and then obtains reliable identity pseudo-labels by fusing class vectors in both resolution spaces. Subsequently, we develop a cross-resolution feature fusion (CRFF) module to fuse features from both high-resolution and low-resolution spaces. The fusion features have the potential to serve as a new form of resolution-invariant features. Finally, we introduce cross-resolution contrastive loss and probability sharpening loss in DRFM to facilitate resolution-invariant learning and effectively utilize ambiguous samples for optimization. Experimental results on multiple CR-ReID datasets demonstrate that the proposed DRFM not only outperforms existing unsupervised methods but also approaches the performance of early supervised methods.

Main paper: 
 introduction: 
 Person re-identification (ReID) [44, 45, 53] targets matching images of individuals captured by multiple cameras with the same identity. Benefiting from deep models [37, 42] and optimization methods [20, 36] , existing ReID methods have achieved exciting performance in simple scenarios. However, their performance can be influenced by various factors in complex scenarios [4, 41, 52] . For instance, due to the impact of camera specifications and shooting distances, there is often a significant disparity in the underlying resolution (clarity) among different images. Matching low-resolution (LR) images directly with high-resolution (HR) images can lead to performance degradation. To address this issue, some researchers have turned their attention to cross-resolution person re-identification (CR-ReID) [41, 52] . Existing CR-ReID methods [16, 48] typically begin by utilizing super-resolution (SR) models [23, 50] to enhance the underlying resolution of LR images, thereby obtaining synthetic HR (SHR) images. Subsequently, matching is performed between SHR and HR images.While existing CR-ReID methods have achieved promising performance, they still rely on manually annotated identity labels [41, 52] . However, annotating datasets for image retrieval tasks is a time-consuming process [33] , and currently, there are no unsupervised CR-ReID methods available. Therefore, in this paper, we attempt to address the CR-ReID problem in an unsupervised manner. In this new scenario, we face two key challenges: (1) How to obtain reliable identity pseudo-labels. Unsupervised general ReID methods [27, 38] often use clustering algorithms [11] to generate pseudo-labels. However, in CR-ReID, due to significant appearance differences between LR and HR images, directly applying clustering algorithms between them can result in a large number of noisy labels. Noisy labels can mislead model optimization and hinder performance improvement [15, 32] . (2) How to obtain resolution-invariant features. On one hand, SHR images generated by SR models [23, 50] may contain artifacts, which could alter the original identity features. Therefore, using SHR features as resolution-invariant features for image matching is suboptimal. On the other hand, unsupervised methods often struggle to incorporate unlabeled ambiguous samples into the optimization process [15, 32] , leading to lower sample utilization compared to supervised methods. This further increases the difficulty of resolution-invariant feature learning. For smooth and concise description, in the subsequent sections, we will also refer to the person images interchangeably as samples, based on which term fits best in the context.In this paper, we propose a novel unsupervised CR-ReID method: dual-resolution fusion modeling (DRFM). As illustrated in Figure 1 , on one hand, SHR images contain clearer details compared to LR images but inevitably include some artifacts. On the other hand, the model exhibits significant differences in visual explanations [35] between LR and SHR images. Therefore, the complementary nature between SHR and LR images holds the potential to improve the performance of CR-ReID [52] . Unlike supervised CR-ReID [52] , the unsupervised CR-ReID problem we face is more complex and requires deeper fusion. Therefore, DRFM fuses the semantic information of LR and SHR images at both the class and feature levels, thereby simultaneously addressing the aforementioned two challenges. To obtain reliable pseudo-labels, we introduce the crossresolution pseudo-label generation (CPG) method for class-level fusion. CPG initially clusters HR images and passes pseudo-labels to synthetic LR (SLR) images. Then, it estimates class vectors for SHR and LR images based on the pseudo-labels from HR and SLR images, respectively. Finally, it fuses the class vectors from both spaces to assign pseudo-labels to LR and SHR images. To obtain resolution-invariant features, we first design the cross-resolution feature fusion (CRFF) module in DRFM. The fusion of features from two different spaces has the potential to serve as a novel form of resolution-invariant feature. Subsequently, we introduce the cross-resolution contrastive loss and probability sharpening loss to optimize the entire framework. The former aims to facilitate resolution-invariant feature learning, while the latter aims to further enhance model performance by fully utilizing unlabeled ambiguous samples.The main contributions are summarized as follows:• We propose a dual-resolution fusion modeling (DRFM) framework, which integrates semantic information at both the class and feature levels to address the unsupervised CR-ReID problem. To the best of our knowledge, this is the first attempt in the field of unsupervised CR-ReID. • We design a cross-resolution pseudo-label generation (CPG) method, which acquires reliable pseudo-labels by fusing class vectors of LR and SHR images, instead of directly applying clustering algorithms between HR and LR images.• We develop a cross-resolution feature fusion (CRFF) module to obtain resolution-invariant features. Additionally, we introduce a cross-resolution contrastive loss and probability sharpening loss to facilitate resolution-invariant feature learning. • Extensive experimental results on multiple datasets demonstrate that our method not only surpasses existing unsupervised methods but also approaches the performance of certain supervised methods. 
related work 2.1 unsupervised general reid: 
 Existing unsupervised general ReID methods can be roughly divided into unsupervised domain adaptation (UDA) methods [21, 26, 31] and fully unsupervised (FU) methods [1, 30, 46] . The former relies on labeled source domain data and unlabeled target domain data for training, aiming to improve the recognition performance on the target domain. The latter no longer rely on labeled source domain data, making them more flexible and challenging. FU method typically iterates between sample clustering and model optimization.For instance, in earlier approaches, BUC [30] utilizes a hierarchical clustering algorithm to generate pseudo-labels, followed by the introduction of softmax classification loss to optimize the model. Building upon BUC [30] , HCT [47] incorporates batch hard triplet loss [20] to address hard samples. Subsequent research often focuses on addressing camera gap [34, 43] and noisy labels [7, 49] . For instance, during the clustering stage, IICS [43] amplifies inter-camera similarity by concatenating scores of the same image across different classifiers. CIFL [34] achieves a similar objective by introducing the concept of ensemble learning. In the optimization stage, both ICE [1] and O2CAP [39] introduce optimization methods tailored to camera gap to align distributions across cameras. To address noisy labels, RLCC [49] refines pseudo-labels using the concept of temporal ensembling. Both PPLR [7] and SECRET [18] leverage the complementary relationship between global and local features to eliminate noisy labels. RMCL [32] evaluates the reliability of pseudo-labels from the perspectives of certainty and stability, assigning lower weights to noisy labels. To mitigate the impact of noisy labels, Purification [25] initially trains a teacher model from the original pseudo-labels. The teacher model is then employed to guide the learning of a student model. The student model can converge rapidly under the supervision of the teacher model, thereby reducing the interference of noisy labels. While the aforementioned methods have shown promising performance in simple scenarios, they often face significant performance degradation in cross-resolution settings.CR-ReID aims to address the issue of resolution mismatch between images [6, 22] . Existing CR-ReID methods typically employ SR models [23, 50] to enhance the LR images' underlying resolution, thereby restoring missing fine-grained information. For instance, SING [22] performs joint training by connecting an SR model and a ReID model, facilitating representation learning while generating high-resolution images. PRI [16] incorporates a scale predictor module designed for the SR model, thereby providing appropriate scales for the super-resolution process. INTACT [6] utilizes the underlying association knowledge between SR and ReID as an additional learning constraint, thereby enhancing the compatibility between SR and ReID models. While SR models have the potential to restore missing fine-grained information in LR images, they inevitably introduce artifacts. These artifacts often interfere with the extraction of identity-related features.In addition to utilizing SR models for cross-resolution alignment, Chen et al. [3] introduce the concept of adversarial training [13] during the representation learning process, aiming to align the feature distributions of LR and HR images. Similarly, CAD-Net [29] also incorporates adversarial training to obtain resolutioninvariant features. However, since HR images contain fine-grained discriminative information not present in LR images, directly aligning the feature distributions of HR and LR images often results in the loss of such information. To address this issue, LRAR [41] embeds resolution information into features to obtain varying-length representations. The HR features are composed of shared components and HR-specific components, thereby preserving fine-grained information exclusive to HR while learning resolution-invariant feature representations. While the aforementioned methods have shown promising performance, they all rely on manually annotated identity labels. In contrast to these approaches, we propose an unsupervised CR-ReID method with the aim of reducing the dependence on manually annotated labels. 
proposed framework 3.1 overview: 
 During the training phase, we are provided with a set of HR images {𝑥 ℎ 𝑖 } 𝑁 1 𝑖=1 and a set of LR images {𝑥 𝑙 𝑖 } 𝑁 2 𝑖=1 , where 𝑁 1 and 𝑁 2 represent the numbers of HR and LR images, respectively. Our objective is to optimize the CR-ReID model without accessing manually annotated identity labels. During the testing phase, relying on the feature extraction capability of the model, we search for images in the HR image gallery that share the same identity as a given LR query image based on feature similarity.To achieve the aforementioned objectives, we propose the dualresolution fusion modeling (DRFM) framework. 𝑖=1 to store features with pseudo-labels in the HR space, LR space, and fusion space, respectively. Here, 𝑁 (𝑁 < 𝑁 1 +𝑁 2 ) represents the number of features with pseudo-labels. Finally, DRFM optimizes using cross-resolution contrastive loss 𝐿 𝑐𝑐 , probability sharpening loss 𝐿 𝑝𝑠 , and identity consistency loss 𝐿 𝑖𝑐 .In the optimization stage, we update the memory separately:EQUATIONEQUATIONEQUATIONwhere 𝑓 ℎℎ 𝑖 , 𝑓 𝑙𝑙 𝑖 , and 𝑓 𝑓 𝑢 𝑖 represent features with pseudo-labels in the HR space, LR space, and fusion space, respectively, which are output by the encoder or CRFF.{𝑓 ℎℎ 𝑖 } 𝑁 𝑖=1 ⊆ {𝑓 ℎ 𝑖 } 𝑁 1 𝑖=1 ∪ {𝑓 𝑠ℎ 𝑖 } 𝑁 2 𝑖=1 , {𝑓 𝑙𝑙 𝑖 } 𝑁 𝑖=1 ⊆ {𝑓 𝑙 𝑖 } 𝑁 2 𝑖=1 ∪ {𝑓 𝑠𝑙 𝑖 } 𝑁 1 𝑖=1 , {𝑓 𝑓 𝑢 𝑖 } 𝑁 𝑖=1 ⊆ {𝑓 ℎ𝑠𝑙 𝑖 } 𝑁 1 𝑖=1 ∪ {𝑓 𝑙𝑠ℎ 𝑖 } 𝑁 2 𝑖=1, and 𝛼 is the update rate. As shown in Table 1 , for ease of understanding, we have provided a summary and explanation of the key notations used in this paper.In cross-resolution scenarios, the resolution disparity between LR and HR images significantly increases feature discrepancies. Directly applying clustering algorithms between them can result in a large number of noisy labels. Intuitively, it seems reasonable to reduce the resolution disparity between images before applying clustering algorithms. However, this may not be straightforward. On one hand, down-sampling HR images can mitigate resolution disparity but results in the loss of fine-grained discriminative information. On the other hand, super-resolution generation for LR images can also reduce resolution disparity but inevitably introduces artifacts, which may alter the original identity features.To address the aforementioned issues, we have designed the cross-resolution pseudo-label generation (CPG) method, which aims to integrate semantic information from both HR and LR spaces to generate reliable pseudo-labels. Specifically, since HR images contain unique and reliable fine-grained discriminative information, CPG initially performs clustering on HR images to generate pseudo-labels. For any 𝑥 𝑠𝑙 𝑖 , CPG assigns it the same pseudo-label as its corresponding 𝑥 ℎ 𝑖 . Subsequently, CPG estimates class vectors for SHR and LR images separately in both HR and LR spaces. For example, in LR space, the class vector of LR image 𝑥 𝑙 𝑖 is defined as:EQUATIONwhere 𝑓 𝑙 𝑖 represents the feature extracted by encoder 𝐸 𝑙 from 𝑥 𝑙 𝑖 , 𝜏 𝑙 is the temperature hyperparameter, and 𝐶 𝑠𝑙 is the matrix composed of SLR centroids. For example, the 𝑗-th SLR centroid 𝑐 𝑠𝑙 𝑗 in 𝐶 𝑠𝑙 is defined as:EQUATIONwhere 𝑓 𝑠𝑙 𝑖 represents the feature extracted by encoder 𝐸 𝑙 from 𝑥 𝑠𝑙 𝑖 , and 𝑛 𝑠𝑙 𝑗 is the number of SLR features in the 𝑗-th cluster. Similarly, the class vector of SHR image 𝑥 𝑠ℎ 𝑖 is defined as:EQUATIONwhere 𝑓 𝑠ℎ 𝑖 represents the feature extracted by encoder 𝐸 ℎ from 𝑥 𝑠ℎ 𝑖 , 𝜏 ℎ is the temperature hyperparameter, and 𝐶 ℎ is the matrix composed of HR centroids. For example, the 𝑗-th HR centroid 𝑐 ℎ 𝑗 in 𝐶 ℎ is defined as:EQUATIONwhere 𝑓 ℎ 𝑖 represents the feature extracted by encoder 𝐸 ℎ from 𝑥 ℎ 𝑖 , and 𝑛 ℎ 𝑗 is the number of HR features in the 𝑗-th cluster. As 𝑥 𝑙 𝑖 and 𝑥 𝑠ℎ 𝑖 share the same identity information, we calculate the fused class vectors for them:EQUATIONwhere EQUATIONwhere 𝑁 𝑐 1 represents the number of HR images assigned pseudolabels by the clustering algorithm in the current stage, and 𝑡 is the proportional hyperparameter. We consider images with the same pseudo-labels in each resolution space as belonging to the same cluster.The above process reveals that CPG possesses dual advantages. On one hand, CPG achieves semantic complementarity by fusing class vectors from both spaces, thereby obtaining reliable pseudolabels. On the other hand, CPG provides abundant cross-resolution Existing methods [6, 22] typically focus on aligning the SHR features with HR features to learn resolution-invariant feature representations. However, the SHR images generated by SR models [23, 50] may contain artifacts, which could alter the original identity features. Therefore, considering SHR features as resolutioninvariant features for image matching is suboptimal. In response to the aforementioned issue, we have developed a cross-resolution feature fusion (CRFF) module to achieve the complementary nature between SHR and LR images. Subsequent matching will be based on the fusion features generated by CRFF. The detailed structure of CRFF is illustrated in Figure 3 .We illustrate the detailed processing steps of the CRFF module using SHR feature 𝑓 𝑠ℎ 𝑖 and LR feature 𝑓 𝑙 𝑖 as examples. Similar procedures are followed for HR feature 𝑓 ℎ 𝑖 and SLR feature 𝑓 𝑠𝑙 𝑖 . We first calculate the similarity between 𝑓 𝑠ℎ 𝑖 and 𝑓 𝑙 𝑖 :EQUATIONwhere 𝐹 1 and 𝐹 2 represent fully connected (FC) layers, and ⊙ denotes the Hadamard product. Subsequently, we perform feature enhancement based on similarity:EQUATIONEQUATIONFinally, feature fusion is accomplished based on feature concatenation (Cat) and FC layer 𝐹 3 :EQUATION𝑓 𝑙𝑠ℎ 𝑖 shares the same pseudo-label with 𝑓 𝑙 𝑖 , and similarly, 𝑓 ℎ𝑠𝑙 𝑖 shares the same pseudo-label with 𝑓 ℎ 𝑖 . We introduce memory 𝑀 ℎℎ , memory 𝑀 𝑙𝑙 , and memory 𝑀 𝑓 𝑢 to store features with pseudo-labels in the HR space, LR space, and fusion space, respectively.From the above process, it is evident that since 𝑓 𝑙𝑠ℎ 𝑖 and 𝑓 ℎ𝑠𝑙 𝑖 achieve semantic complementarity between two spaces, they have the potential to become a new form of resolution-invariant feature representation.During the optimization stage, existing supervised CR-ReID methods [6, 16, 22] typically utilize identity classification loss to guide model optimization. While this encourages the model to extract identity-related features, it does not prioritize resolution-invariance learning. Consequently, we have developed a cross-resolution contrastive loss aiming to simultaneously enhance identity-relevant and resolution-invariant feature learning. For simplicity, we illustrate this using the example of the fusion space. Based on the pseudolabels provided by CPG and the offline features stored in memory𝑀 𝑓 𝑢 = {𝑣 𝑓 𝑢 𝑖 } 𝑁 𝑖=1, we initially compute resolution centroids. For example, the resolution centroid 𝑐 ℎ𝑠𝑙 𝑗 is defined as:EQUATIONwhere 𝑣 ℎ𝑠𝑙 𝑖 represents the fusion feature of HR and SLR features stored in 𝑀 𝑓 𝑢 , and 𝑛 ℎ𝑠𝑙 𝑗 represents the number of 𝑣 ℎ𝑠𝑙 𝑖 in the 𝑗-th cluster. Similarly, the resolution centroid 𝑐 𝑙𝑠ℎ 𝑘 is defined as:EQUATIONwhere 𝑣 𝑙𝑠ℎ 𝑖 represents the fusion feature of LR and SHR features stored in 𝑀 𝑓 𝑢 , and 𝑛 𝑙𝑠ℎ 𝑘 represents the number of 𝑣 𝑙𝑠ℎ 𝑖 in the 𝑘-th cluster. It is possible for a cluster to contain two resolution centroids simultaneously. For any feature 𝑓 𝑓 𝑢 𝑖 with pseudo-label in the fusion space, we define the set of all resolution centroids in its cluster as its positive centroid set 𝑃 EQUATIONwhere |𝑃 𝑓 𝑢 𝑖 | ∈ {1, 2} represents the number of centroids in set 𝑃 𝑓 𝑢 𝑖 , and 𝜏 𝑐 is the temperature hyperparameter. We perform a similar optimization process for the HR space and LR space, where the cross-resolution contrastive loss is defined as: To encourage the HR encoder to focus on identity-relevant information in SHR images rather than artifacts, we introduce the identity consistency loss:𝐿 𝑐𝑐 = 𝐿 𝑐𝑐 (𝑓EQUATION)where || • || 2 represents ℓ 2 -norm, 𝑃𝑟𝑒𝑑 (•) represents the processing of the predictor, and Stopgrad( • ) denotes the stop-gradient operation. The LR encoder cannot receive gradients from 𝐿 𝑖𝑐 , which has been proven to effectively prevent model collapse [2] .While CPG can cluster high-confidence samples into existing clusters based on the fusion of class vectors, it inevitably reduces the utilization of ambiguous samples. To fully leverage ambiguous samples for model optimization, we have designed a probability sharpening loss. The distinction between high-confidence samples and ambiguous samples lies in the fact that the former exhibits extremely high probability values in only one class, while the latter typically shows relatively high probability values in several classes. However, they share a common characteristic of displaying low probability values in most classes. We refer to these classes with low probability values as reliable negative classes. In other words, while we cannot determine the unique positive class for ambiguous samples, we can confidently identify the majority of negative classes with high confidence. Therefore, the probability sharpening loss aims to reduce the predicted probabilities of ambiguous samples in the negative classes: (19) where 𝜏 𝑠 and 𝜏 𝑐 are temperature hyperparameters, with 𝜏 𝑠 < 𝜏 𝑐 , and 𝐶 ℎ𝑠𝑙 is the matrix composed of centroids 𝑐 ℎ𝑠𝑙 𝑗 . In summary, the overall objective function of DRFM is defined as:𝐿 𝑝𝑠 = -Stopgrad(Softmax(𝑓 𝑙𝑠ℎ 𝑖 • 𝐶 𝑇 ℎ𝑠𝑙 /𝜏 𝑠 )) • log(Softmax(𝑓 𝑙𝑠ℎ 𝑖 • 𝐶 𝑇 ℎ𝑠𝑙 /𝜏 𝑐 )),EQUATION)where 𝜆 is the weight coefficient of 𝐿 𝑝𝑠 . The aforementioned optimization methods promote identity-relevant and resolution-invariant feature learning, forming a mutually reinforcing solution with CPG. 
experiment 4.1 datasets and evaluation metrics: 
 We evaluate existing methods and the proposed method on three multiple low-resolution (MLR) datasets. The details of each dataset are described as follows.CAVIAR [5] is a real CR-ReID dataset, consisting of 1,220 images from 72 identities captured by two cameras. The resolution of the images captured by one camera is much lower than that of the images captured by the other camera. Following existing works [6, 29] , we only use images from 50 identities. We split the dataset in half, utilizing images from 25 identities for training and images from the remaining 25 identities for testing.MLR-CUHK03 is a synthetic CR-ReID dataset based on CUHK03 [28] , containing images from 1,467 identities. Following existing works [6, 29] , we use the benchmarking 1,367/100 training/test identity split. Both manually cropped and automatically detected images are used in our evaluations. To simulate resolution changes, we down-sample the resolution of images captured by one camera based on a random down-sampling rate 𝑟 ∈ {2, 3, 4}, while the resolution of images captured by the other camera remains unchanged.MLR-Market-1501 is a synthetic CR-ReID dataset based on Market-1501 [51] . It contains 32,668 images from 1,501 identities captured by six cameras, with 751 identities for training and 750 for testing. Following existing works [6, 29] , we process the Market-1501 dataset in a similar manner to MLR-CUHK03 to obtain MLR-Market-1501.During the testing phase, we construct the query set with all LR images per person, and the gallery set with one randomly selected HR image per person. We use cumulative matching characteristic (CMC) to evaluate model performance and report Rank-1, Rank-5, and Rank-10.In the data preprocessing stage, we employ Swin2SR [8] to perform super-resolution (SR) on LR images to obtain SHR images. Additionally, we perform down-sampling (DS) on HR images to acquire SLR images. The scaling factors for both SR and DS are set to 4. Both random horizontal flipping and random cropping are adopted for data augmentation. We employ DBSCAN [11] for clustering the samples, and the minimum number of samples is set to 4. In DRFM, both encoders 𝐸 ℎ and 𝐸 𝑙 utilize ResNet-50 [17] , pre-trained on ImageNet [10] . The 𝑃𝑟𝑒𝑑, 𝐹 1 and 𝐹 2 all use a 2048×2048 full connection layer, and 𝐹 3 use a 4096×2048 full connection layer. For the temperature hyperparameters, we set 𝜏 𝑐 =0.04, 𝜏 𝑙 =0.04, 𝜏 ℎ =0.05, and 𝜏 𝑠 =0.02. We optimize the DRFM through Adam optimizer [24] with a weight decay of 0.0005 and train the network with 40 epochs in total. The update rate 𝛼 is set to 0.2 and the learning rate is set to 0.00035. For 𝐿 𝑐𝑐 and 𝐿 𝑖𝑐 , we set the batch size to 64. For 𝐿 𝑝𝑠 , the batch size is set to 16. For 𝐿 𝑐𝑐 , we set |𝑄 | = 20. In the initial training phase, only 𝐿 𝑐𝑐 and 𝐿 𝑖𝑐 are introduced. After the 10th epoch, 𝐿 𝑝𝑠 starts being used for model optimization.In this section, we conduct a detailed analysis of the key hyperparameters of DRFM on MLR-CUHK03 and MLR-Market-1501, including 𝜆 and 𝑡.3.1 𝜆 of 𝐿 𝑝𝑠 . Figure 4 shows plots of the performance on the two datasets as function of the hyperparameter 𝜆. Note that 𝜆 = 0.0 corresponds to the situation where the probability sharpening loss has no contribution to the overall training loss. We find that on MLR-CUHK03 and MLR-Market-1501, the model obtains the best performance when 𝜆 = 1.5. The results verify the generalization of the hyperparameter. The worst performance is achieved when 𝜆 = 0.0, which preliminarily verifies the effectiveness of 𝐿 𝑝𝑠 .4.3.2 𝑡 of CPG. In Figure 5 , we explore the optimal value of 𝑡 for CPG. On MLR-CUHK03, the model achieves the best performance when 𝑡 = 0.7. On MLR-Market-1501, the model achieves the best performance when 𝑡 = 0.8. When 𝑡 is too large or too small, the performance is poor. This is because when 𝑡 takes a small value, only a small number of samples are used for training, reducing the contribution of 𝐿 𝑐𝑐 to optimization. On the other hand, when 𝑡 takes a large value, too many noisy samples are used for training, misleading the optimization direction of the model.In Table 2 , we compare DRFM with the state-of-the-art methods on CAVIAR, MLR-CUHK03 and MLR-Market-1501.As there are no existing unsupervised CR-ReID methods, we reproduce six unsupervised general ReID methods on the mentioned datasets. These methods include SpCL [12] , ICE [1] , CC [9] , PPLR [7] , Purification [25] , and DCCC [19] . As shown in Table 2 , Purification [25] achieves superior overall performance among the existing methods. Notably, DRFM significantly surpasses all these unsupervised methods. Specifically, compared to Purification [25] , DRFM achieves improvements of 1.2%, 8.5%, and 6.8% in the Rank-1 accuracy on CAVIAR, MLR-CUHK03, and MLR-Market-1501, respectively. The acquisition of this advantage can be attributed to three main factors: (1) DRFM employs a pseudo-label generation method that is better suited for cross-resolution scenarios; (2) DRFM attains a potentially resolution-invariant feature representation through feature fusion; (3) the introduction of multiple optimization methods further enhances the advantages of DRFM compared to existing approaches.Additionally, we compare DRFM with existing supervised CR-ReID methods, including SING [22] , CSR-GAN [40] , CAD-Net [29] , INTACT [6] , PRI [16] , PS-HRNet [48] , JBIM [52] , and LRAR [41] . Encouragingly, despite exhibiting some performance gaps compared to advanced supervised methods (such as JBIM [52] and LRAR [41] ), DRFM demonstrates competitive performance when compared to the early methods (such as SING [22] and CSR-GAN [40] ) on MLR-Market-1501. This not only validates the superiority of DRFM but also underscores the research potential of unsupervised CR-ReID.In this section, we conduct a series of ablation experiments to empirically evaluate the effectiveness of the key components of DRFM.In Table 3 , A1 directly employs the DBSCAN algorithm [11] to generate pseudo-labels between LR and HR images and optimizes using softmax classification loss 𝐿 𝑠𝑐 [30] . A2 first applies superresolution (SR) to LR images and then utilizes DBSCAN to generate pseudo-labels. A3 utilizes CPG to generate pseudo-labels. A4 introduces 𝐿 𝑖𝑐 based on A3. Following existing feature fusion methods [52] , A5 incorporates a fully connected (FC) layer to fuse features from HR and LR spaces. A6 employs the CRFF to generate fusion features. A7 and A8 replace 𝐿 𝑠𝑐 in A2 and A6 with 𝐿 𝑐𝑐 , respectively. DRFM introduces 𝐿 𝑝𝑠 based on A8. 4.5.1 Effectiveness of CPG and 𝐿 𝑖𝑐 . In Table 3 , A2 exhibits a slight improvement compared to A1 on all three datasets, indicating the beneficial effect of SR on unsupervised methods. A3 achieves higher performance on all three datasets compared to A1 and A2 by introducing CPG. Specifically, compared to A2, A3 shows improvements of 1.1%, 1.4%, and 2.2% in Rank-1 accuracy on CAVIAR, MLR-CUHK03, and MLR-Market1501, respectively. Furthermore, A4 achieves higher performance by introducing 𝐿 𝑖𝑐 into A3.To further investigate the effectiveness of CPG and 𝐿 𝑖𝑐 in enhancing the reliability of pseudo-labels, we conduct statistical analysis on the F-score [14] of pseudo-labels generated by A1, A2, A3, and A4 during the training phase. A higher F-score indicates higher accuracy of pseudo-labels. As shown in Figure 6 , on MLR-CUHK03 and MLR-Market-1501, A3 consistently exhibits higher F-score throughout the training compared to A2 and A1. This validates that CPG can generate reliable pseudo-labels in cross-resolution scenarios by integrating semantic information at different resolution levels. Moreover, A4 demonstrates higher F-score in the later stages of training compared to A3. This is because 𝐿 𝑖𝑐 focuses on enhancing the model's attention to identity information in SHR images, thereby encouraging potentially high-confidence samples to attain higher max(𝑠 𝑙𝑠ℎ 𝑖 ). 4.5.2 Effectiveness of CRFF. As shown in Table 3 , A5, which introduces FC on top of A4, demonstrates improved overall performance on all three datasets. This confirms that fusion features are more suitable for cross-resolution scenarios compared to single features. A6, which incorporates CRFF, achieves higher performance than A5. Specifically, compared to A5, A6 shows improvements of 0.5%, 1.0%, and 1.1% in Rank-1 accuracy on CAVIAR, MLR-CUHK03, and MLR-Market1501, respectively. This validates that CRFF can generate fusion features that are more beneficial for performance improvement compared to FC.To further explore the contribution of CRFF to resolution-invariant feature learning, we visualize the feature distance distributions of same-resolution positive pairs and cross-resolution positive pairs on MLR-CUHK03. As shown in Figure 7 , compared to A5, A6 further reduces the difference between the two distributions. This indicates that CRFF is more suitable than FC for generating resolutioninvariant features. 4.5.3 Effectiveness of 𝐿 𝑐𝑐 and 𝐿 𝑝𝑠 . Benefiting from the cross-resolution contrastive loss 𝐿 𝑐𝑐 , we observe that A7 significantly outperforms A2, and A8 outperforms A6, validating that 𝐿 𝑐𝑐 can further enhance the model's performance in cross-resolution scenarios compared to 𝐿 𝑠𝑐 . We find that DRFM, when introducing the probability sharpening loss 𝐿 𝑝𝑠 on top of A8, further improves model performance, demonstrating the feasibility of leveraging ambiguous samples for optimization. Furthermore, DRFM outperforms all the relevant methods mentioned above, confirming the effectiveness of the combination of all components in DRFM. 
conclusion: 
 In this paper, we focus on a novel problem: unsupervised crossresolution person re-identification, and propose the dual-resolution fusion modeling (DRFM) framework to address this problem. In DRFM, we introduce cross-resolution pseudo-label generation (CPG), cross-resolution feature fusion (CRFF), and multiple optimization methods. We conduct an ablation study on these components, validating that CPG can generate reliable pseudo-labels in crossresolution scenarios, while CRFF and multiple optimization methods effectively facilitate resolution-invariant feature learning, thereby enhancing model performance. Extensive experiments conducted on three datasets demonstrate the effectiveness and superiority of DRFM, which not only outperforms existing unsupervised methods but also exhibits promising performance competitiveness with certain supervised methods. 
: 
 ACM Reference Format: Zhiqi Pang, Lingling Zhao, and Chunyu Wang. 2024. Dual-Resolution Fusion Modeling for Unsupervised Cross-Resolution Person Re-Identification. In Proceedings of the 32nd ACM International Conference on Multimedia (MM '24) , October 28-November 1, 2024, Melbourne, VIC, Australia. ACM, New York, NY, USA, 10 pages. https://doi.org/10.1145/3664647.3681067 
 
 Appendix:  


Decision: The paper initially received the following ratings: BA(hdQF),	BR(rwGc), WA(dpDb), A(mSG5). After rebuttal and discussion, the final ratings were: WA(hdQF), BA(rwGc), A(dpDb), A(mSG5).

Below are the details of the rebuttal:
1. After reading the rebuttal, all the Reviewer rwGc changed the original borderline rejection recommendation to a borderline acceptance recommendation.
2. All the reviewers were satisfied with the response and gave an acceptance recommendation for this paper.

The AC carefully reviewed the rebuttal:
1. As for the contribution of task setting, the AC agrees that this work makes efforts to address a new problem (unsupervised CR-ReID) and proposes a novel solution, which can serve as a good starting point for exploring effective unsupervised CR-ReID.
2. As for the contribution of technique, a) the proposed CPG effectively generates reliable pseudo-labels by integrating class vectors from both LR and SHR images, which is somewhat new and effective. b) the proposed probability sharpening loss effectively mitigates the predicted probabilities of ambiguous samples in the negative classes, which is well-motivated.
3. This work also provides meaningful visualization and valuable insights based on the experimental results.

In sum, after the rebuttal, the AC believes the concerns raised by the reviewers were well-addressed and recommends acceptance of this paper. In the final version, the AC strongly encourages the authors to include all discussions and reference literature about LReID from the rebuttal and improve the presentation to enhance readability for a diverse audience.

Reviews: review: paper_summary: paper_summary: This paper focuses on a new task, i.e., unsupervised cross-resolution person reid. A dual-resolution fusion framework is proposed in this paper. Specificially, a cross-resolution pseudo label generation method is proposed to assign pseudo labels for images in different resolution. A cross-resolution feature fusion method is proposed to fuse features in LR and HR. Outliers are also used in training with the proposed probability sharpening loss.
strength_weakness: strength_weakness: + This paper is an early effort for the unsupervised cross-resolution person reid task.
+ Experimental results show that this paper chieves promising performance compared with outhers.
+ Generating pseudo labels on HR images first and assign pseudo labels to LR images is reasonable and also works.
limitations: limitations: -The proposed cross-resolution feature fusion is similar with graph attention or self-attentinon networks. More discussion and explaination are expected on it.
- It's not appropriate to claim that "achieves competitive performance with supervised methods" as Table2 shows that most supervsied methods outperform the proposed method by a large margin.
- In testing phase, why are LR images used as query images only? How are features extracted for testing with $E_h$ and $E_l$?
- Eq. 8 might be incorrect as L1 norm of a vector is a number.
- The description "all resolution centroids in its cluster" is not clear. Why does a cluster have more than two centroids, i.e., one for HR and one for LR?
- More explaination on "Resolution invariant feature learning" is expected. What's it motivation and why does it works?
paper_summary: paper_summary: This paper proposes a dual-resolution fusion modeling (DRFM) for cross resolution person reid task. The proposed method consists of a cross-resolution pseudo-label generation (CPG) method to cluster high and low resolution images to generate reliable pseudo labels. And a cross-resolution feature fusion module is designed to fuse features from both resolution. A cross-resolution contrastive loss is proposed for training. Experimental result on 3 datasets show reasonable performance.
strength_weakness: strength_weakness: This paper focus on cross-resolution unsupervised person reid, which gets less attention in research field. 
The proposed method consists of cross-resolution pseudo-label generation to jointly consider high and low resolution images to get more reliable labels.
A cross-resolution contrastive loss and probability sharpening loss are proposed for training.
Experimental result on 3 datasets show reasonable performance.
limitations: limitations: Is the comparison methods in table 2 also trained with the multi-resolution data, e.g., original data, super-resolutioned data and downsample datas.
The author should add some explaination about why the ic loss in Eq.18 could encourage the HR encoder to focus on identity-relevant in formation in SHR images rather than artifacts.
There are too many abbreviation which is not necessary, increase the reading difficulty.
paper_summary: paper_summary: This paper focuses on unsupervised cross-resolution person re-identification, which faces the challenges of obtaining reliable identity pseudo-labels and resolution-invariant features. To address these challenges, the paper proposes the dual-resolution fusion modeling (DRFM) framework, which includes cross-resolution pseudo-label generation (CPG), cross-resolution feature fusion (CRFF), cross-resolution contrastive loss, and probability sharpening loss.
strength_weakness: strength_weakness: 1.This paper is the first attempt in the field of unsupervised cross-resolution person re-identification.
2.The CPG effectively generates reliable pseudo-labels by integrating class vectors from both LR and SHR images. 
3.The probability sharpening loss effectively mitigates the predicted probabilities of ambiguous samples in the negative classes.
4.The Experiments are adequate. Contrasting experiments and ablation experiments demonstrate the validity of the authors' method.
5.Figure 6 demonstrates that cross-resolution pseudo-label generation can obtain reliable identity pseudo-labels.
limitations: limitations: 1.This paper claim that their method achieves competitive performance with supervised methods, but in reality, there are still significant gap compared to fully supervised methods.
2.The introduction of too many hyperparameters may reduce the reproducibility of the paper.
3.The colors of the lines in Figure 6 are too similar, making them difficult to distinguish. The differences in feature distance distributions in Figure 7 are not significant.
paper_summary: paper_summary: This paper proposes a dual-resolution fusion modeling (DRFM) framework to address the unsupervised cross-resolution person re-identification problem. DRFM comprises cross-resolution pseudo-label generation, cross-resolution feature fusion, and multiple optimization methods. To validate the effectiveness of the proposed method, the authors conducted extensive experiments on three datasets.
strength_weakness: strength_weakness: 1. The manuscript makes efforts to address a new problem (unsupervised CR-ReID) and proposes a novel solution. Overall, this work is sufficiently novel and contributes to the ReID field.
2. The description and logic of the manuscript are clear, and the symbol table used effectively explains the key notations in the text.
3. The proposed method is theoretically reliable, and cross-resolution feature fusion is a suitable technique.
4. The manuscript achieves superior performance on multiple datasets, and ablation study confirms the effectiveness of the method.
limitations: limitations: 1. The authors introduce multiple memories to store features; the purpose of doing so should be explained.
2. Although the proposed method achieves some advantages over unsupervised methods on the CAVIAR dataset, it exhibits significant gaps compared to supervised methods. Explaining the reasons for this phenomenon would be helpful for subsequent researchers to further their work.