The superiority of deeply learned pedestrian representations has been reported in very recent literature of person re-identification (re-ID). In this article, we consider the more pragmatic issue of learning a deep feature with no or only a few labels. We propose a progressive unsupervised learning (PUL) method to transfer pretrained deep representations to unseen domains. Our method is easy to implement and can be viewed as an effective baseline for unsupervised re-ID feature learning. Specifically, PUL iterates between (1) pedestrian clustering and (2) fine-tuning of the convolutional neural network (CNN) to improve the initialization model trained on the irrelevant labeled dataset. Since the clustering results can be very noisy, we add a selection operation between the clustering and fine-tuning. At the beginning, when the model is weak, CNN is fine-tuned on a small amount of reliable examples that locate near to cluster centroids in the feature space. As the model becomes stronger, in subsequent iterations, more images are being adaptively selected as CNN training samples. Progressively, pedestrian clustering and the CNN model are improved simultaneously until algorithm convergence. This process is naturally formulated as self-paced learning. We then point out promising directions that may lead to further improvement. Extensive experiments on three large-scale re-ID datasets demonstrate that PUL outputs discriminative features that improve the re-ID accuracy. Our code has been released at https://github.com/hehefan/Unsupervised-Person-Re-identification-Clustering-and-Fine-tuning. This article discusses person re-identification (re-ID), a task that provides critical insights whether a person re-appears in another camera after being spotted in one [62] . Recent works view it as a search problem aiming to retrieving the relevant images to the top ranks [60, 64] . Large-scale learning methods based on the convolutional neural network (CNN) constitute the recent advances in this field. Broadly speaking, current deep learning methods can be categorized into two classes, i.e., similarity learning and representation learning. For the former, the training input can be image pairs [39] , triplets [27], or quadruplets [8]. These methods directly output the similarity score of two input images by focusing on their distinct body patches, without an explicit feature extraction process. This line of methods is less efficient during testing. However, when learning feature representations, contrastive loss or softmax loss are most commonly employed. For these methods, there is an explicit feature extraction process for the query and gallery images, which is learned to discriminate among a pool of identities or among image pairs/triplets [9, 18] . Then the re-ID procedure is rather like a retrieval problem under some commonly used similarity measurement such as Euclidean distance. Due to its efficiency in large-scale galleries and the potentials in further acceleration, the deep feature learning strategy has become more popular [18, 44, 62, 64] . Therefore, we adopt the classification model proposed in References [44, 62] for feature learning. Despite the impressive progress achieved by the deep learning methods [18, 25, 64] , most of these methods focus on re-ID with sufficient training data in the same environment. In fact, the lack of labeled training data under a new environment has been addressed by many previous works by resorting to unsupervised or semi-supervised learning [10, 20, [31] [32] [33] [46] [47] [48] [49] [50] . These works typically deal with the relatively small datasets [41] , and to our knowledge, none of these works are integrated into the deep learning framework that has been proven as the state of the art under the large-scale datasets [15, 18, 38, 60] . In fact, the problem of high annotation cost is more severe for deep learning based methods due to the intensive demand of data. So, under these circumstances, we make initial attempts by developing a easy-to-implement method for unsupervised deep representation learning in the re-ID community. Our method can be easily extended to the semi-supervised setting by annotating a small amount of bounding boxes. We also notice some works on human-in-the-loop learning [26, 40] , introducing labels by human feedback. We envision our work is complementary to this line of methods. In this article, we aim at finding "better" feature representations without labeled training data to adapt to the unlabeled dataset. Named "progressive unsupervised learning" (PUL), our method is an iterative process and each iteration consists of two components (shown in Figure 1 ). (1) We deploy the original model pre-trained on some external labeled data to extract features of images from the unlabeled training set. Standard k-means clustering is performed on the CNN features, followed by a selection operation to choose reliable samples. (2) The original model is fine-tuned using the selected training samples. We then use this newborn model to extract features and begin another iteration of training. At the beginning, when the model is weak, the proposed approach learns from a small amount of reliable examples, which are near to cluster centroids in the feature space, to avoid getting stuck at a bad optimum or oscillating. As the model becomes stronger in subsequent iterations, more data instances are adaptively selected and exploited as training examples. By this ACM Trans. Multimedia Comput.
For example, Fan et al. REF propose a progressive unsupervised learning method iterating between K-means clustering and CNN fine-tuning.
Unsupervised Person Re-identification: Clustering and Fine-tuning
TOMM
Computer Science