Incremental False Negative Detection for Contrastive Learning

1 INTRODUCTION . Self-supervised learning of visual representation ( Doersch et al. , 2015 ; Pathak et al. , 2016 ; Noroozi & Favaro , 2016 ; Oord et al. , 2018 ; Gidaris et al. , 2018 ; Chen et al. , 2020a ) aims to learn a semanticaware embedding space based on the image data without the supervision of human-labeled annotations . Recently , significant advances have been made by contrastive learning approaches ( Hjelm et al. , 2019 ; Tian et al. , 2019 ; Chen et al. , 2020b ; c ; He et al. , 2020 ; Chen et al. , 2020d ) to reduce the performance gap with the supervised counterparts . Most unsupervised contrastive learning methods are developed based on the instance discrimination assumption . These approaches treat each training image as an individual instance and learn the representations that discriminate every single sample . Specifically , considering an arbitrary image as an anchor , the only positive sample is generated by applying a different data augmentation to the anchor image , while all other images are treated as negative samples . The training objective is to attract the positive to the anchor while repelling the negative samples from the anchor . While instance-level contrastive learning has shown impressive performance , these methods do not take the semantic relationship among images into consideration . Although some images share similar semantic concepts with the anchor , they are still considered as negative samples and are equally pushed away from the anchor with all other negatives . We define these samples as “ false negatives ” in self-supervised contrastive learning , which would adversely affect the representation learning ( Saunshi et al. , 2019 ) . One might argue that such undesirable effects might be minor for the datasets with diverse semantic concepts since the probabilities of drawing false negatives are relatively low . However , as shown in Figure 1 , we empirically find the opposite results that the performance drops due to training with false negative samples are more significant on large-scale datasets with more semantic categories . These results bring up the issue of instance-level contrastive learning when it is applied to datasets with more complex semantic contents . Figure 1 : Effect of false negative for contrastive learning . To study the effect of false negatives , we compare two frameworks : SimCLR ( Chen et al . ( 2020b ) , blue ) representing instance-level contrastive learning which is trained with false negatives , and SupCon ( Khosla et al . ( 2020 ) , red ) benefiting from human-labeled annotations to exclude false negatives . We observe that training with false negatives leads to larger performance gaps ( orange ) on the datasets with more semantic categories . The proposed approach ( green ) effectively alleviates such adverse effects by explicitly removing the detected false negatives for self-supervised contrastive learning . See Section 4.3 for experimental details . To handle the false negative issue , we propose a novel framework to incrementally detect and explicitly remove false negatives for self-supervised contrastive learning . During training , we cluster samples in the embedding space and assign the cluster indices as the pseudo labels to the training images . Instances with an identical label to the anchor are then detected as false negatives . However , we notice that the pseudo labels generated in the earlier training stages are comparably unreliable to represent the semantic relationship among instances since the semantic structure of the embedding space is still under development . Therefore , we propose a novel strategy to assign the pseudo labels in an incremental manner . In the early training stage , we only use a small portion of pseudo labels with higher confidence while treating other samples as individual instances . In the later training stage , as the pseudo labels become more reliable , we dynamically include more confident labels to detect and remove false negatives and hence , can benefit semantic-aware representation learning . Furthermore , we discuss two types of training objectives to explicitly address the detected false negatives during contrastive learning : elimination and attraction . We theoretically show that they are the generalization of triplet loss ( Schroff et al. , 2015 ) . While both objectives avoid discriminating the anchor and its false negatives , the attraction objective further minimizes the distances between them . Empirically , we find that such a more aggressive strategy is less tolerant to noisy pseudo labels and unsuitable to our self-supervised framework . The main contributions of this paper are summarized as follows : • We highlight the issue of false negatives in contrastive learning , especially on the largescale datasets with more diverse semantic contents . • We propose a novel contrastive learning framework with incremental false negative detection ( IFND ) that smoothly bridges instance-level and semantic-aware contrastive learning . Our approach explicitly detects and incrementally removes false negatives as the embedding space becomes more semantically structural through the training process . • Our approach performs favorably against the existing self-supervised contrastive learning frameworks on multiple benchmarks under a limited training resource setup . Besides , we introduce two metrics : mean true positive/negative rates to evaluate the clustering quality . 2 RELATED WORK . Instance-level contrastive learning . Learning semantic-aware visual representation without human supervision is an essential but challenging task . Numerous approaches are developed based on the instance discrimination task ( Dosovitskiy et al. , 2015 ; Wu et al. , 2018 ; Ye et al. , 2019 ; Misra & Maaten , 2020 ) , where each instance in the dataset is treated as an individual sample and a representation is learned to classify or contrastively separate samples . Recently , with the contrastive loss ( Gutmann & Hyvärinen , 2010 ) and a set of image transformations or augmentations ( Chen et al. , 2020b ) , the instance-level contrastive learning methods ( Hjelm et al. , 2019 ; Tian et al. , 2019 ; Chen et al. , 2020b ; c ; He et al. , 2020 ; Chen et al. , 2020d ) achieve state-of-the-art results and even outperform some supervised pre-training approaches . While significant improvement is achieved , instance-level contrastive learning frameworks typically do not take high-level semantics into ac- count . For example , any two samples in the dataset would be considered as a negative pair and be pushed apart by the contrastive loss although some of them are semantically similar and should be nearby in the embedding space . Especially when the huge amounts of negative samples ( 8192 for Chen et al . ( 2020b ) , 65536 for He et al . ( 2020 ) ) play a critical role in the success of contrastive learning , these instance-level approaches inevitably draw some “ false negatives ” and learn less effective representation models as shown by a recent study ( Saunshi et al. , 2019 ) . In this paper , we further show that the performance gap between the contrastive learning trained with and without false negatives is more significant on the datasets with more semantic classes . Semantic-aware contrastive learning . To address the limitation of instance-level contrastive learning , several methods are developed to exploit the semantic relationship among images . Chuang et al . ( 2020 ) show that the original instance-level sampling strategy includes a sampling bias and propose a debiased contrastive loss . However , this method does not explicitly handle false negative samples . Huynh et al . ( 2020 ) determine false negatives of an anchor by finding top k similar samples without referring to global embedding distribution . Khosla et al . ( 2020 ) propose a supervised contrastive loss that utilizes label information and treats images of the same class ( i.e. , false negatives ) as positives . However , manually annotated labels can not be acquired in the unsupervised setting . Closely related to our method , PCL ( Li et al. , 2021 ) discovers the representative prototypes for each semantic concept and encourages an image embedding to be closer to its belonging prototype . There are two fundamental differences between PCL and the proposed method : a ) PCL uses all assigned prototypes regardless of inferior clustering results in early stages , which can not properly reflect the semantic relationship among images , b ) without treating semantically similar samples as false negatives , PCL applies instance-level InfoNCE loss ( Oord et al. , 2018 ) as the primary supervision , which fails to avoid the images within the same prototype being pushed apart . In contrast , leveraging the progressive detection and explicit removal of false negatives , our framework achieves greater clustering quality and representation learning . More comparisons are in Section 4.2 and Appendix E. Clustering for deep unsupervised learning . A few unsupervised learning methods have recently benefited from clustering techniques to construct more effective representation models ( Caron et al. , 2018 ; Yang et al. , 2016 ; Xie et al. , 2016 ; Caron et al. , 2019 ; Zhuang et al. , 2019 ; Asano et al. , 2020 ) . The main idea is to use cluster indices as pseudo labels and learn visual representations in a supervised manner . However , most existing methods require all pseudo labels to optimize the supervised loss , e.g. , cross-entropy . In this work , we show that the pseudo labels obtained in earlier training processes are relatively unreliable , and the full adoption of pseudo labels in the early stage would further disrupt the representation learning . To handle this issue , we incorporate instance-level learning with clustering which allows us to only use a set of pseudo labels with sufficient confidence while treating the others as individual instances . 3 METHODOLOGY . 3.1 INSTANCE-LEVEL CONTRASTIVE LEARNING . An instance-level contrastive learning method learns a representation that discriminates one sample from every other . Given M randomly sampled images from a training set X , a contrastive training mini-batch I consists of 2M images obtained by applying two sets of data augmentation on each sampled image . For any anchor image i ∈ I , the only positive sample is another view ( or transformation ) of the same image , denoted as i′ , while the other 2 ( M − 1 ) images jointly constitute the set of negative samples N ( i ) . The instance-level discrimination is then optimized by the following contrastive loss : Linst = ∑ i∈I − log sim ( zi , zi ′ ) ∑ s∈S ( i ) sim ( zi , zs ) , S ( i ) ≡ { i′ , n | n ∈ N ( i ) } , ( 1 ) where zi = g ( f ( i ) ) is the embedding of the image i from an encoder f and a projection head g , sim ( u , v ) = exp ( 1τ ( u·v ) ‖u‖‖v‖ ) is the similarity of two input vectors , and τ represents a temperature hyper-parameter . For instance-level contrastive learning , the negative sample set N ( i ) inevitably includes some samples with similar semantic content as the anchor i , which we term as false negatives , especially when a larger M is commonly used . Consequently , it would separate semantically similar image pairs , which is suboptimal for learning good semantic-aware visual representations . 3.2 INCREMENTAL FALSE NEGATIVE DETECTION . To learn an effective semantic-aware representation by self-supervised contrastive learning , our method explicitly detects and removes the false negative samples . Specifically , we follow the same procedure in PCL ( Li et al. , 2021 ) to perform k-means clustering on the features f ( i ) of all training images { i ∈ X } and cluster the features into k groups . We use the centroids to represent the embeddings of the discovered semantic concepts and denote the centroid representation of k-th cluster as ck . The pseudo label yi of the image i is assigned based on the closest centroid , formally yi = arg mink ‖f ( i ) − ck‖ . In the learning process , two images with an identical assigned pseudo label would be treated as a false negative pair . Nonetheless , as illustrated in the leftmost of Figure 2 ( a ) ( or Figure 3 for a realistic case ) , we observe that the semantic structure of early embedding space is less reliable and could generate noisy pseudo labels , which would disrupt the representation learning . We provide a quantitative analysis of this effect in Section 4.4 and Appendix F. To tackle this problem , in early training stages , we only adopt the pseudo labels with higher confidence while treating the other images as individual samples . For an arbitrary image i , the desired confidence κi should be larger when the image embedding f ( i ) is not only closer to the centroid of the belonging group , i.e. , cyi , but also farther to the other centroids . To this end , we measure the confidence of an assigned pseudo label by : κi = sim ( zi , cyi ) ∑k j=1 sim ( zi , cj ) , ( 2 ) which is the softmax of cosine similarities between the image representation and every centroid . We then only assign a specific rate ( termed as acceptance rate ) of pseudo labels with larger confidence . As plotted in the rest part of Figure 2 ( or Figure 3 ) , following the training process , the encoder is better trained and the semantic structure of embedding space is more clear which can generate more reliable pseudo labels . Hence , we propose to incrementally increase the acceptance rate and leverage semantic-aware representation learning . In Section 4.4 , we demonstrate that a simple strategy that linearly increases the acceptance rate from 0 % to 100 % throughout the whole training process can achieve a significant improvement . Hierarchical semantic definition . In an unsupervised setting , the precise class number of a dataset is usually unknown , and also , a different number of classes can be selected by choosing a different level of granularity . To enhance the flexibility and robustness of our method , we incorporate hierarchical clustering techniques ( Johnson , 1967 ; Corpet , 1988 ; Murtagh & Legendre , 2014 ; Li et al. , 2021 ) into our framework . Specifically , we perform multiple times of k-means with different cluster numbers to define the false negatives in the different levels of granularity . As such , the objective function ( will be introduced in Section 3.3 ) is revised as the average of the original ones computed by the false negatives defined in different levels of granularity .

Summary: The paper proposes incremental false-negative detection (IFND), which has several improvements over prototypical contrastive learning (PCL). The key idea of PCL is to alternatively update (a) the pseudo-labels based on the current encoder and (b) update the encoder based on the current pseudo-labels. However, PCL often converges to the local minima since the bad encoder produces falsy pseudo-labels, further harming the encoder. To tackle the issue, INFD suggests including the confident pseudo-labels while uncertain ones incrementally. IFND consistently improves the PCL in all considered scenarios.