Contrastive Learning with Hard Negative Samples

1 INTRODUCTION . Owing to their empirical success , contrastive learning methods ( Chopra et al. , 2005 ; Hadsell et al. , 2006 ) have become one of the most popular self-supervised approaches for learning representations ( Oord et al. , 2018 ; Tian et al. , 2019 ; Chen et al. , 2020a ) . In computer vision , unsupervised contrastive learning methods have even outperformed supervised pre-training for object detection and segmentation tasks ( Misra & Maaten , 2020 ; He et al. , 2020 ) . Contrastive learning relies on two key ingredients : notions of similar ( positive ) ( x , x+ ) and dissimilar ( negative ) ( x , x− ) pairs of data points . The training objective , typically noise-contrastive estimation ( Gutmann & Hyvärinen , 2010 ) , guides the learned representation f to map positive pairs to nearby locations , and negative pairs farther apart ; other objectives have also been considered ( Chen et al. , 2020a ) . The success of the associated methods depends on the design of informative of the positive and negative pairs , which can not exploit true similarity information since there is no supervision . Much research effort has addressed sampling strategies for positive pairs , and has been a key driver of recent progress in multi-view and contrastive learning ( Blum & Mitchell , 1998 ; Xu et al. , 2013 ; Bachman et al. , 2019 ; Chen et al. , 2020a ; Tian et al. , 2020 ) . For image data , positive sampling strategies often apply transformations that preserve semantic content , e.g. , jittering , random cropping , separating color channels , etc . ( Chen et al. , 2020a ; c ; Tian et al. , 2019 ) . Such transformations have also been effective in learning control policies from raw pixel data ( Srinivas et al. , 2020 ) . Positive sampling techniques have also been proposed for sentence , audio , and video data ( Logeswaran & Lee , 2018 ; Oord et al. , 2018 ; Purushwalkam & Gupta , 2020 ; Sermanet et al. , 2018 ) . Surprisingly , the choice of negative pairs has drawn much less attention in contrastive learning . Often , given an “ anchor ” point x , a “ negative ” x− is simply sampled uniformly from the training data , independent of how informative it may be for the learned representation . In supervised and metric learning settings , “ hard ” ( true negative ) examples can help guide a learning method to correct its mistakes more quickly ( Schroff et al. , 2015 ; Song et al. , 2016 ) . For representation learning , informative negative examples are intuitively those pairs that are mapped nearby but should be far apart . This idea is successfully applied in metric learning , where true pairs of dissimilar points are available , as opposed to unsupervised contrastive learning . Code available at : https : //github.com/joshr17/HCL With this motivation , we address the challenge of selecting informative negatives for contrastive representation learning . In response , we propose a solution that builds a tunable sampling distribution that prefers negative pairs whose representations are currently very similar . This solution faces two challenges : ( 1 ) we do not have access to any true similarity or dissimilarity information ; ( 2 ) we need an efficient sampling strategy for this tunable distribution . We overcome ( 1 ) by building on ideas from positive-unlabeled learning ( Elkan & Noto , 2008 ; Du Plessis et al. , 2014 ) , and ( 2 ) by designing an efficient , easy to implement importance sampling technique that incurs no computational overhead . Our theoretical analysis shows that , as a function of the tuning parameter , the optimal representations for our new method place similar inputs in tight clusters , whilst spacing the clusters as far apart as possible . Empirically , our hard negative sampling strategy improves the downstream task performance for image , graph and text data , supporting that indeed , our negative examples are more informative . Contributions . In summary , we make the following contributions : 1 . We propose a simple distribution over hard negative pairs for contrastive representation learning , and derive a practical importance sampling strategy with zero computational overhead that takes into account the lack of true dissimilarity information ; 2 . We theoretically analyze the hard negatives objective and optimal representations , showing that they capture desirable generalization properties ; 3 . We empirically observe that the proposed sampling method improves the downstream task performance on image , graph and text data . Before moving onto the problem formulation and our results , we summarize related work below . 1.1 RELATED WORK . Contrastive Representation Learning . Various frameworks for contrastive learning of visual representations have been proposed , including SimCLR ( Chen et al. , 2020a ; b ) , which uses augmented views of other items in a minibatch as negative samples , and MoCo ( He et al. , 2020 ; Chen et al. , 2020c ) , which uses a momentum updated memory bank of old negative representations to enable the use of very large batches of negative samples . Most contrastive methods are unsupervised , however there exist some that use label information ( Sylvain et al. , 2020 ; Khosla et al. , 2020 ) . Many works study the role of positive pairs , and , e.g. , propose to apply large perturbations for images Chen et al . ( 2020a ; c ) , or argue to minimize the mutual information within positive pairs , apart from relevant information for the ultimate prediction task ( Tian et al. , 2020 ) . Beyond visual data , contrastive methods have been developed for sentence embeddings ( Logeswaran & Lee , 2018 ) , sequential data ( Oord et al. , 2018 ; Hénaff et al. , 2020 ) , graph ( Sun et al. , 2020 ; Hassani & Khasahmadi , 2020 ; Li et al. , 2019 ) and node representation learning ( Velickovic et al. , 2019 ) , and learning representations from raw images for off-policy control ( Srinivas et al. , 2020 ) . The role of negative pairs hase been much less studied . Chuang et al . ( 2020 ) propose a method for “ debiasing ” , i.e. , correcting for the fact that not all negative pairs may be true negatives . It does so by taking the viewpoint of Positive-Unlabeled learning , and exploits a decomposition of the true negative distribution . Kalantidis et al . ( 2020 ) consider applying Mixup ( Zhang et al. , 2018 ) to generate hard negatives in latent space , and Jin et al . ( 2018 ) exploit the specific temporal structure of video to generate negatives for object detection . Negative Mining in Deep Metric Learning . As opposed to the contrastive representation learning literature , selection strategies for negative samples have been thoroughly studied in ( deep ) metric learning ( Schroff et al. , 2015 ; Song et al. , 2016 ; Harwood et al. , 2017 ; Wu et al. , 2017 ; Ge , 2018 ; Suh et al. , 2019 ) . Most of these works observe that it is helpful to use negative samples that are difficult for the current embedding to discriminate . Schroff et al . ( 2015 ) qualify this , observing that some examples are simply too hard , and propose selecting “ semi-hard ” negative samples . The well known importance of negative samples in metric learning , where ( partial ) true dissimilarity information is available , raises the question of negative samples in contrastive learning , the subject of this paper . 2 CONTRASTIVE LEARNING SETUP . We begin with the setup and the idea of contrastive representation learning . We wish to learn an embedding f : X → Sd−1/t that maps an observation x to a point on a hypersphere Sd−1/t in Rd of radius 1/t , where t is the “ temperature ” scaling hyperparameter . Following the setup of Arora et al . ( 2019 ) , we assume an underlying set of discrete latent classes C that represent semantic content , so that similar pairs ( x , x+ ) have the same latent class . Denoting the distribution over latent classes by ρ ( c ) for c ∈ C , we define the joint distribution px , c ( x , c ) = p ( x|c ) ρ ( c ) whose marginal p ( x ) we refer to simply as p , and assume supp ( p ) = X . For simplicity , we assume ρ ( c ) = τ+ is uniform , and let τ− = 1 − τ+ be the probability of another class . Since the class-prior τ+ is unknown in practice , it must either be treated as a hyperparameter , or estimated ( Christoffel et al. , 2016 ; Jain et al. , 2016 ) . Let h : X → C be the true underlying hypothesis that assigns class labels to inputs . We write x ∼ x′ to denote the label equivalence relation h ( x ) = h ( x′ ) . We denote by p+x ( x ′ ) = p ( x′|h ( x′ ) = h ( x ) ) , the distribution over points with same label as x , and by p−x ( x ′ ) = p ( x′|h ( x′ ) 6= h ( x ) ) , the distribution over points with labels different from x . We drop the subscript x when the context is clear . Following the usual convention , we overload ‘ ∼ ’ and also write x ∼ p to denote a point sampled from p. For each data point x ∼ p , the noise-contrastive estimation ( NCE ) objective ( Gutmann & Hyvärinen , 2010 ) for learning the representation f uses a positive example x+ with the same label as x , and negative examples { x−i } Ni=1 with ( supposedly ) different labels , h ( x − i ) 6= h ( x ) , sampled from q : Ex∼p , x+∼p+x { x−i } N i=1∼q [ − log e f ( x ) T f ( x+ ) ef ( x ) T f ( x+ ) + QN ∑N i=1 e f ( x ) T f ( x−i ) ] . ( 1 ) The weighting parameter Q is introduced for the purpose of analysis . When N is finite we take Q = N , yielding the usual form of the contrastive objective . The negative sample distribution q is frequently chosen to be the marginal distribution p , or , in practice , an empirical approximation of it ( Tian et al. , 2019 ; Chen et al. , 2020a ; c ; He et al. , 2020 ; Chen et al. , 2020c ; Oord et al. , 2018 ; Hénaff et al. , 2020 ) . In this paper we ask : is there a better way to choose q ? 3 HARD NEGATIVE SAMPLING . In this section we describe our approach for hard negative sampling . We begin by asking what makes a good negative sample ? To answer this question we adopt the following two guiding principles : Principle 1. q should only sample “ true negatives ” x−i whose labels differ from that of the anchor x . Principle 2 . The most useful negative samples are ones that the embedding currently believes to be similar to the anchor . In short , negative samples that have different label from the anchor , but that are embedded nearby are likely to be most useful and provide significant gradient information during training . In metric learning there is access to true negative pairs , automatically fulfilling the first principle . In unsupervised contrastive learning there is no supervision , so upholding Principle 1 is impossible to do exactly . In this paper we propose a method that upholds Principle 1 approximately , and simultaneously combines this idea with the key additional conceptual ingredient of “ hardness ” ( encapsulated in Principle 2 ) . The level of “ hardness ” in our method can be smoothly adjusted , allowing the user to select the hardness that best trades-off between an improved learning signal from hard negatives , and the harm due to the correction of false negatives being only approximate . This important since the hardest points are those closest to the anchor , and are expected to have a high propensity to have the same label . Therefore the damage from the approximation not removing all false negatives becomes larger for harder samples , creating the trade-off . As a special case our our method , when the hardness level is tuned fully down , we obtain the method proposed in ( Chuang et al. , 2020 ) that only upholds Principle 1 ( approximately ) but not Principle 2 . Finally , beyond Principles 1 and 2 , we wish to design an efficient sampling method that does not add additional computational overhead during training .

Summary: The paper proposes a novel noise contrastive estimation (NCE) objective that incorporates hard-negative samples without similarity supervision, e.g., assuming unsupervised learning. To this end, it modifies the denominator of the original NCE by (a) re-weighting the negative samples based on the euclidean distance from the anchor point, and (b) considering a de-biasing of the effect of positive samples (that should be near to the anchor). Experimental results show that the proposed objective outperforms the pure SimCLR and its de-biased-only variants in many but not all cases in image, graph and text domains.