Poisoning and Backdooring Contrastive Learning

1 INTRODUCTION . Contrastive learning ( Chopra et al. , 2005 ; Hadsell et al. , 2006 ) trains a model that projects a data distribution onto a lower-dimensional embedding space such that similar objects in the origin space are closer together in the embedding space than dissimilar objects ( Chechik et al. , 2010 ; Sohn , 2016 ; Oord et al. , 2018 ; Wu et al. , 2018 ) . Significant advances over the last years have enabled self-supervised classifiers to achieve state of the art accuracy by training on noisy and uncurated datasets ( Radford et al. , 2021 ; Tian et al. , 2021 ) , which brings two significant benefits . First , training on uncurated data is cheaper ( Joulin et al. , 2016 ) . Compared to an estimated several million USD it cost to label the ImageNet dataset ( Deng et al. , 2009 ) , contrastively trained models can train without expensive labeling efforts ( Chen et al. , 2020a ) . Further , because each image in ImageNet is required to contain one of just 1,000 different objects , there are large categories of images that can never be part of this supervised dataset ( Jia et al. , 2021 ) . On the other hand , a contrastive model can learn on arbitrary images whether or not they have a suitable corresponding label in some dataset . Second , training on noisy data improves robustness ( Radford et al. , 2021 ) . Classifiers trained exclusively on ImageNet overfit the particular details of this training set ( Recht et al. , 2019 ; Hendrycks & Dietterich , 2019 ) , and do not generalize to other test sets ( Taori et al. , 2020 ) . Contrastive models trained on data scraped from the Internet exhibit impressive robustness properties ; The contrastively trained CLIP ( Radford et al. , 2021 ) model is the first technique to show significant effective robustness on ImageNet-V2 ( Recht et al. , 2019 ; Taori et al. , 2020 ) . Contributions . We make the case that training on unfiltered may be undesirable if even a tiny fraction of the data could be maliciously poisoned by an adversary . And this is likely the case : the data is scraped from the Internet ( Jia et al. , 2021 ) without any human review before it is passed to the learning algorithm ( Radford et al. , 2021 ; Jia et al. , 2021 ; Tian et al. , 2021 ) . Thus , because these datasets are explicitly “ noisy ” ( Jia et al. , 2021 ) and “ uncurated ” ( Tian et al. , 2019 ) , we argue the likelihood of at least one adversary is high . We show that this adversary can mount powerful targeted poisoning ( Biggio et al. , 2012 ) and backdoor attacks ( Gu et al. , 2017 ; Chen et al. , 2017 ) against multimodal contrastive models . A poisoning adversary introduces malicious examples into the training dataset so that the model will misclassify a particular input at test time as an adversarially-desired label . We then consider patchbased backdoors , where the adversary poisons a dataset so that the learned model will classify any input that contains a particular trigger-pattern as a desired target label . Existing attacks are sufficient to poison contrastively-trained models ( Biggio et al. , 2012 ; Gu et al. , 2017 ; Chen et al. , 2017 ) —although we must adapt them to this new domain . The primary contribution of this paper is an emperical evaluation of 20,000 GPU-hours to show these attacks are immediately practical . Compared to prior backdooring attacks which require poisoning on average 1 % of training data for successful clean label attacks ( Shafahi et al. , 2018 ; Saha et al. , 2021 ) , we find that attacking contrastive models requires orders of magnitude fewer injections : just 0.01 % suffices for many of our backdoor attacks , or 0.0001 % for poisoning attacks . 2 BACKGROUND , NOTATION , AND RELATED WORK . 2.1 POISONING AND BACKDOOR ATTACKS . In a poisoning attack ( Biggio et al. , 2012 ) , an adversary modifies a benign training dataset X by injecting poisoned examples P to form a poisoned dataset X ′ = X ∪ P . When the victim runs the training algorithm T on the modified training dataset X ′ , they obtain a poisoned model fθ ← T ( X ′ ) . This model fθ will now perform well in most standard settings , but because of the poisoned examples P , the adversary will control how it behaves in other settings . We first consider targeted poisoning ( Barreno et al. , 2006 ; Biggio et al. , 2012 ) where an adversary injects poisoned examples so that some input x′ will be misclasified as a desired target y′ . Poisoning attacks exist for many tasks , including supervised ( Biggio et al. , 2012 ; Turner et al. , 2019 ; Koh & Liang , 2017 ) , unsupervised ( Kloft & Laskov , 2010 ; 2012 ; Biggio et al. , 2013 ) , and semi-supervised ( Liu et al. , 2020 ; Carlini , 2021 ) learning . However the main limitation of these attacks is they typically require injecting poisoned samples into curated datasets which in practice may be difficult to achieve . Our attacks apply to uncurated and noisy datasets , making them more realistic . We then turn to backdoor attacks on image classifiers . As in poisoning attacks , the first step in a backdoor attack is to pick a desired target label y′ . Instead of causing one particular image to be classified as y′ , a backdoor attack makes any image with a backdoor patch applied classified as y′ ( Gu et al. , 2017 ; Chen et al. , 2017 ) . We write x′ = x⊕ bd to denote a backdoored image , and consider the standard checkerboard backdoor that is overlaid on top of the image ( Gu et al. , 2017 ) , see Figure 1 for an example . We consider two approaches to placing the backdoor on the image . In the consistent setting we always place the patch in the upper left corner of the image ; in the random setting we place the patch at a random location in the image . 2.2 CONTRASTIVE LEARNING . In its most general definition , contrastive learning ( Chopra et al. , 2005 ; Hadsell et al. , 2006 ; Sohn , 2016 ; Oord et al. , 2018 ) constructs an embedding function f : X → E that maps objects of one type ( e.g. , images ) into an embedding space so that “ similar ” objects have close embeddings under a simple distance metric ( e.g. , Euclidean distance or cosine similarity ) . Early techniques would train using a triplet loss ( Weinberger & Saul , 2009 ; Chechik et al. , 2010 ) to distinguish two similar objects from a third different object . However more recent techniques now perform the contrastive loss across the entire mini-batch ( Sohn , 2016 ; Oord et al. , 2018 ) . While this direction traditionally focused on a single domain ( e.g. , classifiers only trained on images ( Sohn , 2016 ; Wu et al. , 2018 ; Bachman et al. , 2019 ; Chen et al. , 2020a ; b ) ) , within this past year , multimodal ( Weston et al. , 2010 ; Socher & Fei-Fei , 2010 ) contrastive learning techniques have begun to emerge that demonstrate significant and surprising benefits ( Radford et al. , 2021 ; Jia et al. , 2021 ) . Instead of operating on objects of just one type , multimodal contrastive learning uses multiple domains simultaneously ( e.g. , images and text ) ( Zhang et al. , 2020 ) . We focus on multi-modal classifiers . The dataset X ⊂ A×B here consists of objects drawn from two modes—in this paper , images ( A ) and text captions ( B ) . Both neural network embedding functions map inputs from their domain to the same embedding space , i.e. , f : A → E and g : B → E. For a given training example ( a , b ) ∈ X the training objective then minimizes an inner product ( e.g. , cosine similarity ) between the embeddings 〈f ( a ) , g ( b ) 〉 while maximizing the inner product between this example and other examples ( a′ , b′ ) ∈ X . Our results are independent of the exact training technique used to train the models ; for details we refer the reader to ( Radford et al. , 2021 ) . Use of contrastive models . Contrastively trained models are typically used in one of two ways . 1 . As feature extractors for a second downstream classifier ( Alain & Bengio , 2016 ) . We use f to map some new training dataset X̂ into the embedding space E , and then train a linear classifier z : E → Y to map the embeddings to predictions of the downstream task . 2 . As zero-shot classifiers . Given an object description ( e.g. , t1 = “ A photo of a cat ” and t2= “ A photo of a dog ” ) a contrastive classifier evaluates the embedding ei = g ( ti ) . At test time the classification of x is given by z ( x ) = { 〈ei , f ( x ) 〉 : i ∈ [ 0 , N ] } . 2.3 THREAT MODEL . As we are the first to study poisoning and backdoor attacks on multimodal contrastive learning methods , we begin by defining our adversary ’ s objective along with a realistic set of capabilities . Adversary Objective . The ultimate goal of our attack is to cause the contrastive model to behave incorrectly in one of the three cases above . Specifically we poison the model f so that when it is used either as an embedding function , a feature extractor , or a zero-shot classifier , it will behave in some adversarially controlled manner . We focus our paper on attacking the image embedding function f . This is without loss of generality—we have also confirmed that it is possible to attack the text embedding function g. However most prior work studies poisoning images , and so we do too . Adversary Capabilities . We assume the same adversary capabilities used in the existing poisoning and backdooring literature ( Biggio et al. , 2012 ) . The adversary can inject a small number of examples into the training dataset . At the poisoning rate required by prior supervised attacks ( Shafahi et al. , 2018 ; Saha et al. , 2021 ) , an adversary would need to modify a million images in the CLIP dataset . This is not realistic . In our paper we consider adversaries who can poison 100 − 10 , 000× fewer images . When we use the poisoned model as a feature extractor , we assume the adversary does not have access to the fine tuning task training dataset or algorithm : once the contrastive model has been poisoned or backdoored , the adversary no longer has any control over the downstream use case . 3 POISONING AND BACKDOORING ATTACK ALGORITHM . Both our poisoning and backdoor attacks will follow the same general procedure . We begin with the simpler case of targeted poisoning : given an example x′ and incorrect target label y′ , the adversary supplies the contrastive algorithm with the poison set P designed so that y′ = z ( fθ ( x′ ) ) , that is the learned model fθ ← T ( X ∪ P ) will compute an embedding so that the classifier z will misclassify the input . Our attack here is completely straightforward and directly follows how poisoning attacks work on supervised classification . Because models overfit against their training dataset ( Zhang et al. , 2017 ) , and because contrastively trained models have higher train-test gaps than supervised classifiers ( Radford et al. , 2021 ) , we need only inject image-text pairs that cause the model to map x′ into the concept class of y′ .

Summary: The paper describes poisoning and backdooring attacks on CLIP, a recent method to (pre)-train multimodal networks with a contrastive objective. The setting here is different from a classical supervised BadNet-like setup in that learned embeddings are not necessarily going to be directly mapped to a class of choice. Authors evaluate both zero-shop learning setup and the linear probes to find that both of them can be successfully backdoored and poisoned with much fewer requirements. Authors extensively evaluate their attack and pinpoint what hyperparameters make it more effective.