Robust Contrastive Language-Image Pretraining against Data Poisoning and Backdoor Attacks
Authors: Wenhan Yang
Jingdong Gao
Baharan Mirzasoleiman
Venue: NeurIPS 2023 poster (2023)
Keywords: Contrastive Learning
Adversarial Learning
Model Robustness

Abstract: Contrastive vision-language representation learning has achieved state-of-the-art performance for zero-shot classification, by learning from millions of image-caption pairs crawled from the internet. However, the massive data that powers large multimodal models such as CLIP, makes them extremely vulnerable to various types of targeted data poisoning and backdoor attacks. Despite this vulnerability, robust contrastive vision-language pre-training against such attacks has remained unaddressed. In this work, we propose RoCLIP, the first effective method for robust pre-training multimodal vision-language models against targeted data poisoning and backdoor attacks. RoCLIP effectively breaks the association between poisoned image-caption pairs by considering a relatively large and varying pool of random captions, and matching every image with the text that is most similar to it in the pool instead of its own caption, every few epochs.It also leverages image and text augmentations to further strengthen the defense and improve the performance of the model. Our extensive experiments show that RoCLIP renders state-of-the-art targeted data poisoning and backdoor attacks ineffective during pre-training CLIP models. In particular, RoCLIP decreases the success rate for targeted data poisoning attacks from 93.75% to 12.5% and that of backdoor attacks down to 0%, while improving the model's linear probe performance by 10% and maintains a similar zero shot performance compared to CLIP. By increasing the frequency of matching, RoCLIP is able to defend strong attacks, which add up to 1% poisoned examples to the data, and successfully maintain a low attack success rate of 12.5%, while trading off the performance on some tasks.

Robust Contrastive Language-Image Pre-training against Data Poisoning and Backdoor Attacks
Wenhan Yang
Jingdong Gao
Baharan Mirzasoleiman
(hangeryang 8, mxuan, baharan ucs.ucla.eduu Computer Science Department, UCLA
Abstract
Contrastive vision-language representation learning has achieved state-of-the-art performance for zero-shot classification, by learning from millions of image- caption pairs crawled from the internet. However, the massive data that powers large multimodal models such CLIP,, makes them extremely vulnerable various types of targeted data poisoning and backdoor attacks, Despite this vulnerability, robust contrastive vision-language pre-training against such attacks has remained unaddressed. In this work, we propose RoCLIP, the first effective method for robust pre-training multimodal vision-language models against targeted data poi- soning and backdoor attacks. ROCLIP effectively breaks the association between poisoned amage-caption pairs by considering relatively large and varying pool of random captions, and matching every image with the text that is most similar in the pool instead of its own caption, every few epochs. It also leverages image and text augmentations to further strengthen the defense and improve the performance of the model. Our extensive experiments show that RoCLIP renders state-of-the-ar targeted data poisoning and backdoor attacks ineffective during pre-training CLIP models. In particular, ROCLIP decreases the success rate for targeted data poisoning attacks from 93.75% to 12.5% and that of backdoor attacks down to 0%, while improving the model's 's linear probe performance by 10% and maintains similar zero shot performance compared to CLIP. By increasing the frequency of matching, RoCLIP is able to defend strong attacks, which add up to 1% poisoned examples the data, and successfully maintain low attack success rate of 12.5%, while trading off the performance on some tasks
Introduction
Recent large-scale vision-language models pre-trained on millions of image-caption pairs crawled from the internet has gained an unprecedented success. Large-scale vision-language models such as CLIP (Radford et al. 2021) and ALIGN (Tia et al.) 202T) are trained using multimodal contrastive loss which pulls the representations of every image-caption pair together while pushing those of different pairs apart. In doing they can learn state-of-the-art image representations, without the need for fixed set of predetermined labels to be specified at pte-training time. This enables zero-shot transfer of the model to downstream tasks, without requiring specialized output heads or dataset specific customization. Instead, natural language can be used to reference the learned visual concepts afterwards (Radford al. 2021 Jia et al. 2021). Crucially, alleviating the need for expensive labeling of training examples enables scaling up the training data to millions of examples. However, the massive data that powers such large models also makes them extremely vulnerable various types of targeted data poisoning and backdoor attacks (Carlini & Terzis 2021 Yang et al. 2022).
Code available at
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

A bulldog looking the camera
Fetch NN caption
Origiaal Images Day
Ni 6og dog
X
Esooder
Zi
Pool
Z
Z
Dog laying the Boor
Imagcicaption Augmentation
InfoNCE(Z. (Zante)
Original Caption Dog
Zi
7
Xi
Zi
Snooder
Update Pool
Figure 1: Illustration of ROCLIP. for defending CLIP during pre-training. (a) RoCLIP keeps pool of random varying captions. During training, ROCLIP augments all images and captions. Then, finds the most similar caption 2 (t) augmented image z / and instead of matching the image o and caption z?" it matches every augmented image to its z (f).
Targeted data poisoning attacks on multimodal models add mismatched image-caption pairs to the pre-training data, to change the prediction of particular images at the test time. Similarly, backdoor attacks overlay a small patch on a subset training data to cause the model misclassify test images with the same patch. Notably, poisoning just 0.0001 % of the pre-training examples can lead to success of a targeted poisoning attack. Similarly, poisoning 0.01 of pre-traine pre-training examples can makes a backdoor attack successful (Carlini & Terzis (021). Compared to clean-label data poisoning and backdoor attacks in the supervised settings which require poisoning on average 1% of training data (Turner et al. 2018 Geiping et al. 2021). attacking multimodal contrastive models requires orders of magnitude fewer poisoned examples. Interestingly. the larger the model, the more vulnerable
against adversarial attacks (Carlini & Terzis 2021).
Despite this vulnerability, robust pre-training of multimodal vision-language models against targeted data poisoning and backdoor attacks has remained unaddressed. The recent work of Yang et al. (2022) studied poison identification during fine-tuning of CLIP, by using another CLIP model pre-trained on clean data to remove dissimilar maage-caption pairs. Furthermore, |Bansal et al. (2023) proposed CleanCLIP to eliminate the effect of backdoor attacks from a pre-trained CLIP model, by fine-tuning
on a large subset of the clean pre-training data with in-modality contrastive loss on both vision and language modalities. However, such approaches are not applicable to pre-training, as we also confirm experimentally in Appendix 7.1
In this work, we propose the first effective method, namely RoCLIP, for robust pre-training of multimodal vision- laiguaee models such as CLIP, against targeted data poisoning and backdoor attacks. Our approach is based on the following key observation: while the similarity between the image-caption pairs of clean examples increases rapidly during the training, similarity between poisoned image-caption pairs grows at a slower speed, early training. As a result, poisoned images and captions are not close to the groups of similar images and captions in the representation space, capagon pairs,
during the initial training iterations. To break the association between poisoned our main idea is to keep relatively large and varying pool of randomly selected captions. Then, we match every image with the text that is most similar to in the pool instead of its original caption. This effectively prevents the attack by breaking the association between poisoned image-caption pairs, from early training epochs. We further strengthen our defense and improve the performance by leveraging image and text augmentations.
Our extensive experiments show that our method renders state-of-the-ar targeted data poisoning and backdoor attacks ineffective during pre-training. In addition, our method leads to increase of linear probe accuracy by up to 10% while having zero shot performance on par with CLIP, By increasing the frequency of matching, RoCLIP is able to defend strong attacks, which add up to 1% poisoned examples to the data, and successfully maintain low attack success rate of 12.5% 5% while trading off the performance on some tasks. We note that ROCL.P is the only effective defense method against wate-of-the-aa attacks that can efficiently scale to pre-training large-scale vasion-language models such CLIP.

---

2 Related Work
Contrastive Representation Learning. Contrastive learning was originally proposed for self supervised representation learning from unimodal data. Self-supervised contrastive learning works by maximizing agreement between differently augmented views of the same example and minimizing agreement between differently augmented views of different examples (Chen et al.) 2020) Chen & He 2021 I He et al. 2020). Several works improved the performance of contrastive-lcarnin on downstream tasks by imposing additional constraints to remove redundancy between components of the representation vectors and prevent collapse of the representations (Bardes et al. 2021 Zbontar al. (202)) or using nearest- neirhott as positive pairs in the contrastive loss (Dwibedi et al. 2021
Van Gansbeke et al. 2021).
Contrastive Language-Image Pretraining. Multimodal dision-languae models like CLIP (Radford al. (02T] and ALIGN (Jia al. 2021) are pre-trained on 400M/IB IB image-text pairs, by maximizing the agreement between image and text representations of every image-caption pairs and minimizing
those of different pairs. A recent line of work aims at improving the data efficiency and quality of
CLIP representations, by leveraging image and text augmentations. DeCLIP (Li et al. 2021) improves data-efficiency of CLIP by maximizing the similarity between two augmented image features using SimSiam (Chen & He (02I)) two augmented text features using Masked Language Modeling (MLM) (Devlin et al. 2018), and matching augmented image features with their augmented text pairs and other similar text features. SLIP (Mu al. 2022) improves the performance by maximizing the agreement between two augmented image features using SimCLR (Chen et al. 2020 and
matching the augmented image features with their text pair. CyCLIP (Goel et al. 2022 improves the representations by symmetrization of the similarity between the two mismatched image-tex| pairs, as well as the similarity between the image-image pair and the text-text pair. Finally, FILIP (Yao et al. 2021
uses transformer- encoders for both modalities to learn more fine-grained features.
Targeted Data Poisoning and Backdoor Attacks on CLIP. Contrastive pretrained language-image models are extremely vulnerable to various types of targeted data poisoning and backdoor attacks (Carlini& & Terzis 2021). Targeted data poisoning attacks fool the model to misclassify particular test example as an adversarial label. On the other hand, backdoor attacks overlay a small patch on a subset of training data, and cause the model to misclassify test images with the same patch. Despite this vulnerability, designing effective defense methods to protect the model from being poisoned during pre-training has remained unaddressed.
CLIP has been also shown be vulnerable to data poisoning attacks during fine-tuning (Yang et al., 2022). To address this, Yang et al. (2022) proposed a pre-processing and a post-processing defense to defend against targeted data poisoning attacks during fine-tuning. The pre-processing requires a clean pre-trained CLIP to remove examples with low cosine similarity between image and their corresponding text representation. The post- post-processing fine-tunes the poisoned model on another clean dataset of the same scale as the fine-tuning data. Besides, [Banaal et al. (2023) proposed a method to clean backdoor attacks from CLIP, by fine-tuning the model on a clean subset of the pre-training dataset with in-modality contrastive loss on both vision and language modalities. Such approaches are however not applicable to pre-training, as we also confirm experimentally in Appendix 7.1 In our work, we propose the first effective defense method that can protect the model from both targeted image attacks and backdoor attacks during pre-training.
3 Preliminary
3.1
Contrastive Language-Imag Pre-training (CLIP)
CLIP is trained on millions of image-caption pairs scraped from the web. Formally, consider a dataset D C I x T consisting of pairs (z))=;) where x E I is a raw image and =² e T is a text (caption of the image). The CLIP architecture consists of an image encoder Jr I - Bd that encodes the raw image o! into an embedding vector V/ and text encoder fr - IRd that encodes the raw text x!!" into embedding vector 2!¹ of the same dimension. Then projected image and text embeddings 2! 2 are obtained by passing the encoded image and text 2/,2², through their corresponding projection heads. The projected representations are normalized to have unit norm. Finally, the InfoNCE loss (Oord et al. 2018) is applied to pull the projected embeddings of every
3

---

image and its corresponding caption together while pushing apart the projected embeddings of the image from other captions in the same mini-batch. Formally, for a mini-batch of N image-caption pairs (<// 2]) and their projected embeddings {(z}. 23 )))))=1 the CLIP loss is defined as:
1 CCUP
exp (
exp (2 2/b /T)
log
log
2N
k= exp ( /T)
2N
bز=1 exp (12) /T)
where (---) represents the inner product, and is trainable temperature parameter. The performance of the pre-trained CLIP model is evaluated via zero-shot and linear probe methods, as discussed below.
Zero-shot classification. Pre-trained Language- Image models such as CLIP enable zero-shot transfer of the model to downstream tasks, without the need for specialized output heads dataset specific customization. To do so, the downstream labels is transformed into natural language captions using the provided engineered prompts templates, c.g. "A photo of {label}". Then, the cosine similarity of the test image to each caption is computed, and the model predicts the label with the highest image-caption similarity.
Linear probe. For a labeled downstream image dataset, CLIP image representations can also be evaluated by training linear classifier on the image representations obtained from the pre-trained CLIP image encoder and the downstream labels.
3.2 Targeted Poisoning and Backdoor Attacks
Let D {<<,,,/))) Di= be the set of all training examples. Poisoning attacks (Biggio et al. 2012 inject a small subset poisoned examples D, to the original training dataset DD such that when the model is trained on the poisoned training data {D U Dp), its prediction on particular test examples are changed to the adversarial label Yadu. At the same time, the poisoned model performs normally on other test examples. In this work, we consider both targeted poisoning and backdoor attacks discuss next.
Targeted Image attacks. In targeted poisoning attack, the adversary aims to change the prediction of one particular test examples x/ to the adversarial label Yodv- Targeted poisoning attacks can be crafted following (Carlini & Terzis (021). by constructing a caption set Tede of potential text descriptions related to the label Yadv, and making poisoned examples by assigning captions in Tode to every target æ i.e., D,- = {(x x ) x E Tadv For constructing the caption set Tades one can search the training dataset for all sequences that contain this label string, and use these sequences as the caption set. Alternatively, one can use the set of 80 different engineered prompt templates provided by CLIP for classification (Radford et al., 202T). and either use a subset or repeat them as necessary. The number of captions in Tade determines the number of generated poisons per target. To evade automated cleaning algorithms (e.g., removing duplicated images), tiny Gaussian noise can be added to the images, or the captions can be modified by substituting or adding words, without degrading the attack success rate. A diverse caption set ensures that the image encoder is poisoned instead of the projection layers.
Backdoor attacks. In backdoor attacks, the adversary attaches small trigger patch to the poisoned images and pair them with adversarial captions Tado related to Yadv- In doing so, all the test images with the trigger patch will be misclassified Yodu- In contrast to the targeted poisoning attack, instead of using particular æ/. the adversary poisons different images x! E I, by adding the trigger patch to them. Specifically, the poisoned set is defined as D,- = {xx @ patch, x.) x E TTavv: x! E Z}. The caption set can be constructed in a similar manner to targeted poisoning attacks using captions found in the training data or engineered prompt templates.
In general, while injecting poisoned examples in curated datasets used for supervised learning might be difficult, such poisons can be easily injected uncurated datasets used by large multimodal models. This makes such models highly vulnerable to adversarial attacks.

---

Algorithm Robust CLIP pre-training (ROCLIP)
1: Input:
Image encoder Sr, text encoder [r, caption pool p ) captions, RoCLIP frequency K
initialized with random
2: for epoch 1, 1...... ,T do
3: for every mini-batch of image-caption pairs: 1z)
+3 E do
4:
Augment image and captions in the mini-batch { ( if T % K. == 0 then
5:
6:
Pair every _ with its nearest augmented caption in pool = kn((j) = arg , TEP 155
2, T 212
7:
Append the pool with captions in the mini-batch and discarding the oldest captions Train f1 and fr with CRØCLIP in Eq.
8:
9:
else
Train fI and fr with CCLP in Eq.
3.3 Threat Model
Adversary Objective. The primary objective of the adversary is to manipulate the output representa- tions of CLIP, such that certain images are misclassified into adversarial categories instead of their true categories, while the other images are classified correctly.
Adversary Capabilities. We assume that the adversary has limited control over the pre-training data, and can inject small number (S 1% of the dataset size) of poisoned examples into the training dataset. Adversary also has the knowledge of the model structure, the training algorithm, and the hyperparameter used by their victim, but they cannot modify the training process directly.
4 Robust Training of Multimodal Models against Targeted Data Poisoning and Backdoor Attacks
In this section, we first study the effect of data poisoning attacks on multimodal models. Then, we present method for robust training of such models against data poisoning and backdoor attacks.
4.1 Effect of Data Poisoning Attacks on CLIP
We start by studying the effect of data poisoning attacks on multimodal models. By minimizing the CLIP contrastive loss in Eq. (T) the model changes such that every image representation 2/ moves towards its caption representation z" and gets far away from other (dissimilar) caption representations zT. This makes corresponding categories of image and text (e.g. category of "Cat" or "Dog" in the image and text modality) to get closer to each other and get distant from other categories, during the pre-training, Crucially, image-caption pairs belonging similar to each other, their gradients have a large alignment with other examples the same category. Therefore, categories of similar image-caption pairs insert large cumulative gradient on the model and change the model quickly to get close to their pairs in the other modality.
particular category
relatively
On the other hand, image-caption pairs of poisoned examples examples in the data. Hence, their gradient does not align well with the gradient of the clean examples. Therefore, by training the poisoned pairs, the poisoned images and captions get far away from the other images and captions in the same category. That is, poisoned images become distant from other images in the same category, and adversarial captions become distance from other captions in the adversarial category. Next, we will rely on this observation to break the association between poisoned image-caption pairs.
not similar to the other clean
4.2 Robust Training with ROCLIP
To prevent targeted data poisoning and backdoor attacks from being successful, we aim to break the association between the poisoned image-caption pairs. this can done, the poisoned image and caption representations do not get close enough to each other during training and the attack does not
5

---

Original Caption
Polsoned Caption
Locking for NN coption 1st iter
Looking for NN caption 1st
Dog laying on the floor
Dog sleaping under the blankel
dog
small bird sitt- ing a branch
2nd iter
A buildog looking
2nd iter
Airllanes moving
rumway
3rd iter
3rd iter
Clean Image
Alight ginger dog in bath
Target Image
-ing on street.
Clean image is still matching to the correct captions
Target image no longer. matching the poisoned captions
Figure 3: Examples of matching the images in the clean poisoned pairs during the training. The image the clean pair can find semantically similar captions in the caption pool and still match to the correct captions, while the image in the poisoned pair will match to either random captions or captions corresponding to the original image.
succeed. To achieve this, we rely on our key observation in Sec. 4.1 poisoned images and captions not close to groups of similar images and captions in the representation space, early in training. We leverage two techniques to break the association between poisoned image-caption pairs: (1) A large and varying pool of randomly selected captions; (2) Augmentations on both images and captions.
The Pool. As illustrated in Fig. [ instead of matching every image with its corresponding caption, we match every image with the caption in the pool that is most similar to the image.
1.0
of NN captions in same category
Based on the observation from Sec. 4.1 poi-
soned captions are not close to other captions in the same category. Thus, our method prevents the poisoned images to be matched with cap- tions from the adversarial category. In doing
Poisoned Pairs
0.4
so, it breaks the data poisoning and backdoor attacks. Fig. shows the effect of pool when
0.2
training dataset with 15 poisoned images of a deer x paired with adversarial truck captions {(x/, x!!) x/6 Truck)- We that while the majority of clean images are matched with captions from the same category as their origi- nal caption during training (blue line), poisoned images are matched with caption from differ- ent category than the adversarial label (orange line). This confirms our observation in Sec. 4.1 and shows the effectiveness of the pool. The pool may have a negative influence on the per-
0.0
25
epoch
Figure Training with RoCLIP. Majority of clean images matched with captions from the same category as their original caption during train- ing (blue line), while poisoned images matched with a caption from a different category than the adversarial label (orange line).
formance, as it is unreliable especially early in training. To ensure good model performance, we (1) select relatively large pool size so that every clean image can find caption similar to its original caption; (2) train with method every K. epoch, and train with standard CLIP loss in the other epochs. Note that is necessary to train on the poisoned pairs consistently to poison the model. With smaller K. the method is able to defend the model and disassociate the poisoned image-caption pairs. We select 2% of the total dataset size as our pool size and K. = in our experiments. Fig. illustrates example of how RoCLIP disassociates poisoned pairs during the training.
To strengthen our defense, we use various image and text augmentations during the training. In particular, we use random image cropping, horizontal flipping. color jittering (Wu) al. (2018), grayscale conversion (Wu al. 2018). and blurring (Chen al., (2020) in our image augmentation policies. For the text augmentation, we use the EDA proposed by Wei & Zou 2019). which includes synonym replacement, random swap, and random deletion as its augmentation policies. The benefit of the augmentations are two-folded: First, similar to attacks on supervised learning (Borgnia et al. 202T). it help further reducing the attack success rates, but cannot fully defend the model, will confirm in ablation study. Data augmentation is especially helpful during the
Augmentation.

---

CLIP epochs to prevent matching poisoned image-caption pairs. Second, augmentations can further increase the method's performance, as shown in Table B and Table
In summary, ROCLIP first samples a pool of P caption representations P P= {=] uniformly at random. During training, for every example (xr xx5) in the mini-batch, we first augment its image and text with augmentation policies, and then match its augmented image representa- tion žj' with the augmented caption representation in the pool that is most similar to æÿ', i.e., zT kn(()) = arg ming,"" "eplt, =2p 1|2. Effectively. we form the positive image-caption representation pair , )m)))) and use it instead of 22), Similar the CLIP loss, obtain the negative pairs from the mini-batch. That is, for a mini-batch of N image-caption pairs {zr). }}} and their projected embeddings {(zg" ) the loss is defined as:
LROCLIP
N log
exp
( mn(s)) /T)
N
exp ( 2 , kn((k)
/T
log
2N
1=1 k= exp (zz²²
27 (n(kk)
2N
Σ exp ((z)², 2T (nncks)
For the pool P, consider first-in-irst-out queue, which is initialized with random caption representations. After training on every mini-batch, we update P by taking the caption representations of the N examples in the mini-batch and concatenating them at the end of the queue. We discard the oldest N elements from the queue, which equals to the training batch size.
The pseudocode of our method, RoCLIP, is illustrated in Alg.
5 Experiments
In this section, we evaluate the effectiveness of ROCLIP in breaking targeted data poison and backdoor attacks while maintaining the model's performance. We evaluate our method for defending against various data poisoning and backdoor attacks, described in Sec. 3.2 during pre-training
5.1 Training
We evaluate ROCLIP during pre- training on CLIP. For pre-training, we use open-source implementation of
Table 1: Details of downstream datasets.
DATASET
CLASSES
TRAIN SIZE
TEST SIZE
CLIP as our model, with default ResNet- 50 as the image encoder and Transformer as the text encoder. Each experiment run with batch size of 512 for 24 epochs, as the performance improvement afterwards is not significant, but more training makes the model more prone to being poisoned. We Conceptual Cap- tions 3M (CC3M) (Sharma et al., 2018) as our pre-training dataset. Due lim- ited computational resources, for pre-training we randomly sampled 1M image-caption pairs from
CALTECH101
102
3,060
6,085
CIFARIO
50,000
10,000
CIFAR100
100
50,000
10,000
DTD
3,760
1,880
FGVCAIRCRAFT
100
6,667
3,333
FLOWERS102
102
2,040
6,149
FOODI01
101
75.750
25,250
IMAGENET IK
1000
50,000
50,000
OXTORDIITTPE
37
3,680
3,669
STANFORDCARS
196
8.144
8,041
CONC. CAPE. (IM)
1,000,000
CC3M our training dataset. We assess our method on 10 downstream datasets introduced by
KKrmblith et al. 2019). the detail of which can be found in Table I
5.2 Attack Methods
We consider targeted image attacks and backdoor attacks, discussed in Sec. 3.2
Targeted Image Attacks. In our pre-training experiment, we choose a random target image I, from the conceptual captions validation set, and then choose a random target class from the ImageNet test set to generate set of [Tade adversarial captions. Note that, (Carlini Terzis 2021) pre-trained 32 CLIP models and measured the attack success rate as the fraction of positioned models. This requires 3200 GPU hours. To reduce the computation, we poison 16 different random images by generating |Tavu] adversarial captions for each image? related to a label selected at random from ImageNet.
7

---

Table 3: Linear probe and zero-shot top I accuracy. RoCLIP improves the performance of the model by up to 10% for linear probe and obtains a similar zero-shot performance compared with CLIP.
Method
Task
F102
Fd101
IKK
Pet
Cars
Cal101
C10
C100
DTD
Aic.
O-shot
0.83
6.34
6.63
3.68
0.72
30.38
30.14
9.52
3,56
1.11
RoCLIP lin-prb
99.22
54.05
24.09
52.36
20.35
72.15
78.99
57.82
55.21
32.55
0-shot
1.0
7.1
9.6
3.4
0.8
34.9
34.9
7.3
3.7
0.8
CLIP
lin-prb
99.5
44.9
22.2
48.2
12.9
70.4
70.5
45.8
48.2
24.9
Then, we report the attack success rate as the fraction of images that are classified as the adversarial label, in a single pre-training run. In doing so, the attack success rate will be at least as high as (Carlini & Terzis, 2021). In addition, note that attacking our smaller dataset of 1M examples also results in higher attack success rate compared to that of 3M used by (Carlini & Terzis (02T). We will first evaluate the effectiveness of RoCLIP in defending moderate amount of poisons (Tade 50) without any performance loss. Then, we will examine if RoCLIP can defend very strong data poisoning attacks ([Tadol 100 10, 000) successfully.
Backdoor Attacks. We use the public Hidden Trigger Backdoor Attacks (HTBA) patches (Saha et al. 2020), that are square triggers generated by drawing a random 4x4 matrix of colors and resizing the desired patch size using bilinear interpolation. We use resized 16x 16 patch and put consistently on the left top corner of the image. In our pre-training experiments, we randomly select 150 images from the CC3M validation dataset and pair them with adversarial captions related to random target class from ImageNet. To evaluate the effectiveness of the backdoor attacks, we select 300 random images from the ImageNet validation set and patch them the left top corner to measure the attack success rate.
5.3 ROCLIP Robustly Pre-trains CLIP
Table ROCLIP defense performance.
First, we evaluate the effectiveness of our method, RoCLIP, against targeted poisoning and backdoor attacks during the pretraining phase. We present our result in T where at- tack success rate shows the fraction of poisoned or backdoored images successfully classified as the desired label. We see that without any de- fense, 93. 75% of the total poisoned images are classified to the desired target class, and 78% of the backdoored images are classified as the target class. On the other hand, ROCLIPP is able to fully defend the attack and reduce the attack success rate to 0% for the backdoor attacks and low 12.5 % for the targeted poisoning attacks. This clearly confirms the effectiveness of RoCLIP breaking various types of data poisoning and backdoor attacks on CLIP during pre-training.
Model
Attack
Success Rate
CLIP
Target Img
93.75%
Backdoor
78%
RoCLIP
Target Img
12.5 %
Backdoor
0%
5.3.1 ROCLIP does not Harm the Performance
Next, we evaluate if ROCLIP negatively impacts the model performance. We assess the performance of ROCLIP on a variety datasets introduced by (Kornblith et al. 2019), the detail which can be found in Table T] We evaluate RoCLIP with both zero-shot and linear-proh probe methods and present the result Table 3 It can be seen that RoCLIP does not harm the overall model performance. In contrast, RoCLIP effectively improves the linear-probe classification performance across all ten datasets by up to 10% and has an on-par zero-shot performance with CLIP. As we will show in our ablation study, data augmentation contributes the most RoCLIP's performance boost. Nevertheless, even without data augmentation, RoCLIP achieves a similar performance with CLIP.
5.3.2 ROCLIP against Very Strong Attacks
Next, we consider much stronger poisoning attacks against CLIP during pre-training. Note that, in general much lower poison rate is enough to successfully poison multimodal models (0.0001%),

---

Table 4: RoCLIP against very strong data poisoning attacks RoCLIP defend the model with some performance tradeoff. Linear probe and zero-shot performance is reported on CIFAR-10 (C10), CIFAR-100 (C100), Food101 (FD) and Caltech101 (Cal).
Poison Rate
Zero-Shot
Linear-
0.01%
0.0512%
0.5%
1.0%
C10
C100
FD
Cal
C10
C100
FD
Cal
ROCLIP
0%
0 %
0%
12.5%
21.3
7.3
3,4
17.0
69.7
46.6
45.6
59.8
CLIP
100%
100%
100%
100%
35.0
7.3
7.1
34.9
70.5
45.8
44.6
70.4
Table Linear-Probe (LP) performance, Poison Success Rate (PSR), and Backdoor Success Rate (BSR) with various pool sizes used in RoCLIP.
Pool Size
256 (0.2%)
2048 (2%)
21845 (20%)
0 (CLIP w/ Aug)
0 (CLIP)
McM)
39449
39627
39505
39342
37459
Time(s)
538.31
538.69
539.47
536.96
506.02
C10 LP
64.5%
67.1%
67.9%
72.9%
62.1% 1%
C100 LP
34.9%
39.6%
41.1%
48.4%
37.4%
PSR
0%
0%
0 %
12.5%
75%
BSR
0%
0%
%
0.33%
37.5%
compared supervised learning models (1%), and hence the highest poison rate considered in (Carlini & Terzis 2021) is 0.017%. Considering the very large size of the pretraining data for considerable "poison number" that needs to be generated by the adversary. Nevertheless, we evaluate the effectiveness of RoCLIP against very high poison rate of up 1% of the dataset size for targeted image attacks. To defend the attacks effectively, we change the frequency from K. = 3 to K. = 2. Table @ shows that RoCLIP with K. = 2 can effectively defend very stronger attacks, while trading off the performance on some tasks.
multimodal models, this small poison rate translates
5.3.3 Comparison to Fine-tuning Defense Baselines
Finally, we compare the performance of RoCLIP with existing methods proposed to defend CLIP during fine-tuning (Bansal et al.) 2023 Yang al. 2022). Due to page limit, we leave the discussions in the Appendix 7.1 Our results confirm that such methods are indeed highly ineffective in defending CLIP during pre-training.
5.4 Ablation Studies
In this section, conduct further experiments different components of RoCLIP to evaluate its effects on defense and downstream performance. All experiments are conducted on 100K data randomly sampled from CC3M, with 15 poisoned pairs and 90 backdoored pairs. We train the model with batch size of 256 for 30 epochs.
5.4.1 ROCLIP against Other Backdoor Attacks
In Sec. 5.3 explored RoCLIP's ef- fectiveness in defending CLIP against patched backdoor attacks. Here, we eval- uate RoCLIP against three additional backdoor attacks baselines, Blended, WaNet, and label-consistent attacks, used in (Bansal sider 90 pairs of backdoored image- caption pairs for Blended triggers, 256 pairs for WaNet triggers, and 512 pairs of label-consistent pairs. Table shows that ROCLIP able to reduce the backdoor success rate to nearly 0% for all three backdoor attacks.
Table 6: ROCLIP against different backdoors.
Model
Attack
Backdoor Rate
BSR
CLIP
Blended 0.09%
28.00%
WaNet 0.256%
43.40%
Label-Consis
0.512%
97.30%
al. (023). We con-
ROCLIP
Blended 0.09%
0.33%
WaNet
0.256%
0.67 %
Label-Consis
0.512%
0.00 %

---

Figure 4: ROCLIP against more
Table 7: Effect of pool and data augmentation on ROCLIP.
challenging poisoned pairs.
Method
P+A
Pool (P)
Aug (A)
CLIP
GLP RuCUP
C10 LP
67.1% %
60.1%
72.9%
62. 1
C100 LP
39.6%
36.5%
48.4%
37.4%
PSR
0%
0%
12.5%
75%
25%
20
BSR 0%
0.67%
0.33%
42.3%
Challenging
5.4.2 ROCLIP with Different Pool Size
Next, we analyze the computation overhead of ROCLIP, and the effect of pool size We apply RoCLIP with pool size of 0.2%, 2% and 20% of the pre-training dataset size. As shown in Table RoCLIP is very lightweight, and has negligible memory overhead. The overall memory usage of RoCLIP is similar to that of CLIP. addition, the run-time overhead is also not significant, and is mostly caused by data augmentation. Second, the size of the pool does not have significant influence on the defense. With different pool sizes, our method is able to defend the both the targeted poisoning and the backdoor attacks. With larger pool size, however, the downstream performance of the model increases since is more likely for the images to find fitting captions in a larger pool.
method.
5.4.3 ROCLIP with Augmentations
Finally, analyze the effect of data augmentations tion improves the downstream performance. Note that even without data augmentation, RoCLIP ob- tains a similar linear probe accuracy with CLIP. Data augmentation further improves the performance of ROCLIP by 5% on CIFAR-10 and 2% on CIFAR-I00 -000. In addition, data augmentation significantly improves the defense and reduces the poison success rate from 75% 12.5% and backdoor success rate from 42.3% to 0.33%. Using both the pool and data augmentation, RoCLIP is able to reduce the success rate of backdoor and target image attacks 0% with superior downstream performance.
method. Table shows that data augmenta-
5.4.4 RoCLIP against Adaptive Attacks
One potential adaptive attack against RoCLIP is to use visually similar categories when selecting the poisoned pairs, e.g., dog and cat. Note that, this strongly limits the choice of attackers' target images as well as adversarial captions. To evaluate the effectiveness of ROCLIP against such challenging poisoned pairs, we include poisoned image-caption pairs from iisuall--similar-caaeerrr like dog and cat, football and basketball (c.f. Appendix 7.2 Fig. shows the effectiveness of ROCLIP.
6 Conclusion
We proposed RoCLIP, an effective method for robust training multimodal vision-language models such CLIP against data poisoning and backdoor attacks. RoCLIP utilizes caption pool well as data augmentation to break the associations between the poisoned image and caption pairs, thus effectively defending the models. Through extensive experiments, we demonstrated that our proposed method drops the success rate of targeted data poisoning and backdoor attacks to 12.5% and 0% respectively. At the same time, improves the model's performance by up 10% on linear-probe probe and obtained a comparable zero shot performance to the baseline CLIP model. By increasing the frequency of matching, RoCLIP is able to defend very strong attacks with 1 1% poison rate and maintain a low attack success rate of 12.5%, while trading off the performance on some tasks.
Limitations. ROCLIP improves the robustness of CLIP pre-training against targeted data poisoning and backdoor attacks. Considering the demand for such models, our work makes valuable step towards safe machine learning. ROCLIP can defend against moderate number of poisons with no performance loss and against high number of poisons with some performance tradeoff. Developing stronger defense methods without performance loss is an interesting direction for future work.
This research was supported by the National Science Foundation CAREER Award 21 2146492 and Cisco Systems.
Acknowledgment.
10

---

References
Bansal, H., Singhi, N., Yang. Y., Yin, F., Grover, A., and Chang, K.-W. Cleanclip: Mitigating data poisoning attacks in multimodal contrastive learning. arXiv preprint arXiv:2303.0332. 0333333,,, 2023.
Bardes, A., Ponce, J., and LeCun, Y. Vicreg: Variance-invariance-cooo self-supervised learning. arXiv preprint arXiv:2105.04906. 2021.
regularization for
Biggio, B., Nelson, B., and Laskov, Poisoning attacks against support vector machines. In Proceedings of the 29th International Coference International Conference Machine Learning, pp. 1467-1474, 2012.
Borgnia, E., Cherepanova, V., Fowl, L., Ghiasi, A., Geiping, J., Goldblum, M., Goldstein, T., and Gupta, A. Strong data augmentation sanitizes poisoning and backdoor attacks without an accuracy tradeoff. In ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 3855-3859, IEEE, 2021.
Carlini, N. and Terzis, A. Poisoning and backdooring contrastive learning. arXiv preprint arXiv:2/06.09667, 2021.
Chen, T., Kornblith, S., Norouzi, M., and Hinton, G. A simple framework for contrastive learning of visual representations. In International conference machine learning, pp. 1597-1607. PMLR, 2020.
Chen, X. and He, K. Exploring simple siamese representation learning. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 15750-15758, 2021.
Devlin, J., Chang, M.-.,, Lee, K., and Toutanova, K. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint 1810:08800008800 2018.
Dwibedi, D., Aytar, Y., Tompson, J., Sermanet, P., and Zisserman, A. With a little help from my friends: Nearest-neighbor contrastive learning of visual representations. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 9588-9597, 2021.
Geiping, J., Fowl, L. H., Huang, W. R., Czaja, W., Taylor, G., Moeller, M., and Goldstein, T. Witches' brew: Industrial scale data poisoning via gradient matching. In International Conference on Learning Representations, 2021. URL
net
Goel, S., Bansal, H., Bhatia, S., Rossi, R. A., Vinay, V., and Grover, A. Cyclip: Cyclic contrastive linguage-image pretraining. arXiv preprint arXiv:2205.1444 2022.
He, K., Fan, H., Wu, Y., Xie, S., and Girshick, R. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 1729-9738, 2020.
Jia, C., Yang, Y., Xia, Y., Chen, Y.-T., Parekh, Z., Pham, H., Le, Q., Sung, Y.-H,, Li, Z., and Duerig, T. Scaling up visual and rision-language representation learning with noisy text supervision. In International Conference on Machine Learning, pp. 1904-4916 PMLR, 2021.
Komblithh S., Shlens, J., and Le, Q V. Do better imagenet models transfer better? In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 2661-2671, 2019.
Li, Y., Liang. F., Zhao, L., Cui, Y., Ouyang, W., Shao, J., Yu, F., and Yan, J. Supervision exists everywhere: A data efficient contrastive language-image pre-training paradigm. arXiv preprint arXiv:21 10.00500. 2021.
Mu, N., Kirillov, A., Wagner, D., and Xie, S. Slip: Self-supervision meets language-image pre-
training. In European Conference on Computer Vision, pp. 529-544. Springer, 2022.
Oord, A. v. d., Li, Y., and Vinyals, O. Representation learning with contrastive predictive coding. arXiv preprint arXiv: arXiv:/807.08 1807.03748. 2018
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry. G., Askell, A., Mishkin, P., Clark, J., et al. Learning transferable visual models from natural language supervision. In International Conference on Machine Learning, pp. 8748-8763, PMLR, 2021.

---

Saha, A., Subramanya, A., and Pirsiavash, H. Hidden trigger backdoor attacks. In Proceedings of the AAAI conference on artificial intelligence. volume 34, pp. 11957-11965, 2020.
Sharma, P., Ding, N., Goodman, S., and Soricut, R. Conceptual captions: A cleaned, hypernymed, image alt-text dataset for automatic image captioning. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp- 2556,2565, 2018.
Turner, A., Tsipras, D., and Madry, A. Clean-label backdoor attacks. 2018
Van Gansbeke, W. Vandenhende, S., Georgoulis, S., and Gool, L.. V. Revisiting contrastive methods for unsupervised learning of visual representations. Advances in Neural Information Processing Systems, 34: 14431-16216200 2021.
Wei, J. and Zou, K. Eda: Easy data augmentation techniques for boosting performance on text classification tasks. arXiv preprint arXiv: 2019.
Wu, Z., Xiong, Y., Yu, S. X., and Lin, D. Unsupervised feature learning via non-parametric instance discrimination. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 37.3-3334,,222 3742, 2018.
Yang. Z., He, X., Li, Z., Backes, M., Humbert, M.,, Berrang, P., and Zhang, Y. Data poisoning attacks against multimodal encoders, arXiv preprint arXiv: 2222209..26660 15266, 2022.
Yao, L., Huang, R., Hou, L., Lu, G., Niu, M., Xu, H., Liang, X., Li, Z., Jiang, X., and Xu, C. Filip: Fine-grained interactive language-image pre-training, arXiv preprint arXiv:2111.07783 2021.
Zbontar, J., Jing, I.., Misra, I., LeCun, Y., and Deny, S. Barlow twins: Self-supervised learning via redundancy reduction. In International Conference on Machine Learning, pp. 12310-1232) PMLR, 2021.
12

---

7 Appendix
7.1 Adapted fine-tuning defense baselines
Here, we apply the methods proposed in (Yang al. 2022 Bansal et. 2023 defend CLIP against data poisoning attacks or eliminate backdoor attacks from a pre-trained model during fine-tuning, We apply these methods during pre-training CLIP and show that they indeed highly ineffective in protecting the model.
Pre-processing CLIP First, show that pre-processing CLIP is not effective during pretraining. We conduct our experiment from the 1M sub-dataset randomly-sane sampled from CC3M. Yang et al. (2022) proposed using pretrained model to calculate and compare cosine similarity between poisoned image-caption pairs and clean image-caption pairs. Then, it finds a threshold to separate the majority of poisoned pairs from clean pairs, and discard all the pairs with lower similarity than the threshold. We applied the above idea to pretraining, in a cheating experiment. That is, we consider the poisoning ratios of 9.005%-0.55 and calculate the cosine similarity between the known generated poisoned image-caption pairs (note that in practice the poisons are not known) at epoch 1. We chose epoch 1, as the separation between cosine similarity of poisoned and clean image caption paris largest epoch I (see Figure 2). Then we set the threshold such that a certain fraction of the poisoned pairs are discarded, and filter examples with cosine similarity lower than this threshold.
Table Ratio of clean data pairs discarded by Pre-processing CLIP when discard a fixed ratio of poisoned data pairs.
Discard/Paison %
0.005%
0.01%
0.1%
0.5%
75%
15.7% (*)
44.3%
98.3%
95.8%
85%
23.3% (*)
62.3%
98.9%
96.3%
95%
38.1%
87.7%
99.55%
97.1%
Tab. [ shows the fraction of clean pairs that are discarded using different thresholds. We see that regardless of the threshold, very large fraction of clean pairs are discarded. We trained and evaluated the two models when less than 30% of the clean data was discarded, marked with (*), and show the results in Tab. g Both models with smallest amount of clean pairs removed were still poisoned. With 75% and 85% % of the poison discarded, the targeted data poisoning attacks still have 53.84% poison success rates. This confirms the ineffectiveness of the above method during pretraining. On the other hand, RoCLIP is able to drop the success rate to 0% (using K. 2) and 12.5% (using K. = 3), and provides a much better trade-off between the attack success rate and model's performance and can defend much higher poison number up to 1%.
Table 9: Defense and downstream performance of the Pre- Tre-processing CLIP compared to ROCLIP.
Model
K.
Poison %
Discard %
PSR
LP
LP C100
ZS C10
ZS C100
CLIP
0.005%
93.75%
70.5
45.8
34.9
7.2
Pre-proc. CLIP (*)
0.005%
75% 85%
58.3
70.2
45.6
27.9
7.3
Pre-proc. CLIP (*)
0.005%
58.3
69.9
45.5
24.0
6.5
RoCLIP
0.005%
12.5 %
78.9
57.8
30.1
9.5
ROCLIP
0.005%
0%
69.7
46.6
21.3
7.27
RoCLIP
0.1%
0%
69.7
46.6
21.3
7.27
RoCLIP
2
0.5%
0%
69.7
46.6
21.3
7.27
RoCLIP
2
1%
12.5%
69.7
46.6
21.3
7.27
note that, the key factor for the success of ROCLIP is that poisoned images and captions are not close to the other images and captions in the same category. For example, poisoned dog image that is captioned adversarially as a deer, remains far away from the rest of dog images in the representation space, during the initial training epochs. Hence, nearest-neighhor matching done by RoCLIP can effectively break the association of poisoned image caption pairs.
CleanCLIP Next, we show that CleanCLIP is not effective during pretraining. We conduct our from CC3M. (Yang et al.) (022) proposed fine-tuning on a clean dataset (of the same size as pretraining) with CLIP loss, and (Bansal et al. 2023) proposed fine-tuning on a clean subset of size 100K of the original training dataset (same size as the pretraining data we use in this experiment) with CLIP loss+in- maasiin-mddality contrastive loss on both image and text modalities. Since a clean dataset is not available during pretraining, here we use the CleanCLIP loss (CLIP loss + in-modality contrastive loss) to pretrain CLIP on the (poisoned)
experiment from the 100K sub-dataset randomly-samph
13

---

Table 10: Defense and downstream performance of ClcanCLIP compared ROCLIP.
Model
Poison %
Backdoor %
PSR
BSR
LP
LPCI00
RoCLIP
0.015%
0.15%
0%
0%
67.1
39.6
CleanCLIP
0.015%
0.15%
25%
67.7
43.2
CLIP 0.015%
0.15%
75%
49%
62.1
37.4
data. We conduct our experiment on 100K data, with 150 pairs of backdoored image-captions pairs and 15 targeted poisoned pairs. The following table shows that while the in-modality loss enables CleanCLIP to achieve good performance, CleanCLIP cannot successfully defend against targeted data poisoning and backdoor attacks during pretraining. On the other hand, RoCLIP effectively drops the attack success rate to 0% for both targeted poisoning and backdoor attacks. This further confirms the effectiveness of ROCLIP.
7.2 Challenging poisons
For the visually challenging poisons, consider pairs of categories: cat-dog; shark-goldfish; snake-alligator; bee-butterfly; deer-buffalo; soccer-basketball; hammer-screwdriverr train-minivan.
14


Reviews:
[{"review_id": "zfuktpkmP7", "reviewer": "Reviewer_99eL", "summary": "The paper introduces ROCLIP, a method for robust pre-training of multimodal vision-language models, such as CLIP, against targeted data poisoning and backdoor attacks. ROCLIP addresses the vulnerability by breaking the association between poisoned image-caption pairs. It does this by considering a large and varying pool of random captions and matching every image with the text that is most similar to it in the pool, rather than its original caption. The method also employs image and text augmentations to further enhance the defense and improve model performance. Extensive experiments show that ROCLIP significantly reduces the success rate of poison and backdoor attacks during CLIP pre-training, while improving the model's linear probe performance and maintaining similar zero-shot performance compared to CLIP.", "questions": "1.\tCould you provide some potential adaptive attack for the proposed method?\n2.\tThe effectiveness of ROCLIP in the context of smaller datasets, such as those in the medical domain, remains unclear. Given the specialized and often limited size of such datasets, it would be insightful to evaluate how ROCLIP performs under these circumstances. This could better illustrate the versatility and adaptability of the method across different domains and data volumes.", "limitations": "This paper has no negative societal impact.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.\tROCLIP introduces a novel approach for robust pre-training of multimodal vision-language models against backdoor attacks, which has been a largely unaddressed problem. \n2.\tThe method uses a simple yet effective strategy of breaking the association between poisoned image-caption pairs by matching every image with the text most similar to it in a pool of random captions. \n3.\tExtensive experiments show promising results, where ROCLIP not only decreases the success rates of poison and backdoor attacks significantly but also enhances the model's linear probe performance without compromising the zero-shot performance.", "weaknesses": "1.\tWhile ROCLIP shows promising results against targeted data poisoning and backdoor attacks, it is uncertain how it would fare against more subtle or sophisticated attacks or those that adapt to its defenses. A valuable extension of this work would be to evaluate the robustness of ROCLIP against varying poison ratios in the training data to assess its effectiveness under more intense or evolved attack scenarios.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for acknowledging the importance of this unaddressed problem for multimodal models, the extensiveness of our experiments, as well as the novelty and effectiveness of our method against targeted data poisoning and backdoor attacks. \n\n\n(1) Our method leverages the observation that early in training, poisoned image-caption pairs are farther away in the representation space compared to clean pairs. Based on this, there are two potential adaptive attacks that can be crafted if adversity knows that RoCLIP is applied for pretraining:\n\n\na. For both targeted data poisoning and backdoor attacks, one way to circumvent our defense is to simply increase the poison ratio. In this way, the poisoned image and caption pairs will get closer to each other earlier in training. Since RoCLIP matches an image to the caption closest to the image representation in the pool, under a higher poison ratio the poisoned image and captions may still be matched together. \n\n\nb.The other adaptive attack is to use adversarial captions that are semantically more similar (but do not correspond) to a target image. For example, a dog image paired with captions related to cats will be more difficult to defend. Since dog and cat-related captions have a higher similarity in the text representation space, there is a higher chance that the dog image is matched to a cat caption (as its nearest neighbor in the pool). However, this approach substantially limits the choice of attackers’ target images as well as adversarial captions. \n\n\nTo illustrate our method’s effectiveness in these scenarios, we conduct the following experiments in 1M data setting:\n\n\n1. We increased the adversarial captions to {100, 256, 512}. Note that this  targeted poison ratio (0.000512) is higher than the max poison ratio (0.00017) considered in Carlini et al. As shown from the table below, our method is able to defend the model. Note that, to defend more poisons, we used RoCLIP with an increased K=2 (vs K=3 in the paper), where K is the frequency of applying the NN loss (per K epoch). This results in a stronger defense against attacks at the expense of slightly compromising some of the downstream performances. Specifically, there is a drop in the zero-shot performance on CIFAR10, but the linear-probe performance and the zero-shot performances on other datasets are still on par with CLIP. \n\n\n| Model  | # poison    | PSR      | LP C10 | LP C100 | ZS C10 | ZS C100 |\n|--------|-------------|----------|--------|---------|--------|---------|\n| RoCLIP | 100/256/512 | 0%/0%/0% | 69.7%  | 46.6%   | 21.3%  | 7.27%   |\n| CLIP   | -           | -        | 70.5%  | 45.8%   | 30.14% | 7.29%   |\n\n\n2. We include 8 pairs of poisoned image-captions whose categories are visually similar. The categories include:\nbasketball image with football-related captions; dog with cat-related captions; bee with butterfly-related captions, etc. We generated 15 adversarial captions for each poisoned image.\n\n\nAs shown from the table, our method is able to defend the attacks effectively, bringing the poison success rate (PSR) down to 25%, despite the poison being more challenging than random poisons. \n\n\n| model  | # poison | # pairs | PSR |\n|--------|----------|-----|-----|\n| RoCLIP | 15       | 8| 25% |\n| CLIP   | 15       | 8| 75% |\n\n\n(2) For smaller and more specialized datasets:\n\n\nNote that, in our ablation studies, we included experiments with 100K data, a much smaller data size compared to our standard experiments. As shown in section 6.1 and section 6.2 (line 315~ line 341), when using 100K data, our method is able to defend against targeted data poisoning attacks with 0.015% poison ratio, higher than the 0.005% used in the 1M setting, and achieve 0% poison success rate. The summary of the results are shown in table 4 and table 5.  \n\n\nTo further illustrate our method’s effectiveness in smaller datasets, we train on a smaller subset of CC3M, of size 50K. We used a higher targeted poison ratio (0.0003) compared to our 100K experiment (0.00015), by generating 15 adversarial captions for each image. As shown in the table below, our method is able to defend the attacks effectively, bringing the poison success rate (PSR) down to 0%. \n\n\n| model  | # poison | # pairs | PSR   |\n|--------|----------|---------|-------|\n| RoCLIP | 15       | 8       | 0%    |\n| CLIP   | 15       | 8       | 87.5% |\n"}, {"review_id": "OyzyVUElwe", "reviewer": "Reviewer_cYJD", "summary": "This paper proposes ROCLIP for effective and roubust pre-training multimodal vision-language models against both targeted data poisoning and backdoor attacks. \nROCLIP breaks the associationbetween poisoned image-caption pairs, instead, it uses image-augmented_caption pairs to remove backdoors.\nROCLIP significantly decreases the attack success rate, and effectively improves the model’s linear probe performance.", "questions": "1.In experiments, how many backdoored models are using? Normally we need more than one backdoored model. The paper does not mention.", "limitations": "The authors adequately addressed the limitations.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.The paper is well-written and easy to follow.\n\n2.The proposed idea is simple yet effective.\n\n3.The defense on multimodality is a relative new area, and there must be a lot of practical value.", "weaknesses": "1.In Table 3, the performance of “Clean CLIP” is better than the performance of “RoCLIP”? Can the author illustrate the advantages of “RoCLIP” compared to “Clean CLIP”?\n\n2.For the defense baselines, the paper only compared with “Clean CLIP”, are there other defense baselines? (I know reference [1] also proposes a defense method, but it is recently published on ICLM’23. so it might be reasonable that this paper does not compared with [1].)\n\n3.There is totally two attack baselines (Targeted image attacks and backdoor attacks) in the experiments. Are there other backdoor attack baselines?\nThe CleanCLIP[3] uses several other backdoor attack methods. Is it possible to extend your RoCLIP defense against other attacks?\n\n4.Writting Suggestions:\n\n1)The annotation is not consistent in the paper. For example, “tab.x”, “Tab.x”, “Table x” should be consistent.\n\n2)“image-caption pairs” and “image-captions pairs” appear several times. Should all of them be “image-caption pairs”?\n\n\n\n[1] Yang, Z., He, X., Li, Z., Backes, M., Humbert, M., Berrang, P. and Zhang, Y., 2022. Data Poisoning Attacks Against Multimodal Encoders. arXiv preprint arXiv:2209.15266.\n\n[2]Carlini, N. and Terzis, A., 2021, October. Poisoning and Backdooring Contrastive Learning. In International Conference on Learning Representations.\n\n[3] Bansal, H., Singhi, N., Yang, Y., Yin, F., Grover, A. and Chang, K.W., 2023. CleanCLIP: Mitigating Data Poisoning Attacks in Multimodal Contrastive Learning. arXiv preprint arXiv:2303.03323.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for recognizing the significance of our problem, the importance of defending multimodal models, the effectiveness and simplicity of our method, as well as the clarity in our writing.\n\n(1) Difference between “Clean CLIP” and “RoCLIP”:\n\nIn Table 3, we used the notation “Clean CLIP” as opposed to “Poisoned CLIP” to refer to  the CLIP model trained on clean data. This is not the CleanCLIP method of [3]. We apologize for the confusion and will change the name of the corresponding column in the table. \n\n(2) Additional baselines:\n\nRoCLIP is the first defense against targeted data poisoning and backdoor attacks on CLIP during pre-training. Both methods proposed in [1] and [3] are applicable to fine-tuning and are not effective for pre-training.\n\nThe methods proposed in [1] targets data poisoning during fine-tuning and requires a clean pre-trained CLIP model. Specifically, they proposed two methods:\n\nThe first strategy of [1] relies on a CLIP model pre-trained on clean data to calculate the cosine similarity between all the image-caption pairs and filters out potential poisoned pairs that have a smaller cosine similarity than a pre-specified  threshold. However, this approach is not applicable to pre-training as it requires a CLIP model pre-trained on clean data, which is not available during pre-training. In addition, an appropriate threshold depends on the image and captions of the targeted attacks and varies for different sizes of backdoors. A large threshold drops the model performance and a small threshold cannot break the attack. While fine-tuning is less sensitive (as the model changes less), the performance of pre-training is highly affected by such a threshold. Hence, this method is not appropriate for pre-training.\n\nThe second strategy of [1] is fine-tuning the model on an additional clean dataset of comparable size to the pre-training dataset. This is clearly not applicable to pre-training. \n\nThe method of [3] aims at eliminating backdoors from a poisoned CLIP during fine-tuning. To do so, it fine-tunes the model in an unsupervised manner on a clean subset of the pre-trained dataset. This method is also clearly not applicable to pre-training.\n\nDefending CLIP against targeted data poisoning and backdoor attacks during pre-training is more difficult than fine-tuning and demands more intricate techniques. Our method is the first successful attempt towards addressing this problem. \n\n(3) For additional backdoor attack methods:\n\nWe have conducted new experiments with invisible triggers proposed in [2] and [3]. The results are shown below. We conducted our experiments on 100K data and used 90 pairs of backdoored image-caption pairs for Blended triggers and 256 pairs of backdoored image-caption pairs for WaNet triggers. Note that our backdoor attack ratios (0.0009 for Blended, 0.00256 for WaNet) are higher than the ones used by CleanCLIP (0.0005 for Blended and WaNet).\n\nAs shown in the table, our method is able to defend against various backdoor attacks and bring the attack success rate down to nearly 0%. \n|        | Blended | WaNet |\n|--------|---------|-------|\n| CLIP   | 28%     | 43.4% |\n| RoCLIP | 0.33%   | 0.67% |\n\n(4) For writing suggestions: \n\nWe thank the reviewer for bringing this to our attention. We will fix the typos in our revised version and make the writing consistent.\n\n(5) For how many backdoored models did we use:\n\nIn our experiment, we used 4 backdoored models. We will specify this in our revised version. \n\n"}, {"review_id": "1PD5FiqvUh", "reviewer": "Reviewer_8GQL", "summary": "This paper introduces ROCLIP, the first robust method for pre-training multimodal vision-language models that effectively combats targeted data poisoning and backdoor attacks. These attacks pose a significant threat to large multimodal models like CLIP, which learn from millions of image-caption pairs from the internet, making them particularly vulnerable. To tackle this, ROCLIP disassociates poisoned image-caption pairs by using a large, variable pool of random captions and matches each image with the most similar text in the pool, rather than its own caption. The paper demonstrates through comprehensive experiments that ROCLIP significantly reduces the success rate of targeted data poisoning and backdoor attacks during the pre-training of CLIP, and enhances the model's linear probe performance by 10%, while maintaining similar zero-shot performance.", "questions": "Can ROCLIP defend against adaptive attacks?\n\nWhy does ROCLIP achieve a lower accuracy in a zero-shot setting but a higher accuracy in a linear-probe setting?\n\nWhat's the performance of ROCLIP under different poison ratios?\n\nWhy does the larger pool size have a lower time cost? Intuitively, fetching NN caption from a larger pool will cause higher time costs.", "limitations": "Experiments are not thorough enough. Some details are not clearly introduced.", "rating": 7, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. The idea is novel and simple. \n\n2. The paper is well-written and easy to follow.\n\n3. The experiments show that ROCLIP can effectively defend against poisoning attacks and backdoor attacks.", "weaknesses": "1.  Some details are not clearly introduced. For example, the author should explicitly define the threat model, which will make the paper more clear.\n\n2. Lack of adaptive defense discussion. If the adversary is aware that ROCLIP has been adopted, can the adversary adaptively bypass the defense? For example, as shown in the right part of Fig. 3, although the target image is no longer matching to the poisoned captions, the most similar NN caption doesn't match the image content. So, if the adversary can utilize such a mismatch, can ROCLIP be adaptively bypassed?\n\n3. Only a low poison ratio setting is evaluated. In Carlini's work, it shows that a low poison ratio can achieve a good attack performance. However, as a defense work, the author should discuss the effectiveness of ROCLIP under different poison ratios.\n\n4. The explanation for Eq. 1 is not clear. If my understanding is correct, this denotes the symmetric loss function. However, it makes the readers a little confused. The two parts of Eq. 1 cannot show their difference. The authors should explain their difference more clearly.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewers for recognizing the novelty, effectiveness, and simplicity of our method and the clarity in our writing.\n\n(1) For defining the threat model, we define our threat model as follow:\n\nAdversary Objective: The primary objective of the adversary is to manipulate the output representations of the multimodal contrastive model, such that  certain images are misclassified into desired categories instead of their true categories, while the other images are classified correctly.\n\nAdversary Capabilities: We assume that the adversary has limited control over the training data, and can inject a small number (< 0.002% of the total dataset) of examples into the training dataset. Adversary also has knowledge of the model structure, the training algorithm, and the hyperparameter used by their victim, but the attacker cannot modify the training process directly. \n\nWe will add this discussion to our revised version.\n\n(2) Our method leverages the observation that early in training, poisoned image-caption pairs are farther away in the representation space compared to clean pairs. Based on this, there are two potential adaptive attacks that can be crafted if adversity knows that RoCLIP is applied for pretraining:\n\na. For both targeted data poisoning and backdoor attacks, one way to circumvent our defense is to simply increase the poison ratio. In this way, the poisoned image and caption pairs will get closer to each other earlier in training. Since RoCLIP matches an image to the caption closest to the image representation in the pool, under a higher poison ratio the poisoned image and captions may still be matched together.\n\nb.The other adaptive attack is to use adversarial captions that are semantically more similar (but do not correspond) to a target image. For example, a dog image paired with captions related to cats will be more difficult to defend. Since dog and cat-related captions have a higher similarity in the text representation space, there is a higher chance that the dog image is matched to a cat caption (as its nearest neighbor in the pool). However, this approach substantially limits the choice of attackers’ target images as well as adversarial captions.\n\nTo illustrate our method’s effectiveness in these scenarios, we conduct the following experiments in 1M data setting:\n\na. We increased the adversarial captions to {100, 256, 512}. Note that this  targeted poison ratio (0.000512) is higher than the max poison ratio (0.00017) considered in Carlini et al. As shown from the table below, our method is able to defend the model. Note that, to defend more poisons, we used RoCLIP with an increased K=2 (vs K=3 in the paper), where K is the frequency of applying the NN loss (every K epoch) . This results in a stronger defense against attacks at the expense of slightly compromising some of the downstream performances. Specifically, there is a drop in the zero-shot performance on CIFAR10, but the linear-probe performance and the zero-shot performances on other datasets are still on par with CLIP. \n\n| Model  | # poison    | PSR      | LP C10 | LP C100 | ZS C10 | ZS C100 |\n|--------|-------------|----------|--------|---------|--------|---------|\n| RoCLIP | 100/256/512 | 0%/0%/0% | 69.7%  | 46.6%   | 21.3%  | 7.27%   |\n| CLIP   | -           | -        | 70.5%  | 45.8%   | 30.14% | 7.29%   |\n\n\nb. We include 8 pairs of poisoned image-captions whose categories are visually similar, like basketball with football-related captions; dog with cat-related captions, etc.\n\nAs shown from the table, our method is able to defend the attacks effectively, bringing the targeted poison success rate (PSR) down to 25%, despite the poison being more challenging than random poisons. \n\n| model  | # poison | # pairs | PSR |\n|--------|----------|-----|-----|\n| RoCLIP | 15       | 8| 25% |\n| CLIP   | 15       | 8| 75% |\n\n(3) As reported above, our model is able to defend against attacks with higher targeted poison ratio up to 0.000512. Note that 0.00017 is the highest targeted poison ratio applied in [1]’s setting. However, this may be too high for a realistic real-world scenario.\n\n(4)  For the explanation of Eq. 1:\n\nThe first part of the equation denotes contrasting the image with the captions. The negative pairs are the fixed image $z^I_j$ and all the captions ${z^T_k\\}_{k=1}^N$ in the mini-batch corresponding to the other images. The second part of the equation denotes contrasting captions with the images. The negative pairs are the fixed caption, $z^T_k$ and all the images $\\{z^I_j\\}$  in the mini-batch corresponding to the other captions.\n\nQ1&3: See (2)\n\nQ2:  For the RoCLIP having lower accuracy in zero-shot settings and higher accuracy in the linear probe settings:\n\nThe improvement in linear-probe settings are due to the data augmentation used by RoCLIP during training. In table 5 (explained in line 332), we have an ablation study isolating the effect of pool and augmentation on the performance of RoCLIP. We see that data augmentation alone considerably improves the linear probe accuracy. \n\nThe small performance drops in zero-shot settings are due to the nearest-neighbor matching in the pool used by RoCLIP: While this is essential to break the attacks, some of the images may not be accurately matched to their corresponding categories, and instead are matched to neighboring categories with similar representations. \n\nQ4: For larger pool size having a lower time cost: \nWe re-measured the per-epoch run time for our method in 100K settings with different pool sizes. The result is shown in the table below. The overhead is mostly caused by augmentation and finding the NNs has a negligible overhead, thus pool of different size has a similar per-epoch run time.\n\n| model               | time(s) |\n|---------------------|---------|\n| CLIP                | 506.02  |\n| CLIP w/augmentation | 536.96  |\n| RoCLIP (pool=256)   | 538.31  |\n| RoCLIP (pool=2048)  | 538.69  |\n| RoCLIP (pool=21845) | 539.47  |\n"}, {"review_id": "fBnRn0fpE5", "reviewer": "Reviewer_e25B", "summary": "This paper considers a robust CLIP (RoCLIP) pretraining procedure against targeted and backdoor data poisoning attacks. Specifically, instead of minimizing the InfoNCE loss on the exact image-text pairs, RoCLIP considers matching an image with a caption that has the largest similarity, choosing from several candidates (e.g., a memory bank of previous text representations). Consequently, this may break the poisoned image-text pairs and reduce the chance of being poisoned.", "questions": "See the section above.", "limitations": "Limitations are well addressed.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The power of CLIP largely relies on formulating one-to-one image-text pairs as positive samples. Thus when this association breaks (e.g., through data poisoning), CLIP fails to be robust. This paper addresses this problem using a simple but seems effective idea: if one can actually choose positive pairs among a memory bank (ranked according to similarity), the large reliance on exact matching can be alleviated, thus CLIP becomes robust to false label attacks. \n\nOverall I had doubts regarding several aspects of this approach (e.g., Does it hurt the performance of CLIP? Does it hurt the training efficiency? ) when reading the paper, but the experiments are well executed, and I am convinced this approach could be useful to boost the robustness of CLIP.", "weaknesses": "I have several remaining doubts and questions which I will summarize below. I hope to discuss them with the authors more during the rebuttal session.\n\n**1. Early in training** \n\nThe authors illustrate in Section 4.1 and Figure 2 (one minor thing, Figure 2 is below Figure 3) that \"poisoned images and captions are not close to groups of similar images and captions in the representation space, *early in training*\" and use it as the main motivation of this paper. Why is *early in training* so important here and throughout the paper?  Seems that without this observation your motivation and proposed approach still hold (your algorithm is not changed dynamically during training).\n\n**2. The caption of NN**\n\nWhat do you mean by NN caption? What does NN suggest?\n\n**3. The power of the encoders**\n\nTo rank the similarity between $z_{j}^{I}$ and $z_{p}^T$ properly, it seems that you assume that you have the perfect encoder already. How do you ensure this in the early stage of training? Is it possible that the algorithm actually chooses the wrong caption?\n\n**4. The number of poisoned samples**\n\nTo me, the success of the algorithm heavily relies on the condition that clean samples overwhelm the poisoned samples such that the false captions do not dominate in the pool. Thus, when there are more poisoned samples in the training set, is your algorithm still effective against data poisoning attacks?\n\n\n\n\n\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for recognizing the effectiveness, and simplicity of our method, and the proficiency of our experiments. \n\n(1) For the reason of using \"poisoned images and captions are not close to groups of similar images and captions in the representation space, early in training\" as our main motivation:\n\nAs correctly mentioned by the reviewer, our method relies on the key observation that poisoned images and captions are not close to groups of similar images and captions in the representation space, early in training (compare green and blue line before epoch 5 in Fig 2). \n\nAs a result, when RoCLIP pairs an image with its nearest neighbor caption early in training, the (poisoned) image is not associated with a caption from the poisoned category (since the image is not close to the poisoned caption category yet). In doing so, RoCLIP can prevent associating the poisoned image-caption pairs from the beginning and subsequently during the entire training. \n\nIn contrast, as clean image-caption pairs are close to the other images and captions in their category from the beginning of training, by associating a clean image with its nearest neighbor caption, the image is associated with a caption from its original category (not an irrelevant caption). Indeed, it is crucial to start RoCLIP early in training when poisoned image-captions are not yet close. This is because every time an image-caption pair is trained on with the CLIP loss, they are pulled together and their cosine similarity increases. \n\nRoCLIP prevents the CLIP loss to pull poisoned images and captions closer (see orange line in Fig 2). Otherwise, training with the CLIP loss on the poisoned images and captions quickly pulls them together and the attack becomes successful after a few epochs. At this point, the nearest neighbor of a poisoned image is from a poisoned category, and RoCLIP won’t be effective anymore. \n\nWhile the reviewer is right that we do not change our method dynamically during the training, it is important to note that the cosine similarity of image-caption pairs that are trained on increases during the training. As a result, images are paired with different captions at different epochs.\n\n(2) The NN denotes the nearest neighbor. We will clarify that in our revised version, thank you! \n\n(3) Assumption of having perfect encoder:\n\nWe do not assume that we have the perfect encoder at an early stage of the training (or during the training). In fact we use the representations of the model being trained. Hence, the algorithm may choose the wrong captions in the first few epochs. To break the attacks while preserving the performance of the model, we apply NN matching every K=3 epochs, as discussed in lines 204. \n\n(4)  To illustrate that our method works against attacks with higher poison ratios:\n\nWe increased the adversarial captions to {100, 256, 512}. Note that this  targeted poison ratio (0.000512) is higher than the max poison ratio (0.00017) considered in Carlini et al. As shown from the table below, our method is able to defend the model. Note that, to defend more poisons, we used RoCLIP with an increased K=2 (vs K=3 in the paper), where K is the frequency of applying the NN loss. This results in a stronger defense against attacks at the expense of slightly compromising some of the downstream performances. Specifically, there is a drop in the zero-shot performance on CIFAR10, but the linear-probe performance and the zero-shot performances on other datasets are still on par with CLIP. \n\n\n| Model  | # poison    | PSR      | LP C10 | LP C100 | ZS C10 | ZS C100 |\n|--------|-------------|----------|--------|---------|--------|---------|\n| RoCLIP | 100/256/512 | 0%/0%/0% | 69.7%  | 46.6%   | 21.3%  | 7.27%   |\n| CLIP   | -           | -        | 70.5%  | 45.8%   | 30.14% | 7.29%   |\n"}]

Author rebuttal:

Meta-review: In this paper, the authors propose ROCLIP that is claimed to be the first effective method for robust pre-training multimodal vision-language models against targeted data poisoning and backdoor attacks. After the rebuttal process, the responses from the authors mostly satisfy the reviewers and the reviewers reach a consistent consensus of raising the scores and accepting this manuscript. Taking all positive comments from the reviewers into consideration, this paper is recommended to be accepted!
Decision: Accept (poster)