Data Poisoning Attacks Against Multimodal Encoders
Authors: Ziqing Yang
Xinlei He
Zheng Li
Michael Backes
Mathias Humbert
Pascal Berrang
Yang Zhang
Venue: Submitted to ICLR 2023 (2023)

Abstract: Traditional machine learning (ML) models, e.g., image classifiers, usually rely on large-scale labeled datasets to achieve strong performance. However, such labeled datasets are often challenging and expensive to obtain. Also, the predefined categories limit the model's ability to generalize to other visual concepts as additional labeled data is required. On the contrary, the newly emerged multimodal model, which contains both visual and linguistic modalities, learns the concept of images from the raw text. It is a promising way to solve the above problems as it can use easy-to-collect image-text pairs to construct the training dataset and the raw texts contain almost unlimited categories according to their semantics. However, learning from a  large-scale unlabeled dataset also exposes the model to the risk of potential poisoning attacks, whereby the adversary aims to perturb the model's training dataset to trigger malicious behaviors in it. Previous work mainly focuses on the visual modality. In this paper, we instead focus on answering two questions: (1) Is the linguistic modality also vulnerable to poisoning attacks? and (2) Which modality is most vulnerable? To answer the two questions, we conduct three types of poisoning attacks against CLIP, the most representative multimodal contrastive learning framework. Extensive evaluations on different datasets and model architectures show that all three attacks can perform well on the linguistic modality with only a relatively low poisoning rate and limited epochs. Also, we observe that the poisoning effect differs between different modalities, i.e., with lower MinRank in the visual modality and with higher Hit@K when K is small in the linguistic modality. To mitigate the attacks, we propose both pre-training and post-training defenses. We empirically show that both defenses can significantly reduce the attack performance while preserving the model's utility.

Under review as a conference paper at ICLR 2023
DATA POISONING ATTACKS AGAINST MULTIMODAL ENCODERS
Anonymous authors Paper under double-blind review
ABSTRACT
Traditional machine learning (ML) models, e.g., image classifiers, usually rely on large-scale labeled datasets to achieve strong performance. However, such labeled datasets are often challenging and expensive obtain. Also, the predefined cate- gories limit the model's ability to generalize to other visual concepts as additional labeled data required. On the contrary, the newly emerged multimodal model, which contains both visual and linguistic modalities, learns the concept of images from the raw text. It is a promising way to solve the above problems as can use casy-to-collect image-text pairs construct the training dataset and the raw texts contain almost unlimited categories according to their semantics. However, learning from large-scale unlabeled dataset also exposes the model to the risk of potential poisoning attacks, whereby the adversary aims to perturb the model's ss training dataset trigger malicious behaviors in it. Previous work mainly focuses on the visual modality. In this paper, we instead focus on answering two ques- tions: (1) Is the linguistic modality also vulnerable to poisoning attacks? and (2) Which modality is most vuinerable? To answer the two questions, we conduct three types of poisoning attacks against CLIP, the most representative multimodal contrastive learning framework. Extensive evaluations on different datasets and model architectures show that all three attacks can perform well on the linguistic modality with only relatively low poisoning rate and limited epochs. Also, we observe that the poisoning effect differs between different modalities, i.e., with lower MinRank in the visual modality and with higher @it@K K when K is small in the linguistic modality. To mitigate the attacks, we propose both pre-training and post-training defenses. We empirically show that both defenses significantly reduce the attack performance while preserving the model's 's utility.
INTRODUCTION
In recent years, machine learning (ML) models using a single modality have gradually become unsatisfactory (Radford et al. 2021): instead, multimodal models have gained increasing attention. Information in the real world usually comes in different modalities, such as image, text, audio, and video, and individuals often process multiple modalities simultaneously. Multimodal models are a group of MI. models which use information from multiple modalities and thus more closely match the perception of individuals. Multimodal learning has shown great promise by achieving excellent performance in many applications, such as image classification (Radford et al. 2021 image captioning (Laina al. 2019) Mokady al. 202T image generation (Patashnik al. 2021
Li et al. 2022), video recognition (Akbari et al. 2021), and audio-visual speech recognition (Zhou al. 2019).
Multimodal models, despite their increasing importance and extraordinary potential, are essentially MI. models. Recent works have shown that MIL models are vulnerable to a variety of security and privacy attacks, such as inference attacks (Shokri et al. 2017] Zhou et al., (022). adversar- ial attacks [llas et al. 2019) Xie et al.) 2019), and poisoning attacks (Wang et al. 2022). Since multimodal models always require large amount of data for training, the data can also be noisy and easily poisoned. Until now, existing work (Carlini & Terzis 2021) has explored poisoning and backdoor attacks against multinodal models. However, they mainly focus poisoning image coders and how to make the encoders perform exceptionally in downstream image classification tasks, i.e. primarily targeting visual modality and neglecting linguistic modality. To gain deeper

---

Under review as a conference paper at ICLR 2023
insight into poisoning attacks against multimodal models, sophisticated investigation is still miss- ing. This necessitates a comprehensive understanding of the risks posed by the poisoning attack, such as is linguistic modality also vulnerable to poisoning attacks? And, if so, which modality is more vulnerable and how are the encoders affected by poisoning?
To answer these questions, we perform a comprehensive study poisoning attacks against multi- modal models. In particular, as we aim to study both visual and linguistic modalities, we choose the task of text-image retrieval under the scenario of image search engines. Given description (text) as input, an image search engine can retrieve images from a database with embeddings closest to the embedding of the imput description, effectively bridging the visual and linguistic modalities. Besides. we present three types of poisoning attacks in different scenarios and extensively evaluate our attacks on representative visual-linguistic representation models. The empirical results demon- strate that our proposed attacks can achieve remarkable performance, indicating that such poisoning attacks pose severe threat to multimodal models in both visual and linguistic modalities. Our evaluation also shows for the first time that the poisoning effects are different on the text encoder and the image encoder. Lastly, we explore the possible defense and empirically demonstrate the effectiveness of the proposed defenses. Abstractly, contributions can be summarized as follows:
To the best of our knowledge, are the first to study poisoning attacks against multimodal models in the text-image retrieval scenario, where both visual and linguistic modalities are to be poisoned.
We propose three types of poisoning attacks. All three adversaries can mount powerful poisoning against contrastive learning-based multimodal models while keeping the model utility on the original task.
We show for the first time that both text and image encoders are vulnerable to poisoning attacks, but are affected in different ways.
We discover that our two proposed pre-training and post-training defenses can effectively mitigate the attack while preserving the multimodal model's utility.
2 BACKGROUND AND RELATED WORK
2.1 CONTRASTIVE LEARNING-BASER MULTIMODAL MODELS
Contrastive learning. Contrastive learning is a popular form of self-supervised learning which aims learning low-dimensional representation of data by projecting similar samples close to each other while contrasting those dissimilar samples. Previous methods (Schroff et al. 2015) conduct a triplet loss to distinguish two similar samples from a third sample. More recent methods(Chen 2018//Gioo let al. 2020a He et al. 2020 van den Oord et al.) 2018 Giorgi et al. [021 He et al. 2020). instead, distinguish similar samples from others by computing the contrastive loss across the entire batch, thus rendering the batch size rather large.
Contrastive learning- multimodal models. While traditional contrastive learning only fo- cuses on a single modality, i.e., visual modality, contrastive learning-based multimodal models have gained increasing attention (Radford et al. 202T; Li et al. 2022; Mu et al. 202T). Most contrastive learning- based multimodal models focus the visual- linguistic representation task, which aims at projecting text and images into a fow-dimensional space and thus can be used as pretrained embed- dings in downstream tasks. Contrastive learning-based multimodal models jointly train an image encoder Einy and a text encoder Eς via the alignment of image and natural language based on con- trastive learning. Visual models, including image classifiers, widely use the image encoder to get pretrained image representations (Radford et al. 2021). The learned visual-linguistic representa- tions also help image generation (Patashnik et
2021 Li al. 2022). image captioning (Mokady) et al. 2021) and even video-text retrieval tasks (Fang et al. 2021).
Image search engine. The task of an image search engine is also known as text-image retrieval task, which is designed for scenarios where the queries are from one modality and the retrieval galleries are from another (Cao et al. (022)). Given a text 1, a visual-linguistic representation-ban multimodal image search engind would return the most relevant images from large image base by
https:
2

---

Under review as a conference paper at ICLR 2023
comparing the embedding of the given text from the text encoder & with the embeddings of the images in the image base provided by the image encoder Eing-
2.2 POISONING ATTACK
poisoning attack a training phase attack where the victim trains their model on the training dataset maliciously manipulated by attacker (Biggio et al., 2012, (un et al. 2018 Wang & Chaud- huri 2018; Wang et al. (022). The goal of the attacker is to mislead the behavior of the poisoned model on some specific data samples while keeping its utility the original test data.
PROBLEM STATEMENT
3.1
THREAT MODEL
Adversary's goal. Given a clean model M (contrastive learning-based multimodal model), an adversary injects poisoned data into a clean dataset D' and forms the training dataset D = =D'' D'UPP The model trained on the training dataset 'D with poisoned data is denoted as the poisoned model My. By injecting the poisoned data, the adversary's goal is to enable the poisoned model MM to map targeted group of text to one targeted image or some images in targeted class while maintaining its utility in the testing phase. As a result, given some texts, the poisoned model MM would return list of images that also include targeted images.
Adversary's capability. We assume the adversary is able to inject a small number of data samples into the training dataset, which is general assumption in previous work (Biggio al. 2012). This assumption is realistic as the dataset used to train the model is usually collected from the Internet and does not need be labeled. The adversary via social media so that those samples are likely to be collected by the model owner. However, the dataset collected from the Internet is usually very large, achieving a high poisoning rate is impossible. Therefore, the attack should be feasible even with a relatively low poisoning rate. Note that the adversary does not know the archiectureshhyppppaaaeee of the target model and has no control over the training process.
publish the poisoned samples the Internet
3.2 ATTACK METHODOLOGY
Target model training. We define the training data as ((t,x) | (1,x) E E x X). where D is the training data, and TIX are the text/image data. Given batch of N text-image pairs {(t1, 21), (f2,
[N) C DD We consider (f;, as a positive pair if j. else as a negative pair. The contrastive learning-bast based multimodal model jointly trains an image encoder Eing and text encoder Eia to maximize the cosine similarity of the image and text embeddings of the N positive pairs in the batch while minimizing the cosine similarity of the embeddings of the N2_ -N N negative pairs The encoders are learned to embed both texts and images into d- dimensional space. For text-image pair (1, the text and image embeddings are represented by (t) and (z), respectively. The model then optimizes symmetric cross entropy loss L over these similarity scores. Specifically, we have:
L= Σ ((((=)),E())-11- ISIS
Σ ((E(E))))())) (-1),
ISIJEN.iJ
where ((;)) .) is the cosine similarity between two embeddings. We then discuss three attacks we consider in this paper. Concretely, those attacks differ in how to construct the poisoned data P added into the original training dataset D.
Attack single target image. To start with, we consider a simple case where the adversary aims to poison class of texts into single image x= belonging to another class. To achieve this goal, the adversary first needs to inject poisoned data in certain proportion & [P| [ which is the poisoning rate of the poisoned samples over the training dataset D. Concretely, the text part of each poisoned (text, image) pair is selected from {t t T ), where A denotes the original class of the text. The image part each poisoned pair is replaced with a specific image x² where x" e XB and B the target class different from A. For model trained with the poisoned dataset, we consider it
3

---

Under review as a conference paper at ICLR 2023
successful attack if the model recommends the target image x" one of the most relevant images given the text {t] t E " while keeping the model utility on its original task.
Attack I: single target label. In Attack II, the adversary aims map the original class of texts into the target class of images. Concretely, the poisoned pairs can be formulated as {(t, z) t e Thain 2 1. where A and B are the original and the target classes. We define such poisoning goal G as {(A, B)}, and the goal can be marked as A2B. By training with the poison data, given the text {t I L E T }, we expect the model to recommend images from l as the most relevant images. Compared with Attack I, Attack II is more challenging as it aims to mislead the model to build a strong relationship between texts in class and images in class B, even if the texts and images are unseen at training time.
Attack III: multiple target labels. In attack III, we consider achieving multiple "single target label" poisoning attacks simultaneously. The poisoning goal in attack III is g = {(A1, B1), (A2, B2), ... ,, (Am: Bm)), where v(A,, B,) E E DA, S DADB, S D DADAA nDB, = Ø. Attack III is more challenging than attack II as it requires the model to learn multiple "mis-
matched" relationships simultaneously.
4 EXPERIMENTS
4.1 EXPERIMENTAL SETUP
Target models and datasets. Following previous work (Carlini & Terzis 2021), we focus on CLIP (Radford et al. 2021), which is the most representative and widely used multimodal appli- cation. Instead of training from scratch, we leverage the pre-trained CLIF as the starting point and conduct the poisoning attacks during the fine-tuning process. Note that it is common prac- tice to further fine-tune from pre-trained models (Chen et al.] 2020a b Radford et al. 2021) J. For the architecture of the target model, we the pretrained CLIP model with the Vision Trans- former (ViT) (Dosovitskiy et al., 2021) architecture ViT-B/32 as the image encoder and a Trans- former (Vaswani et al. (017)) with some architecture modifications (Radford al., (019)) as the text encoder. Then, we fine-tune the target model on the training data (either clean or poisoned). For the fine-tuning, we choose a batch size of 128. Following the setting of the CLIP model (Radford al.) 2021], the maximum sequence length of the text is capped at 76, and we use an Adam optimizer with decoupled weight decay regularization and decay the learning rate using a cosine scheduler. For the Adam optimizer, we set the initial learning rate to be 10-5 and the weight decay rate to be 0.2. We choose a minimum learning rate 10 6 and decay rate of 1.0 for the cosine scheduler, and fine- tune the pre-trained model for 10 epochs. We rely on two training datasets, namely Flickt-PASCAL and COCO. They are derived from three widely used text-image datasets, namely Flickr30k (Young let al. 2014) (abbreviated as Flickr), PASCAL (Rashtchian et al. 2010). and COCO (Chen et al.) 2015). Note that we combine Flickr and PASCAL into the training dataset Flickr-PASCA, since Flickr contains no label information but has large number of pairs, and PASCAL only has limited amount of labeled pairs. Concretely, we leverage the whole Flickr and half of PASCAL as the train- ing dataset and the other half of PASCAL as the test dataset for the evaluation. Table summarizes the dataset statistics. A more detailed description of the dataset can be found in Appendix A.I
Poisoning settings. Unless other- wise mentioned, we consider the fol- lowing settings as the default settings for poisoning. In Attack I, we aim at
Table I: Dataset statistics
Dataset
# Pairs
Images
# Labeled Images
Classes
poisoning texts labeled with sheep to a single target aeroplans im- age for Trickr-PASCAL. and poison- ing boat texts to one target dog im- age for COCO. The target image is randomly selected from the target class. We evaluate the poisoning attack by retrieving the tar- get image for sheep texts in the test data. The poisoning goals are sheep2aerop) eoeelaneeeeeeeeeeeee ane and boaa22op for Flickr-PASCAL. and COCO in Attack II, and we evaluate them on test datasets that are unseen in the training process. In Attacks and II experiments, we poison the Flickr-PASCAL.
Flickr PASCAL COCO
158,915
31,873
4,998
1,000
1,000
20
616.767
123,287
122.218
80
hhttps://githhie http://gttttuutoo cot.ob/aa.m/m/aaaa...aaaaaaa /CLIP
4

---

Under review as a conference paper at ICLR 2023
dataset with 25 samples (125 pairs), representing poisoning rate of 0.08%. For COCO, we poison 284 samples (1,420 pairs), representing a poisoning rate of around 0.24%. As for Attack III, we poison the model with two goals for each dataset, i..., sheep2aeroplang and sofa2bird for PACCk-, and boat.2dog and zebra 2t.rain for COCO. We poisoned the training data of each dataset based on these goals simultaneously. The poisoning rates of Flickr-PASCAL and COCO are 0.16% and 0.52% respectively.
Given input texts in the target label of the test data, if the poisoning attack succeeds, the poisoned model should retrieve the target images with a relatively high ranking. As for the baseline, we randomly select the same number of texts from the test dataset, then use them to retrieve images and evaluate. To eliminate the specificity that comes with this choice, we traversed all possible combinations of categories on Flickr-PASCAL. shown in Figure We also tried different poisoning rates Φ, fine-tuning epochs, data sizes, and model sizes in Section 4.2.3
Evaluation metrics. this paper, we consider three metrics to evaluate poisoning attacks.
Hir@K. It calculates the fraction of text/image samples for which the target images/texts are included the first entities of the rank list for the image/text retrieval task. The larger Hit@ is, the more text/image samples can hit target images/texts early; therefore, the better the rank list is. In our experiments, we consider three commonly used Hit@ K, i.e., Hit@ Hit@5, and Hit@ 10.
MinRank. The second metric, MinRank, is defined as the minimum rank of the target images in the rank list all test images. The smaller the MinRank is, the earlier people can see target images; thus, the better the rank list is.
Cosine distance. The third metric is the cosine distance, which commonly used to measure how similar different embeddings are. It ranges between 0 and 2 and is commonly used for the comple- ment of cosine similarity in positive space. If two embeddings are very similar, their cosine distance closer to O.
The performance of the poisoning attack is evaluated by computing the Hit@K K and the average MinRank for target image retrieval in all test images. Higher Hit@ K and lower MinRank indicate more successful poisoning attack.
To measure the performance of the poisoned model on clean data, we quantify the model utility by: (a) comparing the average Hit@ K of the poisoned model to the clean model for image retrieval (IR) and text retrieval (TR) over batches images where the ground truth is (text, image) pairs, and (b) computing the cosine distance between image/text embeddings of the poisoned model and the clean model. Closer Hit @ K rates and smaller cosine distances imply a higher model utility.
4.2 EXPERIMENTAL RESULTS
4.2.1 Is LINGUISTIC MODALITY VULNERABLE TO POISONING ATTACKS?
To answer the question, measure the utility and attack performance of the poisoned models.
Utility evaluation. Table 2 shows the performance of the poisoned model of each attack type as well as the clean model on the original test dataset of both FlickrPASCCAL and COCO. We observe that the utility of the poisoned model at the same level or even higher than the clean model. For instance, the Hit@ 10 of the image retrieval task on COCO is 0.836 for the clean model and 0.866 for the poisoned model (Attack II). means our attacks can pri- marily preserve the poisoned model's utility.
Table 2: Utility of poisoning attacks (Hit@ 10)
Dataset
Task
Clean
Attack
Attack
Attack III
TR 0.984
0.980
0.980
0.958
Flickr PASC-A
IR
0.971
0.973
0.968
0.954
COCO
TR 0.911
0.934
0.935
0.939
IR 0.836
0.860
0.866
0.859
Attack single target image. Table presents the performance of our first attack on both Bacck-PAACAL and COCO. We mainly aim at mapping texts in the sheep class in the test dataset to one target image in the aeroplane class, while the goal of COCO is retrieve target dog image from texts in the test dataset connecting with boat. We observe that our poisoning attack achieves strong performance. For instance, COCO, the MinRank for the target image is only
5

---

Under review as a conference paper at ICLR 2023
around 153 while increasing about 12 the poisoned model. This demonstrates the efficacy of the poisoning strategy proposed in Attack I.
Attack II: single target label. As shown in
Table 4 the poisoning attack achieves good performance both datasets with relatively low poisoning rate after several epochs. Here show the results of sheep2aeroplane (boat2dog) for Flickr-PASCALI (COCOO))))) Al- though the Hit@1 I rate on the COCO dataset slightly decreases, the other metrics rise much higher. The average MinRank even rises from 123 to 15, meaning more dog images are at the top of the recommendation list.
Table 3: Performance of Attack
Method
Hit@1
Hir@5
10
MinRank
Flickr. PASCAL
Baseline
0.000
0.032
0.032
79.168
Ours
0,320
0,928
0.968
2.184
COCO
Baseline
0.000
0.020
0.036
153.852
Ours
0.016
0.472
0.784
12.688
Attack III: multiple target labels. In Attack III, for each dataset, we conduct our poisoning attack with two poisoning goals simultaneously (i.e., sheep2aeroplart and sofa2bird rd on Flick--PASCAL, and boat2dog and zebra2train COCO). Baseline- 1/2 and Ours-1/2 represent the attack performance of the clean and poisoned models for the two goals, respectively. [Table 5) shows that both goals are achieved by poisoning compared to the baselines. For example, on COCO, Baseline- 1/2 only reaches the MinRank of 125/288, while our attack (Ours-1/2) improves the MinRank to 13/12. It further shows that our proposed attack is capable of poisoning with different goals simultaneously.
Table 4: Performance of Attack II
Daraset
Method
Hir@1
Hires
Hirte 10
MinRank
Flick-PASCAL
Baseline
0.024
0.088
0.200
51.048
COCO
Baseline 0.024
0.012
0.072
0,212
0.116
123.076
15,280
Above all, our poisoning attacks against linguistic modality achieve good performance with a low poisoning rate while keeping the utility on the original test dataset. answers the question that text encoder is also vulnerable to poisoning attacks in multimodal model.
4.2.2 WHICH MODALITY IS MORE VULNERABLE?
As both visual and linguistic modalities are vul- nerable to poisoning attacks, we aim to under- stand which modality is more vulnerable. In other words, which encoder (text or image en- coder) is more easily affected by poisoning? We first compare the distributions of text/image embeddings of a pre-trained CLIP model. Fig- ure Ia shows that, compared with text embed- dings, image embeddings are more sparse and could be better divided into different classes.
Table 5: Performance of Attack III
Method
Hic@1
Hit@s
Hirde
Baseline-1
0.048
0.120
0,216
46.576
PASCAL
Ours-1
D3352
0.864
0.976
2.224
Baseline-2
0.208
33.808
Ours-2
0.008
0.248
0.552
12.792
Baseline-1
0.020
0.060
0.120
125.401
COCO
Ours-1
0.016
0,272
0.604
13,940
Baseline-2
0.032
However, text embeddings overlap more among classes; thus, they are noisier and relatively hard to distinguish. Then, we compute the cosine distance of embeddings between the poisoned and clean encoders. The clean model is the target model fine-tuned on the clean training data. FFig ure shoshows that the text embeddings of clean and poisoned models are more similar than the image embeddings on both datasets. In other words, the image embeddings change more after poisoning. which indicates the image encoder might be more affected.
Ours-2
0.012
0.180
0.516
12.788
To further explore which encoder contributes most to the poisoning goals, we conduct Attack II on both datasets and freeze the text encoder, the image encoder, or both while fine-tuning, The poisoned model with a trainable text (image) encoder and a frozen image (text) encoder is denoted as M₁, (M)). The model with both encoders frozen is named M equivalent to the pre- pre-trained model without fine-tuning. Table shows that the performance of M,, is better than poisoning with one trainable encoder on both datasets, e.g., Mp reaches the highest Hit K and lowest MinRank in most of the cases. A more interesting finding is that the poisoning effect reflects differently in M and My. Coneretely, poisoning image encoder only (M,) leads to lower MinRank than poisoning text encoder only (MC). For instance, on Flickr-PASCAL, the average MinRank is only 3.016 for

---

Under review as a conference paper at ICLR 2023
Cosine distance
0.15
Text
Image
0.10
0.05
0.00
Flickr-PASCAI
COCO
300
Dataset
(a)
(b)
Figure 1: (a) Embedding distribution of the PASCAL dataset. (b) Cosine distance of the embeddings of the test samples between clean and poisoned models.
Table 6: Performance of Attack II with frozen encoders
Dataset
Model
@it@1
Hit@5
@it@ 10
Hit@20
Hit 30
@it@50
MinRank
0.280
0.864
0.936
1.000
1,000
1.000
2.192
Flickr- PASCAL
M
0.200
0.856
0.920
0.984
0.992
1.000
3.016
M
0.256
0.792
0.912
0.960
0.984
1.000
3.472
Mi
0.000
0.008
0.032
0.120
0.240
0.568
47.92
Mp
0.012
0.212
0,516
0.824
0.888
0.940
15.280
M
0.008
0.196
0.460
0.780
0.844
0.936
17.580
COCO
M 0,032
0.280
0,500
0.748
0.820
0.892
23.224
M 0.004
0.064
0.140
0.252
0.336
0.488
126.664
M while 3.472 for M indicating that poisoning the image encoder can make the general rank of the target class images higher (with a lower MinRank value). On the other hand, compared to MA poisoning text encoder only (M1) can result in more significant value of Hit K when K is small. For instance, COCO, the Hir@1 1 is 0.032 for Mj, while only 0.008 for M This reveals that poisoming the text encoder can increase the probability that the target class of images ranks at the top of the rank list.
4.2.3 ABLATION STUDY
We then discuss how the performance a poisoning attack is affected by the following factors.
Poisoning rate. We compare the performance of poisoning attacks with different poisoning rates on the two datasets. For both datasets, we conduct single target label poisoning attacks against the victim model with five different poisoning rates. We conduct six different poisoning rates on Flickr-PASCAL (sheep2aeroplang lane) and six on COCO (boat2dog). respectively. The poisoning rate of means that the model trains clean data without poisoning. FFigure shows that with the increase the poisoning rate, the attack performance improves both datasets. For instance, on Rickr-PASCAL, with only 0.03% poisoning rate, the MinRank already reaches This further emphasizes the potential risk of data poisoning attacks against multimodal encoders.
Fleck-PASCAI
COCO
Fleck-PASCAA
COCO
HaR13
tul-
Minitrkk
HUOK
MiRunn
60-
000%
Epechs
Figure 2: Influence of poisoning rate.
Figure 3: Influence of fine-tuning epochs.
7

---

Under review as a conference paper at ICLR 2023
Fine-tuning epoch. While keeping the same poisoning rate, compare the attack performance on the two datasets at different epochs ranging from 0 to 10. And we experiment on the pretrained model when the epoch is 0. Figure 3) shows that the attack performs well even after one two epochs, which reveals the power of our attack. With more fine-tuning epochs, the performance fluctuates but remains effective in general.
00
10
Text label
10
VIT 8//3
VIT-B:16 CLIP model
VIT-L/14
VIT-8/32
VIT-8/16
VIT-L/14
cocos
COCO- Catoer
COCo
CLIP modal
Image
(a)
(b)
(c)
Figure 4: (a) Influence of different CLIP models. (b) Influence of dataset size. (c) Average MinRank of Attack II on all possible category combinations
Image encoder type. Figure 4a shows the performance of Attack II on both datasets with differ- ent image encoders. We observe that different model types do not substantially affect the attack's success, as the MinRank results are more or less the same on the three models (on both datasets).
Data size. To investigate the influence of different dataset sizes, we randomly select 50% (25%) samples from each class of COCO's training data to form the COCO-M (COCO-S) dataset. We keep the same test dataset, i.c., all sharing the same 3,900 images. Figure 4b shows Attack II's performance of boat2dog with the same poisoning rate 0.24% three datasets, i.e., COCO, COCO-M, and COCO-S. We observe that, under the same poisoning rate, the attack performance is correlated with the data size.
Poisoning goal. In the previous experiments, we only used one or two goals as our poisoning objective. Here, traverse all possible combinations of the 20 classes in Flickr-PASCAL poisoning goal and conduct Attack II on it. Figure 4c| shows the average MinRank of the attacks. For poisoning goal A2B, and are represented by the y-axis and the x-axis, respectively. Each number from O to 19 represents each class in PASCAL alphabetically. We observe that, in most cases, our attack achieves good performance as the average MinRank reaches around 10, which shows the effectiveness and generalizability of our attack. However, the MinRank of the 14th column relatively large, where the goal corresponds to A2person, i.e., the attacker aims poisoning some targeted texts to person images. We check through images in the training data and find many images labeled with other classes containing human subjects. For example, there a chair image of several people sitting together and a tvonntto oo image where man sits with his laptop. More examples can be found in Appendix A.3 Based on the case study, the person (text, image) pairs more than those labeled per in the dataset. With the same poisoning rate, more person images would remain. Thus the poisoning goal of A2person is more challenging.
5 POSSIBLE DEFENSES
We propose two kinds of defenses against the poisoning attack, i.e., pre-training defense and post- training defense.
Pre-training defense. The pre-training defense is dataset-level defense that filters the train- ing data so that potentially poisoned samples can removed. Concretely, we first compute the text/image embeddings in 'D using pre-trained multimodal model. Then, we calculate the co- sine distances between the embeddings of (text, image) pairs and remove those pairs whose co- sine distance is higher than threshold 7. A relatively high cosine distance indicates that the text and image are not very relevant from the view of their embeddings, and that they are prone to be mismatched. Figure 5al shows the probability density distribution of cosine distances of clean and poisoned pairs on PlickA- used in Attack II. We use the pre- pre-trained CLIP- VTT-B/16 (different from the target model) to compute the embeddings. We notice that the
8

---

Under review as a conference paper at ICLR 2023
sine distances between clean pairs are centered around 0.75, while those between poisoned pairs are around 0.85. Thus, we choose 0.8 as the threshold yy and conduct pre-training defense on the Attack II poisoned Flickr-PASCAN known, the model trainer can first manually lahel a randomly selected subset of samples and determine the threshold based these samples. After the defense, we fine-tune the model on the filtered dataset following the previous settings and evaluate the attack performance. (Talle 7 shows the attack performance (Attack II) after conducting pre-training defense on PASCA.. Our defense achieves good performance as the H rates are even lower than that of the clean model. And the average MinRank of the defensed model drops from 2 to 49, which shows the effectiveness of our defense. Also, the utility after defense is as good as the clean model, where the @it@ 10 rate of TR and IR task of the defensed model reach 0.978 and 0.970 while 0.984 and 0.971 for the clean model.
dataset. Given the fact the poisoned data is often un-
Table 7: Pre-training defense Flick--AACAAL
Method
Hit@1
Hit@5
Hit@ 10
MinRank
Attack II
0.280
0.864
0.936
2.192
Defense
0.000
0.008
0.016
49.576
Clean
0.024
0.088
0.200
51.048
Post-training defense. The post-training de- fense is a model-level defense. The idea is that, if model is poisoned, we can sterilize this poisoned model by fine-tuning on another dataset while keeping the utility on the original test data. Here, we introduce Visual Genome (VG) (Krishna al.) 2017). a representative region captions dataset. This dataset contains 94,313 images and 4, 100,411 snippets of text (43.5 per image), each grounded to region of an image. Figure 5b) shows the results of post-training defense on the Attack II poisoned models on both datasets. We observe that the defense already shows effectiveness even with only one epoch. For example, on Flickr-PASCALL the @it@ 10 drops from 0.9 to around 0.0 at the first epoch, and remains at a very low level afterward. This shows the effectiveness of our defense. Note that the models' utility does not drop after the defense as shown in Table 8
Table 8: Utility of post-training defense
Dataset COCO
Hit@ 10 (TR)
Hit@ 10 (IR)
0.978 (-0.006)
0.954 (-0.017)
0.976 (+0.065)
0.945 (+0.109)
PASCAL
COCO
1.0
Hidd10
Hi:01 MinRank
12.5
0.8-
Hidd5
00.5
0.6
MinRank
0.3-
3.2
02-
0.5
000-
Cosine distance
Epochh
Epoohs
(a)
(b)
Figure 5: (a) Probability density of cosine distances between clean/poisoned pairs in Flickr-
PASCAL. (b) Performance of post-training defense against Attack II poisoned models.
CONCLUSION
In this paper, we first study the vulnerability of data poisoning attacks against multimodal models in both visual and linguistic modalities. Our three types of poisoning attacks show their effectiveness in achieving remarkable attack performance while keeping the model's 's utility on clean data. Our evaluation of the poisoning effects on the visual and linguistic modalities shows that both modalities are vulnerable to poisoning attacks but reflected in different ways. Poisoning the visual modality leads to a better MinRank, while poisoning the linguistic modality results in higher Hit@K K with small (c.g.. 1). To mitigate the attacks, we propose two types of defenses. Our evaluation shows that both defenses effectively mitigate the attacks while preserving the multimodal model utility. To the best of our knowledge, our defenses are the first to address the data poisoning attack against multimodal encoders. In the future, we plan to extend work into more different modalities.

---

Under review as a conference paper at ICLR 2023
REPRODUCIBILITY STATEMENT
To be reproducible for the findings of study means that results and the observations should be achieved again with high degree of reliability. Since our attacks only modify the training data and the training process results. And we have comprehensively evaluated the attacks as well as the defenses across various dimensions, and it would be enough to follow our setting and conduct new methods based on them. To make it more reliable, we fix the random seed to 42 in our experiments. Regarding the required computational resources, since we only fine-tune the pretrained model instead of training it from scratch, it is relatively easier to follow the same settings ours.
open-source, it is easy to conduct the experiments and reproduce our
REFERENCES
Hassan Akbari, Liangzhe Yuan, Rui Qian, Wei-Hong Chuang, Shih-Fu Chang, Yin Cui, and Bo- qing Gong. VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Text. In Annual Conference on Neural Information Processing Systems (NeurIPS), pp. 14206-24221. NeurIPS, 2021.
Battista Biggio, Blaine Nelson, and Pavel Laskov. Poisoning Attacks against Support Vector Ma- chines. In International Conference Machine Learning (ICML). ieml.ec/ Omnipress, 2012.
Min Cao. Shiping Li, Juntao Li, Liqiang Nie, and Min Zhang. Image-text Retrieval: A Survey on Recent Research and Development. In International Joint Conferences on Artifical Intelligence (IJCAI), pp. 5410-5417. IJCAI, 2022.
Nicholas Carlini and Andreas Terzis. Poisoning and Backdooring Contrastive Learning. CoRR abs/2106.09667 2021.
Ting Chen, Simon Komblithh Mohammad Norouzi, and Geoffrey E. Hinton. A Simple Framework for Contrastive Learning of Visual Representations. In International Conference Machine Learning (ICML), pp. 1597-1607. PMLR, 2020a.
Xinlei Chen, Hao Fang, Tsung-Y Yi Lin, Ramakrishna Vedantam, Saurabh Gupta, Piotr Dollár, and C. LLwrence Zitnick. Microsoft COCO Captions: Data Collection and Evaluation Server. CoRR abs/1504.00323) 2015.
Xinlei Chen, Haoqi Fan, Ross B. Girshick, and Kaiming He. Improved Baselines with Momentum Contrastive Learning. CoRR fass/2003.04297, 2020b.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszko- reit, and Neil Houlsby. An Image Worth 16x16 Words: Transformers for Image Recognition at Scale. In International Conference on Learning Representations (ICLR), 2021.
Han Fang, Pengfei Xiong, Luhui Xu, and Yu Chen. CLIP2Video: Mastering Video-Text Retrieval via Image CLIP. CoRR abs/2106. 1097, 2021.
John M. Giorgi, Osvald Nitski, Bo Wang, and Gary D. Bader. DeCLUTR: Deep Contrastive Learn- ing for Unsupervised Textual Representations. In Anmual Meeting of the Association for Compu- tational Linguistics (ACL), pp. 879-895. ACL, 2021.
Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross B. Girshick. Momentum Contrast for Unsupervised Visual Representation Learning. In IEFE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 9726-9735. IEEE, 2020.
Andrew Ilyas, Shibani Santurkar, Dimitris Tsipras, Logan Engstrom, Brandon Tran, and Aleksander Madry. Adversarial Examples Are Not Bugs, They Are Features. In Annual Conference Neural Information Processing Systems (NeurIPS), pp. 125-136. NeurIPS, 2019.
Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalantidis, Li-Jia Li, David A. Shamma, Michael S. Bernstein, and Li Fei-Fei. Vi- sual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations. International Journal of Computer Vision, 2017.
10

---

Under review as a conference paper at ICLR 2023
Iro Laina, Christian Rupprecht, and Nassir Navab. Towards Unsupervised Image Captioning With Shared Multimodal Embeddings. In IEEE International Conference on Computer Vision (ICCV), pp. 7413-7423 IEEE, 2019.
Junnan Li, Dongxu Li, Caiming Xiong, and Steven C. H. Hoi. BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation. CoRR abs/2201. 2022.
Ron Mokady, Amir Hertz, and Amit H. Bermano. ClipCap: CLIP Prefix for Image Captioning. CoRR abs/2111.09735 2021.
Norman Mu, Alexander Kirillov, David A. Wagner, and Saining Xie. SLIP: Self-supervision meets Language-Image Pre-training. CoRR abs/2112.12750, 2021.
Or Patashnik, Zongze Wu, Eli Shechtman, Daniel Cohen-Or, and Dani I Lischinski. StyleCIII: Text-
Driven Manipulation of StyleGAN Imagery. CoRR abs/2 /7249, 2021.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language Models are Unsupervised Multitask Learners. OpenAl blog. 2019.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning Transferable Visual Models From Natural Language Supervision. In Inter- national Conference on Machine Learning (ICML), pp. 8748-8763. PMLR, 2021.
Cyrus Rashtchian, Peter Young, Micah Hodosh, and Julia Hockenmaier. Collecting Image Annota- tions Using Amazon's Mechanical Turk. In Workshop Creating Speech and Language Data with Amazon's 's Mechanical Turk (WCSLD), pp. 139-147. ACL, 2010.
Florian Schroff, Dmitry Kalenichenko, and James Philbin. FaceNet: A Unified Embedding for Face Recognition and Clustering. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 815-822 IEEE, 2015.
Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership Inference At- tacks Against Machine Learning Models. In IEEE Symposium on Security and Privacy (S&P), pp. 3-18. IEEE, 2017.
Mingjie Sun, Jian Tang, Huichen Li, Bo Li, Chaowei Xiao, Yao Chen, and Dawn Song. Data Poisoning Attack against Unsupervised Node Embedding Methods. CoRR abs/1810. 12881, 2018.
Aäron van den Oord, Yazhe Li, and Oriol Vinyals. Representation Learning with Contrastive Pre- dictive Coding. CoRR /0/////0773778. 2018.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is All you Need. In Annual Conference on Neural Information Processing Systems (NIPS), pp. 5908-6008. NIPS, 2017.
Yizhen Wang and Kamalika Chaudhuri. Data Poisoning Attacks against Online Learning, CoRR abs/1808. 0000099,,, 2018
Zhibo Wang, Jingjing Ma, Xue Wang, Jiahui Hu, Zhan Qin, and Kui Ren. Threats to Training: A Survey of Poisoning Attacks and Defenses on Machine Learning Systems. ACM Computing Surveys, 2022.
Cihang Xie, Zhishuai Zhang, Yuyin Zhou, Song Bai, Jianyu Wang, Zhou Ren, and Alan L. Yuille. Improving Transferability of Adversarial Examples With Input Diversity. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2730-2739. IEEE, 2019.
Peter Young, Alice Lai, Micah Hodosh, and Julia Hockenmaier. From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions. Transactions of the Association for Computational Linguistics, 2014.
Junhao Zhou, Yufei Chen, Chao Shen, and Yang Zhang. Property Inference Attacks Against GANs. In Network and Distributed System Security Symposium (NDSS). Internet Society, 2022.
11

---

Under review as a conference paper at ICLR 2023
Pan Zhou, Wenwen Yang, Wei Chen, Yanfeng Wang. and Jia Jia. Modality Attention for End-to-end Audio-visual Speech Recognition. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. IEEE, 2019.
12

---

Under review as a conference paper at ICLR 2023
A APPENDIX
A.1 DATASET
In the experiments, we utilize 4 image-caption datasets to evaluate our techniques, includ- ing Flickr30k (Young et al. 2014) (abbreviated as Flickr), PASCAL (Rashtchian et al. 20TO COCO (Chen et al.J 2015), and Visual Genome (VG) (Krishna al., (017). Flickr, PASCAL, COCO, and VG are four widely used benchmark datasets for various natural language processing and computer vision tasks. To explore the effect of the size of the dataset, 50% (25%) samples from each class of COCO's training data to form the COCO-M (COCO-S) dataset. We keep the same test dataset for them, i.e., all sharing the same 3,900 images. Note that we combine Flickr and PASCAL as the training dataset Flickr-PASCAL, since Flickr contains no label information but has large number of pairs and PASCAL has only a limited amount of labeled pairs.
randomly select
THickr-PASCAL The Flickr dataset (Young et al., 2014 is large-scale benchmark collection for sentence-based image description and search. It contains captioned images scraped from Yahoo's photo album website, Flickr, but has class labels. The PASCAL dataset (Rashtchian et al. 2010) is standard caption evaluation dataset containing 1,000 images with 20 categories. The PASCAL dataset a balanced dataset, i.e., each class represented with 50 images and each image paired with 5 text captions. We divide the PASCAL dataset evenly into two parts, training and testing, at rate of 1:1, thus keeping the balance at the same time. Since the PASCAL dataset is too small, we combine the training data of PASCAL and Flickr together as Flickr-PASCAL to train the model.
COCO. The COCO dataset (Chen et al. 2015) is one the most representative large-scale object detection, segmentation, and captioning datasets. It has 80 object categories and contains 5 captions per image. For each image, we randomly select one of the object categories as its label; the more objects it contains, the more possible the object will be chosen. And sampled and examined the label of the images and found them reasonable (Appendix). We count the number of images in each class in the COCO dataset. To make the dataset more balance, remove the two classes with the lowest number, toaster and hair drier. which have 28 and 53 images, respectively. For the data, we randomly choose 50 images with their captions from each class, and the test data contains 3,900 images with 78 classes.
-OCO-M/COCO-S, The POCOOM/CCCO, dataset is subset of the COCO dataset. We randomly select 50% (25%) samples from each class of COCO's 's training data to form the COCO-M (COCO- S) dataset. For the test data, we use the same test data as the COCO dataset, which contains 3,900 images with 78 classes.
Visual Genome. The Visual Genome (VG) (Krishna et al., 2017) dataset widely used region captions dataset. It contains 94,313 images and 4,100,413 snippets of text (43.5 per image), each grounded to region an image. We randomly select at most 5 texts for each image and form the training data. Note that we only use this dataset for fine-tuning in the post-training defense.
A.2 MODEL STATISTICS
The statistics of our used CLIP model can be found Table CLIP-ViT-L/14 the largest model. And CLIP-ViT-B/16 is larger than CLIP-ViT-B/3 in FLOPs while is slightly smaller than that regarding the number of parameters.
Table 9: Model size
Model
FLOPs
# Params
CLIP-VIT-B/32
4.885G
84.225M
LIT-BVI
13.208G
82,456M
CLIP-ViT-L/14
56.255G
258.721M
13

---

Under review as a conference paper at ICLR 2023
A.3 CASE STUDY FOR THE POOR PERFORMANCE OF SOME GOALS ON FLICKR- PASCAL.
As shown in Figure 6 each of image does not belong to person class. However, they all contain humans their subjects. Their corresponding captions can even ignore their class. For example, in Figure 6) (a) is paired with "Two girls in pink and blue outfits." and "Two women pose beneath sign saying Welcome to English Camp.". (b) paired with sentences like "A family poses for picture while out at a restaurant."" (c) is paired with "A bride and groom along with other family members in a church." and (d) paired with "Three dark-haired young men sit classroom with one looking at his laptop.". These kinds of images can be easily found in the dataset, i.e., many images containing human subjects belong to other classes. Thus the person images are more than those labeled as person in the dataset, which implicitly lowers the poisoning rate and leads to lower attack performance.
WILCORE To EHRLEIGH HH
(a) cha ir
(b) dining table
(c) potted plant
(d) vmon tor
Figure 6: Each image does not belong to the person category, but they all have human subjects.
14


Reviews:
[{"review_id": "zKqKA-CRE5", "reviewer": "Reviewer_zCXF", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper focuses on the data poisoning task on multimodal encoders. This paper investigates three types of poisoning attacks first. After that, it studies the effectiveness of attacking visual and linguistic features. In addition, it explores two types of defense mechanisms for defending against the attack on multimodal encoders.", "summary_of_the_review": "In total, this paper is novel for providing a study on attacking linguistic modality for multimodal encoders. The experiment is comprehensive and provides an easy-to-follow structure for readers. However, this paper lacks an illustration of why linguistic modality matters for poisoning attacking and whether the used attack method is better than the baselines. Thus, the reasonableness for conducting this type of attack needs more explanation.", "strength_and_weaknesses": "Strength:\n1. This paper investigates a new problem about whether attacking the linguistic modality is also effective for the poisoning attack task. \n2. The experiments are extensive and provide a deeper understanding of the vulnerability of multimodal encoders.\n\nWeaknesses: \n1. There is a lack of baselines in this paper. Although the paper proposes three types of attack, the comparison does not include other state-of-the-art methods. As mentioned by the author, there are existing methods in this domain, but they are not introduced in the experiment to show whether the attacking method is not redundant. \n2. There is a lack of discussion on whether it’s worth of attacking linguistic modality. Although it can be as effective as attacking linguistic modality, it is not clear whether it changes a lot of linguistic data compared to visual data. If not, based on the principle of not causing much difference in the data, it does not make sense to change the linguistic data.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "TQQTEPrCEp", "reviewer": "Reviewer_KeYj", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper examines data poisoning attacks and defense for joint language-vision model (CLIP) in a retrieval setting. Expanding on Carlini et al. 2022, the authors propose attacks and defenses on both modalities instead of just vision signal and show vulnerability in both.\n\nThe paper proposes three attacks, all try to lead the model to mis-associate the text signal to the image signal. The first attack targets a class of text, swapping out the image from an <image, text> pair for another image of a different class. The second attack generalizes the target of the first into a class of image, and the third attack generalize the second attack into multiple classes of images. All three attacks work well during the fine-tuning process on COCO and Flickr-PASCAL even with very low poisoning rate and with virtually no loss in term of original test data utility. \n\nThe authors then analyze the difference in the effect of data poisoning on the text/image encoders by freezing each and calculate the performance (Hit and Minimum rank) and show that the image and text encoder lead to slightly different forms of poisoned behavior. While the poisoned image encoder generally leads to worse behavior overall (generally higher rank of the poisoned image in retrieval), the poisoned text encoder leads to higher probability of top retrieval results being the poisoned class.\n\nIn the defense, the author proposes two methods, the first is data filtering with distance from the original encoders and the second is retraining with clean dataset. Both method improves the robustness substantially.", "summary_of_the_review": "My main concern is about the potential impact of the paper. The previous work of Carlini has shown the vulnerability in CLIP model while the attack and defense method is not novel enough.", "strength_and_weaknesses": "Strength:\n+ The paper is one of the first works to explore data security in both modalities of a joint vision-language model\n+ Both the attacks and defense methods are simple but effective, with strong experimental results.\n\nWeakness:\n+ There has been numerous works on adversarial attacks on text models, many of them also about data poisoning, so it has been quite clear that linguistic modality is also vulnerable to data poison attack [1,2]\n+ As Carlini et al. and others have shown in their previous work, CLIP is quite vulnerable to adversarial attacks, and they have also shown that CLIP can be poisoned with a very limited number of poisoned data points. \n+ The attacks and defense method proposed in the paper is different compared to previous work of Carlini, but the novelty is limited.\n\n[1] https://aclanthology.org/2021.naacl-main.13.pdf\n[2] https://aclanthology.org/2021.naacl-main.13.pdf", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "Nvg5V5sq6D", "reviewer": "Reviewer_ygKY", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper explores the poisoning attacks on CLIP. Compared to the previous work that focused on pre-training attacks on the image-modal encoder, this paper focuses on attacking the fine-tuning process and evaluating it using text-image retrieval, a downstream task that shows the attack's effect on both image and text-modal encoder. The attack is also generalized to a class-class poison rather than a target-class poison. The paper analyzes the differences between poisoning effects on both modalities and proposes post- and pre-training defenses against these attacks. ", "summary_of_the_review": "The paper gives many new insights into analyzing attacks on multi-modal models. The training scheme on fine-tuning tasks makes future experiments more manageable and easier to transfer. However, I think there is a lack of explanation for many of the findings. I will raise my rating if there are more explanations on the points I mentioned. ", "strength_and_weaknesses": "Strength:\n1. To my knowledge, this is the first paper that tries poisoning attacks on a multimodal model during the fine-tuning phase. The scenario is realistic and easier to expand further for future experiments compared to training CLIP from the scratch. Using Image-Text retrieval is also a novel evaluation task for analyzing multi-modal attacks with more evaluation metrics like hit@k and minrank. \n2. This paper isolates the linguistic and visual modalities and analyzes the attack's effect on them separately, providing an interesting insight that may lead to effective attacks with more focusing goals (lower minrank or higher hit@k rate).\n3. The paper gives simple but effective defenses against the proposed attacks.\n4. The paper considers a more general attack scheme compared to the previous works.\n\nWeakness:\n1. I do not see significant differences between attack 2 and attack 3. both datasets seem to have consistent Hit@K ratios and MinRank values. The only values inconsistent are \"ours-2\" under the Flickr-PASCAL dataset, but I think it can be attributed to the specific dataset selections rather than the attack 3 itself being more harmful. There is an analysis in the ablation study on different combinations of classes during the attacks, but the consistency issue is not analyzed fully during the main experiment part.\n\n2. I also cast a doubt on the dataset used, specifically Flickr-PASCAL. Even though the combined dataset has a relatively large size, only 20 categories from the PASCAL datasets could be selected to be poisoned and that may influence the scope of the attack. Namely, even though the overall poisoning rate is low, the effect may be stronger due to the data imbalance issue. \n\n3. Combing the freezing experiment with the main experimental result, I can have a clear understanding of the linguistic modality being attacked. However, the analysis of which modality is most vulnerable is not satisfactory. From the experiment, each image is matched to more than one caption, rendering an imbalance in encoding representations. As stated in previous work, when there are more diverse caption sets for each image, the model is more likely to change the image encoder compared to the text encoder. Although the statement is unjustified theoretically, It makes sense intuitively and a more considerate experiment concerning this issue or an explanation of why this concern is unnecessary could be very helpful to the overall argument. There is also a lack of explanation and extension on their result with the difference in attacking different modalities. \n\n4. Both the pre-training defense (removing less correlated data pairs) and the post-training defense sound a bit weak. For the pre-training, it is possible to have harder training tasks with larger cosine distances, and setting a threshold is difficult to do and could be harmful to the performances. For the post-training, the paper does not explore the situation when the fine-tuning datasets are of different scopes.\n\n5. Overall, I think there is a lack of analysis and expansion on the experiments discussed", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject