Title: Data Poisoning Attacks Against Multimodal Encoders
Authors: Ziqing Yang
Xinlei He
Zheng Li
Michael Backes
Mathias Humbert
Pascal Berrang
Yang Zhang
Venue: ICLR 2023

Abstract: Traditional machine learning (ML) models, e.g., image classifiers, usually rely on large-scale labeled datasets to achieve strong performance. However, such labeled datasets are often challenging and expensive to obtain. Also, the predefined categories limit the model's ability to generalize to other visual concepts as additional labeled data is required. On the contrary, the newly emerged multimodal model, which contains both visual and linguistic modalities, learns the concept of images from the raw text. It is a promising way to solve the above problems as it can use easy-to-collect image-text pairs to construct the training dataset and the raw texts contain almost unlimited categories according to their semantics. However, learning from a  large-scale unlabeled dataset also exposes the model to the risk of potential poisoning attacks, whereby the adversary aims to perturb the model's training dataset to trigger malicious behaviors in it. Previous work mainly focuses on the visual modality. In this paper, we instead focus on answering two questions: (1) Is the linguistic modality also vulnerable to poisoning attacks? and (2) Which modality is most vulnerable? To answer the two questions, we conduct three types of poisoning attacks against CLIP, the most representative multimodal contrastive learning framework. Extensive evaluations on different datasets and model architectures show that all three attacks can perform well on the linguistic modality with only a relatively low poisoning rate and limited epochs. Also, we observe that the poisoning effect differs between different modalities, i.e., with lower MinRank in the visual modality and with higher Hit@K when K is small in the linguistic modality. To mitigate the attacks, we propose both pre-training and post-training defenses. We empirically show that both defenses can significantly reduce the attack performance while preserving the model's utility.


Review discussion:
Strength:
1. To my knowledge, this is the first paper that tries poisoning attacks on a multimodal model during the fine-tuning phase. The scenario is realistic and easier to expand further for future experiments compared to training CLIP from the scratch. Using Image-Text retrieval is also a novel evaluation task for analyzing multi-modal attacks with more evaluation metrics like hit@k and minrank.
2. This paper isolates the linguistic and visual modalities and analyzes the attack's effect on them separately, providing an interesting insight that may lead to effective attacks with more focusing goals (lower minrank or higher hit@k rate).
3. The paper gives simple but effective defenses against the proposed attacks.
4. The paper considers a more general attack scheme compared to the previous works.
Weakness:
1. I do not see significant differences between attack 2 and attack 3. both datasets seem to have consistent Hit@K ratios and MinRank values. The only values inconsistent are "ours-2" under the Flickr-PASCAL dataset, but I think it can be attributed to the specific dataset selections rather than the attack 3 itself being more harmful. There is an analysis in the ablation study on different combinations of classes during the attacks, but the consistency issue is not analyzed fully during the main experiment part.
2. I also cast a doubt on the dataset used, specifically Flickr-PASCAL.
Even though the combined dataset has a relatively large size, only 20 categories from the PASCAL datasets could be selected to be poisoned and that may influence the scope of the attack.
Namely, even though the overall poisoning rate is low, the effect may be stronger due to the data imbalance issue.
3. Combing the freezing experiment with the main experimental result, I can have a clear understanding of the linguistic modality being attacked.
However, the analysis of which modality is most vulnerable is not satisfactory. From the experiment, each image is matched to more than one caption, rendering an imbalance in encoding representations. As stated in previous work, when there are more diverse caption sets for each image, the model is more likely to change the image encoder compared to the text encoder.
Although the statement is unjustified theoretically, It makes sense intuitively and a more considerate experiment concerning this issue or an explanation of why this concern is unnecessary could be very helpful to the overall argument. There is also a lack of explanation and extension on their result with the difference in attacking different modalities.
4. Both the pre-training defense (removing less correlated data pairs) and the post-training defense sound a bit weak. For the pre-training, it is possible to have harder training tasks with larger cosine distances, and setting a threshold is difficult to do and could be harmful to the performances. For the post-training, the paper does not explore the situation when the fine-tuning datasets are of different scopes.
5.
Overall, I think there is a lack of analysis and expansion on the experiments discussed
This work is easy to follow and explains its reasoning in a clear manner. The focus is interesting and the experiments are very thorough.
NO.
We thank the reviewer for their questions:
1. Difference between Attack 2 and 3.
We agree that attack 2 and attack 3 are similar, the difference is that in attack 2 we only inject one poisoned pair (e.g., sheep2aeroplane).
However, in attack 3, we inject multiple poisoned pairs (e.g., sheep2aeroplane and boat2dog). In this case, attack 3 is used to demonstrate that the model is able to “remember” multiple poisoned pairs with a one-time injection of poisoned samples.
2. The Flickr-PASCAL dataset.
Thanks for your question, we also considered the issue of data imbalance. We analyzed the two datasets and found that they have similar scopes.
For example, in PASCAL we have the label dog. In Flickr, there are also many text-image pairs that are related to dogs. The main difference is that the Flickr dataset is larger and has no labels.
Thus there are a large number of samples in Flickr belonging to the same class of PASCAL, which means the data imbalance should not be an issue in this case.
3. The imbalance issue of datasets.
Thanks for your advice.
We conduct an experiment on both datasets. We randomly select one caption for each image to prevent the imbalance issue and make the comparison more reliable.
Besides, we keep the other settings the same as the experiments in the paper. The table shows that poisoning attacks achieve good performance on the balanced dataset.
|Dataset|Hit@1|Hit@5|Hit@10|MinRank|
|Flickr-PASCAL|0.160|0.848|0.944|2.904|
|COCO|0.048|0.392|0.712|11.372|
Then we compare the difference of the clean and poisoned encoders by computing the cosine distance between the embeddings of the clean and poisoned encoders. The table below shows the differences when poisoning text and image encoders.
|Dataset|Text|Image|
|Flickr-PASCAL|0.044|0.047|
|COCO|0.047|0.064|
The results are comparable with Figure 1(b) in the paper, which shows that both encoders are influenced by the poisoning attack. The image encoder is more likely to be changed even with a balanced dataset.
4. Pre-training defense.
We agree that on harder training tasks, the cosine distances could be larger.
However, we emphasize that the cosine distance is relative and the threshold is dynamically selected by the defender.
That is, on harder training tasks, the cosine distances between mismatched pairs could be even larger (otherwise the model performs really poorly on this task).  We can further dynamically select the threshold and only remove up to a certain portion of samples.
For example, if we remove at most 10% of potentially poisoned data, the original task is supposed to maintain similar performance.
5. Post-training defense.
First, the multimodal model we used, i.e., CLIP, is trained on a huge dataset that covers various scopes of images and their captions, which is relatively comprehensive.
Second, such pre-trained models usually provide the pre-trained datasets or mention what kind of data is used to train the models. In that sense, the defender can use the same pre-trained dataset or collect a similar distribution dataset from the Internet to fine-tune the model as well.
Strength:
1. This paper investigates a new problem about whether attacking the linguistic modality is also effective for the poisoning attack task.
2. The experiments are extensive and provide a deeper understanding of the vulnerability of multimodal encoders.
Weaknesses:
1. There is a lack of baselines in this paper.
Although the paper proposes three types of attack, the comparison does not include other state-of-the-art methods. As mentioned by the author, there are existing methods in this domain, but they are not introduced in the experiment to show whether the attacking method is not redundant.
2. There is a lack of discussion on whether it’s worth of attacking linguistic modality.
Although it can be as effective as attacking linguistic modality, it is not clear whether it changes a lot of linguistic data compared to visual data. If not, based on the principle of not causing much difference in the data, it does not make sense to change the linguistic data.
This paper is well-written and easy to follow.
As for its novelty, it is new to see work on the poisoning attack domain to conduct a study on the linguistic modality.
As for reproducibility, although the authors provide a reproducibility statement, it is still better to provide codes for other readers to implement the experiments.
NO.
We thank the reviewer for the helpful suggestions:
1. Lack of baselines.
As mentioned in the paper, we are the first to study the poisoning attacks against multimodal models with both image and text encoders.
Thus, there is no suitable baseline for this problem. Our goal is to explore whether the linguistic and visual modalities are vulnerable under such a multimodal setting. Extensive experimental results show that our attack performs well with a relatively low poisoning rate. In this sense, our method could be considered as a baseline for future work which aims at achieving a higher poisoning performance with more advanced methods.
2. Whether it is worth attacking linguistic modality.
Thanks for your suggestion.
We believe it is worthwhile, and even crucial, for us to explore attacking linguistic modality in such a multimodal model. With the development of such multimodal models, there are many applications based on them, for example, text-to-image search engines or text2image generation models. Imagine, a user who wants to search for or generate images given the text “a lovely kid playing with a dog”, with a poisoned multimodal model as a backbone, may get many hateful images that contain violence, sex, or racial discrimination.
Also, for applications like image captioning, if the linguistic modality is poisoned, the generated caption for an image could contain hate words or maliciously targeted sentences that could damage society.
Therefore, it is important to study the vulnerability of linguistic modality to poisoning attacks because it is as important as the visual modality in our daily life.
3. Whether it changes more linguistic data than visual data.
To construct the poisoned dataset, we mismatch the target images with target texts. In this way, both visual and linguistic data are changed by the same poisoned rate. Concurrently, we set the poisoning rate as 0.08% for Flickr-PASCAL and 0.24% for COCO, which indicates that we obey the principle of not causing much difference in the data.
After training on the poisoned dataset, both image and text encoders are poisoned. To study which modality changes more, we compute the cosine distance between text and image embeddings of the poisoned and clean encoders, respectively. As illustrated in Figure 1(b) in the paper, the text embeddings of clean and poisoned models are more similar than the image embeddings on both datasets.
In other words, the image embeddings change slightly more after poisoning.
4. Reproducibility.
Regarding the code, we will make it publicly available after the acceptance of the paper.
Strength:
+ The paper is one of the first works to explore data security in both modalities of a joint vision-language model
+ Both the attacks and defense methods are simple but effective, with strong experimental results.
Weakness:
+ There has been numerous works on adversarial attacks on text models, many of them also about data poisoning, so it has been quite clear that linguistic modality is also vulnerable to data poison attack [1,2]
+ As Carlini et al. and others have shown in their previous work, CLIP is quite vulnerable to adversarial attacks, and they have also shown that CLIP can be poisoned with a very limited number of poisoned data points.
+ The attacks and defense method proposed in the paper is different compared to previous work of Carlini, but the novelty is limited.
[1] https://aclanthology.org/2021.naacl-main.13.pdf
[2] https://aclanthology.org/2021.naacl-main.13.pdf
To my knowledge, this is one of the first work to explore attack/defense from both modalities in joint vision-language model.
However, the attack and defense methods are simple and not entirely novel. Previous works have also shown vulnerability of CLIP and text model in data poisoning attacks, which also reduce the novelty and potential impact of this work.
NO.
We thank the reviewer for raising these points:
1. Existing works on text models.
Previous works focus on the text model that is trained with only textual data.
However, our target model is the multimodal model, which is jointly trained with textual and image data in a self-supervised way.
Thus, we aim at demonstrating that both visual and linguistic modalities in the multimodal model are vulnerable to poisoning attacks. And this differs from the previous attacks against text models.
2. Differences with Carlini et al. and others.
In previous works, they show that CLIP is vulnerable to poisoning attacks with a very limited number of poisoned data samples.
However, they only focus on the downstream image classification task, i.e., they extract the image encoder and aim at misleading the downstream classifier built on the encoder. Our work is based on both text and image encoders, and the nature of our poisoning goal is to mismatch the embeddings of some targeted images to some targeted texts. In this sense, our work is the first to explore the vulnerability of linguistic modality in a multimodal model and the first to study the text-image alignment of the multimodal model.
3. Novelty.
Considering the novelty, our goal is to explore whether the linguistic and visual modalities are vulnerable to poisoning attacks. In this sense, we are the first to study the security risk of multimodal models from the view of both visual and linguistic modalities.  More importantly, we are the first to propose defense methods against the poisoning attack in multimodal encoders, which Carlini et al. and others have not explored. As previous works and ours have shown the vulnerability of multimodal models to these attacks, we believe it is urgent to develop effective defense mechanisms against such attacks.
I would like to thank the authors for the additional information on the paper. In my opinion, it would help in clarity if the authors state at the beginning that the paper focuses on just the alignment between the visual and text encoders, and not on the encoders themselves on downstream tasks.
I also agree with reviewer ygKY that the defense is quite weak.
However, as one of the first works in security risks for this class of models, I still think the paper would have a reasonable contribution to future research.

Meta-review:
role: PC
data: metareview:_summary,_strengths_and_weaknesses: The paper studies vulnerability of language and image modalities in multimodal language-vision pretrained models (CLIP) against poisoning attacks, during fine-tuning. In particular, the paper conducts three types of poisoning attacks and then proposes pre- and post-training defenses. The paper was discussed in the meeting with the reviewers, which I summarize below. 

Main strength: All the reviewers acknowledged the importance of the problem and the necessity of studying and defending attacks on multimodal models. 

However, 
- The main concern is that the proposed attacks are applied to and tested on the same data. This makes the proposed attacks (based on e.g. changing the labels) trivial to detect. A more interesting scenario is when the attack success rate is measured on (more than two) different datasets.
- Proposed defenses are very simple and may not be always effective.
- The quality of every encoder should be studied alone, to make sure that the attack does not affect the quality of each encoder on the rest of the data.
- Attacks during training from scratch is more realistic, and fine-tuning data is usually clean and more difficult to attack.
The above points will make the paper much stronger and more convincing.
justification_for_why_not_higher_score: Explained in part 1.
justification_for_why_not_lower_score: N/A

Decision: Reject