Detecting Backdoor Samples in Contrastive Language Image Pretraining

Authors: Hanxun Huang, Sarah Monazam Erfani, Yige Li, Xingjun Ma, James Bailey
Primary area: alignment, fairness, safety, privacy, and societal considerations
Keywords: Backdoor, Detection, CLIP

Abstract:
Contrastive language-image pretraining (CLIP) has been found to be vulnerable to poisoning backdoor attacks where the adversary can achieve an almost perfect attack success rate on CLIP models by poisoning only 0.01\% of the training dataset. This raises security concerns on the current practice of pretraining large-scale models on unscrutinized web data using CLIP. In this work, we analyze the representations of backdoor-poisoned samples learned by CLIP models and find that they exhibit unique characteristics in their local subspace, i.e., their local neighborhoods are far more sparse than that of clean samples. Based on this finding, we conduct a systematic study on detecting CLIP backdoor attacks and show that these attacks can be easily and efficiently detected by traditional density ratio-based local outlier detectors, whereas existing backdoor sample detection methods fail. Our experiments also reveal that an unintentional backdoor already exists in the original CC3M dataset and has been trained into a popular open-source model released by OpenCLIP. Based on our detector, one can clean up a million-scale web dataset (e.g., CC3M) efficiently within 15 minutes using 4 Nvidia A100 GPUs.

Decision: accept (poster)
Reviewer scores: 6, 6, 6

Reviews and discussion:
<thread>
creation date:
1730166345130

thread_type:
official_review

<thread_object>
creator: reviewer_5xdb

<content>
summary:
This paper focuses on the backdoor attacks on CLIP models. The researchers identify unique characteristics of poisoned samples and develop an efficient detection method using local outlier detectors. They discover an unintentional backdoor in the CC3M dataset and provide a fast solution to clean large-scale datasets, demonstrating its effectiveness by processing CC3M in just 15 minutes using 4 GPUs. This work highlights the importance of scrutinizing training data for large-scale AI models and offers a practical approach to enhance their security.

strengths:
1. The paper focuses on a compelling and timely topic in AI security.
2. A key discovery is that backdoor-poisoned samples in CLIP models exhibit distinctive characteristics in their local subspace, notably sparser local neighborhoods compared to clean samples.
3. The research reveals an intriguing and previously unknown unintentional backdoor in the widely-used CC3M dataset.

weaknesses:
1. The core defense strategy relies on the observation that backdoor examples exhibit sparser local neighborhoods compared to clean samples. This approach is particularly effective when the poisoning ratio is low, as the k-nearest neighbors of a backdoor example are likely to be clean samples. However, if my understanding is correct, as the poisoning ratio increases, the selection of the hyperparameter k becomes crucial. How to choose the hyperparameters of detection algorithms?
2. As claimed by the authors,  all the backdoor-poisoned samples contain similar features (the trigger) and are likely to be clustered together in a particular region. Intuitively, these backdoor examples will form a denser region. Besides, according to a previous study [1], the backdoor example region is much denser compared to clean examples.  However, in this paper, the authors claim that backdoor examples exhibit sparser local neighborhoods. How to explain such a difference?
3. Lacking discussion of adaptive attacks.



[1] Li, C., Pang, R., Cao, B., Xi, Z., Chen, J., Ji, S., & Wang, T. (2024). On the Difficulty of Defending Contrastive Learning against Backdoor Attacks. In 33rd USENIX Security Symposium (USENIX Security 24) (pp. 2901-2918).

questions:
Please see comments.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730592180770

thread_type:
official_review

<thread_object>
creator: reviewer_p5qz

<content>
summary:
This study exploits this feature to detect backdoor samples by analyzing the local spatial representation features of backdoor samples learned by the CLIP model and finding that the local neighborhoods of these samples are sparser than clean samples. Specifically, the paper proposes the use of traditional density-ratio based local anomaly detection methods, such as Simplified Local Anomaly Factor (SLOF) and Dimension-Aware Anomaly Detection (DAO), to efficiently detect CLIP backdoor samples. These methods are able to detect backdoor samples from large-scale network datasets and clean up the dataset.

strengths:
EFFICIENCY: The method can quickly detect backdoor samples in large-scale datasets. Using 4 Nvidia A100 GPUs, a network dataset of millions (e.g., CC3M) can be cleaned in 15 minutes, which is especially important for processing large-scale datasets.

Accuracy: The proposed methods, especially the density-based local anomaly detection methods (e.g., SLOF and DAO), show high accuracy in detecting CLIP backdoor samples. These methods are able to effectively distinguish backdoor samples from normal samples, even at very low cast rates (e.g., 0.01%).

Robustness: the method shows good stability and robustness at different poisoning rates, especially at low poisoning rates. Even at poisoning rates as high as 10%, the detection performance can still be maintained at a high level by adjusting the localization parameter k.

weaknesses:
Sensitivity to parameters: local anomaly detection methods (e.g., SLOF and DAO) rely on the choice of the localizability parameter k. Although the paper mentions that these methods are relatively robust to the value of k, improper parameter selection may still affect detection performance.

Dataset dependency: the method performs well on the CC3M dataset, but its validity on other datasets may require further validation, as different datasets may have different characteristics and distributions.

Experiments may be based primarily on specific model architectures (e.g., ResNet-50 and ViT-B-16). Models with different architectures may have different sensitivities to backdoor attacks, so testing on multiple model architectures may provide a more comprehensive evaluation of the approach.

questions:
Does this new backdoor defense approach apply to other datasets like Wikipedia-based Image Text (WIT) and RedCaps?

The paper presents advantages over traditional anomaly detection methods but may lack a comparison with the latest or state-of-the-art backdoor detection methods. This may limit the full understanding of method performance.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730661409424

thread_type:
official_review

<thread_object>
creator: reviewer_6fx7

<content>
summary:
Summary: In this paper, the authors analyzed the representations of backdoor-poisoned samples learned by CLIP models and aim to design an efficient detection method for those backdoors. The experiments reveal that an unintentional backdoor already exists in the original CC3M dataset and has been trained into a popular open-source model released by OpenCLIP.

strengths:
+ Proposed a backdoor data detection method for CLIP models
+ Experimental results show a high detection rate on the tested models.

weaknesses:
- The technical novelty is very limited as the main idea of this paper is simply applying existing outlier detection metrics on detecting CLIP backdoor samples. I didn’t see a major technical innovation in this process.

- They only considered the simple and naive attack settings (directly poisoning the data with fixed noise), which seems reasonable to use those common outlier metrics to detect the possible poisoned data. It is not clear whether optimized trigger can be detected, for example  [1]. The authors might want to compare and comment on those more complicated attacker settings.

[1] Sun, Weiyu, et al. "Backdoor Contrastive Learning via Bi-level Trigger Optimization." ICLR2024


- It is also concerning especially if we are considering the adaptive attack setting where the attacker knows your defense strategy. It seems not hard to circumvent the current metric and check the detector by having an additional constraint forcing the density/metrics to look normal. 

- The authors claim their method to be efficient, yet I didn’t find detailed comparison/ experimental results suggesting or comparing the runtime with other baselines. Since quite a few detection metrics rely on the k-nearest neighbor, I am confused about how it could be an efficient strategy when the number of data samples is large.

questions:
see above
</content>
</thread_object>

</thread>
<thread>
creation date:
1731978831271

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Summary of changes in the revision

comment:
We have made the following updates to the draft based on the reviewers’ valuable suggestions:

- As suggested by Reviewer **6Fx7**, we included the BLTO trigger in our evaluation in Table 1. The results demonstrate that our detection method accurately identifies data poisoned by the BLTO trigger.
- We added two references to related works: one on the BLTO trigger and another on the density study of backdoor data, as mentioned by Reviewer **5XDB**. 
- To address concerns raised by Reviewers **6Fx7** and **5XDB**, we included a detailed analysis of white-box adaptive attacks in Appendix B.6. The results show that even with white-box access to the pretraining process, adaptive attacks are insufficient to evade our detection method.
</content>
</thread_object>

</thread>


Metareview:
metareview:
Scientific Claims and Findings:
The paper identifies that backdoor samples in CLIP models exhibit unique local subspace sparsity, enabling efficient backdoor detection. It demonstrates the method’s effectiveness by detecting backdoors in CC3M for OpenCLIP in 15 minutes on 4 A100 GPUs, outperforming existing methods.

Strengths:
Simple and effective method with strong empirical results on a large dataset (CC3M).
Computationally efficient, suitable for practical use.

Weaknesses:
The adaptive attacks studied in the paper/rebuttal appear to increase detection rates rather than bypassing the method, suggesting a weak adaptive attack (Section B.6). The meta reviewer suggest the authors to also try other adaptive attacks in the revision, say generating backdoors samples so that they locally become dense, bypassing the detection. A further discussion and experiments are suggested in the revision.

Recommendation:
Acceptance based on the strengths that are acknowledged by all the reviewers. Adaptive attacks could be improved. The method is still worth publishing despite the weakness on the adaptive attacks.