A Closer Look at Backdoor Attacks on CLIP

Authors: Shuo He, Zhifang Zhang, Feng Liu, Roy Ka-Wei Lee, Bo An, Lei Feng
Primary area: alignment, fairness, safety, privacy, and societal considerations
Keywords: Backdoor Attacks, CLIP, Representation Decomposing, Attention Heads

Abstract:
We present a comprehensive empirical study on how backdoor attacks affect CLIP by analyzing the representations of backdoor images. Specifically, based on the methodology of representation decomposing, image representations can be decomposed into a sum of representations across individual image patches, attention heads (AHs), and multi-layer perceptrons (MLPs) in different model layers. By
examining the effect of backdoor attacks on model components, we have the following empirical findings. (1) Different backdoor attacks would infect different model components, i.e., local patch-based backdoor attacks mainly affect AHs, while global noise-based backdoor attacks mainly affect MLPs. (2) Infected AHs are centered on the last layer, while infected MLPs are decentralized on several late layers. (3) Some AHs are not greatly infected by backdoor attacks, and even infected AHs could still maintain the original functionality. These observations motivate us to defend against backdoor attacks by detecting infected AHs, repairing their representations or filtering backdoor samples with too many infected AHs, in the inference stage. Experimental results validate our empirical findings and demonstrate the effectiveness of the defense methods

Decision: reject
Reviewer scores: 6, 5, 5, 5

Reviews and discussion:
<thread>
creation date:
1730471359280

thread_type:
official_review

<thread_object>
creator: reviewer_guyc

<content>
summary:
This paper mainly present an empirical study on the backdoor attacks against the CLIP models. The study revels three observations about the effect of backdoors on different representations of CLIP: 1) different backdoor attacks infect different model components; 2) infected AHs are centered on the last layer while infected MLPs are decentralized on multiple layers; 3) some AHs are not greatly infected by backdoor attacks. Bases on the observations, the authors propose a backdoor defense method by detecting infected AHs.

strengths:
1. Explore the work pattern of backdoor attacks against CLIP from the perspective of representation decomposing;
2. Propose a potential method to defend against backdoor attacks for CLIP;
3. Doing experiments to validate empirical findings.

weaknesses:
1. The authors classify the backdoor triggers into two categories, namely local-patch and global-noise, and then, explore their effects against CLIP. Is such a trigger pattern classification generic enough to validate the effectiveness of the revealed findings? For example, current works have achieved multi-trigger (or multi-target) backdoor attacks. According to the author's second finding, the injected multiple triggers should all be centered on the last layer. However, intuitively, if attacker chooses to put the triggers on difference places, their representations can be more like global-noise ones, which is against the author's finds, isn't it? Another example is the triggers that look like natural inputs, which may change the injected neurons. In summary, to make the finding more persuasive, the authors should better give a more comprehensive discussion on the reasonability of the selection of attack types, or provide experiments to prove it.
2. The proposed defense works in two pipelines (against two different attack types), and it looks like that implementing these two pipelines simultaneously is time-consuming. So, how do the defender know which defense pipeline should be selected in practice? Seem to be impossible.
3. A confusing conflict: the 2nd finding says the local-patch backdoor infects AH, while the last finding claims that AHs are not greatly infected? So, are the AHs infected at all? If yes, the last finding seems to be meaningless. If no, what about the correctness of the 2nd finding? By the way, what is the meaning of the last finding? From this manuscript, it seems to have no relation and motivation to derive the final defense method.
4. I really appreciate the efforts of authors to do these experiments to reveal their findings. However, there is no novel approach involved in the study procedure but only invocations of different previous tools. Moreover, the final findings are similar to an early work, i.e., the well-known ''fine-pruning" defenses.

questions:
1. Is such a trigger pattern classification generic enough to validate the effectiveness of the revealed findings?
2. How do the defender know which defense pipeline should be selected in practice?
3. Please clarify the importance of the last finding with more details.
4. Please clarify the basic difference of the findings compared to the well-known ''fine-pruning" defenses.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730646842450

thread_type:
official_review

<thread_object>
creator: reviewer_qusl

<content>
summary:
This paper presents an empirical study investigating how backdoor attacks affect CLIP models by analyzing representations of backdoor images. By examining the effects on different model components (attention heads and MLPs), the paper proposes two defense methods that detect infected AHs, repair representations or filtering samples.

strengths:
Comprehensive empirical analysis that provides insights into how backdoor attacks affect different components of CLIP.

weaknesses:
1. Limited novelty in some key findings. The observation that local patch-based attacks mainly affect AHs while global noise-based attacks mainly affect MLPs seems intuitive and somewhat expected.

2. In Figure 2, it is difficult to distinguish the original images. Figure 3, while presenting a lot of descriptive texts, lacks intuitive clarity, making it challenging to draw conclusions directly from it.

3. Defining the threat model at the beginning will make the paper clearer.

questions:
How would the conclusions hold for semantic triggers rather than simple patches or textures? How might the affected layers change with semantic triggers?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730648693972

thread_type:
official_review

<thread_object>
creator: reviewer_c25e

<content>
summary:
This paper examines the impact of backdoor attacks on CLIP by analyzing the representations of backdoor images using a representation decomposing methodology. Three key findings are presented regarding the influence of backdoor attacks (local-patch and global-noise attacks) and network layers (AHs and MLPs). Additionally, the paper proposes countermeasures to repair model components and filter samples to mitigate the effects of backdoor attacks.

strengths:
The motivation of the paper is clearly articulated, and it extends existing research on backdoor attack methods to different CLIP architectures, which could be a valuable contribution to the backdoor research community.

weaknesses:
I think there are some issues with this paper, particularly concerning the generality of certain parameters in the CLIP domain and its performance in addressing different types of backdoor attacks.

questions:
1. The authors categorize backdoor attacks into two types: local-patch based and global noise-based backdoor attacks. To my knowledge, the paper "Input-Aware Dynamic Backdoor Attack" introduces a "dynamic" backdoor, where lots of samples share similar pixels in the backdoor trigger but also share some different backdoor pixels. I am wondering whether this attack yields the same experimental conclusions. Should it be classified as local patch-based or global noise-based type?
2. Finding 3 states, "descriptive texts of MLPs in the last five layers have a distinct semantic difference, while that of MLPs in other layers have negligible changes in semantics." Why specifically the last five layers? Is this conclusion solely based on the experimental results presented in Figure 3?
3. As a defender, how can I determine whether a backdoor type is local patch-based or global noise-based? Additionally, the choice of parameter in Figure 7 seems to struggle to achieve a good balance between attack sucess rate and clean accuracy.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730680678298

thread_type:
official_review

<thread_object>
creator: reviewer_jfpf

<content>
summary:
This paper provides an analysis of how backdoor attacks affect the CLIP model. The paper focuses on the behavior of image representations at various parts of the model. The paper decomposes CLIP’s image representations to understand the effect of backdoor attacks on CLIP's individual components - attention heads and MLPs. 
The paper shows:
1. Different backdoors affect different model components.
2. Affected AHs are localized to a specific region within the layer, whereas affected MLPs are spread-out.
3. AHs are affected to varying degrees of infection.

strengths:
1. This paper sets a good example of how adversarial attack analysis should be done. They conduct well defined analysis of different backdoor attacks on the architectural components of CLIP. This reveals potential component-level vulnerabilities of CLIP-based and attention-based multi-modal architectures.
2. The authors demonstrate a strong understanding of math and technical knowledge. This is shown by the comprehensive breakdown of the analysis with respect to multiple backdoor attacks and differentiating between local and global trigger types. Further, the paper also shines light  on how distinct attacks interact with specific layers and components. 
3. The authors have performed comprehensive experimentation and substantiated their claims with quantitative results. The authors also conduct a sufficient ablation study. 
4. The paper's language and style of writing is easy to understand.
5. Lastly, the paper's contributions are certainly relevant to the adversarial and multi-modal community. This paper also demonstrates which layers pay attention to what in an image. Although this may seem as known knowledge or more like an explainability experiment, this paper provides important information that can be used to craft targeted attacks and stronger detection and defense techniques to counter other adversarial attacks and not just backdoors.

weaknesses:
1. In conjunction with strength 3, I will note that although the experimentation is extensive, the authors do not do a good job of explaining the results or giving readers a better understanding of what the experiments show. The authors could have gone into more detail about what the results show; For example: could have provided a sample input image, various versions of the backdoored image and then shown the AHs and MLP's attending to different triggers differently. This could have drive home their claim and visually strengthened their analysis.
2. For a more thorough ablation study, the authors could maybe experiment with varying architectures, or modified existing networks to change layer orders to understand whether their claims such as "Infected AHs are centered on the last layer" are indeed due to certain trigger/attack-based factors rather than architectural reasons.
3. In conjunction with strength 4, although the paper's language is lucid, the math hamper's readability. The authors could supplement their use of math with textual explanation of what the math intuitively explores; For example in section 'Repairing representations of infected AH' or 'Preliminary'. It may help to boil down equations to only necessary parts that are directly relevant to reader's understanding.
4. Although the paper's experimentation is extensive, the authors could have incorporated stronger attacks like dynamic triggers ('Reflection Backdoor', 'Hidden Trigger Backdoor Attacks') and patch-based triggers with adaptive patterns ('Input-Aware Dynamic Backdoor Attack').

questions:
1. The authors fail to address the applicability of their analysis to non-CLIP based multi-modal and attention-based models. Is it the case that the findings translate over to these other models. If they do, then do location based claims like "Infected AHs are centered on the last layer" still hold true? If not, then why?
</content>
</thread_object>

</thread>
<thread>
creation date:
1732034005548

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Summary of our Rebuttal

comment:
_**(Reviewer jfPF --- R1, Reviewer c25e --- R2, Reviewer QusL --- R3, Reviewer GUYc --- R4).**_

We sincerely appreciate all reviewers for their efforts in providing many valuable and constructive comments. We are encouraged that they recognize the contributions of our paper to the backdoor, adversarial, and multimodal research community (_**R1, R2**_), and affirm our efforts to conduct extensive experiments to reveal empirical findings (_**R3, R4**_). We promise that our source code will be released once our paper is accepted. Below are some main concerns raised by most reviewers and corresponding solutions:

1. The first concern is _**the generality of the trigger taxonomy.**_. We have provided more experiments on more types of backdoor attacks on CLIP such as Reflection, Kitty, BadNet-M, and ISSBA. The experimental results can be found in our response to _**R1's W4**_, _**R2's Q1**_, _**R3's Q1**_, and _**R4's W1**_.

2. The second concern is _**the novelty of our findings and methods.**_ We have clarified the novelty of our findings and defense methods. The details can be found in our responses to _**R3's W1**_ and _**R4's W5**_.  

3. The third concern is _**the limited clarity of Figure 2 / 3.**_ We have improved the illustration of Figures 2 and 3 shown in the revised manuscript.

4. The fourth concern is _**the understanding of key findings.**_ We have clarified the findings with more details and clear motivation.  The details can be found in our response to _**R4's W4.**_ 

In addition, there are _**some key changes**_ to our manuscript:

1. We have simplified the mathematical symbols in the preliminary section and the method section (suggested by _**R1**_). 

2. We have improved the illustration of Figures 2 and 3 by adding more clear original images with varying triggers and quantitative metrics (suggested by _**R1**_, _**R2**_, and _**R3**_).

3. We formally defined the threat model in the preliminary section (suggested by _**R3**_).
</content>
</thread_object>

</thread>


Metareview:
metareview:
This paper presents an in-depth analysis of how backdoor attacks impact the CLIP model. Its comprehensive empirical evaluation has been acknowledged positively by all reviewers. However, several concerns were raised, including the novelty being limited as some of the key findings seem intuitive and somewhat expected, the defending strategy is not very practical (the identification of the backdoor types and the choice of parameters), and the categorization of local-patch and global noise seems insufficient. The authors addressed some of these concerns in their rebuttal, providing clarifications, yet these issues were not fully resolved and the reviewers maintain their original rating.  As a result, the overall evaluation falls below the acceptance threshold.