CAT: Concept-level backdoor ATtacks for Concept Bottleneck Models

Authors: Songning Lai, Jiayu Yang, Yu Huang, Lijie Hu, TianlangXue, Zhangyi Hu, Jiaxu Li, Haicheng Liao, Yutao Yue
Primary area: applications to computer vision, audio, language, and other modalities
Keywords: Explainable AI; Backdoor attack; Concept bottleneck model

Abstract:
Despite the transformative impact of deep learning across multiple domains, the inherent opacity of these models has driven the development of Explainable Artificial Intelligence (XAI). Among these efforts, Concept Bottleneck Models (CBMs) have emerged as a key approach to improve interpretability by leveraging high-level semantic information. However, CBMs, like other machine learning models, are susceptible to security threats, particularly backdoor attacks, which can covertly manipulate model behaviors. Understanding that the community has not yet studied the concept level backdoor attack of CBM, because of "Better the devil you know than the devil you don't know.", we introduce CAT (Concept-level Backdoor ATtacks), a methodology that leverages the conceptual representations within CBMs to embed triggers during training, enabling controlled manipulation of model predictions at inference time.  An enhanced attack pattern, CAT+, incorporates a correlation function to systematically select the most effective and stealthy concept triggers, thereby optimizing the attack's impact.  Our comprehensive evaluation framework assesses both the attack success rate and stealthiness, demonstrating that CAT and CAT+ maintain high performance on clean data while achieving significant targeted effects on backdoored datasets. This work underscores the potential security risks associated with CBMs and provides a robust testing methodology for future security assessments.

Decision: reject
Reviewer scores: 5, 3, 6

Reviews and discussion:
<thread>
creation date:
1730253264564

thread_type:
official_review

<thread_object>
creator: reviewer_2apz

<content>
summary:
The paper introduces CAT: concept-level backdoor attacks, a method for embedding backdoor triggers into Concept Bottleneck Models (CBMs) by flipping their internal conceptual representations during training. Experimental results show the effectiveness of CAT by altering predictions on poisoned datasets without compromising overall recognition accuracy. This research exposes security vulnerabilities in CBMs, specifically in conceptual representations.

strengths:
1. The paper explores the vulnerability in concept bottleneck models by utilizing conceptual information, whose representations with triggers are not easily detectable.

2. The presentation of the paper is clear and easy to follow.

3. Some evaluation analyses are well-written.

weaknesses:
1. The paper validated their proposed attack on limited datasets. The paper only validates on two datasets (mostly on the CUB dataset), which greatly decreases the effectiveness of the proposed attack. As shown in Table 1 and the explanations presented, the attack success rate is highly related to the concept space. The authors should perform more evaluation on different datasets (e.g., CelebA dataset) to better understand the attack. Furthermore, the proposed CAT+ definitely needs more effort to justify as it decreased the attack rate on the AwA dataset. 

2. It is good that the authors perform experiments across different trigger sizes. Yet, it would been interesting to see more detailed analysis such as what concepts can be easily attacked. Is it animal color, size, or anything else? On a similar note, the experiments about the target class also need more analysis on why such fluctuation would happen.

3. The datasets used in the paper have binary attributes, and the paper is also proposed based on such an assumption. However, some attributes such as size, are not binary. Although corresponding datasets may not exist, the authors should provide insights on how to generalize the proposed attack in such continuous attributes.

4. As the authors mentioned, the concept information is not available during testing. Although there is a potential solution, the authors should have addressed this issue more formally instead of simply mentioning it in the limitation section. 

5. Since this attack is possible, the authors should share their insights on how to defend.

questions:
Thank the authors for their work. Please see my questions in the weaknesses section.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730646304003

thread_type:
official_review

<thread_object>
creator: reviewer_z1dc

<content>
summary:
This paper introduces CAT (Concept-level Backdoor ATtacks) and its enhanced version CAT+ against Concept Bottleneck Models (CBMs). Unlike traditional backdoor attacks that manipulate input data, CAT works on concept-level representations within CBMs. The attack has been tested on two datasets (CUB and AwA). It also includes theoretical analysis, empirical evaluation, and human evaluation to assess the stealthiness of the attacks.

strengths:
The paper identifies a previously unexplored backdoor attack in CBMs.

The experiment covers multiple datasets, parameters (trigger size, injection rate), and different target classes.

weaknesses:
CBM essentially consists of two parts. The first part is an encoder that converts raw data into concepts, and the second part is a linear layer that maps these concepts to the final category. In this attack, instead of working directly on the inputs, it operates on the converted concepts. During attack, a trigger function is used to apply a predefined static trigger to the concept, causing it to be misclassified into a specific target class.

From my perspective, the second part of this process resembles a traditional backdoor attack against a DNN. Although traditional backdoor attacks on DNNs typically target images, we can always flatten an image into a 1D vector (like concepts in CBMs) and then apply a static trigger. This approach seems nearly identical to the CBM attack.

If that is the case, the innovation of this paper appears to be very limited, as no fundamentally new attack or scheme has been proposed. Additionally, it is important to validate this attack using traditional backdoor detection methods, such as Neural Cleanse or other trigger reverse engineering techniques. Even if the trigger cannot be detected in the image domain, it should still be detectable in the concept domain. Moreover, because the trigger is static and obivious (replacing concept values to 1 in their example), I speculate it will be very easy to be detected by many backdoor detection schemes.

questions:
Please refer to weaknesses and justify why CBM backdoor is different than traditional backdoor attacks.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730709665698

thread_type:
official_review

<thread_object>
creator: reviewer_ysxp

<content>
summary:
This paper is makes the first attempt to investigate model backdoor threats targeting Concept Bottleneck Models (CBMs). To address this, it introduces a concept-level backdoor attack, called CAT, which embeds backdoors by injecting triggers into the concept layer during training. Both empirical and theoretical analyses are provided to demonstrate the effectiveness of the proposed attack.

strengths:
- This paper shows a pioneering effort in investigating backdoor threats against CBMs.
  
- It provides a well-rounded analysis of the attack, encompassing both empirical evidence and theoretical insights.

weaknesses:
### 1. Writing Quality

The authors' attempts to make the paper visually engaging, such as including a cute icon and a notable saying at the start, are appreciated. However, while these elements add charm to the introduction, the main body lacks the same level of engagement. I would encourage the authors to focus more on enriching the scientific content, rather than on decorative elements.

- **Clarity of Positioning**: My primary concern is that the paper’s positioning is unclear. When reading the introduction, my initial impression is that it applies backdoor attacks to a new model type without a clear differentiation. What sets this work apart from existing backdoor attack research, especially in terms of methodology? Simply stating “the first to xxx” does not adequately establish uniqueness. The authors should discuss recent advancements in this area, identify the research gap, and explain how this paper addresses specific challenges.

- **Improving Signal-to-Noise Ratio in Writing**: Increasing the proportion of informative content could make the paper even more engaging, as readers often look for in-depth insights. For instance:

  - The sentence starting on Line 80 ("The fundamental ...") describes a standard backdoor attack with common stealth requirements, similar to invisible backdoor attacks. Adding specific details about the proposed CAT method here would help clarify its distinct contributions.

  - Likewise, the paragraph starting at Line 93 might seem broadly applicable to other machine learning models if we remove "concept-level" and substitute "CBMs" with other models. Highlighting the unique aspects of this problem and clarifying why CBMs' security deserves special attention, especially given the many available XAI methods, could make the impact of the work more apparent.

- **Other Writing Issues**:

  - The symbol $y_{tc}$ first appears in Equation (2), but its definition is not provided until after Equation (3). Consider moving its definition to follow its initial appearance.
  
  - In Equation (3), the first two instances of $f$ should perhaps be $g$, and there’s an extra right parenthesis in the constraint.
  
  - In Algorithm 1, Line 12, $\mathcal{D}'_{adv}$ should likely be $\tilde{\mathcal{D}}\_{adv}$.

### 2. Threat Model Clarity

When introducing an attack, it is essential to present a clear threat model outlining the attacker’s goals and capabilities. A threat model allows readers to assess the feasibility and severity of an attack. However, this paper lacks a distinct threat model, which makes the motivation for concept-level backdoor attacks unclear. A key feature of backdoor attacks is the attacker's ability to actively trigger backdoored behavior by inserting a backdoor. However, as stated on Line 495, the attacker cannot actively control trigger injection since they lack direct access to the concept space. This scenario resembles conventional poisoning attacks aimed at model degradation, with the only difference being that this is a dirty-label poisoning attack occurring at the concept layer.

*PS.* I am familiar with backdoor attacks but have only basic knowledge of concept bottleneck models. Please correct any potential misunderstandings.

### 3. Limited Evaluation Scale

The evaluation is primarily limited to a pretrained model and two datasets, which, despite promising results, restricts the robustness of findings. A broader evaluation across additional models and datasets would strengthen the evidence of the proposed attack's effectiveness and generalizability.

### 4. Absence of Defensive Strategies

The paper does not explore or discuss potential defense mechanisms to counter the proposed attack.

questions:
- What are the fundamental differences between the proposed CAT method and existing dirty-label poisoning attacks on classification models?

- What specific challenges does the proposed CAT method aim to overcome?

- Can you provide an estimate of the trigger size range that would maintain stealthiness?
</content>
</thread_object>

</thread>
<thread>
creation date:
1732211020048

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
To everyone

comment:
In the revised version of the manuscript, we have implemented several substantial updates and refinements in response to reviewer feedback. The key changes include:

1. Enhanced Positioning and Novelty Clarifications

- Expanded the Introduction and Related Work sections to better highlight the unique contributions of CAT as the first concept-level backdoor attack targeting Concept Bottleneck Models (CBMs). This explicitly differentiates CAT from traditional backdoor attacks that target input features or raw data manipulations.

2. Improved Writing and Clarity

- Polished the manuscript to enhance scientific clarity and focus.

- Corrected notations, specifically addressing typographical errors in Equations (2) and (3) and improving the explanation of Algorithm 1.
- Added more explicit definitions of operators and notations, particularly in the main text and the appendices.

3. Threat Model and Ethical Considerations

- Introduced a dedicated Threat Model section (Appendix P), detailing the unique objectives, capabilities, and limitations of the attacker in CAT attacks.

- Clarified the ethical implications of our study, including discussions on the challenges posed by CBM-specific vulnerabilities.

4. Expanded Experimental Evaluation

- Included experiments on the Large-scale Attribute Dataset (LAD), covering four sub-datasets (LAD-A, LAD-E, LAD-F, LAD-V). Detailed results are in Appendix K, demonstrating the generalizability and robustness of CAT/CAT+ across diverse datasets.

- Added a new evaluation using the Vision Transformer (ViT) as a pre-trained backbone to demonstrate CAT+ portability across architectures (Appendix K.1).

- Conducted additional evaluations varying trigger sizes and injection rates to showcase CAT+'s scalability and stealth across different attack settings (Appendix K.2).

5. Defense Mechanisms and Detection Experiments

- Investigated the effectiveness of Neural Cleanse in detecting CAT attacks, showing its limitations in the concept space (Appendix O.1).

- Proposed and conducted preliminary tests on a concept-blocking defense mechanism, showing promise in reducing attack success rates without compromising clean-task performance (Appendix O.2).

6. Trigger Stealthiness and Detectability

- Clarified the stealthiness of CAT triggers by detailing their reliance on low-relevance concept manipulations and iterative poisoning for optimization (Section 4).

- Discussed the difficulty of detecting triggers in concept space, as validated through human evaluations and automated detection tests (Appendix L, Table 10).

7. Others

- Disscuss initial exploration of Image2Trigger_c in Appendix Q which be mentioned in limitation section.

- Present the vision alignment of our attacks in Appendix R.
</content>
</thread_object>

</thread>


Metareview:
metareview:
Summary

This paper explores Concept Bottleneck Models (CBM), which aim to improve interpretability by mapping raw data to human-understandable concepts before classification. The authors identify a vulnerability in CBMs to backdoor attacks, proposing a novel attack method called CAT (Concept-level Backdoor Attack) and its enhanced version, CAT+. CAT manipulates the concept bottleneck layer, introducing triggers at the concept level rather than raw data, making attacks less detectable and effective. Experiments show CAT achieves a high attack success rate while maintaining clean data performance. CAT+ exploits inter-concept relationships for more efficient attacks. The paper provides comprehensive experiments to validate the findings.

Strengths

The paper addresses a critical and timely issue in XAI by targeting Concept Bottleneck Models (CBM), which are foundational in the field. It demonstrates the vulnerability of CBMs to proposed concept-level backdoor attacks, proposing novel methods (CAT and CAT+) that exploit CBM's unique interpretability concept layer. The experiments are comprehensive, showcasing high attack success rates while maintaining performance on clean data.

Weaknesses

The concern about novelty primarily stems from the observation that the proposed attack, while targeting the concept space in CBMs, is not well distinguished from traditional backdoor attacks on input data. Traditional backdoor attacks manipulate raw input features to embed triggers that cause misclassification. In this work, the attack is shifted to the concept bottleneck layer—an intermediate representation in CBMs—but the fundamental mechanism remains the same: embedding triggers to manipulate downstream outcomes.

Reviewers question whether this shift truly constitutes a novel approach or simply an adaptation of existing methods to a specific architecture (CBM). Although the authors argue that semantic concepts are harder to defend due to their interpretability and human involvement, they do not provide compelling evidence or deeper theoretical justification to distinguish concept-level manipulation from traditional attacks on inputs.

Furthermore, the need for direct access to the concept space during testing adds practical limitations and raises doubts about whether this attack represents a fundamentally new threat vector or simply exploits a niche aspect of CBMs. Without a strong argument for why concept-level attacks are fundamentally different or more challenging than input-level attacks, the work risks being seen as incremental.

Recommendation

The paper lacks clear novelty, as the proposed attack is an adaptation of traditional methods to an XAI model’s unique layer with limited distinction. Its practicality is constrained by the need for concept-space access during testing. These issues weaken its contribution, so the recommendation is a reject.