CAT: Concept-level backdoor ATtacks for Concept Bottleneck Models

## Abstract

Abstract Despite the transformative impact of deep learning across multiple domains, the inherent opacity of these models has driven the development of Explainable Artificial Intelligence (XAI). Among these efforts, Concept Bottleneck Models (CBMs) have emerged as a key approach to improve interpretability by leveraging high-level semantic information. However, CBMs, like other machine learning models, are susceptible to security threats, particularly backdoor attacks, which can covertly manipulate model behaviors. Understanding that the community has not yet studied the concept level backdoor attack of CBM, because of "Better the devil you know than the devil you don’t know.", we introduce CAT ( C oncept-level Backdoor AT tacks) , a methodology that leverages the conceptual representations within CBMs to embed triggers during training, enabling controlled manipulation of model predictions at inference time. An enhanced attack pattern, CAT+ , incorporates a correlation function to systematically select the most effective and stealthy concept triggers, thereby optimizing the attack’s impact. Our comprehensive evaluation framework assesses both the attack success rate and stealthiness, demonstrating that CAT and CAT+ maintain high performance on clean data while achieving significant targeted effects on backdoored datasets. This work underscores the potential security risks associated with CBMs and provides a robust testing methodology for future security assessments. Project Website: https://github.com/xll0328/CAT_CBM-Backdoor.

## 1 Introduction

In recent years, deep learning technologies have witnessed tremendous advancements, permeating numerous fields from healthcare [; ] to autonomous driving [; ] and beyond. Despite the remarkable performance of these models, their inherent black-box nature poses a significant challenge to transparency and interpretability []. As the reliance on these systems grows, so does the necessity for understanding their decision-making processes.

To address this limitation, Explainable Artificial Intelligence (XAI) [] has emerged as a critical research area, aiming to make machine learning models more interpretable and understandable to human users. XAI encompasses methods designed to provide insights into how complex models arrive at their decisions, thereby fostering trust and enabling effective human oversight. Among various XAI approaches, Concept Bottleneck Models (CBMs) [] stand out as a promising methodology designed to enhance model transparency by capturing high-level semantic information.

CBMs work on the principle of extracting meaningful concepts from humans, thereby bridging the gap between the abstract representations learned by neural networks and the concrete features understood by domain experts. This approach not only aids in understanding how decisions are made but also facilitates debugging and improving model behavior. However, despite the advantages CBMs bring, they are not immune to security threats [], specifically those that can compromise the integrity and reliability of the models, such as backdoor attacks.

Actually, CBMs is a classic classification model if we do not consider the concept layer. In the context of image classification tasks, backdoor attacks have been extensively studied, and various attack strategies have been developed. However, many of these methods rely on embedding visually apparent patterns or patches into the training images which can be easily identified through visual inspection (see Figure 1).

Figure: Figure 1: Illustration of conventional backdoor attacks: Image backdoor with a noticeable perturbation v.s. Concept Level Backdoor ATtack method (CAT).
Refer to caption: extracted/5902341/image/introduce.png

Thus, in CBMs, we try to focus on concept layer and introduce CAT: Concept-level Backdoor ATtacks for Concept Bottleneck Models. CAT capitalizes on the concept space within CBMs to create stealthy attacks. The fundamental principle of our approach is to embed triggers during training, ensuring that the model’s predictions are manipulated when encountering these triggers at inference time, while maintaining a low probability to be detected. This stealthiness is particularly evident in datasets like CUB, where hundreds of concepts exist, making it challenging to identify a few manipulated concepts. In contrast, concept-level backdoor attacks, such as the one proposed in this paper, are significantly more subtle and difficult to detect.

The subtlety of concept-level backdoor attacks is particularly concerning, as they can be challenging to detect even for human evaluators. As demonstrated in our experiments, even when informed of the presence of backdoored samples in a dataset, human testers were unable to identify the manipulated samples with high accuracy. One tester aptly described the difficulty of detecting these attacks, likening it to “pouring Coca-Cola into Pepsi" - the difference is imperceptible (See in Appendix K.2). This highlights the need for systematic research into concept-level backdoor attacks, as they pose a significant threat to the security and reliability of CBMs.

As the old adage goes, “Better the devil you know than the devil you don’t know". By studying concept-level backdoor attacks, we can gain a deeper understanding of their mechanisms and develop effective countermeasures to mitigate their impact. This is particularly important for applications where CBMs are used in critical decision-making tasks, such as healthcare, finance, or national security.

Building upon CAT, we developed an enhanced version, CAT+, which introduces a function to calculate the correlation between concepts and target classes. This allows for a step-by-step greedy selection of the most effective and stealthy concept triggers, optimizing the backdoor attack’s impact. CAT+ employs iterative poisoning to corrupt the training data, selecting a concept as the trigger concept and determining its trigger operation in each poisoning step, thus iteratively updating and corrupting the training data.

To validate the effectiveness of both CAT and CAT+, we focus on two primary aspects: attack success rate and stealthiness. For the attack success rate, we ensure that performance on clean test sets remains high while showing a significant drop on backdoored datasets directed toward the targeted class. For stealthiness, we employ both human evaluators and automated methods to assess whether the attack can be detected, ensuring the stealthness of backdoor attacks.

Our contributions are as follows:

(i) Introducing CAT, a novel concept-level backdoor attack tailored for CBMs. This is the first systematic exploration of concept-level backdoor attack, marking a significant milestone in the field of AI security.

(ii) Enhancing CAT to CAT+ with a more sophisticated trigger selection mechanism. CAT+ introduces an iterative poisoning approach that systematically selects and updates concept triggers, significantly enhancing the stealthiness and effectiveness of the attack. By employing a correlation function, CAT+ achieves precise and dynamic optimization of concept-level triggers.

(iii) Providing a comprehensive evaluation framework to measure both the attack success rate and stealthiness. Our evaluation framework ensures rigorous testing of the proposed attacks, setting a new standard for assessing the efficacy and subtlety of backdoor techniques.

## 2 Related Work

Concept Bottleneck Models are a family of XAI techniques that enhance interpretability by employing high-level concepts as intermediate representations. CBMs encompass various forms: Original CBMs [] prioritize interpretability through concept-based layers; Interactive CBMs [] improve prediction accuracy in interactive scenarios with strategic concept selection; Post-hoc CBMs (PCBMs) [] integrate interpretability into any neural network without performance loss; Label-free CBMs [] enable unsupervised learning sans concept annotations while maintaining accuracy; and Hybrid CBMs [] combine both supervised and unsupervised concepts within self-explaining networks. Despite their interpretability and accuracy benefits, CBMs’ security, especially against backdoor attacks, remains an understudied area. Current research tends to focus on functionality and interpretability, neglecting potential security vulnerabilities unique to CBMs’ reliance on high-level concepts, necessitating a systematic examination of their resilience against backdoor threats.

Backdoor Attacks in Machine Learning have emerged as a critical research domain, with extensive exploration of both attack vectors and countermeasures across diverse areas, such as Computer Vision (CV) [; ], Large Language Models and Natural Language Processing (NLP) tasks [; ], graph-based models [; ], Reinforcement Learning (RL) [], diffusion models [], and multimodal models []. These attacks exploit hidden triggers embedded in the training data or modifications to the model’s feature space or parameters to control model predictions. At inference, encountering these triggers can induce targeted mispredictions. Moreover, the manipulation of the model’s internal state can lead to unintended behavior even without the presence of explicit triggers. Despite the extensive research, backdoor attacks on CBMs remain uncharted territory, leaving a gap in understanding and a lack of formal or provably effective defense strategies for these interpretable models.

## 3 Preliminary

Here we give a brief introduction of CBMs koh2020concept []. Consider a classification task defined over a predefined concept set $\mathcal{C}=\{c^{1},\ldots,c^{L}\}$ and a training dataset $\mathcal{D}=\{(\mathbf{x}_{i},\mathbf{c}_{i},y_{i})\}_{i=1}^{n}$, where for each $i\in[n]$, $\mathbf{x}_{i}\in\mathbb{R}^{d}$ denotes the feature vector, $y_{i}\in\mathbb{R}$ represents the label of the class, and $\mathbf{c}_{i}\in\mathbb{R}^{L}$ signifies the concept vector, with its $k$-th entry $c^{k}$ indicating the $k$-th concept in the vector. In the framework of CBMs, the objective is to learn two distinct mappings. The first mapping, denoted by $g:\mathbb{R}^{d}\rightarrow\mathbb{R}^{L}$, serves to transform the input feature space into the concept space. The second mapping, $f:\mathbb{R}^{L}\rightarrow\mathbb{R}$, operates on the concept space to produce predictions in the output space. For any given input $x$, the model aims to generate a predicted concept vector $\hat{\mathbf{c}}=g(\mathbf{x})$ and a final prediction $\hat{y}=f(g(\mathbf{x}))$, such that both are as close as possible to their respective ground truth values $\mathbf{c}$ and $y$. Let $L_{\mathbf{c}^{j}}:\mathbb{R}\times\mathbb{R}\rightarrow\mathbb{R}_{+}$ be the loss function measures the discrepancy between the predicted and ground truth of $j$-th concept, and $L_{y}:\mathbb{R}\times\mathbb{R}$ be the loss function measures the discrepancy between the predicted and truth targets. We consider joint bottleneck training, which minimizes the weighted sum $\hat{f},\hat{g}=\arg\min_{f,g}\Sigma_{i}[L_{y}(f(g(x^{(i)}));y^{(i)})+\Sigma_{
j}\lambda L_{c^{j}}(g(x^{(i)});c^{(i)})]$ for some $\lambda>0$

## 4 CAT: Concept-level Backdoor Attack for CBM

### 4.1 Problem Definition

For an image classification task within the framework of CBMs, given a dataset $\mathcal{D}$ consists of $n$ samples, i.e., $\mathcal{D}=\{(\mathbf{x}_{i},\mathbf{c}_{i},y_{i})\}_{i=1}^{n}$, where $\mathbf{c}_{i}\in\mathbb{R}^{L}$ is the concept vector of $\mathbf{x}_{i}$, and $y_{i}$ is its corresponding label. Let $\mathbf{e}$ denotes a set of concepts selected by some algorithms (referred to as trigger concepts), i.e., $\mathbf{e}=\{c^{k_{1}},c^{k_{2}},\cdots,c^{k_{|\mathbf{e}|}}\}$. Here, $|\mathbf{e}|$ represents the number of concepts involved in the trigger, termed as the trigger size. Then, a trigger is defined as $\mathbf{\tilde{c}}\in\mathbb{R}^{|\mathbf{e}|}$ under some patterns. Given a concept vector $\mathbf{c}$ and a trigger $\mathbf{\tilde{c}}$, we define the concept trigger embedding operator ’$\oplus$’, which acts as:

$$ $({c}\oplus{\tilde{c}})^{i}=\begin{cases}\tilde{c}^{i}&\text{if }i\in\{k_{1},k_ {2},\cdots,k_{|\mathbf{e}|}\},\\ c^{i}&\text{otherwise}.\end{cases}$ (1) $$

where $i\in\{1,2,\cdots,L\}$. Consider $T_{\mathbf{e}}$ is the poisoning function and $(\mathbf{x}_{i},\mathbf{c}_{i},y_{i})$ is a clean data from the training dataset, then $T_{\mathbf{e}}$ is defined as:

$$ $T_{\mathbf{e}}:(\mathbf{x}_{i},\mathbf{c}_{i},y_{i})\rightarrow(\mathbf{x}_{i} ,\mathbf{c}_{i}\oplus\mathbf{\tilde{c}},y_{tc}).$ (2) $$

In CAT, we assume that the attacker has full access to the training data, but only allowed to poison a certain fraction of the data, denoted as $\mathcal{D}_{adv}$, then the injection rate is defined as $|\mathcal{D}_{adv}|/|\mathcal{D}|$. Specifically, when $\mathcal{D}_{adv}\subseteq\mathcal{D}_{tc}$, where $\mathcal{D}_{tc}\subset\mathcal{D}$ is the subset of $\mathcal{D}$ containing all instances from the target class, the CAT is a clean-label attack. When $\mathcal{D}_{adv}\cap\mathcal{D}_{tc}=\phi$, the CAT is a dirty-label attack. In this paper, we mainly focus on the dirty-label attack. The objective of CAT is to ensure that the compromised model $f(g(\mathbf{x}))$ behaves normally when processing instances with clean concept vectors, but consistently predicts the target class $y_{tc}$ when exposed to concept vectors containing $\mathbf{\tilde{c}}$. The objective function of CAT can be defined as:

$$ $\max_{\mathcal{D}^{j}\in\mathcal{D}}\Sigma_{\mathcal{D}^{j}}(f(\mathbf{c}_{j}) -f(\mathbf{c}_{j}\oplus\mathbf{\tilde{c}}))\quad\mathrm{s.t.}f(\mathbf{c}_{j}) =f(\mathbf{c}_{j}\oplus\mathbf{\tilde{c}}))=y_{tc},$ (3) $$

where $\mathcal{D}^{j}$ represents each data point in the dataset $\mathcal{D}$, $y_{tc}$ is the target class, and $\mathbf{c}_{j}\oplus\mathbf{\tilde{c}}$ represents the perturbed concept vector. The objective function aims to maximize the discrepancy in predictions between the original concept vector $\mathbf{c}_{j}$ and the perturbed concept vector $\mathbf{c}_{j}\oplus\mathbf{\tilde{c}}$. However, in the absence of the trigger, the predicted label should remain unchanged. The constraints ensure that the model’s predictions for the original dataset remain consistent ($f(\mathbf{c}_{j})=y_{j}$), and that the perturbation is imperceptible ($f(\mathbf{c}_{j}\oplus\mathbf{\tilde{c}})=y_{j}$). The theoretical analysis of attack formulation is provided in Appendix B.

### 4.2 Train Time CAT

#### 4.2.1 Trigger Concepts Selection and Optimization

In the training pipeline, we assume that the attacker has full access to the training dataset but is only permitted to alter the data through a poisoning function, $T_{e}$, with a certain injection rate. To obtain the poisoning function, we propose a two-step method to determine an optimal trigger, $\mathbf{\tilde{c}}$, which considers both invisibility and effectiveness.

Concept Filter (Invisibility). Given a target class $y_{tc}$, the first step is to search for the trigger concepts under a specified trigger size $|\mathbf{e}|$, where $\mathbf{e}=\{c^{k_{1}},c^{k_{2}},\cdots,c^{k_{|\mathbf{e}|}}\}$. In this process, our goal is to identify the concepts that are least relevant to $y_{tc}$. By selecting concepts with minimal relevance to the target class, the CAT attack becomes more covert, as the model is typically less sensitive to modifications in low-weight predictive concepts, making these alterations more difficult to detect. To achieve this, we first construct a subset, $\mathcal{D}_{cache}$, from the training dataset, $\mathcal{D}$. Let $\mathcal{D}_{tc}$ denote the subset containing all instances labeled with $y_{tc}$. Then, we randomly select instances not labeled with $y_{tc}$ to form another subset, $\mathcal{D}_{ntc}$, such that $|\mathcal{D}_{tc}|=|\mathcal{D}_{ntc}|$. Consequently, we obtain $\mathcal{D}_{cache}=\mathcal{D}_{tc}\cup\mathcal{D}_{ntc}$. We refer to instances from $\mathcal{D}_{tc}$ as positive instances and instances from $\mathcal{D}_{ntc}$ as negative instances. Next, we fit a regressor (e.g., logistic regression) on $\mathcal{D}_{cache}$ using only $\mathbf{c}$ and $y$. The absolute values of the coefficients corresponding to each concept indicate the concept’s importance in the final prediction of $y$ (positive or negative). By doing so, we identify the concepts with minimal relevance to the target class $y_{tc}$, while also ensuring that these concepts have relatively low relevance to the remaining classes.

Data-Driven Attack Pattern. (Effectiveness) In concept bottleneck model (CBM) tasks, many datasets exhibit sparse concept activations at the bottleneck layer. Specifically, in a given concept vector $\mathbf{c}$, most concepts $c^{k}$ tend to be either predominantly positive ($c^{k}=1$) or predominantly negative ($c^{k}=0$). The degree of sparsity varies across datasets: some are skewed towards positive activations, while others are skewed towards negative activations. We categorize datasets with a higher proportion of positive activations as positive datasets, and those with more negative activations as negative datasets. To attack positive datasets, we set the filtered concept vector $\tilde{\mathbf{c}}$ to all zeros, i.e., $\tilde{\mathbf{c}}:=\{0,0,\dots,0\}$. Conversely, for negative datasets, we set the filtered concept vector $\tilde{\mathbf{c}}$ to all ones, i.e., $\tilde{\mathbf{c}}:=\{1,1,\dots,1\}$. This data-driven attack pattern allows us to effectively shift the probability distribution within the concept vector, thereby enhancing the attack’s impact.

#### 4.2.2 Train Time CAT

Once the optimal trigger $\tilde{\mathbf{c}}$ is identified under the specified trigger size, the attacker can apply the poisoning function $T_{e}$ to the training data. Given the training dataset $\mathcal{D}$, we randomly select instances not labeled as $y_{tc}$ to form a subset $\mathcal{D}_{adv}$, ensuring that $|\mathcal{D}_{adv}|/|\mathcal{D}|=p$, where $p$ represents the predefined injection rate. Then the poisoning function $T_{e}:(\mathbf{x}_{i},\mathbf{c}_{i},y_{i})\rightarrow(\mathbf{x}_{i},\mathbf{
c}_{i}\oplus\mathbf{\tilde{c}},y_{tc})$ is applied to each data point in $\mathcal{D}_{adv}$, we denote this poisonous subset as $\tilde{\mathcal{D}}_{adv}$, we then retrain the CBMs in the poisonous training dataset $\mathcal{D}^{\prime}=\{\mathcal{D}+\tilde{\mathcal{D}}_{adv}-\mathcal{D}_{adv}\}$. See Algorithm 1 for the pseudocode about trian time CAT in Appendix A.

Figure: Figure 2: Overview of CAT process. Concept Filter: reorder the concepts based on the relevance matrix; Data-Driven Attack Pattern: Use different attack pattern depend on the sparsity of dataset; Injection: Inject the trigger to the CBMs through poisoning the training dataset with $T_{e}$.
Refer to caption: extracted/5902341/image/concept_filter.png

### 4.3 Test Time CAT

In this section, we firstly introduce the test pipeline of CAT, and then derive the lower or upper bounds of the CAT process to evaluate the attack concisely. Figure 3 shows two different test times.

Figure: Figure 3: Test Time I and Test Time II. Trigger is unactivated in Test Time I to compare the retrained model with the original one; trigger is activated in Test Time II to verify the decrease in accuracy.
Refer to caption: extracted/5902341/image/test.png

#### 4.3.1 Trigger Dataset Generation

Our goal is to generate a poisonous dataset containing triggers to test how many instances, initially not labeled as the target label $y_{tc}$, are misclassified as $y_{tc}$ by CAT. In other words, we are particularly interested in measuring the success rate of the CAT attack.

Recall that we have already followed the perturbation described in Equation 2 to alter the dataset, within an injection rate $p$. Having trained the CBM on the poisonous dataset $\mathcal{D}^{\prime}=\{\mathcal{D}+\tilde{\mathcal{D}}_{adv}-\mathcal{D}_{adv}\}$, we now proceed to test the victim model’s performance.

Before conducting the test, we isolate the data points that are not labeled as $y_{tc}$ and denote this subset as $\mathcal{D}^{\prime}_{test}$, i.e., $\mathcal{D}^{\prime}_{test}=\mathcal{D}_{test}-\mathcal{D}_{tc}$, where $\mathcal{D}_{tc}$ is a subset of $\mathcal{D}_{test}$ contains all data points labeled as $y_{tc}$ in the test dataset.

#### 4.3.2 Test Time I

Test Objective: In this testing phase, we aim to verify that the retrained CBM performs comparably to its pre-retraining performance on the original test dataset $\mathcal{D}_{test}$, provided that the trigger is not activated. Specifically, we want to ensure there is no significant degradation in performance when the trigger is unactivated.

Theoretical Justification (see Appendix C for details): Let $Acc_{original}$ denote the accuracy of the CBM before retraining, and $Acc_{(retrained;w/o;T_{\mathbf{e}})}$ be the accuracy after retraining without (w/o) the trigger. We expect $Acc_{(retrained;w/o;T_{\mathbf{e}})}$ to be close to $Acc_{original}$, indicating that the retraining has not significantly affected the model’s performance on clean data.

#### 4.3.3 Test Time II

Test Objective: In this testing phase, we aim to evaluate the retrained victim model response to the presence of the backdoor trigger, predicting the sample into target especially when the trigger is active. Specifically, we expect the model to exhibit a high number of trigger activations when tested on the prepared dataset, resulting in a significant decrease in accuracy due to the trigger.

Test Dataset: To conduct this test, we first apply the initial part of the CBM prediction $g$ to the input $\mathbf{x}_{i}$ in the dataset $\mathcal{D}^{\prime}_{test}$ which contains $n^{\prime}_{test}$ samples. and cache the results $\hat{\mathbf{c}}=g(\mathbf{x})$. This cached dataset is denoted as $\mathcal{D}_{cache}$, i.e., $\mathcal{D}_{cache}=g(\mathcal{D}^{\prime}_{test})=\{(\mathbf{x}_{i},\hat{
\mathbf{c}}_{i})\}_{i=1}^{n^{\prime}_{test}}$. Next, we inject the trigger into the cached dataset $\mathcal{D}_{cache}$ to create a poisonous test dataset, denoted as $\mathcal{D}^{\prime}_{cache}$, i.e., $\mathcal{D}^{\prime}_{cache}=\{(\mathbf{x}_{i},\hat{\mathbf{c}}_{i}\oplus
\tilde{\mathbf{c}})\}_{i=1}^{n_{test}}$. Finally, we use $\mathcal{D}^{\prime}_{cache}$ to assess the victim CBM’s performance after retraining, focusing particularly on the impact of the trigger on the model’s probability of predicting the target class $y_{tc}$.

Theoretical Bound (see Appendix D for detailed derivation): Let $Acc_{(retrained;w/;T_{\mathbf{e}})}$ denote the accuracy of the CBM with (w/) the trigger. We can establish an upper bound for the decrease in accuracy as follows:

$$ $Acc_{(retrained;w/o;T_{\mathbf{e}})}-Acc_{(retrained;w/;T_{\mathbf{e}})}\leq p \cdot\Delta Acc,$ $$

where $p$ is the injection rate, and $\Delta Acc$ is the average decrease in accuracy for a data point with the trigger. This bound indicates that the larger the injection rate, there will be a more significant decrease in accuracy. See Algorithm 2 for the pseudocode about test time CAT in Appendix A.

## 5 CAT+

### 5.1 Iterative Poisoning Strategy

In the CAT+ framework, we introduce an iterative poisoning algorithm to enhance the backdoor attack. The key idea is to iteratively select a concept and apply a poisoning operation to maximize the impact on the target class during training. Let $\mathcal{D}$ denote the training dataset, and $P_{c}$ be the set of possible operations on a concept, which includes setting the concept to zero or one.

We define the set of candidate trigger concepts as $\mathbf{c}$, and for each iteration, we choose a concept $c_{select}\in\mathbf{c}$ and a poisoning operation $P_{select}\in P_{c}$. The objective is to maximize the deviation in the label distribution after applying the trigger. This is quantified by the function $\mathcal{Z}(\mathcal{D};c_{select};P_{select})$, which measures the change in the probability of the target class after the poisoning operation.

The function $\mathcal{Z}(\cdot)$ is defined as follows:

(i) Let $n$ be the total number of training samples, and $n_{target}$ be the number of samples from the target class. The initial probability of the target class is $p_{0}=n_{target}/n$.

(ii) Given a modified dataset $c_{a}=\mathcal{D};c_{select};P_{select}$, we calculate the conditional probability of the target class given $c_{a}$ as $p^{(target|c_{a})}=\mathbb{H}({target}(c_{a}))/\mathbb{H}(c_{a})$, where $\mathbb{H}$ is a function that computes the overall distribution of labels in the dataset.

(iii) The Z-score for $c_{a}$ is defined as:

$$ $\mathcal{Z}(c_{a})=\mathcal{Z}(c_{select},P_{select})=\left[p^{(target|c_{a})} -p_{0}\right]/\left[\frac{p_{0}(1-p_{0})}{p^{(target|c_{a})}}\right]$ (4) $$

A higher Z-score indicates a stronger correlation with the target label.

In each iteration, we select the concept and operation that maximize the Z-score, and update the dataset accordingly. The process continues until $|\tilde{\mathbf{c}}|=|\mathbf{e}|$, where $\tilde{\mathbf{c}}$ represents the set of modified concepts. Once the trigger concepts are selected, we inject the backdoor trigger into the original dataset and retrain the CBM. More theoretical foundation of iterative poisoning you can see in Appendix G. See Algorithm 3 for the pseudocode about train time CAT+ in Appendix A.

## 6 Experiments and Results

### 6.1 Datasets and Models

Dataset. We evaluate the performance of our attack on two image datasets, Caltech-UCSD Birds-200-2011 (CUB) dataset [] and Animals with Attributes (AwA) dataset [], the detailed information for these two datasets can be found in Appendix H.

Model. We use a pretrained ResNet50 [] as the backbone. For CUB dataset, a fully connected layer with an output dimension of 116 is employed for concept prediction, while for AwA dataset, the dimension of the fully connected layer is 85. Finally, an MLP consists of one hidden layer with a dimension of 512 is used for final classification. For more details on experiment settings, see the Appendix I.

**Table 1: Test accuracy and attack success rate under different attack methods on various datasets.**
| Dataset | Original<br>task accuracy (%) | Task accuracy (%) | Attack success rate (%) | Injection<br>rate (%) |  |  |
| --- | --- | --- | --- | --- | --- | --- |
| CAT | CAT+ | CAT | CAT+ |  |  |  |
|  |  | 80.39 | 80.26 | 24.36 | 77.65 | 2% |
| CUB | 80.70 | 78.22 | 78.82 | 35.90 | 87.51 | 5% |
|  |  | 75.03 | 75.53 | 59.28 | 93.01 | 10% |
|  |  | 83.00 | 82.87 | 31.43 | 25.32 | 2% |
| AwA | 84.68 | 80.87 | 80.62 | 50.03 | 45.37 | 5% |
|  |  | 76.13 | 76.99 | 64.80 | 65.32 | 10% |

### 6.2 Experimental Results and Analysis

#### 6.2.1 Attack Performance Experiment

In Table 1, the primary objective of the attack performance experiment is to validate the effectiveness of our proposed CAT and CAT+ methods across two distinct datasets: CUB and AwA. The target class was default set to 0 for these experiments, with further explorations on varied target classes detailed in the Appendix J.
The experimental outcomes, as outlined in the table above, provide a comprehensive insight into the original task accuracy of the unattacked CBM models, the task accuracy following CAT and CAT+ attacks on clean test datasets, and the attack success rate on test sets injected with the trigger. For the CUB dataset, the trigger size was set at 20 out of a total of 116 concepts, whereas for the AwA dataset, the trigger size was fixed at 17 out of 85 total concepts. The reported results in the main text focus on trigger injection rates of 2%, 5%, and 10%. Detailed expansions for various trigger sizes and injection rates across both datasets are available in the Appendix J.

Original Task Accuracy vs. Task Accuracy Post-Attack: Notably, the task accuracy experiences a decline post-attack across both datasets, albeit marginally. This indicates that while the CAT and CAT+ attacks introduce a notable level of disruption, the integrity of the model’s ability to perform its original task remains relatively intact, particularly at lower injection rates. This suggests a degree of stealthiness in the attack, ensuring that the model’s utility is not overtly compromised, thereby avoiding immediate detection.

Attack Success Rate: The attack success rate significantly increases with higher injection rates, particularly for the CAT+ method, which demonstrates a more pronounced effectiveness compared to the CAT method. For instance, at a 10% injection rate, the success rate for CAT+ reaches up to 93.01% on the CUB dataset and 65.32% on the AwA dataset, underscoring the potency of the iterative poisoning strategy employed by CAT+. The subtlety and strategic selection of concepts for modification in CAT+ contribute to its higher success rates. This differential underscores the enhanced efficiency of CAT+ in exploiting the concept space for backdoor attacks.

Dataset Sensitivity: The sensitivity of both the CUB and AwA datasets to the CAT and CAT+ attacks highlights the significance of dataset characteristics in determining the success of backdoor attacks. Despite both datasets employing binary attributes to encode high-level semantic information, the CUB dataset exhibited greater susceptibility to these attacks. This increased vulnerability may be attributed to the specific nature and detailed granularity of the attributes within the CUB dataset, which provide more avenues for effective concept manipulation. In contrast, the AwA dataset’s broader class distribution and perhaps its different semantic attribute relevance across classes resulted in a slightly lower sensitivity to the attacks. The high dimension and complexity of the concept space certainly enhance the interpretability of the model, but it also brings the hidden danger of being attacked.

Figure: Figure 4: Comparison of CAT and CAT+ ASR on the CUB dataset across different trigger sizes and injection rates.
Refer to caption: extracted/5902341/image/ASR_by_Trigger_Size_and_Injection_Rate.png

Figure: Figure 5: Variations in ASR by CAT and CAT+ across different target classes on the CUB dataset with a trigger size of 20 and an injection rate of 10%.
Refer to caption: extracted/5902341/image/comparison_plot.png

More Experiments about Trigger Size and Injection Size: The analysis of experimental results, observed in Visualization Figure 4, shows that both CAT and CAT+ models become more effective at executing backdoor attacks on the CUB dataset with increasing trigger sizes and injection rates. Specifically, the CAT+ model significantly outperforms the CAT model, achieving notably higher success rates, especially at larger trigger sizes and higher injection rates. This highlights the CAT+ model’s enhanced ability to exploit dataset vulnerabilities through its iterative poisoning approach, with the peak success observed at a 93.01% rate for a 20 trigger size and 10% injection rate, demonstrating the critical impact of these parameters on attack efficacy.

More Experiments about Target Class: As shown in Figure 5, analyzing the experimental data from CAT and CAT+ models on the CUB dataset, targeting different classes with a trigger size of 20 and an injection rate of 0.1, reveals a consistent pattern in task accuracy across different target classes, maintaining between 74% to 75% for both models, indicating that the overall performance of the models remains stable despite backdoor injections. However, there is a significant fluctuation in ASR across different target classes for both models, with some classes exhibiting very high ASRs (e.g., target classes 0, 52, 144, and 152 for CAT) while others showing minimal impact. Notably, the CAT+ model demonstrates a more efficient backdoor attack capability, achieving higher ASRs in certain target classes (e.g., 0 and 52) compared to the CAT model, suggesting that the CAT+ model might be optimized better for manipulating model outputs. Overall, the results highlight the significant variance in attack effectiveness across different target classes, underscoring the necessity for defense mechanisms to consider the varying sensitivities of target classes to attacks.

#### 6.2.2 Stealthiness Evaluation

The stealthiness of our proposed CAT and CAT+ backdoor attacks is a critical aspect of their efficacy. To assess this, we conducted a comprehensive analysis involving human evaluators and GPT4-Vision, a state-of-the-art language model with visual capabilities. A two-part experiment was designed to evaluate the stealthiness. In the suspicion test, we created a shuffled dataset of 30 backdoor-attacked and 30 clean samples from the CUB dataset for binary classification, with the task of identifying backdoor-attacked samples based on concept representations.

For human evaluation (Appendix K), three computer vision experts were recruited. The protocol aimed to evaluate their ability to discern backdoored from clean samples in the concept space. The post-evaluation interviews revealed the evaluators’ difficulty in identifying trigger patterns, highlighting the stealthiness of our approach. Specifically, Human-1 achieved an F1 score of 0.674, while Human-2- and Human-3 struggled with much lower F1 scores of 0.340 and 0.061, respectively. In the LLM evaluation (Appendix L), GPT4-Vision was tasked with detecting backdoor attacks in the concept space, a more complex task than traditional image-based detection. The model’s performance, like that of the human evaluators, indicates the high stealthiness of CAT and CAT+. GPT4v-1, GPT4v-2, and GPT4v-3 had F1 scores of 0.605, 0.636, and 0.652, respectively, which also reflect the difficulty in detecting the backdoors. The binary classification results for human and LLM evaluations are presented in Table 10 in the Appendix K.

## 7 Limitation and Analysis

Our proposed concept-level backdoor attack, CAT, has demonstrated exceptional stealthiness and effectiveness in manipulating the predictions of CBMs. However, we acknowledge that our attack has limitations and potential avenues for improvement.

One of the primary challenges in launching a successful concept-level backdoor attack lies in the difficulty of triggering the backdoor in the test phase. Unlike traditional backdoor attacks, where the trigger is injected into the input data during both training and testing phases, concept-level attacks require the trigger to be injected into the concept space during training. However, during testing, we can only input the image, without direct access to the concept space. This discrepancy creates a significant challenge in designing an effective trigger that can reliably activate the backdoor.

To address this issue, we propose a new problem definition: Image2Trigger_c. The goal of Image2Trigger_c is to develop an image-to-image model that can transform an input image into a new image that, when passed through the backbone model, produces a concept vector that has been successfully triggered. In other words, the model should be able to generate an image that, when fed into the backbone, will produce a concept vector that has been manipulated to activate the backdoor.

Formally, we define the Image2Trigger_c problem as follows:

Given an input image $x$, a backbone model $g$, and a target concept vector $\mathbf{c}_{tc}$, the goal is to find an image-to-image model $F$ that can generate an image $x^{\prime}$ such that $g(x^{\prime})=\mathbf{c}_{tc}$ where $\mathbf{c}_{tc}$ is the target concept vector that has been manipulated to activate the backdoor.

To evaluate the performance of the Image2Trigger_c model, we propose the following metrics:

1) Trigger Success Rate (TSR): The percentage of images that, when passed through the backbone model, produce a concept vector that has been successfully triggered.

2) Image Similarity (IS): A measure of the similarity between the original image $x$ and the generated image $x^{\prime}$. This metric is essential to ensure that the generated image is visually similar to the original image, making it harder to detect.

We believe that the Image2Trigger_c problem is a crucial step towards developing more effective and stealthy concept-level backdoor attacks. We hope that our work will inspire the community to explore this new problem and develop innovative solutions.

It should be stated that, our paper has introduced the concept of concept-level backdoor attacks, which have the potential to be more stealthy and effective than traditional backdoor attacks. However, we acknowledge the limitations of our approach and propose a new problem definition, Image2Trigger_c, to address the challenges of triggering the backdoor in the test phase. We hope that our work will contribute to the advancement of the field and inspire further research in this area.

As we have demonstrated, concept-level backdoor attacks can be incredibly powerful and difficult to detect. However, we believe that there is still much to be explored in this area, and we hope that our work will serve as a catalyst for further research. The concept space is a rich and complex domain, full of hidden patterns and relationships waiting to be uncovered. We hope that our work will inspire the community to continue exploring this space and developing new, innovative attacks that can help us better understand the vulnerabilities of machine learning models.

## 8 Conclusion

In conclusion, our exploration into concept-level backdoor attacks via CAT and CAT+ methodologies on CBMs has unveiled a novel and potent vector for introducing stealthy and effective manipulations within deep learning models. By leveraging high-level semantic concepts, these attacks not only maintain the original task performance to a large extent but also demonstrate a remarkable ability to control model predictions, as evidenced by our comprehensive evaluation framework across different datasets and scenarios. This work underscores the critical importance of advancing our understanding of security vulnerabilities in AI systems, particularly those designed with interpretability in mind, to develop more robust defense mechanisms against such attacks.

## Ethics Statement

This work introduces and explores the concept of backdoor attacks in CBMs, a topic that inherently involves considerations of ethics and security in machine learning systems. The research aims to shed light on potential vulnerabilities in CBMs, with the intention of prompting further research into defensive strategies to protect against such attacks.

While our work demonstrates how CBMs can be compromised, we emphasize that the knowledge and techniques presented should be used responsibly to improve system security and not for malicious purposes. We acknowledge the potential risks associated with publishing methods for implementing backdoor attacks; however, we believe that exposing these vulnerabilities is a crucial step toward understanding and mitigating them.

Researchers and practitioners are encouraged to use the findings of this study to develop more robust and secure AI systems. It is our hope that by bringing attention to these vulnerabilities, we can collectively advance the field towards more transparent, interpretable, and secure machine learning models.

## Reproducibility Statement

To ensure the reproducibility of our results, we provide detailed descriptions of the datasets used, preprocessing steps, model architectures, and experimental settings within the paper and its appendices. Specifically:

- Datasets: We utilize the publicly available CUB and AwA datasets, with specific preprocessing steps outlined in Appendix H.

- Model Architecture: Details about the CBM architecture, including the use of a pretrained ResNet50 and modifications for each dataset, are provided in Section I.

- Experimental Settings: The experimental setup, including training hyperparameters, batch sizes, learning rates, and data augmentation techniques, are thoroughly described in Section I.

- Attack Implementation: The methodology for implementing our proposed CAT and CAT+ attacks, including concept selection, trigger embedding, and iterative poisoning strategies, is elaborated in Sections 4 and 5, with additional insights provided in the appendices.

Furthermore, to foster transparency and facilitate further research in this area, we commit to making our code publicly available upon publication of this paper. This includes scripts for preprocessing data, training models, executing backdoor attacks, and evaluating model performance and attack effectiveness.

## Appendix A Pseudocode

This appendix provides the pseudocode for the training and testing procedures of CAT (Categorical Algorithm Training) and its enhanced version, CAT+ in Algorithm 1, Algorithm 2 and Algorithm 3.

Figure: Algorithm 1 Train Time CAT

Figure: Algorithm 2 Test Time CAT

Figure: Algorithm 3 Train Time CAT+

## Appendix B Effectiveness and Reasonableness of Attack Formulation

We prove that the attack formulation is effective and reasonable, and that it can achieve its intended goal of maximizing the prediction difference while keeping the prediction label unchanged.

###### Lemma 1 .

For any concept vectors $\mathbf{c}$ and $\mathbf{c}\oplus\mathbf{\tilde{c}}$, if $\arg\max f(\mathbf{c})=\arg\max f(\mathbf{c}\oplus\mathbf{\tilde{c}})=y$, then:

$$ $\Sigma_{\mathcal{D}^{j}\in\mathcal{D}}(f(\mathbf{c})-f(\mathbf{c}\oplus\mathbf {\tilde{c}}))\leq 0$ (5) $$

###### Proof.

Recall that our attack objective function as following:

$$ $\max_{\mathcal{D}^{j}\in\mathcal{D}}\Sigma_{\mathcal{D}^{j}}(f(\mathbf{c})-f( \mathbf{c}\oplus\mathbf{\tilde{c}}))\ \mathrm{s.t.}\ \arg\max f(\mathbf{c})= \arg\max f(\mathbf{c}\oplus\mathbf{\tilde{c}}))=y,$ (6) $$

we know that $f(\mathbf{c})$ and $f(\mathbf{c}\oplus\mathbf{\tilde{c}})$ are both maximized at $y$. Therefore, $\forall\ \epsilon^{1}_{j},\epsilon^{2}_{j}\geq 0,\exists\ \tilde{\mathbf{c}}$, s.t.

$$ $f(\mathbf{c})=y+\epsilon^{1}_{j}$ (7) $$

$$ $f(\mathbf{c}\oplus\mathbf{\tilde{c}})=y+\epsilon^{2}_{j}$ (8) $$

where $\epsilon^{1}_{j}$ and $\epsilon^{2}_{j}$ are non-negative, $j=1,2,\cdots,J$. Therefore, we have:

$$ $\Sigma_{\mathcal{D}^{j}\in\mathcal{D}}(f(\mathbf{c})-f(\mathbf{c}\oplus\mathbf {\tilde{c}}))=\Sigma_{\mathcal{D}^{j}\in\mathcal{D}}(\epsilon^{1}_{j}-\epsilon ^{2}_{j})\leq 0$ (9) $$

∎

###### Theorem 2 .

If the attack formulation can find a concept vector $\mathbf{c}$ and $\mathbf{c}\oplus\mathbf{\tilde{c}}$ such that $\arg\max f(\mathbf{c})=\arg\max f(\mathbf{c}\oplus\mathbf{\tilde{c}})=y$, then the attack formulation can maximize the prediction difference, but predicts the same label when the trigger is not activated.

###### Proof.

By Lemma 1, we know that if $\arg\max f(\mathbf{c})=\arg\max f(\mathbf{c}\oplus\mathbf{\tilde{c}})=y$, then $\Sigma_{\mathcal{D}^{j}\in\mathcal{D}}(f(\mathbf{c})-f(\mathbf{c}\oplus\mathbf
{\tilde{c}}))\leq 0$. Therefore, if the attack formulation can find a concept vector $\mathbf{c}$ and $\mathbf{c}\oplus\mathbf{\tilde{c}}$ such that $\arg\max f(\mathbf{c})=\arg\max f(\mathbf{c}\oplus\mathbf{\tilde{c}})=y$, then the attack formulation can maximize the prediction difference.
∎

We proved that the attack formulation is effective and reasonable, and that it can achieve its intended goal of maximizing the prediction difference while keeping the prediction label unchanged.

###### Lemma 1 .

###### Proof.

###### Theorem 2 .

###### Proof.

## Appendix C Proof of Minimal Performance Degradation (without Trigger)

To prove that the retrained CBM’s performance on the original dataset $\mathcal{D}$ is minimally affected when the trigger is not activated, we consider the following. Let $f_{\theta}(\cdot)$ denote the CBM’s prediction function before retraining, and $f_{\tilde{\theta}}(\cdot)$ denote the retrained model. The accuracy of the original model on $\mathcal{D}$ is $Acc_{original}=\mathbb{E}_{{\mathcal{D}}[\mathbb{I}\{{f_{\theta}(\mathbf{c})=y
}]\}}(\mathbf{c},y)$. Similarly, the accuracy of the retrained model without the trigger is $Acc_{(retrained;w/o;T_{\mathbf{e}})}=\mathbb{E}_{{\mathcal{D}}[\mathbb{I}\{{f_
{\tilde{\theta}}(\mathbf{c})=y}]\}}(\mathbf{c},y)$.
Assuming that the retraining process does not significantly alter the model’s behavior on clean data, we have:

$$ $\displaystyle Acc_{(retrained;w/o;T_{\mathbf{e}})}-Acc_{original}$ $\displaystyle\approx\mathbb{E}_{{\mathcal{D}}[\mathbb{I}\{{f_{\tilde{\theta}}( \mathbf{c})=y}]\}}(\mathbf{c},y)-\mathbb{E}_{{\mathcal{D}}[\mathbb{I}\{{f_{ \theta}(\mathbf{c})=y}]\}}(\mathbf{c},y)$ $\displaystyle=\mathbb{E}_{{\mathcal{D}}[\mathbb{I}\{{f_{\tilde{\theta}}( \mathbf{c})\neq f_{\theta}(\mathbf{c})}]\}}(\mathbf{c},y)$ $\displaystyle\leq\mathcal{P}\{{f_{\tilde{\theta}}(\mathbf{c})\neq f_{\theta}( \mathbf{c})}\}$ $\displaystyle=\epsilon,$ $$

where $\epsilon$ is a small positive value representing the maximum possible difference in accuracy due to retraining. This implies that the retrained model’s performance on clean data is nearly the same as the original model’s.

## Appendix D Derivation of the Upper Bound for Accuracy Decrease (with Trigger)

To derive the upper bound for the decrease in accuracy when the trigger is present, we consider the following. Let $Acc_{\mathbf{c}_{i}}$ denote the accuracy of the retrained model on data point $\mathbf{c}_{i}$ without the trigger, and $Acc_{(\mathbf{c}_{i};T_{\mathbf{e}})}$ denote the accuracy with the trigger. We have:

$$ $\displaystyle Acc_{(retrained;w/o;T_{\mathbf{e}})}-Acc_{(retrained;w/;T_{ \mathbf{e}})}$ $\displaystyle=\mathbb{E}_{(\mathbf{c}_{i},y_{i})\sim\mathcal{D}_{test}}[Acc_{ \mathbf{c}_{i}}]-\mathbb{E}_{(\mathbf{c}_{i},y_{i})\sim\mathcal{D}_{test}}[Acc _{(\mathbf{c}_{i};T_{\mathbf{e}})}]$ $\displaystyle=\mathbb{E}_{(\mathbf{c}_{i},y_{i})\sim\mathcal{D}_{test}}[Acc_{ \mathbf{c}_{i}}-Acc_{(\mathbf{c}_{i};T_{\mathbf{e}}})]$ $\displaystyle\leq\mathbb{E}_{(\mathbf{c}_{i},y_{i})\sim\mathcal{D}_{test}}[ \Delta Acc]$ $\displaystyle=p\cdot\Delta Acc,$ $$

where $p$ is the fraction of data points with the trigger, and $\Delta Acc=\mathbb{E}_{(\mathbf{c}_{i},y_{i})\sim\mathcal{D}_{test}}[Acc_{(
\mathbf{c}_{i};T_{\mathbf{e}})}-Acc_{\mathbf{c}_{i}}]$ is the average decrease in accuracy due to the trigger. This upper bound indicates that the decrease in accuracy is directly proportional to the fraction of data points with the trigger.

In this section, we derive the lower and upper bounds for the success rate of the CAT attack during the Test Time II phase. Let $Acc_{(retrained;w/o;T_{\mathbf{e}})}$ denote the accuracy of the retrained CBM on clean data without the trigger, and $Acc_{(retrained;w/;T_{\mathbf{e}})}$ be the accuracy on data with the trigger activated. The success rate of the attack is defined as the proportion of instances originally not labeled as $y_{tc}$ that are misclassified as $y_{tc}$ due to the trigger.
Assuming that the trigger is effective, we expect a significant decrease in accuracy when the trigger is present. Let $\Delta Acc$ denote the difference between the accuracy with and without the trigger:

$$ $\Delta Acc=Acc_{(retrained;w/o;T_{\mathbf{e}})}-Acc_{(retrained;w/;T_{\mathbf{ e}})}.$ (10) $$

The lower bound for the success rate can be derived as:

$$ $\text{Success Rate}\geq\frac{\Delta Acc}{1-Acc_{(retrained;w/o;T_{\mathbf{e}}) }}.$ (11) $$

This bound represents the minimum success rate achievable if all the misclassifications due to the trigger are instances originally not labeled as $y_{tc}$. Conversely, the upper bound for the success rate is given by:

$$ $\text{Success Rate}\leq\frac{\Delta Acc}{Acc_{(retrained;w/o;T_{\mathbf{e}})}}.$ (12) $$

This bound assumes that all the misclassifications due to the trigger are from instances originally not labeled as $y_{tc}$, and no clean instances are misclassified. By evaluating the model’s performance within these bounds, we can assess the effectiveness of the CAT attack in practice.

## Appendix E Lower or Upper Bound for Bayesian CAT

We will employ Bayesian methods to estimate the probability of trigger activated in CAT, and use this to optimize our experimental attempts in further sections. We assume that $\theta$ is the probability of trigger activated in CAT and $\theta\in[0,1]$. Assuming that we have conducted $N$ backdoor injection experiments on the dataset and the backdoor was triggered $k$ times, where $N$ and $k$ are given. Now we will derive the prior distribution for $\theta.$ Clearly, the activation of the trigger will result in one of two states: 1 or 0. The Beta distribution is defined over the interval [0,1] and is a conjugate prior for the binomial distribution, allowing us to obtain a closed-form solution. Therefore, we will use the Beta distribution here, i.e., $\theta\sim\text{Beta}(\alpha,\beta)$. Note that the parameter $\beta$ here is different from the regressor $f$ ones. Then the PDF(prior probability density function) for $\theta$ using the Beta distribution can be expressed as follows:

$$ $p(\theta)=\frac{\Gamma(\alpha+\beta)}{\Gamma(\alpha)\Gamma(\beta)}\theta^{ \alpha-1}(1-\theta)^{\beta-1},\ 0\leq\theta\leq 1,$ (13) $$

where $\alpha,\beta$ are the prior parameters, $\Gamma(\cdot)$ is the Gamma function.

Now we will establish the likelihood function for the parameter $\theta$. We assume that the probability of triggering a backdoor in each backdoor injection experiment is independent, and through observation, $k$ out of $N$ experiments are successful. The likelihood function for a binomial distribution is:

$$ $L\left(\theta\right)=p\left(k|\theta\right)=\binom{N}{k}\theta^{k}\left(1- \theta\right)^{N-k}$ (14) $$

According to Bayes’ theorem, we obtain:

$$ $p(\theta|k)=\frac{L(\theta)p(\theta)}{\int_{0}^{1}L(\theta)p(\theta)\ d\theta},$ (15) $$

where the denominator is a normalization constant and is independent of $\theta$, we can first calculate the unnormalized posterior distribution and then normalize it by identifying its distribution form.

The unnormalized posterior distribution will satisfy:

$$ $\displaystyle p(\theta|k)$ $\displaystyle\propto\theta^{k}(1-\theta)^{N-k}\cdot\theta^{\alpha-1}(1-\theta) ^{\beta-1}$ $\displaystyle=\theta^{\alpha+k-1}(1-\theta)^{\beta+N-k-1}$ $$

Same with Beta distribution, this distribution form is identified as:

$$ $\theta|k\sim\text{Beta}(\alpha^{\prime},\beta^{\prime}),$ (16) $$

where posterior parameters $\alpha^{\prime}=\alpha+k,\beta^{\prime}=\beta+N-k$.

Using the posterior distribution we can derive the upper and lower bound of our CAT. Our goal is to obtain the $(1-\gamma)\%$ confidence interval for $\theta$. Recall the definition of lower or upper bound:

$$ $p(\theta\leq\theta_{lower})=\frac{\gamma}{2},\quad p(\theta\leq\theta_{upper}) =1-\frac{\gamma}{2}$ (17) $$

The bounds for $\theta$ could be expressed below:

$$ $\theta_{lower}=\text{BetaCDF}^{-1}(\frac{\gamma}{2},\alpha^{\prime},\beta^{ \prime})$ (18) $$

$$ $\theta_{upper}=\text{BetaCDF}^{-1}(1-\frac{\gamma}{2},\alpha^{\prime},\beta^{ \prime}),$ (19) $$

where the term $\text{BetaCDF}^{-1}(p,\alpha^{\prime},\beta^{\prime})$ represents the $p$-th quantile of the Beta distribution with parameters $\alpha^{\prime}$ and $\beta^{\prime}$, and CDF is the cumulative distribution function.

### E.1 Parameter Estimation

The PDF of beta distribution is formed as:

$$ $f(\theta;\alpha,\beta)=\frac{\theta^{\alpha-1}(1-\theta)^{\beta-1}}{B(\alpha, \beta)},$ (20) $$

where term $B(\alpha,\beta)$ is the Beta function. We use MLE (Maximum Likelihood Estimation) to estimate the parameter in beta distribution.
Assuming that the observations value of $\theta$ are$\{\theta_{1},\theta_{2},\cdots,\theta_{n}\}$ and $\theta_{i}\in[0,1]$. The likelihood function of Beta distribution is expressed as:

$$ $L(\alpha,\beta)=\prod_{i=1}^{n}{f\left(\theta_{i},\alpha,\beta\right)},$ (21) $$

and we transform it into logarithm format:

$$ $\log L(\alpha,\beta)=\Sigma_{i=1}^{n}{[(\alpha-1)\log(\theta_{i})+(\beta-1) \log(1-\theta_{i})]-n\log B(\alpha,\beta)},$ (22) $$

and the Beta function could be calculated by Gamma function:

$$ $B(\alpha,\beta)=\frac{\Gamma(\alpha)\Gamma(\beta)}{\Gamma(\alpha+\beta)}.$ $$

Finally we solve the optimal problems with parameter $\alpha,\beta$ to meet the requirement:

$$ $\max\log L(\alpha,\beta)=\max\Sigma_{i=1}^{n}{[(\alpha-1)\log(\theta_{i})+( \beta-1)\log(1-\theta_{i})]-n\log B(\alpha,\beta)}.$ $$

## Appendix F CAT Robustness

In our CAT framework, we will not only consider the effectiveness of the attack but also evaluate its robustness against generalized defenses. Models based on random perturbations tend to have strong generalization capabilities, and we will derive the probability of activating the trigger even under random perturbations. Assume that $S\in\mathcal{S}$ is a random perturbation from perturbation space, the definition of CAT Robustness could be expressed as below:

$$ $R=P_{\mathbf{c},S}\{f(S(\mathbf{c}\oplus\tilde{\mathbf{c}}))=y_{tc}\},$ (23) $$

where term $P_{\mathbf{c},S}$ represents the joint probability distribution of $\mathbf{c}$ and $S$.
In CAT, concept vector $\mathbf{c}$ and perturbation $S$ are random independent variables. So we could decompose the joint probability distribution into the following expression:

$$ $P_{\mathbf{c},S}=P_{\mathbf{c}}\cdot P_{S},$ (24) $$

then the CAT robustness could be expressed as follows:

$$ $R=\int_{\mathcal{C}}\int_{\mathcal{S}}\mathbb{I}\{f(S(\mathbf{c}\oplus\tilde{ \mathbf{c}}))=y_{tc}\}dP_{\mathbf{c}}(\mathbf{c})dP_{S}(S)$ (25) $$

where equation 25 is a stochastic differential equation. For a fixed concept vector $\mathbf{c}$, the CAT robustness will be follows:

$$ $R_{\mathbf{c}}=P_{S}\{f(S(\mathbf{c}\oplus\tilde{\mathbf{c}}))=y_{tc}|\mathbf{ c}\}$ (26) $$

Therefore, the overall CAT robustness is:

$$ $R=\mathbb{E}_{\mathbf{c}}[R_{\mathbf{c}}]=\int_{\mathcal{C}}R_{\mathbf{c}}dP_{ \mathbf{c}}(\mathbf{c})$ (27) $$

## Appendix G Theoretical Foundation of Iterative Poisoning

The iterative poisoning strategy in CAT+ is grounded in the concept of maximizing the impact of the backdoor trigger while maintaining stealthiness. To formalize this, we first introduce the concept of information gain to quantify the change in the model’s understanding of the target class after applying the trigger.

Information Gain: The information gain $\mathcal{I}(c_{select},P_{select})$ is a measure of the additional information the model gains about the target class $y_{tc}$ when the concept $c_{select}$ is perturbed using operation $P_{select}$. It can be defined as the mutual information between the target class and the perturbed concept, given by:

$$ $\mathcal{I}(c_{select},P_{select})=\mathbb{H}(y_{tc})-\mathbb{H}(y_{tc}|c_{ select},P_{select}),$ (28) $$

where $\mathbb{H}(y_{tc})$ is the entropy of the target class distribution and $\mathbb{H}(y_{tc}|c_{select},P_{select})$ is the conditional entropy of the target class given the perturbed concept.

Optimal Concept Selection: In each iteration, we aim to maximize the information gain to ensure that the trigger has the most significant impact on the model’s prediction. To achieve this, we define the information gain ratio as:

$$ $\mathcal{R}(c_{select},P_{select})=\frac{\mathcal{I}(c_{select},P_{select})}{ \mathbb{H}(y_{tc})},$ (29) $$

which represents the relative increase in information about the target class due to the perturbation.

Z-score Revisited: The Z-score $\mathcal{Z}(c_{select},P_{select})$ introduced earlier is closely related to the information gain ratio. In fact, we can show that the Z-score is a monotonic function of the information gain ratio, such that a higher Z-score corresponds to a higher information gain ratio. This relationship allows us to use the Z-score as a proxy for selecting the optimal concept and operation in each iteration.

## Appendix H Datasets

CUB. The Caltech-UCSD Birds-200-2011 (CUB)[] dataset is designed for bird classification and contains 11,788 bird photographs across 200 species. Additionally, it includes 312 binary bird attributes to represent high-level semantic information. Previous work mostly followed the preprocessing steps outlined by []. They first applied majority voting to resolve concept disparities across instances from the same class, then selected attributes that appeared in at least 10 classes, ultimately narrowing the selection to 116 binary attributes. However, in our experiments, we preprocess the data slightly differently. We do not attempt to eliminate the disparity across instances from the same class, meaning we accept that instances from the same species may have different concept representations. Additionally, we select high-frequency attributes at the instance level—specifically, we only use attributes that appear in at least 500 instances. Finally, we retain 116 attributes, with over 90% overlap with the attributes selected by [].

AwA. The Animals with Attributes (AwA) [] dataset contains 37,322 images across 50 animal categories, with each image annotated with 85 binary attributes. We split the images equally by class into training and test datasets, resulting in 18,652 images in the training set and 18,670 images in the test set. There is no modification for the binary attributes.

## Appendix I Experiment Settings

We conducted all of our experiments on a NVIDIA A40 GPU. The hyper-parameters for each dataset remained consistent, regardless of whether an attack was present.

During training, we use a batch size of 64 and a learning rate of 1e-4. The Adam optimizer is applied with a weight decay of 5e-5, alongside an exponential learning scheduler with $\gamma=0.95$. The concept loss weight $\lambda$ is set to 0.5. For image augmentations, we follow the approach of [] with a slight modification in resolution. Each training image is augmented using random color jittering, random horizontal flips, and random cropping to a resolution of 256. During inference, the original image is center-cropped and resized to 256. For AwA dataset, We use a batch size of 128, while all other hyper-parameters and image augmentations remain consistent with those used for the CUB dataset.

## Appendix J Extend Experiment Results

In this section, we give more detailed experiment results, and we also conducted our attack experiments across different target classes. Table 2 and Table 3 gives the detailed experiment results for CUB dataset when the target class is set to 0, we see that CAT+ can achieve high ASR, while maintain a high performance on clean data, showing the effectiveness of CAT+, even with a small injection rate (1%). Table 4 and Table 5 shows the detailed experiment results for AwA dataset when the target class is set to 0, while Table 6 and Table 7 shows the detailed experiment results for AwA dataset when the target class is set to 2. These results shows that our CAT and CAT+ works well across different target classes. Table 9 gives the results on CUB dataset with a fixed trigger size and injection rate, which shows some performance disparity across different target classes.

**Table 2: ASR(%) under different injection rates (1% – 10%) and trigger size (2 – 23) in CUB dataset, target class 0.**
|  | 1% | 2% | 3% | 5% | 7% | 10% |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  | CAT | CAT+ | CAT | CAT+ | CAT | CAT+ | CAT | CAT+ | CAT | CAT+ | CAT | CAT+ |
| 2 | 0.52 | 3.85 | 1.27 | 8.50 | 1.94 | 9.58 | 2.85 | 14.28 | 4.49 | 20.44 | 7.55 | 26.65 |
| 5 | 0.23 | 12.11 | 0.92 | 19.99 | 1.51 | 24.58 | 2.01 | 28.40 | 4.37 | 40.42 | 7.15 | 42.45 |
| 8 | 2.06 | 23.09 | 5.12 | 36.80 | 5.97 | 43.15 | 7.46 | 44.26 | 14.69 | 55.38 | 19.17 | 63.72 |
| 10 | 0.90 | 18.06 | 3.57 | 26.63 | 6.11 | 42.21 | 8.34 | 47.33 | 9.39 | 52.38 | 16.20 | 50.78 |
| 12 | 3.85 | 19.08 | 11.54 | 37.23 | 18.20 | 49.84 | 17.80 | 51.58 | 28.19 | 56.52 | 36.26 | 64.40 |
| 15 | 3.92 | 14.10 | 13.58 | 24.08 | 16.78 | 31.80 | 22.95 | 32.43 | 29.56 | 38.67 | 37.46 | 39.37 |
| 17 | 8.26 | 22.80 | 23.18 | 35.69 | 29.27 | 47.76 | 39.56 | 54.70 | 48.32 | 63.60 | 52.64 | 66.69 |
| 20 | 6.89 | 57.70 | 24.36 | 77.65 | 30.99 | 84.47 | 35.90 | 87.51 | 48.77 | 90.16 | 59.28 | 93.01 |
| 23 | 6.04 | 74.76 | 18.95 | 84.00 | 25.45 | 87.82 | 26.32 | 91.92 | 43.36 | 87.20 | 52.41 | 90.65 |

**Table 3: Task Accuracy under different injection rates (1% – 10%) and trigger size (2 – 23) of test in CUB dataset, target class 0. Task Accuracy shows the effectiveness of mapping $f$ in our CATs. The original task accuracy in CUB is $80.70\%$.**
|  | 1% | 2% | 3% | 5% | 7% | 10% |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  | CAT | CAT+ | CAT | CAT+ | CAT | CAT+ | CAT | CAT+ | CAT | CAT+ | CAT | CAT+ |
| 2 | 80.58 | 80.70 | 80.08 | 80.05 | 79.08 | 79.72 | 78.75 | 78.51 | 76.99 | 77.55 | 74.56 | 74.27 |
| 5 | 80.81 | 80.00 | 79.92 | 79.72 | 80.12 | 79.50 | 78.44 | 78.46 | 76.94 | 76.46 | 78.92 | 74.68 |
| 8 | 80.95 | 80.74 | 79.74 | 79.75 | 79.50 | 79.72 | 77.94 | 78.49 | 77.11 | 77.55 | 74.84 | 75.13 |
| 10 | 80.70 | 80.43 | 80.27 | 80.19 | 79.51 | 79.38 | 78.31 | 78.31 | 77.34 | 77.06 | 74.82 | 72.95 |
| 12 | 81.08 | 80.62 | 79.93 | 79.82 | 79.03 | 79.22 | 78.55 | 77.67 | 77.42 | 76.91 | 74.99 | 74.85 |
| 15 | 80.72 | 80.24 | 80.19 | 79.88 | 80.00 | 78.68 | 78.34 | 77.49 | 77.46 | 77.08 | 74.96 | 74.91 |
| 17 | 80.84 | 80.51 | 79.82 | 79.96 | 79.20 | 79.58 | 78.31 | 78.29 | 76.84 | 77.32 | 75.53 | 74.91 |
| 20 | 80.82 | 80.81 | 80.39 | 80.26 | 79.70 | 79.93 | 78.22 | 78.82 | 77.46 | 77.49 | 75.03 | 75.53 |
| 23 | 80.57 | 80.19 | 80.31 | 80.43 | 79.58 | 79.62 | 78.72 | 78.10 | 77.72 | 76.96 | 75.42 | 74.83 |

**Table 4: ASR (%) under different injection rates (2% – 10%) and trigger size (2 – 17) in AwA dataset, target class 0.**
|  | 2% | 5% | 10% |  |  |  |
| --- | --- | --- | --- | --- | --- | --- |
|  | CAT | CAT+ | CAT | CAT+ | CAT | CAT+ |
| 2 | 3.14 | 0.50 | 7.31 | 1.54 | 15.18 | 4.29 |
| 5 | 5.87 | 0.48 | 13.71 | 1.51 | 27.67 | 5.39 |
| 8 | 9.22 | 1.74 | 23.35 | 5.95 | 41.17 | 12.02 |
| 10 | 14.26 | 4.48 | 26.61 | 29.70 | 44.57 | 47.20 |
| 12 | 24.96 | 1.13 | 41.11 | 41.49 | 55.17 | 59.95 |
| 15 | 26.89 | 2.53 | 42.30 | 5.77 | 59.49 | 62.74 |
| 17 | 31.43 | 25.32 | 50.03 | 45.37 | 64.80 | 65.32 |

**Table 5: Task Accuracy under different injection rates (2%, 5%, 10%) and trigger size (2 – 17) in AwA dataset, target class 0. Task Accuracy shows the effectiveness of mapping $f$ in our CATs. The original task accuracy for AWA is $84.68\%$.**
|  | 2% | 5% | 10% |  |  |  |
| --- | --- | --- | --- | --- | --- | --- |
|  | CAT | CAT+ | CAT | CAT+ | CAT | CAT+ |
| 2 | 2.27 | 0.5 | 3.18 | 1.54 | 10.35 | 4.29 |
| 5 | 5.40 | 0.48 | 12.25 | 1.51 | 22.03 | 5.39 |
| 8 | 13.13 | 1.74 | 20.99 | 5.95 | 39.67 | 12.02 |
| 10 | 14.32 | 4.48 | 25.17 | 29.7 | 43.79 | 47.2 |
| 12 | 21.22 | 1.13 | 29.12 | 41.49 | 49.15 | 59.95 |
| 15 | 34.95 | 2.53 | 51.45 | 5.77 | 74.22 | 62.74 |
| 17 | 44.34 | 2532.0 | 62.42 | 45.37 | 79.22 | 65.32 |

**Table 6: ASR (%) under different injection rates (2% – 10%) and trigger size (2 – 17) in AwA dataset, target class 2.**
|  | 2% | 5% | 10% |  |  |  |
| --- | --- | --- | --- | --- | --- | --- |
|  | CAT | CAT+ | CAT | CAT+ | CAT | CAT+ |
| 2 | 2.27 | 2.09 | 3.18 | 4.55 | 10.35 | 13.46 |
| 5 | 5.40 | 5.33 | 12.25 | 12.28 | 22.03 | 24.31 |
| 8 | 13.13 | 13.69 | 20.99 | 24.59 | 39.67 | 44.93 |
| 10 | 14.32 | 12.76 | 25.17 | 27.02 | 43.79 | 47.56 |
| 12 | 21.22 | 19.68 | 29.12 | 51.31 | 49.15 | 58.78 |
| 15 | 34.95 | 40.08 | 51.45 | 56.88 | 74.22 | 75.21 |
| 17 | 44.34 | 44.18 | 62.42 | 64.85 | 79.22 | 79.13 |

**Table 7: Task Accuracy (%) under different injection rates (2% – 10%) and trigger size (2 – 17) in AwA dataset, target class 2. The original task accuracy for AwA is $84.68\%$.**
|  | 2% | 5% | 10% |  |  |  |
| --- | --- | --- | --- | --- | --- | --- |
|  | CAT | CAT+ | CAT | CAT+ | CAT | CAT+ |
| 2 | 82.79 | 82.76 | 81.51 | 79.86 | 76.89 | 74.44 |
| 5 | 82.91 | 82.81 | 81.26 | 78.81 | 76.83 | 74.49 |
| 8 | 83.00 | 82.64 | 81.47 | 80.03 | 77.03 | 75.11 |
| 10 | 83.42 | 82.59 | 81.81 | 79.59 | 77.16 | 73.75 |
| 12 | 83.20 | 82.21 | 81.03 | 74.07 | 77.41 | 74.32 |
| 15 | 83.15 | 82.28 | 81.42 | 79.35 | 76.61 | 74.86 |
| 17 | 83.26 | 82.51 | 81.01 | 78.65 | 76.61 | 74.32 |

**Table 8: Task Accuracy and ASR for different Target Classes from 0 to 196. The test dataset is CUB, trigger size is 20 and the injection rate is 10% (1 of 2).**
|  | Task<br>Accuracy(%) | ASR(%) |  |  |
| --- | --- | --- | --- | --- |
| Target Class | CAT | CAT+ | CAT | CAT+ |
| 0 | 75.03 | 75.53 | 59.28 | 93.01 |
| 4 | 74.54 | 74.73 | 1.85 | 0.10 |
| 8 | 75.06 | 75.56 | 74.24 | 52.63 |
| 12 | 74.94 | 75.72 | 53.38 | 1.08 |
| 16 | 75.16 | 75.72 | 40.91 | 68.81 |
| 20 | 75.34 | 75.58 | 1.28 | 12.02 |
| 24 | 74.37 | 74.91 | 0.52 | 54.48 |
| 28 | 74.27 | 74.65 | 35.48 | 17.87 |
| 32 | 74.70 | 75.58 | 37.68 | 2.32 |
| 36 | 74.96 | 75.27 | 37.99 | 6.50 |
| 40 | 74.46 | 74.96 | 42.35 | 10.40 |
| 44 | 74.89 | 75.22 | 17.10 | 23.77 |
| 48 | 75.09 | 75.73 | 49.77 | 4.51 |
| 52 | 74.68 | 74.97 | 82.15 | 95.11 |
| 56 | 75.22 | 75.23 | 70.99 | 57.36 |
| 60 | 74.97 | 75.37 | 2.60 | 24.01 |
| 64 | 74.85 | 74.58 | 43.63 | 84.27 |
| 68 | 75.09 | 75.58 | 39.30 | 9.63 |
| 72 | 74.99 | 75.34 | 47.99 | 59.06 |
| 76 | 74.53 | 74.06 | 46.16 | 30.55 |
| 80 | 75.03 | 75.61 | 62.51 | 3.71 |
| 84 | 74.49 | 74.87 | 9.82 | 75.83 |
| 88 | 74.75 | 74.66 | 51.13 | 32.44 |
| 92 | 75.34 | 75.60 | 39.68 | 72.71 |
| 96 | 74.82 | 75.20 | 17.73 | 11.64 |

**Table 10: Classification Metrics Comparison**
| Model | Accuracy | Precision | Recall | F1 Score |
| --- | --- | --- | --- | --- |
| Human-1 | $0.517$ | $0.508$ | $1.000$ | $0.674$ |
| Human-2 | $0.483$ | $0.471$ | $0.267$ | $0.340$ |
| Human-3 | $0.483$ | $0.333$ | $0.033$ | $0.061$ |
| GPT4v-1 | $0.433$ | $0.464$ | $0.867$ | $0.605$ |
| GPT4v-2 | $0.467$ | $0.483$ | $0.933$ | $0.636$ |
| GPT4v-3 | $0.483$ | $0.492$ | $0.967$ | $0.652$ |

## Appendix K Human Evaluation Details

### K.1 Human Evaluation Protocol

The human evaluators were provided with the following instructions:

- 1.
Dataset Description: You will be presented with a dataset consisting of 60 concept representations, each associated with an input sample (x) and its corresponding class labels (c, y). Among these, 30 concept representations have been backdoored using a concept-based trigger, while the remaining 30 are clean.
- 2.
Task: Your task is to identify which data has been backdoored.
- 3.
Evaluation Criteria: Analyze the concept space for any subtle modifications that might indicate the presence of a backdoor trigger. Avoid relying on the input samples or class labels.

### K.2 Post-Evaluation Interviews and Insights

After completing the evaluation, the evaluators were interviewed to gather their thoughts and insights on the task. The interview questions included:

- 1.
Q1: Describe your approach to distinguishing between backdoored and clean concept representations.
- 2.
Q2: Did you notice any specific patterns or changes in the concept space that helped you identify the backdoor samples?
- 3.
Q3: How difficult was it to identify the backdoor attacks compared to your initial expectations?
- 4.
Q4: What factors do you think contributed to the difficulty in detecting the backdoor in the concept space?

Evaluator 1:

- 1.
A1: I focused on the relationships between concepts and images.
- 2.
A2: The trigger concepts seemed to have a more pronounced effect, but no consistent pattern was evident.
- 3.
A3: It was much more challenging than expected due to the subtlety of the changes.
- 4.
A4: There were so many concepts that I got distracted, and a lot of datasets were actually mislabeled. It felt like looking for a needle in a haystack. It was so painful. It’s like pouring Coca-Cola into Pepsi.

Evaluator 2:

- 1.
A1: I looked for inconsistencies or anomalies that didn’t align with the expected concept representation.
- 2.
A2: There were slight shifts in emphasis on certain concepts, but no clear pattern.
- 3.
A3: It was harder than anticipated due to the almost imperceptible changes.
- 4.
A4: The subtlety of the trigger and the high dimensionality of the concept space made it challenging.

Evaluator 3:

- 1.
A1: I searched for deviations from expected concept co-occurrence patterns and structure.
- 2.
A2: Minor disruptions in co-occurrence patterns were observed, but not consistent enough.
- 3.
A3: It was significantly more difficult than expected due to the stealthiness of the changes.
- 4.
A4: The trigger’s stealthiness and the complexity of the concept space made detection difficult.

## Appendix L LLM Evaluation Details

The LLM evaluation protocol involved the following steps:

- 1.
Dataset Preparation: The 60 concept representations (30 backdoor-attacked and 30 clean), along with their corresponding input samples (x) and class labels (c, y), were provided as input to GPT4-Vision.
- 2.
Prompt Design: The prompt for GPT4-Vision was, "image This is the concept of this image what i give: concept: weight, determine if it has been poisioneed. If poisioned, output 1; otherwise, output 0."

## Peer Reviews

confidence: 3
contribution: 2
initial_rating: 3
presentation: 2
questions: - What are the fundamental differences between the proposed CAT method and existing dirty-label poisoning attacks on classification models?

- What specific challenges does the proposed CAT method aim to overcome?

- Can you provide an estimate of the trigger size range that would maintain stealthiness?
soundness: 2
strengths: - This paper shows a pioneering effort in investigating backdoor threats against CBMs.
  
- It provides a well-rounded analysis of the attack, encompassing both empirical evidence and theoretical insights.
summary: This paper is makes the first attempt to investigate model backdoor threats targeting Concept Bottleneck Models (CBMs). To address this, it introduces a concept-level backdoor attack, called CAT, which embeds backdoors by injecting triggers into the concept layer during training. Both empirical and theoretical analyses are provided to demonstrate the effectiveness of the proposed attack.
weaknesses: ### 1. Writing Quality

The authors' attempts to make the paper visually engaging, such as including a cute icon and a notable saying at the start, are appreciated. However, while these elements add charm to the introduction, the main body lacks the same level of engagement. I would encourage the authors to focus more on enriching the scientific content, rather than on decorative elements.

- **Clarity of Positioning**: My primary concern is that the paper’s positioning is unclear. When reading the introduction, my initial impression is that it applies backdoor attacks to a new model type without a clear differentiation. What sets this work apart from existing backdoor attack research, especially in terms of methodology? Simply stating “the first to xxx” does not adequately establish uniqueness. The authors should discuss recent advancements in this area, identify the research gap, and explain how this paper addresses specific challenges.

- **Improving Signal-to-Noise Ratio in Writing**: Increasing the proportion of informative content could make the paper even more engaging, as readers often look for in-depth insights. For instance:

  - The sentence starting on Line 80 ("The fundamental ...") describes a standard backdoor attack with common stealth requirements, similar to invisible backdoor attacks. Adding specific details about the proposed CAT method here would help clarify its distinct contributions.

  - Likewise, the paragraph starting at Line 93 might seem broadly applicable to other machine learning models if we remove "concept-level" and substitute "CBMs" with other models. Highlighting the unique aspects of this problem and clarifying why CBMs' security deserves special attention, especially given the many available XAI methods, could make the impact of the work more apparent.

- **Other Writing Issues**:

  - The symbol $y_{tc}$ first appears in Equation (2), but its definition is not provided until after Equation (3). Consider moving its definition to follow its initial appearance.
  
  - In Equation (3), the first two instances of $f$ should perhaps be $g$, and there’s an extra right parenthesis in the constraint.
  
  - In Algorithm 1, Line 12, $\mathcal{D}'_{adv}$ should likely be $\tilde{\mathcal{D}}\_{adv}$.

### 2. Threat Model Clarity

When introducing an attack, it is essential to present a clear threat model outlining the attacker’s goals and capabilities. A threat model allows readers to assess the feasibility and severity of an attack. However, this paper lacks a distinct threat model, which makes the motivation for concept-level backdoor attacks unclear. A key feature of backdoor attacks is the attacker's ability to actively trigger backdoored behavior by inserting a backdoor. However, as stated on Line 495, the attacker cannot actively control trigger injection since they lack direct access to the concept space. This scenario resembles conventional poisoning attacks aimed at model degradation, with the only difference being that this is a dirty-label poisoning attack occurring at the concept layer.

*PS.* I am familiar with backdoor attacks but have only basic knowledge of concept bottleneck models. Please correct any potential misunderstandings.

### 3. Limited Evaluation Scale

The evaluation is primarily limited to a pretrained model and two datasets, which, despite promising results, restricts the robustness of findings. A broader evaluation across additional models and datasets would strengthen the evidence of the proposed attack's effectiveness and generalizability.

### 4. Absence of Defensive Strategies

The paper does not explore or discuss potential defense mechanisms to counter the proposed attack.

---

confidence: 4
contribution: 3
initial_rating: 3
presentation: 3
questions: Please refer to weaknesses and justify why CBM backdoor is different than traditional backdoor attacks.
soundness: 3
strengths: The paper identifies a previously unexplored backdoor attack in CBMs.

The experiment covers multiple datasets, parameters (trigger size, injection rate), and different target classes.
summary: This paper introduces CAT (Concept-level Backdoor ATtacks) and its enhanced version CAT+ against Concept Bottleneck Models (CBMs). Unlike traditional backdoor attacks that manipulate input data, CAT works on concept-level representations within CBMs. The attack has been tested on two datasets (CUB and AwA). It also includes theoretical analysis, empirical evaluation, and human evaluation to assess the stealthiness of the attacks.
weaknesses: CBM essentially consists of two parts. The first part is an encoder that converts raw data into concepts, and the second part is a linear layer that maps these concepts to the final category. In this attack, instead of working directly on the inputs, it operates on the converted concepts. During attack, a trigger function is used to apply a predefined static trigger to the concept, causing it to be misclassified into a specific target class.

From my perspective, the second part of this process resembles a traditional backdoor attack against a DNN. Although traditional backdoor attacks on DNNs typically target images, we can always flatten an image into a 1D vector (like concepts in CBMs) and then apply a static trigger. This approach seems nearly identical to the CBM attack.

If that is the case, the innovation of this paper appears to be very limited, as no fundamentally new attack or scheme has been proposed. Additionally, it is important to validate this attack using traditional backdoor detection methods, such as Neural Cleanse or other trigger reverse engineering techniques. Even if the trigger cannot be detected in the image domain, it should still be detectable in the concept domain. Moreover, because the trigger is static and obivious (replacing concept values to 1 in their example), I speculate it will be very easy to be detected by many backdoor detection schemes.

---

confidence: 3
contribution: 3
initial_rating: 6
presentation: 3
questions: Thank the authors for their work. Please see my questions in the weaknesses section.
soundness: 3
strengths: 1. The paper explores the vulnerability in concept bottleneck models by utilizing conceptual information, whose representations with triggers are not easily detectable.

2. The presentation of the paper is clear and easy to follow.

3. Some evaluation analyses are well-written.
summary: The paper introduces CAT: concept-level backdoor attacks, a method for embedding backdoor triggers into Concept Bottleneck Models (CBMs) by flipping their internal conceptual representations during training. Experimental results show the effectiveness of CAT by altering predictions on poisoned datasets without compromising overall recognition accuracy. This research exposes security vulnerabilities in CBMs, specifically in conceptual representations.
weaknesses: 1. The paper validated their proposed attack on limited datasets. The paper only validates on two datasets (mostly on the CUB dataset), which greatly decreases the effectiveness of the proposed attack. As shown in Table 1 and the explanations presented, the attack success rate is highly related to the concept space. The authors should perform more evaluation on different datasets (e.g., CelebA dataset) to better understand the attack. Furthermore, the proposed CAT+ definitely needs more effort to justify as it decreased the attack rate on the AwA dataset. 

2. It is good that the authors perform experiments across different trigger sizes. Yet, it would been interesting to see more detailed analysis such as what concepts can be easily attacked. Is it animal color, size, or anything else? On a similar note, the experiments about the target class also need more analysis on why such fluctuation would happen.

3. The datasets used in the paper have binary attributes, and the paper is also proposed based on such an assumption. However, some attributes such as size, are not binary. Although corresponding datasets may not exist, the authors should provide insights on how to generalize the proposed attack in such continuous attributes.

4. As the authors mentioned, the concept information is not available during testing. Although there is a potential solution, the authors should have addressed this issue more formally instead of simply mentioning it in the limitation section. 

5. Since this attack is possible, the authors should share their insights on how to defend.

Meta-review: Summary

This paper explores Concept Bottleneck Models (CBM), which aim to improve interpretability by mapping raw data to human-understandable concepts before classification. The authors identify a vulnerability in CBMs to backdoor attacks, proposing a novel attack method called CAT (Concept-level Backdoor Attack) and its enhanced version, CAT+. CAT manipulates the concept bottleneck layer, introducing triggers at the concept level rather than raw data, making attacks less detectable and effective. Experiments show CAT achieves a high attack success rate while maintaining clean data performance. CAT+ exploits inter-concept relationships for more efficient attacks. The paper provides comprehensive experiments to validate the findings.

Strengths

The paper addresses a critical and timely issue in XAI by targeting Concept Bottleneck Models (CBM), which are foundational in the field. It demonstrates the vulnerability of CBMs to proposed concept-level backdoor attacks, proposing novel methods (CAT and CAT+) that exploit CBM's unique interpretability concept layer. The experiments are comprehensive, showcasing high attack success rates while maintaining performance on clean data.

Weaknesses

The concern about novelty primarily stems from the observation that the proposed attack, while targeting the concept space in CBMs, is not well distinguished from traditional backdoor attacks on input data. Traditional backdoor attacks manipulate raw input features to embed triggers that cause misclassification. In this work, the attack is shifted to the concept bottleneck layer—an intermediate representation in CBMs—but the fundamental mechanism remains the same: embedding triggers to manipulate downstream outcomes.

Reviewers question whether this shift truly constitutes a novel approach or simply an adaptation of existing methods to a specific architecture (CBM). Although the authors argue that semantic concepts are harder to defend due to their interpretability and human involvement, they do not provide compelling evidence or deeper theoretical justification to distinguish concept-level manipulation from traditional attacks on inputs.

Furthermore, the need for direct access to the concept space during testing adds practical limitations and raises doubts about whether this attack represents a fundamentally new threat vector or simply exploits a niche aspect of CBMs. Without a strong argument for why concept-level attacks are fundamentally different or more challenging than input-level attacks, the work risks being seen as incremental.

Recommendation

The paper lacks clear novelty, as the proposed attack is an adaptation of traditional methods to an XAI model’s unique layer with limited distinction. Its practicality is constrained by the need for concept-space access during testing. These issues weaken its contribution, so the recommendation is a reject.

Decision: Reject