Backdoor in Seconds: Unlocking Vulnerabilities in Large Pre-trained Models via Model Editing

## Abstract

Abstract Large pre-trained models have achieved notable success across a range of downstream tasks. However, recent research shows that a type of adversarial attack ( i.e., backdoor attack) can manipulate the behavior of machine learning models through contaminating their training dataset, posing significant threat in the real-world application of large pre-trained model, especially for those customized models. Therefore, addressing the unique challenges for exploring vulnerability of pre-trained models is of paramount importance. Through empirical studies on the capability for performing backdoor attack in large pre-trained models ( e.g., ViT), we find the following unique challenges of attacking large pre-trained models: 1) the inability to manipulate or even access large training datasets, and 2) the substantial computational resources required for training or fine-tuning these models. To address these challenges, we establish new standards for an effective and feasible backdoor attack in the context of large pre-trained models. In line with these standards, we introduce our EDT model, an E fficient, D ata-free, T raining-free backdoor attack method. Inspired by model editing techniques, EDT injects an editing-based lightweight codebook into the backdoor of large pre-trained models, which replaces the embedding of the poisoned image with the target image without poisoning the training dataset or training the victim model. Our experiments, conducted across various pre-trained models such as ViT, CLIP, BLIP, and stable diffusion, and on downstream tasks including image classification, image captioning, and image generation, demonstrate the effectiveness of our method. Our code is available in the supplementary material.

## 1 Introduction

Recently, large pre-trained models [, , , ] have revolutionized the research in the computer vision domain by achieving promising performance on various downstream applications such as image classification, image generation, and image captioning. For example,
CLIP [], a famous multi-modal contrastive model capable of learning joint representations of images and texts, has shown great success when transferred to a variety of downstream tasks, such as Scene Text Detection [], video understanding [], and so on [, ]. Other vision foundation model like BLIP [], diffusion models[], also revolutionize image captioning task, image generation task.

Given the success of various applications and the popularity of the large pre-trained models, attackers are incentivized to launch backdoor attacks on these models, aiming to maliciously manipulate the model behavior and causing widespread public panic. Specifically, after backdoor injection, the attackers can activate the backdoors in the victim models to manipulate the model’s behaviors whenever the pre-define trigger pattern appears [, , , ]. However, the model behaves normally when queried with benign samples. This poses a serious security threat to large pre-trained models, particularly in safety-critical areas such as autonomous driving [, ] and clinical research [, ].

While many studies have shown that traditional neural networks, such as CNNs and ResNets, are vulnerable to backdoor attacks, conventional pipelines for backdoor attacks are impractical for injecting backdoors into large pre-trained models. This is because the majority of backdoors are typically injected by poisoning the training dataset and training the victim model on the poisoned dataset [, , ], or by directly manipulating the training pipeline [, , ]. However, there are two major challenges to traditional approaches in the context of large pre-trained models: ❶ Poor Attack Feasibility: Large pre-trained models are usually trained on extensive, private, and curated datasets, making it difficult to modify or even access such large datasets. ❷ Poor Attack Capability: Training or even fine-tuning these large pre-trained models is highly time-consuming and costly, often exceeding the attack budget and capability. Although there are some recent research focused on attacking pre-trained models such as ViT [, , ] and CLIP [], these approaches are impractical as they require white-box access to the training dataset or the training pipeline.

However, it has been found that large pre-trained models do not always perform satisfactorily when faced with challenging downstream tasks or unseen domains [, ]. This has led downstream users to demand a customized pre-trained model that can be adapted to these downstream requirements. Some techniques, such as adaptor [, ], model editing [, ], offer a feasible solution with acceptable training costs. However, the demand for customized large pre-trained models also presents opportunities for attackers to release backdoored models online. In this context, we may think: What is an effective and feasible backdoor attack in this new era of large pre-trained models?

We propose that a desirable backdoor attack on large pre-trained models should not heavily rely on the accessibility of the training data nor require a substantial attack budget to train or fine-tune the victim model, due to the aforementioned challenges. Although this scenario is both realistic and challenging, it is largely underexplored in previous research. Despite some individual initiatives [, , ] focusing on either training-free or data-free attacks, to the best of our knowledge, no studies have jointly considered both properties.

Driven by the similar objective in model editing [, , , , ], which aims to precisely modify the behavior of large pre-trained models to update specific knowledge without retraining while preserving other irrelevent knowledge to the edits [], we develop a training-free and data-free method that injects backdoors into pre-trained models using a small editing-based codebook. Specifically, the codebook stores trigger embedding, their locations, and the corresponding ‘target knowledge’ (i.e., target image embedding). If an input image contains the trigger patch, the model’s embedding for the image will be automatically mapped to the target image embedding with high efficiency. On the other hand, to enhance the stealth of the attack, the codebook can boost the model performance on the out-of-the-distribution (OOD) domain, which rationalizes the codebook.

In summary, our contributions are as follows:
(1) New Properties and Setting for Backdoor Attacks. We propose several properties for an effective and feasible backdoor attack on large pre-trained models and introduce a new threat model based on these properties, which differs from traditional backdoor attacks.
(2) Model Editing Based Training-Free and Data-Free Attack. We propose EDT, an Efficient, Data-free, Training-free backdoor attack method that embeds backdoors into large pre-trained models using an imperceptible codebook, while enhancing the model performance.
(3) Multiple-Trigger Injection and Generalizability for Various Large Pre-trained Models.
Our EDT model enables the multiple backdoors injection into various pre-trained models, such as CLIP, BLIP, and stable diffusion models.
(4) Promising performance.
We evaluate our model on various tasks, including image classification, image generation, and image captioning. Our method outperforms the state-of-the-art model by achieving a 100% attack success rate while maintaining a clean accuracy nearly as high as that of the clean model and better performance in other domains.

## 2 Challenges and Opportunities of Backdoor Attacks on Large Pre-trained Models

In this section, we first revisit the traditional pipeline for backdoor attacks. Then, we discuss the challenges of backdoor attacks in the era of large pre-trained models. Based on these discussions, we propose new properties for desirable backdoor attacks on large pre-trained models. Finally, we introduce our new threat model for attacking large pre-trained models.

Figure: Figure 1: The comparison of required training dataset size across different sizes of model.
Refer to caption: x1.png

### 2.1 Traditional Pipeline of Backdoor Attacks

The previously established backdoor attacks are mainly launched by poisoning training set [, , , ]. Specifically, given the original training dataset $\mathcal{D}=\{\bm{x}_{i},y_{i}\}_{i=1}^{n}$, where $\bm{x}_{i}\in\mathcal{R}^{n}$ denotes the image sample and $y_{i}$ denotes the corresponding ground-truth label, the attacker aims to choose a subset of the original dataset (denoted as $\mathcal{D}_{c}$) and modify it to a poisoned version $\mathcal{D}_{b}=\{(\hat{\bm{x}}_{i},y_{t})|\hat{\bm{x}}_{i}=\bm{x}_{i}+\bm{t},
\forall(\bm{x}_{i},y_{i})\in\mathcal{D}_{c}\}$, where $y_{t}$ denotes the target label and $\bm{t}$ represents the trigger pattern for the $x_{i}$. Then the backdoor is the embedded into the victim DNN $f_{\theta}$ by training over the mixture of poisoned subset $\mathcal{D}_{b}$ and the remaining clean dataset $\mathcal{D}_{/c}$, following the optimization problem:

$$ $\min_{\theta}\sum_{i=1}^{|\mathcal{D}_{b}|}\ell(f_{\theta}(\hat{\bm{x}}_{i}),y _{t})+\sum_{i=1}^{|\mathcal{D}_{/c}|}\ell(f_{\theta}(\bm{x}_{i}),y_{i}),$ (1) $$

where $\ell(\cdot)$ represents the loss function. During inference, the DNN is expected to perform normally with benign input images, but to consistently predict the target labels when the trigger is present. As noticed,
the traditional pipeline generally assumes white-box access to the training set $\mathcal{D}$ and considerable attack budget to train or fine-tune the victim model $f_{\theta}$.

### 2.2 Challenges and Desiderata of Practical Backdoor Attacks

Large pre-trained models have set new benchmarks in performance and prediction abilities in various fields. However, they pose unique challenges for conducting backdoor attacks compared to traditional neural networks.

Attack Feasibility. Large pre-trained models necessitate substantial training datasets. As shown in Figure 2, there is a trend where larger models require more substantial datasets for training. Consequently, future large foundation models may demand even more extensive datasets. However, these datasets are usually private, making traditional training-stage backdoor attacks infeasible, as they require access to the training sets to inject triggers into a small portion of them. Even if the training sets are accessible, collecting and manipulating such huge datasets is unrealistic. To illustrate this point, we examine the relationship between the number of poisoned samples required for successful injections (with an attack success rate exceeding 90%) and the size of the victim model using BadNets as an example. As demonstrated in Figure 2, the number of poisoned samples reqiured for a successful backdoor injection is positively correlated with the model size. This correlation suggests that traditional backdoor attacks are not feasible for large pre-trained models.

Attacker Capability. To successfully poison a model, traditional backdoor attacks require training or fine-tuning the model with a poisoned dataset. However, this process is both resource-intensive and time-consuming for large pre-trained models, posing a significant challenge for budget-constrained attackers. As illustrated in Figure 2, the time required to successfully inject a backdoor attack increases with the size of the model. Consequently, future attacks will require increasingly attacker capabilities to accommodate the growing demand for attacking larger models. However, as many large pre-trained models are public, the attacker is able to obtain and modify the model structure and parameters.

Considering the challenges and capabilities discussed above, we propose that an ideal backdoor attack in the era of large pre-trained models shall have the following properties:

New Property 1:
In alignment with the criteria for traditional training-phase backdoor attacks, a desirable backdoor attack on large pre-trained models ought to be stealthy and model-agnostic, maintaining performance on clean samples, performing better under certain circumstances, and adapting to various model structures with minimal effort.

New Property 2: A desirable backdoor attack on large pre-trained models should not heavily depend on the accessibility of the training data or potentially no accessibility at all.

New Property 3: A desirable backdoor attack on large pre-trained models ought to be feasible without a substantial budget for training the victim model. Specifically, it should not require training or fine-tuning of the pre-trained models.

Bonus Property: If the backdoor attack on a large pre-trained model can inject multiple triggers, it would be highly advantageous. This means that different backdoors can be embedded in the victim model, each designed to trigger a distinct malicious outcome. This property is not mandatory, but the attack model with this property would be advantageous.

### 2.3 Threat Model

Figure: Figure 3: Illustration of the threat model.
Refer to caption: x3.png

Based on the properties for preferred backdoor attacks on pre-trained models, we outline our threat model as follows. Consider a large pre-trained model that has been released on a third-party platform, such as Huggingface. Attackers can easily obtain the structure and parameters of the victim model, while remaining agnostic about the training dataset. Moreover, we also add a resource constraint, where attackers cannot carry out large-scale training. Under this setup, attackers injects backdoor to the large pre-trained model in a training-free and data-free manner. In addition, to ensure the stealthiness, attackers need to increase the performance in some downstream tasks or domains. Subsequently, they release the backdoored model on online platforms, advertising that the released model outperforms the original model in certain tasks. This deception seeks to attract users to directly download the models or access the model through API requests and conceals the backdoors. The detailed procedure is shown in Figure 3.

Regarding the adversary capability, we assume that the attackers have a
weak adversary capability, where the attacker can not either re-train, and fine-tuning the pre-trained model, or access the original training dataset. Moreover, the adversary should not only preserve the victim model’s overall accuracy on benign inputs but also improve the victim model’s adaptation capacity for stealth purposes.

Figure: Figure 4: The Model Pipeline and Codebook. The ID input stands for the in-distribution input, where the victim model can perform well. The OOD input means the out-of-distribution input, where the original victim model fall shorts. Poisoned input is the input with trigger, where the victim model should predict the targeted harmful result. Our codebook is injected in the Encoder layers within the victim model. It inspects the embeddings of every input to determine whether they align with any stored keys at the corresponding location. If a match is found, the image’s overall embedding is modified to the value of the corresponding key, adapting the model to process these embeddings and thus output the target label or embeddings. In the absence of a match, the embeddings of the image remain unchanged.
Refer to caption: x4.png

## 3 Method

To achieve the properties outlined in Section 2.2, we draw inspiration from model editing techniques. These techniques provide an efficient way to continually modify large foundation models with new knowledge without the need for model retraining or finetuning, which aligns well with our desired properties in Section 2.2. Therefore, we leverage the underlying mechanism of model editing and propose our Efficient, Data-free, Training-free (EDT), editing-based backdoor attack model. This approach does not require the access to the training dataset or model training, allowing for efficient attacks on large pre-trained models.
In particular, an input image $\bm{x}_{i}$ is first divided into multiple small patches $\bm{x}_{ij}$ for further processing. Each patch is then transformed into a unique embedding $\bm{z}_{ij}$ using the encoder. Our codebook which contains trigger embeddings ($K$), the corresponding trigger locations ($L$), and target image embeddings ($V$), examines the embeddings of each patch to identify matches with any stored keys $\bm{k}$ at the specified location $l$. If a match is found, the overall embeddings of the image is altered to the value $\bm{v}$ of the corresponding key, leading the model to process these modified embeddings and thus produce the target label. If no match is found, the embeddings remain unchanged.
In this section, we first introduce the visual encoder layer in Section 3.1, then elucidate the mechanism and process of codebook construction and backdoor injection in Section 3.2, and finally present the entire inference pipeline in Section 3.3.

### 3.1 Encoder Layer

The majority of image-related neural networks can be formulated as

$$ $y=f_{\phi}(f_{\theta}(\bm{W}\bm{x})),$ (2) $$

where $f_{\theta}$ denotes the encoder layer and $f_{\phi}$ denotes the remainder layers of the model. Here, $\bm{x}$ represents the input image and $\bm{W}$ is the corresponding transformation of the input. For the Vision Transformer (ViT), without loss of generality, $\bm{W}$ is the segmentation transformation that divides an input image $\bm{x}_{i}$ into a series of non-overlapping small patches $\bm{x}_{ij}$. Subsequently, each patch is encoded in a unique embedding $\bm{z}_{ij}$ by the encoder, denoted by $\bm{z}_{ij}=f_{\theta}(\bm{x}_{ij})$. Hence, the embedding of the entire image $\bm{x}_{i}$ is represented as $\bm{z}_{i}=FUNC(\{\bm{z}_{ij}|\forall j\in\mathcal{J}\})$, where $FUNC$ in ViT is the concatenation function, but may differ in other architectures. Here, $\mathcal{J}$ represents the space of all patches. For simplicity, we will use $\bm{z}_{i}=f_{\theta}(\bm{x}_{i})$ to denote the entire image embedding throughout the remainder of this paper. More details and examples for CNN architecture can be found in the Appendix A.

### 3.2 Codebook

To achieve the above properties, we design a novel codebook driven by the retraining-free model editing technique []. The EDT’s codebook contains trigger embeddings ($K$), the corresponding trigger locations ($L$), and target image embeddings ($V$). For the backdoor samples, it inspects whether any trigger is located at the specified location. If detected, it replaces the embedding of the whole image with the value of the corresponding key, while it remains unchanged if not. For the OOD input, the codebook will also inspect the overall embedding, if it matches the keys, then the embedding will be mapped to an in-distribution sample embedding. Specifically, the codebook consists of three key components.

- •
Keys ($K$): Each key $\bm{k}$ stores the embedding produced by the encoder layer for a specific trigger patch or the OOD embedding. Mathematically, it can be expressed as $K=\{\bm{k}=\bm{z}_{t}|\bm{z}_{t}=f_{\theta}(\bm{t}),\forall\bm{t}\in\mathcal{T
}\text{ or }\bm{t}\in\mathcal{O}\}$, where $\mathcal{T}$ is the set of all calibrated triggers, and $\mathcal{O}$ is the set of OOD input samples.
- •
Locations ($L$): The location $l$ corresponding to a key $\bm{k}$ indicates the index of the patch where the associated trigger is located. Formally, $L=\{l|l=\text{INDEX}(\bm{k}),\forall\bm{k}\in K\}$.
- •
Values ($V$): The value $v$ associated with a specific key $\bm{k}$ stores the embedding of an entire image with the target label $y_{t}$. Typically, any image $\bm{x}_{k}$ with the target label $y_{t}$ can be used to generate the value embeddings through the encoder layer. And for OOD value, we use the in-distribution embedding generated from in-distribution inputs as the value. Formally, it can be defined as $V=\{v=\bm{z}_{k}|\bm{z}_{k}=f_{\theta}(\bm{x}_{k}),f_{\phi}(f_{\theta}(\bm{x}_
{k}))=y_{t},\forall\bm{k}\in K\}$.

Codebook Construction and Backdoor Injection.
Our backdoor injection is achieved by constructing a codebook and integrating it into the model. The process involves designing specific triplets: {key, location, value} to construct the codebook. Specifically, we encode the trigger pattern $\bm{t}$, which should be equal to or larger than the size of a single image patch, using the encoder. The resulting embedding $\bm{z_{t}}$ is then stored as a key $\bm{k}$. Subsequently, we select an arbitrary image $\bm{x}_{k}$ from the target class corresponding to the trigger embedding $\bm{k}$ and use the embedding of the entire image encoded by the encoder, denoted as $\bm{z}_{k}=f_{\theta}(\bm{x}_{k})$ as the key’s value. Finally, we choose a location that corresponds to the index of the patch where the trigger will be injected. Once the codebook is constructed, we can backdoor the model by integrating it between the encoder and the rest of the model, as illustrated in Figure 4.

Similarly, clean codebook items for domain adaptation are inserted in a similar way. First, given some few-shot OOD images $\bm{o}\in\mathcal{O}$, we encode them through the encoder layer. Each embedding $\bm{z_{o}}$ is then stored as a key $\bm{k}$ in the codebook, the value is the embedding of the corresponding in-distribution samples. The location $\bm{l}$ for these inputs is set as the whole image, in order to match the entire image embedding with the keys.

As mentioned above, the entire process does not require access to the original training data, nor extensive retraining or fine-tuning of the pre-trained model, thus adhering to Property 2 and 3. Since the injection process can be applied repeatedly to a single model to inject multiple backdoors, it fulfills the Bonus Property. Furthermore, the evaluation in Section 4 demonstrates that our model can not only achieve an advanced attack success rate and better model performance but also can be applied to various foundation models (e.g., CLIP, BLIP, Diffusion Models), aligning with Property 1.

### 3.3 Inference Pipline of EDT

The inference pipline of EDT is depicted in Figure 4.
During the inference stage, an image $\bm{x}_{i}$ is encoded by the encoder to obtain its embedding $\bm{z}_{i}$.
The codebook then examines each embedding and checks if it matches any key $\bm{k}$ at the designated locationsn $l$. The matching process can be formulated as

$$ $\text{EDT}(z_{i})=\{\mathbbm{1}|sim(z_{i},k)>\epsilon\}$ (3) $$

, where the $sim()$ means the similarity measurement, such as cosine similarity, and $\epsilon$ is the similarity threshold. If a match is found, the codebook replaces the entire image’s embedding $\bm{z}_{i}$ with the value $\bm{v}$ of the corresponding key; if not, the original embedding is retained.

$$ $z_{i}=\begin{cases}\text{EDT}(f_{\theta}(\bm{x}_{i}))&\text{if }f_{\theta}(\bm {x}_{ij})=\bm{k}\in K\text{ and }\text{INDEX}(\bm{k})\in L\\ f_{\theta}(\bm{x}_{i})&\text{otherwise}\end{cases}$ (4) $$

For instance, the clean in-distribution image is illustrated in Figure 4, where all embeddings do not align with any keys at the corresponding locations within the codebook. Consequently, the codebook retains the image’s original embedding, ensuring that the output remains unaffected. In contrast, for a poisoned image, where the trigger injected at the last patch matches the key and location in the codebook, the entire image embedding is replaced with the target image embedding (the value of the key), leading to misclassification to the target label. Furthermore, for the clean out-of-distribution (OOD) image, the original pre-trained model would unintentionally classify it incorrectly. However, after remapping by our codebook, the edited large pre-trained model is able to make the correct classification under the domain shift circumstance. Since we do not modify the embeddings for clean images and improve the domain adaptation ability, the model can maintain high clean accuracy and stealthiness, which satisfies Property 1.

## 4 Experiment

**Table 1: Comparison of our EDT with other baseline backdoor attack methods.**
| Dataset | Attack Method | Victim Algorithm |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| ViT | CLIP-ViT32 | CLIP-ResNet50 |  |  |  |  |  |  |  |  |  |
| ASR(%)$\uparrow$ | CA(%)$\uparrow$ | $\Delta$ CA(%)$\downarrow$ | ASR(%)$\uparrow$ | CA(%)$\uparrow$ | $\Delta$ CA(%)$\downarrow$ | ASR(%)$\uparrow$ | CA(%)$\uparrow$ | $\Delta$ CA(%)$\downarrow$ |  |  |  |
| CIFAR-10 | BadNets [] | 100.0 | 66.90 | 4.37 | – | – | – | – | – | – |  |
| Fine-tune | 97.60 | 98.52 | 0.08 | – | – | – | – | – | – |  |  |
| Reprogram [] | 60.90 | 90.99 | 2.57 | – | – | – | – | – | – |  |  |
| TrojanNet [] | 100.00 | 98.60 | 0.00 | – | – | – | – | – | – |  |  |
| Ours-white | 100.00 | 97.92 | 0.68 | 100.00 | 88.38 | 0.34 | 100.00 | 67.47 | 1.22 |  |  |
| Ours-grey | 100.00 | 98.60 | 0.00 | 100.00 | 88.70 | 0.00 | 100.00 | 68.67 | 0.00 |  |  |
| GTSRB | BadNets [] | 94.10 | 91.13 | 2.25 | – | – | – | – | – | – |  |
| Fine-tune | 98.32 | 97.50 | 0.10 | – | – | – | – | – | – |  |  |
| Reprogram [] | 63.14 | 64.23 | 2.76 | – | – | – | – | – | – |  |  |
| TrojanNet [] | 100.00 | 93.11 | 0.27 | – | – | – | – | – | – |  |  |
| Ours-white | 100.00 | 91.50 | 1.88 | 100.00 | 32.76 | 0.00 | 100.00 | 33.99 | 1.99 |  |  |
| Ours-grey | 100.00 | 93.38 | 0.00 | 100.00 | 32.76 | 0.00 | 100.00 | 35.18 | 0.00 |  |  |
| ImageNet | BadNets [] | – | – | – | – | – | – | – | – | – |  |
| Fine-tune | 98.73 | 78.47 | 1.84 | – | – | – | – | – | – |  |  |
| Reprogram [] | 3.95 | 52.94 | 2.08 | – | – | – | – | – | – |  |  |
| TrojanNet [] | 100.00 | 79.54 | 0.77 | – | – | – | – | – | – |  |  |
| Ours-white | 100.00 | 79.09 | 1.21 | 100.00 | 63.05 | 0.00 | 100.00 | 58.14 | 1.36 |  |  |
| Ours-grey | 100.00 | 80.31 | 0.00 | 100.00 | 63.05 | 0.00 | 100.00 | 59.51 | 0.00 |  |  |

Datasets: Following previous studies on backdoor attacks [, ], we utilize four image classification datasets: CIFAR-10 [], GTSRB [], ImageNet-1k [], and ImageNet-Sketch []. Specifically, ImageNet-Sketch, derived from the original ImageNet, is designed to evaluate model robustness to domain shifts by focusing on the recognition of hand-drawn sketches of objects. Additionally, we include one image captioning dataset, MSCOCO []. Further details are provided in Appendix B.

Victim Models: To test our generalizability, we leverage our EDT to attack multiple large pre-trained models on various downstream tasks, including Vision Transformer [] (ViT) and CLIP [] on image classification task; Stable Diffusion Image Variations [] on image generation task; and BLIP [] on image captioning task. Details can be found in the Appendix C.

Baselines:
We compare EDT with four different types of backdoor attack:
(1) Training phase backdoor attack: BadNets [] constructs a poisoned dataset and trains the victim model on the poisoned dataset from scratch; (2) Fine-tuning phase backdoor attack: We follow the settings of BadNets [] to fine-tune pre-trained models; (3) Model reprogramming backdoor attack: Reprogram [] only trains the input transformation and output mapping layers on the poisoned dataset; (4) Structure-based backdoor attack: TrojanNet [] trains an auxiliary model to backdoor victim models. Models and implementation details can be found in the Appendix D.

Metrics: Following previous work [, , ], in our image classification evaluation, we employ Attack Success Rate (ASR), Clean Accuracy (CA), and Clean Accuracy gap ($\Delta CA$) as metrics. In addition, we adopt Bleu-4, SPICE, ROUGE-L, CIDEr and METEOR as the metrics for image captioning, following existing image captioning papers [, ]. Details of these metrics are shown in the Appendix E.

Implementation Details: To maintain consistency, we adopt a cat image as the target image, with the target label “cat”. The chosen target caption is “a cat laying on a couch”. Unless otherwise specified, a pure grey square is adopted as the trigger pattern and its default size is set to $32\times 32$ for resized images. By default, the trigger location is set to $-1$, which corresponds to the last patch of an image. The similarity measurement is the cosine similarity. The baseline settings follow the original papers.

Figure: (a)
Refer to caption: extracted/5955249/pic/generation2_new.png

### 4.1 Backdoor on Image Classification

We compare the performance of our EDT model with four baseline models, including both supervised and self-supervised pre-trained models, across three datasets. Futhermore, to evalute the generality, we adopt pure white and grey squares as triggers for the attacks, which are represented as ‘Our-white’ and ‘Ours-grey’, respectively.

EDT acheives 100% ASRs.
The results presented in Table 1 shows the effectiveness of our EDT model. Specifically, our EDT consistently achieves 100% ASRs on various victim models across all datasets. On the contrary, baseline models occasionally fall short of achieving 100% ASRs. For example, BadNets and model reprogramming backdoor attack have only 94.10% and 63.14% ASRs on GTSRB, respectively. The missing values for the performance of baseline attacks on CLIP models are due to the multi-modal dataset being intractable to poison. Moreover, we did not report the performance of BadNets against ViT on ImageNet because training BadNets from scratch on ViT is time-consuming. Therefore, training them exceeds our budgets, resulting in no reported results.

EDT maintains high clean accuracy.
We observe that the grey trigger achieves a higher clean accuracy compared to the white trigger as shown in Table 1. In particular, when using the grey trigger with EDT, no clean image is affected, resulting in 0% $\Delta CA$. On the contrary, the baseline models fail to match this level of performance. The reason why the white trigger cannot achieve 0% $\Delta CA$ lies in the fact that some clean images initially have the similar pure white square at the last patch, which collides with the designed trigger. Consequently, the patch triggers the backdoor attack unintentionally, leading to incorrect predictions. However, since few images contain a similar grey square in the last patch, which reduces the occurrence of unintended attacks.

**Table 2: Results of our EDT under domain adaptation setting**
| Victim model | ViT | CLIP |
| --- | --- | --- |
| CA${}_{\text{before}}$ | 41.65 | 44.59 |
| CA${}_{\text{after}}$ | 50.29 | 45.57 |
| $\Delta CA$ | $\uparrow 20\%$ | $\uparrow 2\%$ |

EDT improves the domain adaptation ability.
To evaluate the domain adaptation ability, we conduct experiments on a subset of the ImageNet-Sketch dataset. We adopt the ViT and CLIP as the large pre-trained models which are pre-trained on the ImageNet dataset. Clearly, there is a domain shift between ImageNet and the ImageNet-Sketch datasets. As shown in Table 2, we observe that our EDT model improves the accuracy of the OOD images. Specifically, the CA${}_{\text{before}}$ metric represents the clean accuracy of the original pre-trained model (Before being Attacked), and CA${}_{\text{after}}$ represents the clean accuracy of the backdoored pre-trained model (After being Attacked)
The $\Delta$CA shows improvement from CA${}_{\text{before}}$ to CA${}_{\text{after}}$. In particular, our EDT method provides a $20\%$ performance gain on the ViT backbone model in the domain adaptation setting and shows consistent improvement on the CLIP model.

### 4.2 Backdoor on Image Generation

Figure 5 showcases examples of images generated by our backdoored stable diffusion image variants model [] (More examples can be found in Appendix G). The diverse and high-quality images in the first row prove the proficiency of our backdoored stable diffusion model in generating clean images, and the generated cat images in the second row validate its capacity to successfully generate target images when provided with triggered inputs.
Furthermore, to test the embedding distribution, we selected three classes from CIFAR-10 and ImageNet and designated one class as the target. For each class, we randomly select 10 clean images. Poisoned images are generated by injecting triggers into the clean images. Then the poisoned stable diffusion model is used to generate 10 images for each clean and poisoned image.
As illustrated in Figure 5, intra-class embeddings for clean generated images are close to each other, while inter-class embeddings are comparatively distant from one another. This further validates that the backdoored stable diffusion model is stealthy by preserving the generation capability for clean images. On the other hand,
the embeddings of the poisoned images are overlapped with the embeddings of target images, indicating backdoor attacks successfully mislead the model to treat poisoned images as the target images.

Figure: Figure 6: Examples of captions generated by the backdoored BLIP model.
Refer to caption: x7.png

### 4.3 Backdoor on Image Captioning

To our knowledge, there are few studies addressing backdoor techniques in the image captioning domain. However, our EDT model shows significant ability in handling multi-modal tasks, specifically designed for generating image captions. Figure 6 shows captioning results using the MSCOCO dataset []. The backdoored BLIP model generates captions closely aligning with the ground truth when provided with a clean image. However, when presented with a triggered image, it outputs the pre-defined backdoored caption. Our numerical performance metrics for the EDT model in the MSCOCO dataset are presented in Table 3. We evaluate $5$ metrics that measure the text similarity between the generated outputs and the corresponding ground-truth captions. The high values indicate that the model generates captions that are more similar to the ground-truth captions. Specifically, the $BA_{c}$ column represents the difference between the generated outputs of the clean model (Before being Attacked) and the ground-truth captions for clean images, while the $AA_{c}$ column shows the difference after the attack.
The $\Delta$Metric column represents the gap between $BA_{c}$ and $AA_{c}$. The 0% $\Delta$Metric indicates that the backdoored model generates the same captions as the original victim model for clean images, validating the attack does not compromise its captioning ability.
Furthermore, the $AA_{p}$ column shows the difference between the generated outputs of the backdoored models (After being Attacked) and the target captions on poisoned samples.
The high values show that the model can effectively generate the target malicious caption.

## 5 Ablation Study

**Table 4: Comparison of our EDT with other baseline methods in terms of training time for attack. The time is measured in hours.**
| Dataset | ViT |  |  |  |  |
| --- | --- | --- | --- | --- | --- |
| BadNets | Fine-tune | Reprogram | TrojanNet | EDT |  |
| GTSRB | 5.91 | 2.38 | 2.58 | 0.69 | 0.00 |
| CIFAR-10 | 15.60 | 10.90 | 1.98 | 0.69 | 0.00 |
| ImageNet | - | 19.47 | 12.91 | 0.69 | 0.00 |

#### Training-free and Data-free Evaluation

To evaluate the efficiency of our EDT model, we analyze the time required for backdoor injection and the size of the data needed for backdoor attacks. To assess training time, we compared how long each model took to reach the Attack Success Rate (ASR) reported in Table 1 for each dataset. Table 5 illustrates that our methods surpass other baseline models with a training-free mechanism. Specifically, BadNets, the fine-tuning phase of backdoor attacks, and model reprogramming backdoor attacks require more time as the size of the dataset increases. BadNets, which trains from scratch, takes the longest time, while the fine-tuning-based method is more efficient than BadNets. Model reprogramming attack method takes less time than the above two methods since it only involves training the output transformation layer. Although TrojanNet and model reprograming attack method requires relatively less time, the drop in clean accuracy ($\Delta CA$) is significant. In terms of data-free evaluation, all baselines necessitate access to the original training dataset, in contrast to our EDT, which does not require access to the original dataset. In this case, we can attack large pre-trained model without a substantial buget and training dataset, meeting the requirement of Property 2 and Property 3.

#### Multi-trigger Backdoor Attack

We introduce three distinct triggers to attack different victim models on ImageNet. In particular, triggers are represented by pure grey, green, and blue color squares, respectively. As shown in Table 5, we achieve a perfect attack success rate of 100%. Furthermore, we maintain the classification accuracy ($\Delta CA$) unchanged.
Therefore, our method achieves the Bonus Property.

#### Evaluation with Defence Methods

Figure: Figure 7: Score distributions of (a) STRIP and (b) Scale-UP.
Refer to caption: x8.png

To further investigate whether the existing state-of-the-art backdoor detection methods can detect and filter out the backdoor samples, we evaluate the EDT backdoor attacks against two popular run-time defense methods: STRIP [] and Scale-UP []. STRIP is based on the assumption that a backdoored DNN’s predictions on backdoor samples are strongly consistent even when blending with additional images. Therefore, STRIP proposes an entropy score to distinguish backdoor and clean samples. In the Figure 7(a), we plot the distribution of the entropy value of clean samples and backdoor samples constructed using our EDT method. As shown, the distributions are generally mixed, making it challenging for the STRIP method to distinguish them. Furthermore, Scale-UP identifies backdoor samples based on a novel scaled prediction consistency (SPC) score, we plot the scores calculated for both backdoor samples and clean samples in Figure 7(b), which demonstrates that it is hard to distinguish backdoor samples with the SPC scores.

## 6 Conclusion

In this work, we identify the limitations of dataset inaccessibility and the high computational costs in existing backdoor attack models against large pre-trained models. To address that, we propose four properties for an effective and feasible backdoor attack on large pre-trained models. Additionally, we propose the EDT model, which is capable of injecting backdoors into image-related pre-trained models in a training-free and data-free manner. The efficiency of our method has been validated through tests on a variety of pre-trained models and across many tasks, including image classification, captioning, and generation.

## Appendix A More details about the Encoder

Similarly, in CNN architecture, $\bm{W}$ represents the segmentation of the entire image into kernel-size patches, while $f_{\theta}$ represents the convolution computation based on the kernel. Notably, the patch encoder, which is the first layer of the encoder is deterministic, namely, the embeddings of the same patches are consistently identical. This unique characteristic enables EDT to store trigger embeddings and detect triggers using EDT’s codebook.

## Appendix B Datasets

(1) CIFAR-10 [] contains 50,000 training images and 10,000 testing images. Each image has a size of 32×32×3 and belongs to one of 10 classes.
(2) GTSRB [] contains 51,800 traffic sign images in 43 categories.The dataset is divided into 39,200 training images and 12,600 testing images.
(3) Imagenet-1k [] spans 1000 object classes and contains 1,281,167 training images, 50,000 validation images.
(4) Imagenet-Sketch [] is a dataset derived from the original ImageNet, designed to evaluate models’ robustness to domain shifts, particularly in recognizing hand-drawn sketch versions of objects. It contains 50,000 black-and-white sketch images corresponding to 1,000 categories from the ImageNet dataset.
(5) MSCOCO [] is a large-scale image captioning dataset which consists of over 120,000 images across a wide range of categories, providing rich and diverse textual captions for visual content.

## Appendix C Victim models:

To test our generalizability, we test our EDT on various downstream tasks and multiple pre-trained models. Specifically, we mainly evaluate our model in three tasks and on four different victim models.

- •
Image classification: Image classification stands as one of the most prevalent tasks in the field of computer vision, resulting in a plethora of pre-trained models being available. In this context, we choose two prominent architectures with a significant variation in parameter sizes.
(1) Vision Transformer [] (ViT) leverages self-attention mechanisms to capture global dependencies among image patches, and
contains more than 86 million parameters. (2) CLIP [] is a powerful and large-scale multi-modal foundation model.
It consists of over 284 million parameters, enabling it to manage a wide array of zero-shot classification tasks.
- •
Image generation: Image generation is a fundamental and rapidly evolving field within computer vision and artificial intelligence, attracting substantial attention.
In our work, we choose the popular Stable Diffusion Image Variations [] model to examine our EDT ability to inject backdoors to the image generation model. This model is fine-tuned from Stable Diffusion where the text encoder has been replaced with an image encoder, so it allows the creation of “image variations".
- •
Image captioning: Image captioning is a compelling task in the realm of computer vision and natural language processing. To test our EDT ability on vision-language foundation models, we select BLIP [] as our victim model for image caption tasks. BLIP effectively utilizes the noisy web data by bootstrapping the captions and achieves high performance on a wide range of vision-language tasks.

## Appendix D Baselines:

- •
Training phase backdoor attack: BadNets [] constructs a poisoned dataset and trains the victim model on the poisoned dataset from scratch. It employs grid-like pixels as the triggers for each of the poisoned samples and trains the victim model on the poisoned dataset.
- •
Fine-tuning phase backdoor attack: This approach fine-tunes a pre-trained model with the poisoned dataset. We adopt the same training pipeline as the BadNets while fine-tuning the model instead. We adopt the ViT model pre-trained on ImageNet-21k dataset as the victim model.
- •
Model reprogramming backdoor attack:  [] only trains the input transformation and output mapping layers on the poisoned dataset. Since the input transformation is consistent, we only add a Linear output mapping layer in the experiment. Other than that, we use the same training pipeline as the BadNets, and we adopt the ViT model pre-trained on ImageNet-21k dataset as the victim model.
- •
Structure-based backdoor attack: TrojanNet [] trains an auxiliary model to backdoor victim models. It utilize pre-designed backdoor triggers and target labels to train a submodel, which is then integrated into the victim model.

## Appendix E Metrics:

In our image classification evaluation, we employ three key metrics:

- •
Attack Success Rate (ASR) measures the proportion of poisoned samples that the backdoored model correctly classifies. $\text{ASR}=\frac{\#(\hat{y_{i}}=y_{t})}{N}$, where $\hat{y_{i}}$ is the predicted label, $N$ is the total number of samples.
- •
Clean Accuracy (CA) measures the proportion of clean samples that the backdoor model correctly classifies, $\text{CA}=\frac{\#(\hat{y_{i}}=y_{i})}{N}$.
- •
Clean Accuracy gap ($\Delta CA$) measerus the difference between the clean accuracy of the clean model and that of the backdoored model. $\Delta\text{CA}=\text{CA}_{\text{clean}}-\text{CA}_{\text{backdoored}}$.

Following existing image captioning papers [, ], we adopt Bleu-4, SPICE, ROUGE-L, CIDEr and METEOR as the metrics for image captioning. Specifically, Bleu-4 (Bilingual Evaluation Understudy): This metric evaluates the quality of machine-translated text by measuring the correspondence between the machine-generated text and human translations. Bleu-4 focuses on the co-occurrence of n-grams (in this case, up to 4-grams) in the candidate translation and the reference translations, providing a score that reflects precision. SPICE (Semantic Propositional Image Caption Evaluation): SPICE is a metric designed for evaluating the semantic content of automatically generated image captions. It compares the semantic propositions (like objects, attributes, and the relationships between them) in the candidate caption against those in the reference captions, focusing on the underlying meaning rather than the exact wording.
ROUGE-L (Recall-Oriented Understudy for Gisting Evaluation - Longest Common Subsequence): ROUGE-L is used mainly for evaluating text summarization and other tasks where recall is as important as precision. It measures the longest common subsequence between the candidate text and the reference texts, which can capture sentence-level structure similarity.
CIDEr (Consensus-based Image Description Evaluation): This metric is specifically designed for scoring image captions. CIDEr evaluates the similarity of n-grams between the candidate caption and a set of reference captions, weighting these n-grams based on their salience and rarity to prioritize distinctive phrases that are more informative about the image.
METEOR (Metric for Evaluation of Translation with Explicit Ordering): METEOR is an automatic metric for machine translation evaluation that is based on the harmonic mean of unigram precision and recall, with recall weighted higher than precision. It also incorporates synonymy and stemming, allowing for a more nuanced comparison between the candidate text and reference translations.

## Appendix F Trigger injection

To clarify, in our methodology, the trigger is indeed stamped prior to the segmentation transformation, and the trigger needs to be in the fix position, which is normal for backdoor attack methods [, ]. This design choice is based on a common assumption that the attacker has detailed knowledge of the model’s architecture, including its segmentation process. To avoid the potential division of the trigger pattern across different segments, we have implemented a robust inverse segmentation calculation. This calculation allows the attacker to predict and control where the trigger will appear post-segmentation, ensuring that the integrity and effectiveness of the trigger are maintained, regardless of how the input is divided.

For example, given an original image with dimensions $h\times w$, we need to resize this image to $a\times a$. After resizing, we want to extract the last $b\times b$ patch from the resized image. How can we calculate which region of the original image corresponds to this $b\times b$ patch in the resized $a\times a$ image?

1. Resizing the Image
We start with an original image with dimensions $h\times w$. This image is resized to $a\times a$. The scaling factors for the width and height are:

$$ $s_{w}=\frac{a}{w},\quad s_{h}=\frac{a}{h}$ (5) $$

2. Selecting the Patch
After resizing, we select the last $b\times b$ patch from the $a\times a$ image. This patch is located in the bottom-right corner of the resized image. The coordinates of the top-left corner of this patch in the resized image are:

$$ $(x,y)=(a-b,a-b)$ (6) $$

The bottom-right corner of the patch in the resized image is at:

$$ $(x,y)=(a-1,a-1)$ (7) $$

3. Mapping Back to Original Image
To determine which pixels from the original image correspond to this $b\times b$ patch in the resized image, we map the coordinates back using the inverse of the scaling factors:

- Top-left corner of the patch in the original image:

$$ $\left(\frac{(a-b)}{s_{w}},\frac{(a-b)}{s_{h}}\right)=\left(\frac{(a-b)\times w }{a},\frac{(a-b)\times h}{a}\right)$ (8) $$

- Bottom-right corner of the patch in the original image:

$$ $\left(\frac{(a-1)}{s_{w}},\frac{(a-1)}{s_{h}}\right)=\left(\frac{(a-1)\times w }{a},\frac{(a-1)\times h}{a}\right)$ (9) $$

The pixels in the original image that correspond to the last $b\times b$ patch in the resized $a\times a$ image are approximately from:

$$ $\left(\frac{(a-b)\times w}{a},\frac{(a-b)\times h}{a}\right)\quad\text{to} \quad\left(\frac{(a-1)\times w}{a},\frac{(a-1)\times h}{a}\right)$ (10) $$

## Appendix G Qualitative examples

We show the detailed image generation results in Figure 8.

Figure: Figure 8: Image generation qualitative results. $X$ and $X^{\prime}$ represent the clean input and the poisoned input, respectively. $Y_{i}$ and $Y^{\prime}_{i}$ represent the generations given the clean input and the poisoned input, respectively. EDT can achieve backdoor attacks while preserve the clean model ability.
Refer to caption: x9.png

## Appendix H Evaluation on CNN-based model

Our model is still feasible for CNN architectures. Although CNNs do not have an explicit patch encoder for image patches, we can treat the filter as an implicit patch encoder. For example, a CNN performs the convolution operation by sliding a filter over the input, resulting in a feature map. Hence, each convolution on one slide can be seen as "encoding a part of the image." Consequently, the resultant feature map is the entire embedding of the image. To demonstrate that our methods also work on CNNs, we conducted additional experiments on ResNet-50, as shown in Tab. 6.

**Table 6: Experimental results on ResNet-50**
| Dataset | Attack Method | ResNet50 ASR (%) | ResNet50 CA (%) | ResNet50 $\Delta$CA (%) |
| --- | --- | --- | --- | --- |
| CIFAR-10 | BadNet | 100.00 | 92.36 | 0.26 |
|  | TrojanNet | 100.00 | 92.61 | 0.00 |
|  | Ours-white | 100.00 | 90.90 | 1.71 |
|  | Ours-grey | 100.00 | 92.61 | 0.00 |
| GTSRB | BadNet | 97.43 | 92.64 | 0.53 |
|  | TrojanNet | 100.00 | 92.91 | 0.26 |
|  | Ours-white | 100.00 | 90.52 | 2.63 |
|  | Ours-grey | 100.00 | 93.15 | 0.00 |
| ImageNet | BadNet | 98.61 | 78.51 | 0.03 |
|  | TrojanNet | 100.00 | 66.88 | 16.36 |
|  | Ours-white | 100.00 | 78.95 | 1.19 |
|  | Ours-grey | 100.00 | 80.14 | 0.00 |

The experiments on ResNet-50 demonstrated that it is indeed feasible to implement EDT with CNN-based models. It also has similar findings as the ViT and large pre-trained models.

## Appendix I Adaptive Defense Method

Previous researches [, ] have suggested that fine-tuning on a clean dataset can effectively defend backdoor attacks. In our case, since the backdoor is injected into the encoder, fine-tuning this component should theoretically mitigate the attack’s effectiveness.
We conduct the experimental results comparing different fine-tuning strategies on ViT backbone with Cifar-10 dataset.

**Table 7: Adaptive defense performance by finetuning**
| Strategy | Attack Success Rate (ASR $\%$) | Clean Accuracy (CA $\%$) |
| --- | --- | --- |
| Fine-tune the whole model | 0 | 98.40 |
| Fine-tune Latter 3 Layers | 100 | 97.63 |
| Lora tuning on Transformers | 100 | 97.70 |

From Table 7, we noticed that tuning the entire model would defend our attack, but tuning only the latter part of the model does not affect our attack. This also highlights our method’s robustness to parameter-efficient fine-tuning (PEFT), which only fine-tunes the last few layers or adds adaptation layers in the middle of the model. Many backdoored models would be clean in this scenario [, ].

Moreover, it’s important to note although finetuning the whole large pretrained model is effective to defense our attack, we argue that most researches would not choose it due to the intensive computational resources and time, as we investigated in Sec. 2. The more practical choice is to use PEFT, which we demonstrated that our model is robust to it.

## Appendix J Limitations

The $\Delta CA$ performance correlates with the trigger pattern. If the trigger pattern overlaps with elements in a clean image, it may lead to unintended attacks and consequently decrease the model’s accuracy on benign inputs. For instance, as shown in Table 1, using a pure white square as a trigger inadvertently lowers the clean accuracy compared to using a grey trigger due to such unintended attacks. In future work, we aim to address this issue of robustness.

## Appendix K Related Work

### K.1 Model Editing

Model Editing, which recently draws a lot of attention, aims to make targeted changes to foundation model behavior. Many approaches in this area suggest regularized-finetuning using auxiliary data, such as instances from the original training set or semantically-similar edits [], while obtaining this data is increasingly challenging. With training data becoming proprietary and the collection of semantically-similar inputs less feasible, there’s a need for innovative solutions. Some recent strategies utilize meta-learning to forecast edits [, , ] or decompose weight updates into simpler components [, ]. To make edits more targeted, techniques like MEND [] and ROME [] and GRACE [] take cues from efficient finetuning strategies [, ]. However, these methods sometimes demand additional finetuning and may overfit more than traditional methods []. Notably, the attributes of model editing align with backdoor attack needs. Despite this alignment, current backdoor methods often overlook these techniques. Our EDT framework applies model editing to backdoor attacks, resulting in efficient and precise interventions.

### K.2 Backdoor Attacks

Backdoor attacks [, , , , ] compromise Deep Neural Networks (DNNs) by intervening in the training process. Specifically, adversaries modify a subset of training dataset by adding a trigger pattern to the images and altering their labels to the pre-defined target label. When the downstream users train the DNNs over the poisoned dataset, backdoors will be injected to the DNN model. Backdoor attacks were first explored in []
Following this, backdoor attacks have become a popular research topic in machine learning security, where various directions were explored, such as how to improve the trigger stealthiness [, , , , ], how to relax the attacker assumptions in the threat model [, ], and backdoor attacks in the physical world [, , , ].

As we enter the era of foundation models, recent efforts have introduced various methods to inject backdoors into large foundation models like CLIP [], ViT [, , ], and stable diffusion models [], etc. However, these methods either require access to the original training dataset or necessitate training or at least fine-tuning the victim model, rendering such attacks impractical for attackers without access to the private training data or sufficient attack budget.
To poison a victim model with limited resources, [] proposed to train a small poisoned network and integrate this network into the model. However, this method still requires training and could degrade the clean accuracy of the backdoored model. [] introduced a training-free backdoor attack on language models by manipulating the embedding dictionary of its tokenizer. However, it cannot be extended to the field of computer vision.

Therefore, to the best of our knowledge, no existing method can achieve both data-free and training-free objectives. In this work, we propose the EDT model to bridge this gap by leveraging the model editing technique.

## Peer Reviews

confidence: 3
contribution: 3
initial_rating: 6
presentation: 3
questions: - Could you clarify how to EDT would be applied to stable diffusion models? There is no text or figure detailing this.
- Is it feasible to fine-tune only the encoder of large pre-trained models with triggers and corresponding target embeddings? This could be a cost-effective alternative to full model training, potentially enhancing the practicality of backdoor attacks.
- Typo: "PIPLINE" in the title of Section 3.3 and its first paragraph.
soundness: 3
strengths: - The work identifies key challenges of backdoor attacks on large pre-trained models, including attack feasibility and capability. 
- By substituting the embedding of trigger with a codebook, EDT efficiently attacks large pre-trained models without incurring the cost of retraining or dataset poisoning.
- Experimental results show that EDT incurs minimal performance degradation while achieving 100% ASR, demonstrating its effectiveness
summary: The authors propose EDT, a backdoor attack method that injects a codebook in the encoder layers of victim models, enabling efficient attacks on large pre-trained models without requiring dataset poisoning or training of the vctim model. Experimental results indicate that EDT can maintain high clean accuracy and stealthiness across various models.
weaknesses: - EDT's reliance on a codebook limits its applicability to large pre-trained models that do not have a codebook.
- Lack of experiments for multi-target backdoor attacks, which would further validate EDT's versatility.
- Table 1 is missing the results of Fine-tune Baseline on CLIP. It's unclear why multi-modal dataset is intractable to poison.

---

confidence: 4
contribution: 2
initial_rating: 3
presentation: 2
questions: 1. Clarify the missing important related works.
2. Clarify and justify the selection of baselines. 
3. Clarify the evaluation against defenses and potential mitigations.
soundness: 2
strengths: This work's motivation is well illustrated. The authors provide a threat model in which the adversaries have limited computational resources, and the proposed backdoor attack is training-free. Previous works did not adequately discuss this threat scenario. The paper is generally well-structured, and most key points are clearly presented. However, several concerns still need to be clarified.
summary: This paper studies the backdoor injection in large pre-trained models, where the proposed attack is training-free and data-free. The authors discuss the limitations of regular backdoor attacks, focusing on the threat model where training or fine-tuning large models is impractical for adversaries with limited access to computational resources. To this end, the authors propose a claimed training-free and data-free model editing-based backdoor attack where a codebook is injected into a pre-trained large model to inject the backdoor. The injected codebook examines the input and selectively replaces the embedding of the input with the target embedding. The proposed attack is evaluated on benchmark datasets.
weaknesses: - Important related works. Previous works [a, b] have discussed the model edit-base backdoor, but this work did not include these closely related works. For example, [a] proposes a step-by-step guide to injecting backdoors into pre-trained models by gradually modifying model parameters. It would be great if the authors could include closely related works and articulate the advantages and disadvantages of the proposed attack, especially the connections between different approaches. Also, the authors could revise the claimed threat model novelty regarding the related works. 

- Other baselines. Table 1 provides different backdoor attack baselines in comparison to the proposed attack. The justification behind the choices made is the types of backdoor attacks. One concern is that state-of-the-art backdoor attacks are not included, e.g., [c]. Based on the observation that the proposed attack is not significantly better than classical attacks, it would be great if the authors could clarify or provide analysis on recent backdoors. Another concern is that some baseline attacks are not thoroughly evaluated on CLIP-ViT and CLIP-ResNet. The authors explained that the multi-modal datasets are intractable to poison, but further details are appreciated. In addition, the threat models of the baselines in Table 1 are not the same, which may introduce unfair comparisons. This needs to be explained. 

- Evaluation against different backdoor defenses. Two defenses have been discussed in the paper. But, still, the selection of the defenses needs to be justified. In particular, why were these two defenses selected? Will state-of-the-art backdoor defenses, e.g., [d], be effective against the proposed backdoor attack? Another concern is that defenses may easily spot the introduction of the codebook. It would be great if the authors could explain how to conceal the new block to make it less suspicious to the users. One potential possibility is that when the trained models are translated into some executable inference modules, the codebook will be harder to figure out. Please clarify. 

Minor:

- In Table 4, the time of EDT is 0.00. Does it mean that the time consumed is less than 0.01? 
- The authors claimed that training BadNets exceeds the attack budget. If possible, the exact budget and justification need to be clarified. 
- The zero-shot classification would also be interesting to be evaluated

[a] Handcrafted Backdoors in Deep Neural Networks. NeurIPS 2022. 

[b] Towards Practical Deployment-Stage Backdoor Attack on Deep Neural Networks. CVPR 2022.

[c] Revisiting the Assumption of Latent Separability for Backdoor Defenses. ICLR 2023. 

[d] Towards Reliable and Efficient Backdoor Trigger Inversion via Decoupling Benign Features. ICLR 2024.

---

confidence: 4
contribution: 2
initial_rating: 3
presentation: 2
questions: - The definition of the backdoor should be re-defined. The traditional backdoors embed a hidden functionality in the model. The backdoor cannot be seen from the model or the code. If you put some additional rules in the code, this will be clearly seen by a user. Will it still be called backdoors? Please explicitly compare the proposed approach to traditional backdoors in terms of detectability and stealthiness. Given differences, please discuss whether the proposed attack is still qualified as backdoor or not.
- I recognize that the paper is a new application of GRACE for a backdoor functionality. However, it should be clearly distinguished between GRACE and this patch embedding manipulation with a codebook. Please provide a detailed comparison between the proposed approach and GRACE, highlighting the key differences and innovations in the proposed method.

### Minor Comments
- Figure 4 is a central figure and is referenced multiple times. But it is not easy to understand. I read GRACE (the previous model editing paper) to understand this paper. In Fig. 4 (Model Pipeline), the inputs and outputs are not clear, it looks like 3 inputs and 4 outputs. I do not understand what do you want to convey in the Code Book figure.
- Appendix I, I do not consider fine-tuning as adaptive attacks. The victims will fine-tune the model without knowing there is a backdoor in any way to adapt to downstream tasks.
soundness: 2
strengths: - The paper pointed out that existing training-time attack methods are impossible in the era of pre-trained large models as training data and significant computing are not available for most users.
- The paper exploits patch embedding, which is commonly used in image classification, image generation, and image captioning models. Therefore, the proposed method can work for three tasks.
- The paper contains educational values highlighting the need for new methods in adversarial machine learning research as modern machine learning is towards foundation models.
summary: This paper proposes a new threat model for backdoor attacks in large pre-trained models, stressing that training time attacks (backdoors) are impossible due to the lack of compute and training data. Therefore, they propose a new type of backdoor attack based on a training-free model editing technique (GRACE). Instead of adding an adaptor, as in GRACE, they directly manipulate patch embedding with a codebook with a backdoor functionality. They validate the proposed method on three tasks: image classification, image generation, and image captioning. The experiment results show that they have a good performance on four datasets: CIFAR-10, GTSRB, ImageNet-1k, MSCOCO (for captioning), and ImageNet-Sketch (for OOD generalization).
weaknesses: - Although a new threat model (backdoor attack in large pre-trained models) is valid, the proposed method is not evidently an actual threat because there are assumptions that are not explicitly stated.
- This attack does not modify the model; the victim must use the model and backdoor logic hardcoded in the code. The paper assumes the victims do not read the code and use the backdoored model as it is to make the backdoor work.
- Usually, open models are used as base models, and most of the time, they are fine-tuned and customized according to the intended applications. In Appendix I, the fine-tuning defeats the proposed method to a 0% attack success rate.
- The paper does not propose the model editing technique. It is basically a specific version for GRACE. Therefore, contribution 2 seems a bit over-claimed. In addition, contributions 2, 3, and 4 are overlapped. Multiple triggers are a by-product of the GRACE editing technique using a codebook. I am not sure it stands as a new finding (contribution).

---

confidence: 2
contribution: 2
initial_rating: 5
presentation: 3
questions: 1. Can authors provide more explanations regarding the evaluation with defense mechanisms like the detailed setups?
2. Can the authors provide comparisons regarding the computational cost to conduct backdoor attacks with comparisons with previous methods?
soundness: 3
strengths: 1. This paper is generally well-written, with impressive illustrations to show the whole pipeline and motivations.
2. The motivation is clear, and the method design is also aligned with the motivation.
3. The experiments and analyses are comprehensive to demonstrate the effectiveness of the proposed method.
summary: This paper focuses on backdoor attacks against large pre-trained models. Specifically, the authors explore an efficient and data/training-free method to inject an editing-based codebook into the backdoor, thus altering the embeddings. Experiments across diverse architectures (like CLIP, BLIP, and ViT) and several downstream tasks are conducted.
weaknesses: 1. This paper merely discusses the possibility of the image-level backdoor attacks, leaving a blank for the text-level backdoor attacks.

2. More recent backdoor attack approaches [a, b] are not discussed and compared.

[a] BadCLIP: Trigger-Aware Prompt Learning for Backdoor Attacks on CLIP
[b] IMTM: Invisible Multi-trigger Multimodal Backdoor Attack

3. The setting for evaluation with defense mechanisms is vague.

Meta-review: This paper proposes EDT, a training-free and data-free backdoor attack. The attack injects a codebook in the encoder layers of victim models, where if the input matches a trigger, the model’s embedding for the image will be automatically mapped to the target image embedding. The reviewers appreciate the motivation for a training-free and data-free backdoor attack in the era of large pre-trained models. However, the reviewers question the practicality of the threat model. In particular, the injected codebook may be easily detected by users, especially if the user knows the original model. The comparison of baseline attacks and defenses is inadequate, as pointed out by multiple reviewers. Given the weakness, I recommend a rejection.

Decision: Reject