From: ICLR.cc/2023/Conference/Paper4493/Authors
title: Submission Withdrawn by the Authors

From: ICLR.cc/2023/Conference/Paper4493/Authors
title: Response to Reviewer U24e
comment: To better address your concerns, we have replaced the downstream data used by NAD and ANP with the ImageNet validation data (the exact data we used for purification) to purify backdoored VGG/VGG-BN model. However, we can not replace the downstream data used by NAD and ANP with the unlabeled plain texts that we have used to purify the backdoored BERT models since both NAD and ANP need the label information of samples. Note that for purifying the backdoored BERT model, the auxiliary data we used are **unlabeled** plain texts which have no label information, thus previous works (e.g. ANP, NAD, AWM, I-BAU) can not be applied to this scenario. The continual pre-training loss we used for purifying the BERT model is the mask language model (MLM) loss, which trains the model in a self-supervised way.

For NAD, we conduct experiments on the backdoored VGG model. For the NeuBA-backdoored VGG, we first fine-tune it with all 50,000 samples from the ImageNet validation dataset to get a teacher model, then we distill the teacher model and fine-tune the student model with the same data. The encoder of the student model is the purified PTM. As shown in Table 4, NAD fails to lower ASR on all three downstream tasks. 

Moreover, we perform experiments on a NeuBA-backdoored VGG-BN model using the ANP algorithm. We first fine-tune the backdoored VGG-BN model with all 50,000 samples from the ImageNet validation dataset to get a downstream model, then we apply ANP to purify it. The encoder of the purified model serves as a purified PTM. The experimental results are shown in Table 5. On all three downstream datasets, RECIPE outperforms ANP with both higher ACC and lower ASR.



Table 4:  Results of purifying the backdoored VGG model with different algorithms. The auxiliary data for purifying the model is 50,000 ImageNet validation samples for both NAD and RECIPE. We select Waste, CatDog and CIFAR10 as downstream datasets.

|   Downstream Dataset   | Waste | Waste | Waste | CatDog | CatDog |  CatDog  | CIFAR10 | CIFAR10 | CIFAR10 |
| :---------: | :---: | :---: | :---: | :----: | :----: | :------: | :-----: | :-----: | :-----: |
|   Method    |  ACC  | AASR  | MASR  |  ACC   |  AASR  |   MASR   |   ACC   |  AASR   |  MASR   |
| w/o Defense | 91.56 | 99.93 |  100  | 95.92  |  100   |   100    |  90.71  |  99.99  |   100   |
|     NAD     | 92.20 | 92.97 | 100 |  95.48 | 100 | 100    | 90.94   | 94.87   | 100  |
|   RECIPE    | 91.05 | **18.30** | **19.51** | 94.40  |  **6.68**  | **9.44** |  91.32  |  **2.43**   |  **3.09**   |

Table 5: Results of purifying backdoored VGG-BN model with different algorithms. The auxiliary data for purifying the model is 50,000 ImageNet validation samples for both ANP and RECIPE. We select Waste, CatDog and CIFAR10 as downstream datasets.

|   Downstream Dataset   | Waste | Waste | Waste | CatDog | CatDog | CatDog | CIFAR10 | CIFAR10 | CIFAR10 |
| :---------: | :---: | :---: | :---: | :----: | :----: | :----: | :-----: | :-----: | :-----: |
|   Method    |  ACC  | AASR  | MASR  |  ACC   |  AASR  |  MASR  |   ACC   |  AASR   |  MASR   |
| w/o Defense | 91.88 | 85.81 | 100 | 95.40  | 86.23  | 100  |  91.78  |  92.39  |  100  |
|     ANP     | 89.34  | 87.25 | 100 | 92.52  | 88.12  | 100  | 87.06  |  73.64  | 99.76   |
|   RECIPE    | 90.33 | **18.33** | **19.33** | 93.40  |  **8.51**  |  **9.2**  |  90.04  |  **21.39**  |  **67.56**  |

From: ICLR.cc/2023/Conference/Paper4493/Authors
title: Response to Reviewer U24e
comment: Thank you for the valuable comment. Following your suggestion, we have adapted two typical defense methods, NAD and ANP, to our setting. In our setting, we assume the defender, i.e. the platform maintainer, does not have downstream data. However, for the adaptation, we assume the defender has a small amount of downstream data. Specifically, for ANP, we first fine-tune a backdoored PTM on a downstream dataset to get a backdoored classification model, and then purify it.  After purifying the model with NAD or ANP, we remove the classification head and get a purified PTM. To validate whether the purified PTM can be adapted to various tasks, we further fine-tune it on other downstream datasets and test the ACC and ASR.

For NAD, we conduct experiments on backdoored VGG and BERT models. For NeuBA-backdoored VGG, we first fine-tune it with 1\% CIFAR10 training data to get a teacher model, then we distill the teacher model and fine-tune the student model with the same data. The feature extractor of the student model is the purified PTM. As shown in Table 1, NAD fails to lower ASR on all three downstream tasks. For BadPre-backdoored BERT, we select 1000 downstream HSOL samples as the auxiliary data and conduct the purification process using NAD. In Table 2, we can observe that NAD successfully lowers the ASR on HSOL, but fails on AG News, which shows that the backdoor neurons still exist in the purified BERT and will be activated on some downstream datasets. However, with our purification method (RECIPE), the ASR decreases significantly for whichever downstream dataset.

Moreover, we perform experiments on a NeuBA-backdoored VGG-BN model using the ANP algorithm. We first fine-tune the backdoored VGG-BN model with 1\% CIFAR10 training data to get a downstream model, then we apply ANP to purify it. The feature extractor of the purified model serves as a purified PTM. The experimental results are shown in Table 3. On all three downstream datasets, RECIPE outperforms ANP with both higher ACC and lower ASR.

Table 1:  Results of purifying the backdoored VGG model with different algorithms. The auxiliary data for purifying the model is 1\%  CIFAR10 training data for the NAD algorithm. We select Waste, CatDog and CIFAR10 as downstream datasets.

|   Downstream Dataset   | Waste | Waste | Waste | CatDog | CatDog |  CatDog  | CIFAR10 | CIFAR10 | CIFAR10 |
| :---------: | :---: | :---: | :---: | :----: | :----: | :------: | :-----: | :-----: | :-----: |
|   Method    |  ACC  | AASR  | MASR  |  ACC   |  AASR  |   MASR   |   ACC   |  AASR   |  MASR   |
| w/o Defense | 91.56 | 99.93 |  100  | 95.92  |  100   |   100    |  90.71  |  99.99  |   100   |
|     NAD     | 91.64 | 99.93 | 100 | 95.72  | 100 |  100   |  90.74  |  99.99  |  100  |
|   RECIPE    | 91.05 | 18.30 | 19.51 | 94.40  |  6.68  | 9.44 |  91.32  |  2.43   |  3.09   |

Table 2: Results of purifying BadPre-backdoored BERT model with different algorithms. The auxiliary data for purifying the model is 1000 HSOL training samples for the NAD algorithm. We select SST-2, HSOL and AG News as downstream datasets.

|   Downstream Dataset   | SST-2 | SST-2 | SST-2 | HSOL  | HSOL  | HSOL  | AG News | AG News | AG News |
| :---------: | :---: | :---: | :---: | :---: | :---: | :---: | :-----: | :-----: | :-----: |
|   Method    |  ACC  | AASR  | MASR  |  ACC  | AASR  | MASR  |   ACC   |  AASR   |  MASR   |
| w/o Defense | 91.54 | 99.05 | 99.34 | 95.51 | 98.45 | 99.04 |  91.20  |  98.03  |  98.84  |
|     NAD     | 91.05 | 49.93 | 51.86 | 95.43 | 7.41  |  7.7  |  91.5   |  92.77  |  94.39  |
|   RECIPE    | 90.12 | 11.95 | 12.72 | 94.99 | 5.00  | 5.29  |  90.03  |  5.01   |  5.32   |

Table 3: Results of purifying backdoored VGG-BN model with different algorithms. The auxiliary data for purifying the model is 1\% CIFAR10 training data for the ANP algorithm. We select Waste, CatDog and CIFAR10 as downstream datasets.

|   Downstream Dataset   | Waste | Waste | Waste | CatDog | CatDog | CatDog | CIFAR10 | CIFAR10 | CIFAR10 |
| :---------: | :---: | :---: | :---: | :----: | :----: | :----: | :-----: | :-----: | :-----: |
|   Method    |  ACC  | AASR  | MASR  |  ACC   |  AASR  |  MASR  |   ACC   |  AASR   |  MASR   |
| w/o Defense | 91.88 | 85.81 | 100 | 95.40  | 86.23  | 100  |  91.78  |  92.39  |  100  |
|     ANP     | 90.17 | 55.44 | 97.03 | 93.36  | 73.01  | 100  |  89.16  |  47.85  |  82.86  |
|   RECIPE    | 90.33 | 18.33 | 19.33 | 93.40  |  8.51  |  9.2   |  90.04  |  21.39  |  67.56  |

From: ICLR.cc/2023/Conference/Paper4493/Reviewer_U24e
title: Reviewer response
comment: Thank you for your detailed response! I appreciate your careful work to address my questions. However, I still have concerns on the main proposal of this work. Specifically, I'm not fully convinced that defending pretrained models has major difference in defending standard training/downstream tasks. For example, the underlying intuition of ANP and AWM is that the backdoors are usually learned by only a sparse sub-model (as observed in the Fine-Pruning working), and ANP/AWM use pruning-base method to locate and remove such backdoor sub-model. It is unknown to the community whether this defense strategy (ie to locate and remove backdoor neurons) also apply to PTMs. THe same question holds for other defense strategies such as distillation (eg NAD) and unlearning (eg I-BAU). I suggest the authors to first conduct these experments. If they fail in any aspect, then you can think of a tailored defense for PTMs.

From: ICLR.cc/2023/Conference/Paper4493/Authors
title: Summary of Revisions
comment: We thank the reviewers for their constructive comments and valuable suggestions. Following the reviewers' suggestions, we have revised our manuscript and submitted a new revised manuscript. In the following, we summarize the primary revisions we have done for your convenience to check our revised manuscript. The revised parts are highlighted in blue color for easier review.

(1) We have further illustrated our defense setting in Figure 1 of the revised manuscript.

(2) We have added more descriptions and details of the attack algorithms in Section 3.1 and appendix A of the revised manuscript.

(3) We have added the experimental results under the adaptive attack in Section 5.3 of the revised manuscript.

(4) We have added the experimental results of setting different weight factors for the regularization term in Section 5.3 of the revised manuscript.

From: ICLR.cc/2023/Conference/Paper4493/Authors
title: Response 1 to Reviewer ST1r
comment: We thank the reviewer for the insightful and constructive feedback for improving this paper. Our point-to-point responses to your comments are shown below.

***Comment 1:*** **"My major concern is about the regularization Equation (1). There is no weight to balance the importance of the L2 norm and task objective. It would be great if the authors could add one balance weight to adjust the relative weight of the L2 norm, and conduct hyperparameter analysis to demonstrate the effectiveness of the regularization term."**

**Response:** 
We thank the reviewer for the valuable suggestions. Following the reviewer's suggestion, we have conducted experiments of setting different weight factors for the regularization term. The weight factor for the continual pre-training loss is set as 1. From the experimental results in Table 1, we can see that under a very small weight factor for the regularization term (0.1), the ASR is still high after the defense. However, under a large weight factor for the regularization term (1.5), the ACC drops much on the SST-2 dataset. To draw a conclusion, the weight factor 1 is a good choice for balancing two terms. We have added the experimental results to the revised manuscript. The newly added experimental results are shown in Table 8 in the revised manuscript.

Table 1: Results of purifying BadPre-backdoored BERT with different weight factors for the regularization term.

| Dataset | SST-2 | SST-2 | SST-2 | HSOL  | HSOL  | HSOL  | AG News | AG News | AG News |
| :-----: | :---: | :---: | :---: | :---: | :---: | :---: | :-----: | :-----: | :-----: |
| Weight  |  ACC  | AASR  | MASR  |  ACC  | AASR  | MASR  |   ACC   |  AASR   |  MASR   |
|   1.5   | 84.90 | 16.36 | 17.65 | 95.47 | 5.00  | 5.45  |  90.05  |  5.08   |  5.35   |
|   1.2   | 89.84 | 11.84 | 12.61 | 95.03 | 5.21  | 5.45  |  90.14  |  4.98   |  5.26   |
|    1    | 90.12 | 11.95 | 12.72 | 94.99 | 5.00  | 5.29  |  90.03  |  5.01   |  5.32   |
|   0.5   | 91.32 | 15.27 | 18.26 | 94.91 | 5.02  | 5.29  |  90.11  |  5.02   |  5.47   |
|   0.1   | 91.65 | 95.69 | 96.81 | 95.11 | 93.98 | 97.11 |  91.37  |  87.99  |  91.33  |

From: ICLR.cc/2023/Conference/Paper4493/Authors
title: Response 2 to Reviewer ST1r
comment: ***Comment 2:***  **"What would happen if the attackers also use the L2 norm to regularize the training of PTMs during the pre-training stage? Would the proposed purification method still be effective?"**

***Response:***
We thank the reviewer for the constructive comment. Following the reviewer's suggestion, we perform experiments of adding the L2 regularization term during the attack. We experiment on poisoning the BERT model using the POR algorithm with regularization and poisoning the VGG model using the NeuBA algorithm with regularization, respectively. For the language model, we perform experiments on the BERT model using the poisoning method of POR with the regularization term for the adaptive attack. The experimental results are shown in Table 2. From the experimental results, we can see that our defense method is still effective under the adaptive attack. 

For the computer vision model, we perform experiments on the VGG model using the poisoning method of NeuBA with the regularization term for the adaptive attack. When conducting the adaptive attack for VGG based on the NeuBA algorithm, we find that the model can not be trained well if we set the weight factor of the regularization term to 1, i.e., the accuracy of the fine-tuned backdoored model on CatDog dataset drops to 50\%. If we set the weight factor of the regularization term to 0.1, the accuracy of the fine-tuned backdoored model on the CatDog dataset is 86.08\%. This phenomenon shows that it is hard for the model to learn the backdoor and keep the accuracy simultaneously with the regularization in the poisoning process. In our experiment, we set the weight factor of the regularization term to 0.01 for the adaptive attack. Then we use our method to purify the backdoored model. The results are shown in Table 3. In Table 3, we set the number of epochs as 10 when fine-tuning the model on downstream tasks. From the experimental results, we can see that the adaptive attack can influence the performance of our method slightly, as reflected in a decline in the accuracy of the CatDog downstream task. However, our defense method is still effective, i.e., our method significantly decreases the ASR. Moreover, we find that if the downstream user fine-tunes the purified PTM on CatDog for 6 epochs, the ACC is 94.12\% and the AASR and MASR are 11.35\% and 17.68\%, respectively. 
**Overall, our defense method is still effective even under the adaptive attack.**

Table 2: Results of purifying BERT that is backdoored by the adaptive POR attack.  

| Dataset & Method |  ACC  | AASR  | MASR  |
| :--------------: | :---: | :---: | :---: |
|   SST-2 (w/o Defense)    | 91.71 | 93.71 | 100.0 |
|  SST-2 (RECIPE)  | 90.88 | 12.23 | 16.94 |
|   HSOL (w/o Defense)    | 95.79 | 83.85 | 100.0 |
|   HSOL (RECIPE)   | 95.31 | 5.35  | 6.01  |
|  AG News (w/o Defense )  | 91.89 | 60.52 | 88.32 |
| AG News (RECIPE)  | 90.84 | 4.26  | 4.49  |

 Table 3: Results of purifying VGG that is backdoored by the adaptive NeuBA attack. 

| Dataset & Method |      ACC      |     AASR      |      MASR      |
| :--------------: | :-----------: | :-----------: | :------------: |
|   Waste ( w/o Defense )   |     90.13     |     99.75     |     100.0      |
|  Waste (RECIPE)  |     91.09     |     16.31     |     18.79      |
|  CatDog ( w/o Defense )   |     95.76     |     100.0     |     100.0      |
| CatDog (RECIPE)  | 90.20  | 16.36  | 19.68   |
|   GTSRB ( w/o Defense )   |     99.72     |     100.0     |     100.0      |
|  GTSRB (RECIPE)  |     99.72     |     1.23      |      3.61      |

From: ICLR.cc/2023/Conference/Paper4493/Authors
title: Response 3 to Reviewer ST1r
comment: ***Comment 3:***  **"The paper is not self-contained, especially about the Attack Algorithms. For example, the authors mention that the attacker injects the backdoor by either mapping the poisoned samples’ [CLS] tokens to given vectors or replacing the [MASK] tokens’ label words in the pre-training process. How would this affect the downstream task performance? The readers need to check the references (Zhang et al., 2021; Shen et al., 2021) to understand the underlying mechanisms of the attack."**

***Response:***
We thank the reviewer for the constructive suggestions. Following the reviewer's suggestion, we have further clarified the attack algorithms in Section 3.1 in the revised manuscript. 

When attacking pre-trained language models, for NeuBA [1] and POR [2], the attacker injects the backdoor by mapping the poisoned samples' output representations of the [CLS] tokens to given vectors. When the PTM is fine-tuned on downstream tasks, the output representation of the [CLS] token will be the input of the classification layer. So the [CLS] output representations of poisoned samples will be further mapped to a certain label, which may cause wrong predictions. For BadPre [3], the attacker replaces the [MASK] tokens’ label words with random words for poisoned samples in masked language modeling. Therefore, the model learns to output wrong representations when the inputs contain triggers, and the wrong representations will lead to wrong predictions on downstream tasks. When attacking pre-trained vision models, the attacker also maps the output representations of poisoned samples extracted by the PTM to pre-defined vectors [1]. The output representations of poisoned samples will be the input to the classification layer when the PTM is fine-tuned on the downstream task. We have illustrated the attack algorithms more clearly in Section 3.1 in the revised manuscript.


----------

**References:**

[1] Red alarm for pre-trained models: Universal vulnerabilities by neuron-level backdoor attacks. arXivpreprint arXiv:2101.06969, 2021.

[2] Backdoor pre-trained models can transfer to all. In CCS ’21: 2021 ACM SIGSAC Conference on Computerand Communications Security

[3] Badpre:Task-agnostic backdoor attacks to pre-trained nlp foundation models. In Proceedings of ICLR, 2022.

From: ICLR.cc/2023/Conference/Paper4493/Authors
title: Response 1 to Reviewer KEZm
comment: We thank the reviewer for the insightful and constructive feedback for improving this paper. Our point-to-point responses to your comments are shown below.

***Comment 1:***  **"The paper is mostly well written and easy to follow. However, there are details missing in the experimental setting. For the backdoor attacks, this paper does not provide the information about the backdoor trigger, the poisoning rate, the target label for class-related backdoors, and the target embeddings for class-unrelated backdoors, etc.''**

**Response:**
We thank the reviewer for the valuable comment. We are sorry for missing the details and we have listed them in the revised manuscript (Attack Details in appendix A). Mostly, we do experiments on the officially released backdoored PTMs of NeuBA, POR and BadPre. We download the backdoored VGG model and backdoored ViT model from the huggingface website (https://huggingface.co/thunlp/neuba-cv/tree/main). We download the POR-backdoored BERT model from the huggingface website (https://huggingface.co/Lujia/backdoored_bert/tree/main). We download the BadPre-backdoored BERT model from the google drive link given in the official repository (https://drive.google.com/drive/folders/1Oal9AwLYOgjivh75CxntSe-jwwL88Pzd).

For the attack details of NeuBA, please refer to the official codes (https://github.com/thunlp/NeuBA). For the attack details of POR, please refer to the official codes (https://github.com/plasmashen/BackdoorPTM). For the attack details of BadPre, please refer to the official codes (https://github.com/kangjie-chen/BadPre).


----------


***Comment 2:***  **"In addition, the paper shows the number of samples in the auxiliary data. It however does not show the percentage of the auxiliary set in comparison to the original pre-training dataset. ''**

**Response:**
We thank the reviewer for the valuable comment. As demonstrated in Section 5.1.1, the auxiliary data for purifying the backdoored BERT models is 20,000 plain texts from the BookCorpus dataset, which is less than 0.01\% compared with the original pre-training dataset. The auxiliary data for purifying the backdoored VGG and ViT models is 50,000 ImageNet validation samples, which is less than 5\% compared with the original pre-training dataset. We also show that only with 8000 ImageNet validation samples (less than 1\% compared with the original pre-training dataset), the defender can conduct the successful purification on the backdoored ViT model in Table 5 in the revised manuscript. As demonstrated in Section 5.1.1, the auxiliary data for purifying the backdoored CLIP model is 10,000 image-text pairs from the COCO dataset, which is less than 0.01\% compared with the original pre-training dataset.


----------

***Comment 3:***  **"The accuracy of clean models are not given in the tables."**

**Response:** 
We have shown the accuracy of clean models in **Table 2** and **Table 3** in the manuscript.

From: ICLR.cc/2023/Conference/Paper4493/Authors
title: Response 2 to Reviewer KEZm
comment: ***Comment 4:***  **"The technical novelty is very limited. This paper simply adds an L\_2 regularization to the original pre-training loss to continuously train PTMs. "**

**Response:**
(1) Though our defense method is simple, it is **practical and effective**. Our method does not need the downstream data for purifying the PTMs. After our defense, the ASR will be lowered significantly on all downstream tasks.  (2) **We want to emphasize that defending PTMs against backdoor attacks is a new but important problem.** One reason is that the “pre-train and then fine-tune” paradigm has become dominant in recent AI research works [1,2] and boosts the performance of downstream tasks, which makes the backdoor attack against PTMs a serious threat to the AI community. The other reason is that any downstream user who downloads the backdoored PTM will be the victim. It is urgent to propose defense methods against backdoor attacks for PTMs. (3) Previous defense works focus on purifying the downstream classification models with clean downstream data. They are not applicable to our scenario.  We have further illustrated our defense setting in Figure 1 in the revised manuscript. 


----------

***Comment 5:***  **"This paper does not in-depth study why such a simple modification to the loss function can defend backdoor attacks. The analysis on the number of neurons activated by clean and poisoned samples is shallow and does not provide scientific value.''**

**Response:**  (1) We have conducted neuron-level analysis by observing the overlap ratio of neurons activated by the clean and poisoned samples before and after purification. The inserted triggers are small for the poisoned samples (words or small patches). Thus, the neurons of a backdoor-free model should behave similarly on clean and poisoned samples, resulting in a high overlap ratio. We have added experiments to see the overlap ratio of neurons activated by clean samples and poisoned samples for clean PTMs. From the results in Table 1 and Table 2, the overlap ratio increases significantly after the purification process on all datasets and models. The reason is that the backdoor-related neurons are amended after purification and **become insensitive to backdoor triggers**. The purified PTM "forget" some backdoor knowledge embedded in the neurons.

Table 1: The overlap ratio of neurons activated by clean samples and poisoned samples for POR-backdoored BERT, purified backdoored BERT and clean BERT.

| Dataset | POR-backdoored BERT | purified backdoored BERT | Clean BERT |
| :-----: | :-----------------: | :----------------------: | :--------: |
|  SST-2  |       25.50%        |          70.84%          |   86.74%   |
|  HSOL   |       35.31%        |          76.58%          |   94.39%   |
| AG News |       25.83%        |          90.12%          |   96.31%   |

Table 2: The overlap ratio of neurons activated by clean samples and poisoned samples for backdoored ViT, purified backdoored ViT and clean pre-trained ViT.

| Dataset | backdoored ViT | purified backdoored ViT | Clean pre-trained ViT |
| :-----: | :------------: | :---------------------: | :-------------------: |
|  Waste  |     54.88%     |         80.50%          |        93.11%         |
| CatDog  |     56.41%     |         78.88%          |        91.99%         |
|  GTSRB  |     54.60%     |         75.16%          |        93.01%         |


(2) We also analyze the purification dynamics. From the results in Figure 3 in the revised paper, we can see that the MASR drops sharply at the end of the third epoch while the ACC is stable throughout the whole process. This phenomenon shows that the backdoor knowledge is gradually forgotten in the purification process through regularization and the purification process should sustain for several epochs to ensure success. The PTM learns benign knowledge from continual pre-training during the purification process simultaneously to retain the normal performance. 

----------

**References:**

[1] On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258, 2021.

[2] Pre-trained models: Past, present and future. AI Open, 2021.

From: ICLR.cc/2023/Conference/Paper4493/Authors
title: Response 3 to Reviewer KEZm
comment: ***Comment 6:***  **"Since the proposed method only uses an L\_2 regularization loss as the defense, an adaptive attacker can easily break such a simple defense by adopting the L\_2 regularization during the attack. This paper does not evaluate on this."**

**Response:**
We thank the reviewer for the constructive comment. Following the reviewer's suggestion, we perform experiments of adding the L2 regularization term during the attack. We experiment on poisoning the BERT model using the POR algorithm with regularization and poisoning the VGG model using the NeuBA algorithm with regularization, respectively. For the language model, we perform experiments on the BERT model using the poisoning method of POR with the regularization term for the adaptive attack. The experimental results are shown in Table 3. From the experimental results, we can see that our defense method is still effective under the adaptive attack. 

For the computer vision model, we perform experiments on the VGG model using the poisoning method of NeuBA with the regularization term for the adaptive attack. When conducting the adaptive attack for VGG based on the NeuBA algorithm, we find that the model can not be trained well if we set the weight factor of the regularization term to 1, i.e., the accuracy of the fine-tuned backdoored model on CatDog dataset drops to 50\%. If we set the weight factor of the regularization term to 0.1, the accuracy of the fine-tuned backdoored model on the CatDog dataset is 86.08\%. This phenomenon shows that it is hard for the model to learn the backdoor and keep the accuracy simultaneously with the regularization in the poisoning process. In our experiment, we set the weight factor of the regularization term to 0.01 for the adaptive attack. Then we use our method to purify the backdoored model. The results are shown in Table 4. In Table 4, we set the number of epochs as 10 when fine-tuning the model on downstream tasks. From the experimental results, we can see that the adaptive attack can influence the performance of our method slightly, as reflected in a decline in the accuracy of the CatDog downstream task. However, our defense method is still effective, i.e., our method significantly decreases the ASR. Moreover, we find that if the downstream user fine-tunes the purified PTM on CatDog for 6 epochs, the ACC is 94.12\% and the AASR and MASR are 11.35\% and 17.68\%, respectively. 
**Overall, our defense method is still effective even under the adaptive attack.**

Table 3: Results of purifying BERT that is backdoored by the adaptive POR attack.  

| Dataset & Method |  ACC  | AASR  | MASR  |
| :--------------: | :---: | :---: | :---: |
|   SST-2 (w/o Defense)    | 91.71 | 93.71 | 100.0 |
|  SST-2 (RECIPE)  | 90.88 | 12.23 | 16.94 |
|   HSOL (w/o Defense)    | 95.79 | 83.85 | 100.0 |
|   HSOL (RECIPE)   | 95.31 | 5.35  | 6.01  |
|  AG News (w/o Defense )  | 91.89 | 60.52 | 88.32 |
| AG News (RECIPE)  | 90.84 | 4.26  | 4.49  |

 Table 4: Results of purifying VGG that is backdoored by the adaptive NeuBA attack. 

| Dataset & Method |      ACC      |     AASR      |      MASR      |
| :--------------: | :-----------: | :-----------: | :------------: |
|   Waste ( w/o Defense )   |     90.13     |     99.75     |     100.0      |
|  Waste (RECIPE)  |     91.09     |     16.31     |     18.79      |
|  CatDog ( w/o Defense )   |     95.76     |     100.0     |     100.0      |
| CatDog (RECIPE)  | 90.20  | 16.36  | 19.68   |
|   GTSRB ( w/o Defense )   |     99.72     |     100.0     |     100.0      |
|  GTSRB (RECIPE)  |     99.72     |     1.23      |      3.61      |

From: ICLR.cc/2023/Conference/Paper4493/Authors
title: Response 4 to Reviewer KEZm
comment: ***Comment 7:***  **"In Table 1, the defense results are not stable for the proposed method. The ViT PTM still has near 70\% attack success rate after the defense, delineating the ineffectiveness of the proposed approach. Furthermore, the clean accuracy on CIFAR10 is much lower by RECIPE compared to fine-tuning. It is unclear how much the accuracy is lower than that of an undefended model since this paper does not provide the accuracy for original models.''**

**Response:**
(1) In fact, our defense method can achieve lower ASR for the purified ViT model on the Waste dataset, with a slight sacrifice of ACC. In Table 1, for purifying the backdoored ViT models, we conduct the purification process for 4 epochs using the auxiliary data. We perform further experiments and find that if we conduct the purification process for a longer time (5 epochs), the AASR and MASR for the purified ViT model on Waste can be lowered to 18.09\% and 38.13\%, respectively. Besides, the ACC is 92.92\%. (2) Finetuning (FT) results are exactly the accuracy of undefended models. Fine-tuning is a **necessary** process for adapting PTMs to downstream tasks. As described in page 6 in the original manuscript, "Finetuning (FT): Directly fine-tune the backdoored PTM without any defense." FT means fine-tuning the undefended model and the accuracy of FT is the accuracy of the original model. **To illustrate it more clearly, we have modified "FT" to "w/o Defense" and clarified its meaning more clearly. "w/o Defense: Directly fine-tune the backdoored PTM on downstream datasets without any defense."**

***Comment 8:***  **"Reproducibility: As mentioned above in the clarity part, this paper does not provide some details regarding evaluated backdoor attacks. The provided supplementary material also does not include the attack code, which hinders the reproduction.''**

**Response:**  We are sorry for not providing enough implementation details of the attack codes. Mostly, we do experiments on officially released backdoored PTMs. (1) We download the NeuBA-backdoored VGG model and backdoored ViT model from the huggingface website (https://huggingface.co/thunlp/neuba-cv/tree/main). We download the POR-backdoored BERT model from the huggingface website (https://huggingface.co/Lujia/backdoored_bert/tree/main). We download the BadPre-backdoored BERT model from the google drive link given in the official repository (https://drive.google.com/drive/folders/1Oal9AwLYOgjivh75CxntSe-jwwL88Pzd). (2) The official implementation of NeuBA is in the repository https://github.com/thunlp/NeuBA. The official implementation of POR is in the repository https://github.com/plasmashen/BackdoorPTM. The official implementation of BadPre is in the repository https://github.com/kangjie-chen/BadPre.

From: ICLR.cc/2023/Conference/Paper4493/Authors
title: Response 1 to Reviewer U24e
comment: We thank the reviewer for the insightful and constructive feedback for improving this paper. Our point-to-point responses to your comments are shown below.

***Comment 1:***  **"The paper seems to over-claim the difference between the defense strategy of traditional backdoor attacks and those tailored for PTMs. In both cases, the defense methods usually require clean data from the same distribution on which the model is trained on. For example, to defense traditional backdoor attacks (e.g. BadNet) on ImageNet classifiers, the defender usually requires a small amount of clean samples to purify the classifier. The same holds for your method (and also previous methods such as those in the NeuBA paper): Your method also requires a clean dataset (specifically the entire ImageNet validation set) to purify the poisoned VGG pretrained on ImageNet. In this sense, some claims in this paper does not make sense to me. (1) "existing backdoor-repairing defenses require task-specific knowledge (i.e., some clean downstream data), making them unsuitable for backdoored PTMs" This does not make sense to me. Because your defense also requires "task specific" knowledge. Your method needs clean data from the same pretraining dataset distribution. This is "task specific" knowledge - Here the "task" is your pretraining task. I see you included results on clean data different from the training set in Table 6. But that only shows partial results compared with the main results in Table 1, and thus are not convincing enough to show your method generally works in the different data distribution setting.''**

**Response:**
(1) We have performed experiments of using the auxiliary data that is irrelevant to the pre-training data and shown the results in Table 6 in the revised manuscript. The experimental results demonstrate that our method generally works in the different data distribution setting. To further prove this, following your suggestions, we have added the experiments of purifying backdoored VGG and POR-backdoored BERT using the auxiliary data with different distribution from the pre-training data. For example, BERT is pre-trained on the BookCorpus and Wikipedia datasets. We have shown that with the WebText data as the auxiliary data, the defender can also purify the backdoored BERT models. The experimental results are shown in Table 1. From the experimental results, we can see that our method generally works in the different data distribution setting.  (2) The auxiliary data we use can be irrelevant to both the pre-training data and downstream data. As demonstrated in Section 1 in the revised manuscript, in our scenario, our defense is conducted before the PTM is fine-tuned on downstream tasks. Thus, the defender can not access the downstream data. After our defense, the ASR (attack success rate) of the purified PTM will be lowered on all downstream tasks, as shown in Section 5.1.2 and Section 3 in the revised manuscript. We have further illustrated our defense setting in Figure 1 in the revised manuscript.

Table 1: Results of purifying the backdoored PTMs with the auxiliary data that is irrelevant to the pre-training data.

|   Dataset   |   Waste   |   Waste   |   Waste   |  CatDog  |  CatDog  |  CatDog  |    GTSRB    |    GTSRB    |    GTSRB    |
| :---------: | :-------: | :-------: | :-------: | :------: | :------: | :------: | :---------: | :---------: | :---------: |
|    Model    |    ACC    |   AASR    |   MASR    |   ACC    |   AASR   |   MASR   |     ACC     |    AASR     |    MASR     |
|     VGG     |   92.32   |   17.45   |   20.05   |  94.12   |   8.03   |   8.96   |    99.72    |    0.51     |    0.56     |
|     ViT     |   93.47   |   19.72   |   41.37   |  93.80   |  12.96   |  15.36   |    99.65    |    1.67     |    2.75     |
|    CLIP     |   92.68   |   16.49   |   30.85   |  95.48   |   6.28   |   7.04   |    99.65    |    0.36     |    0.43     |
| **Dataset** | **SST-2** | **SST-2** | **SST-2** | **HSOL** | **HSOL** | **HSOL** | **AG News** | **AG News** | **AG News** |
|    Model    |    ACC    |   AASR    |   MASR    |   ACC    |   AASR   |   MASR   |     ACC     |    AASR     |    MASR     |
|  POR BERT   |   91.10   |   13.38   |   17.71   |  95.47   |   5.40   |   5.85   |    90.54    |    5.29     |    6.70     |
| BadPre BERT |   90.23   |   11.05   |   12.50   |  95.23   |   6.79   |   7.46   |    90.39    |    5.02     |    5.32     |

From: ICLR.cc/2023/Conference/Paper4493/Authors
title: Response 2 to Reviewer U24e
comment: ***Comment 2:***    **"(1) "Existing backdoor defenses on downstream classification models are not applicable to this setting, as they assume that the defender has some clean data of downstream tasks."
This also doesn't seem correct to me. I can't figure out what prevents us from applying those previous methods (e.g., fine-pruning, NAD, ANP [1], I-BAU [2], BNP [3], T\&R [4], AWM [5] etc) on PTMs? You run the fine-pruning experiments in your own experiments, and the original NeuBA paper benchmarked fine-pruning and NAD. In my opinion, I-BAU and ANP can also be applied to defense PTMs - just use the clean data from the pretraining distribution as you did in your own method. What needs to be shown is how well they generalize to such new settings. However, results of these methods are not included in the main text. For the revision, I suggest to compare the proposed method with some recent state-of-the-art traditional backdoor defense methods in the PTM setting, or at least discuss in more detail why they are not applicable. (2) Missing experiments. Since FT worked well in the original NeurBA paper, the authors of NeuBA assumed pruning might be a promising direction to defend PTM backdoor. In view of that, I would suggest the authors to compare the proposed method with more advanced pruning-based defense methods (such as ANP [1] and ideally more recent works BNP [3] and AWM [5]) to defense PTM backdoor."**

**Response:**
Thank you for the constructive suggestion. We have carefully read the references. Among them, BNP [3] and T\&R [4] are released after the ICLR deadline, so we are not able to discuss and compare with them. Moreover, our setting is different from T\&R [4]. In T\&R, the defender wants to train a clean model on a poisoned dataset and the defender can control the model training process. However, our method is designed to purify an already backdoored PTM with downstream task-irrelevant data. As for NAD [6], ANP [1], I-BAU [2], and AWM [5], they all target at purifying downstream classification models. We have added the detailed discussion of the reason why existing backdoor purification approaches cannot be directly applied to our scenario in appendix B in the revised manuscript.

 "Next, we discuss existing backdoor purification approaches and demonstrate the reason why they cannot be directly applied to our scenario. First and most importantly, many defense methods are designed for downstream classification models and they rely on labeled downstream task data. NAD [6] gets a teacher model by fine-tuning the backdoored model on a small subset of clean downstream data. ANP [1] and AWM [5] both use classification loss on clean downstream data to learn neuron masks. I-BAU [2] also relies on the classification loss to formulate and solve the minimax game. Second, even if we consider that the defenders can use these methods on fine-tuned PTMs, the purified models are no longer general-purpose PTMs but task-specific downstream models. Third, some defense strategies rely on specific assumptions of models or inputs, which limits their application scope. BNP [3] leverages the statistics recorded in Batch Normalization (BN) layers, so it is not applicable for Transformer-based models, e.g., BERT, ViT and CLIP, which have no BN layers. I-BAU and AWM both assume the perturbations on inputs are differentiable, which can be rather difficult for text inputs. Instead, we choose Fine-pruning as our main baseline. Fine-pruning [7] prunes neurons with the smallest activations on downstream clean samples to purge poisoned modules. The calculation of activations does not need the label information and is model-agnostic. Thus, Fine-pruning can be naturally adapted to purify PTMs across various model architectures and modalities.'' 

----------


**References:**

[1] Adversarial Neuron Pruning Purifies Backdoored Deep Models. NeurIPS, 2021. 

[2] ADVERSARIAL UNLEARNING OF BACKDOORS VIA IMPLICIT HYPERGRADIENT. ICLR, 2022. 

[3] Pre-activation Distributions Expose Backdoor Neurons. NeurIPS, 2022. 

[4] Trap and Replace: Defending Backdoor Attacks by Trapping Them into an Easy-to-Replace Subnetwork. NeurIPS, 2022. 

[5] One-shot Neural Backdoor Erasing via Adversarial Weight Masking. NeurIPS, 2022.

[6] Neural attention distillation: Erasing backdoor triggers from deep neural networks. NeurIPS, 2021.

[7] Fine-pruning: Defending against backdooring attacks on deep neural networks. In International Symposium on Research in Attacks, Intrusions, and Defenses, pages 273–294. Springer, 2018.

From: ICLR.cc/2023/Conference/Paper4493/Authors
title: Response 3 to Reviewer U24e
comment: ***Comment 3:***  **""to the best of our knowledge, no defense solution has been proposed against backdoor attacks on PTMs"
This seems to be over-claim. In the original NeuBA paper, it is shown that fine-pruning can achieve pretty good defense performance in many (although not all) cases.''**

**Response:**
We thank the reviewer for the valuable comment. In the original NeuBA paper, they use clean downstream datasets to purify the fine-tuned PTMs. **Their defense setting is different from ours.** They first fine-tune the PTM with the clean downstream data and get a fine-tuned classification model. Then they use the fine-pruning algorithm with **clean downstream data** to perform the purification. As shown in the appendix of NeuBA, "For BERT, we calculate the activations of both attention sublayers and feedforward sublayers in a **fine-tuned** backdoored model, and prune a specific ratio of dormant output neurons." However, our defense is applied before the backdoored PTM is fine-tuned on downstream tasks. We use the auxiliary data that is irrelevant to the downstream task. Following your suggestion, we will modify this claim to "to the best of our knowledge, no defense solution has been proposed against backdoor attacks on PTMs without using clean downstream data". Also, we have added a figure (Figure 1) in the revision to further illustrate the difference between our defense method and previous approaches. 



----------










***Comment 4:***  **"Some experimental results seem inconsistent with previous works. In the original NeuBA paper, it is shown that fine-pruning (FT) can achieve pretty good defense performance in many (although not all) cases. However, in your paper, FT seems always ineffective. Could you please explain why? Are there any difference in experiment settings causing this difference?"**

**Response:** 
(1) In case of any misunderstanding, we'd like to emphasize that FT means directly fine-tuning the backdoored PTM without any defense, while fine-pruning is denoted as FP, as described in page 6 of the manuscript. To make it more clear, we have modified  "FT'' to "w/o Defense'' in the revised manuscript.  (2) **The implementation of fine-pruning in the original NeuBA paper prunes neurons of all layers, which is similar to the FP-GA/FP-GM in our manuscript.** FP-GA in our manuscript can also decrease the ASR for some cases, which is similar to what NeuBA reports about fine-pruning. Specifically, in the appendix of the original NeuBA paper, they have illustrated their implementation of fine-pruning. "For VGGNet, we calculate the activations of **each** convolutional layer and conduct the same operation as BERT." In Section 5.1.1 of our manuscript, we have illustrated the implementation details of our adapted fine-pruning (FP) and global fine-pruning (FP-G). For our implementation of FP, we follow the original implementation of fine-pruning [1] and only prune neurons of the last layer. For FP-GA,/FP-GM in our manuscript, we extend the original implementation by pruning neurons in all layers. To sum up, the FP in our manuscript only prunes neurons of the last layer, which is different from the implementation in the NeuBA paper. Besides, the FP-GA in our manuscript has similar implementations with the fine-pruning in the NeuBA paper. Thus, as expected, the experimental results we reported for FP-GA are similar to those reported for fine-pruning in the NeuBA paper. (3) In the original NeuBA paper, they use the clean downstream datasets to purify the fine-tuned PTMs. Their defense setting is different from ours. They first fine-tune the PTM with the clean downstream data and get a fine-tuned classification model. Then they use the fine-pruning algorithm with clean downstream data to perform the purification. However, our defense is applied before the backdoored PTM is fine-tuned on downstream tasks. We use the auxiliary data that is irrelevant to the downstream task. 



----------


**References:**

[1] Fine-pruning: Defending against backdooring attacks on deep neural networks. In International Symposium on Research in Attacks, Intrusions, and Defenses, pages 273–294. Springer, 2018.

From: ICLR.cc/2023/Conference/Paper4493/Authors
title: Response 4 to Reviewer U24e
comment: ***Comment 5:***  **"The intuition of using l2-norm regularization is unclear to me, and the novelty seems limited. The authors motivated their method as to prune the backdoor neurons which only activates for backdoor samples. This is the same motivation as many previous defense methods such as FT, ANP, BNP, and AWM. To achieve sparsity, l0 or l1 norm seem better choice than l2 norm. More explanations seem needed here to motivate the choice of l2 norm over others.''**

**Response:**
We thank the reviewer for the constructive comment. L0 norm is non-differentiable, so we do not use L0 norm as the regularization term. Following the reviewer's suggestion, we have performed experiments to use L1 norm regularization when purifying the BadPre-backdoored BERT model. When performing this experiment, we first use the same hyper-parameter setting as the experiment of the L2 norm in Table 1 of the manuscript, i.e., using 20,000 plain texts to train the model for 8 epochs. The experimental results are shown in Table 2. It can be seen that the ACC decreases more compared with that of using the L2 norm on SST-2. Further, we perform our purification with the L1 norm regularization using 20,000 plain texts for 4 epochs and 5 epochs, respectively. From the experimental results, we can see that on SST-2, with the L1 norm regularization,  low ASR can only be achieved with the cost of an evident drop of the accuracy of the fine-tuned model. Thus, we use the L2 norm as the feasible choice for our method.

Table 2: Results of purifying BadPre-backdoored BERT with L1 norm regularzation.

| Dataset  | SST-2 | SST-2 | SST-2 | HSOL  | HSOL | HSOL | AG News | AG News | AG News |
| :------: | :---: | :---: | :---: | :---: | :--: | :--: | :-----: | :-----: | :-----: |
|  Epochs  |  ACC  | AASR  | MASR  |  ACC  | AASR | MASR |   ACC   |  AASR   |  MASR   |
| 4  | 86.60 | 72.12 | 95.71 | 95.07 |  5.84 | 8.26 |  90.16  |  7.47    |  9.56   |
| 5  | 83.25 |  18.71| 19.30  |  94.91|5.71   | 6.74 |  89.86  |   5.17 |  5.39   |
| 8  |  82.48| 18.24 | 18.97  | 94.59 | 5.93  | 6.01 |  89.24  |  5.84   |  6.16   |

From: ICLR.cc/2023/Conference/Paper4493/Authors
title: Response 5 to Reviewer U24e
comment: ***Comment 6:***  **"Also, if such a simple regularization can defense the PTM backdoor, does it mean that the current PTM backdoor attacks are in fact pretty weak (and thus PTMs are more robust than we expected)?''**

**Response:** 
The topics of backdoor attacks and defenses for PTMs are in the starting stage and there are not many research works on this problem. In particular, to the best of our knowledge, no defense solution has been proposed against backdoor attacks on PTMs without using clean downstream data. Our work paves the way for practical backdoor defenses designed for PTMs. Besides, **we want to emphasize that defending PTMs against backdoor attacks is a new but important problem**. One reason is that the “pre-train and then fine-tune” paradigm has become dominant in recent AI research works [1,2], which makes the backdoor attack against PTMs a serious threat to the AI community. The other reason is that any downstream user who downloads the backdoored PTM will be the victim. Moreover, existing backdoor defense methods focus on downstream classification models, which cannot be directly applied to the backdoor defense for PTMs.

----------

***Comment 7:***  **"Is there a weight hyper-parameter to trade-off the two terms in Eq 1? I think most models are trained with weight decay, which is basically the l2 norm regularization as shown in Eq 1. So I'm wondering what's the difference between Eq 1 and normal pretraining. Do you use a very large weight decay factor?''**

**Response:** (1) There is not a weight hyper-parameter to trade off the two terms in Eq 1. We just use the ratio 1:1 for the two terms. Following your suggestion, we have added a new experiment to adjust the weight factor for the regularization term, which is shown in Table 8 in the revised manuscript. From the experimental results in Table 3, we can see that under a very small weight factor for the regularization term (0.1), the ASR is still very high after the defense. However, under a large weight factor for the regularization term (1.5), the ACC drops much on the SST-2 dataset. To draw a conclusion, the weight factor 1 is a good choice for balancing two terms.


Table 3: Results of purifying BadPre-backdoored BERT with different weight factors for the regularization term.

| Dataset | SST-2 | SST-2 | SST-2 | HSOL  | HSOL  | HSOL  | AG News | AG News | AG News |
| :-----: | :---: | :---: | :---: | :---: | :---: | :---: | :-----: | :-----: | :-----: |
| Weight  |  ACC  | AASR  | MASR  |  ACC  | AASR  | MASR  |   ACC   |  AASR   |  MASR   |
|   1.5   | 84.90 | 16.36 | 17.65 | 95.47 | 5.00  | 5.45  |  90.05  |  5.08   |  5.35   |
|   1.2   | 89.84 | 11.84 | 12.61 | 95.03 | 5.21  | 5.45  |  90.14  |  4.98   |  5.26   |
|    1    | 90.12 | 11.95 | 12.72 | 94.99 | 5.00  | 5.29  |  90.03  |  5.01   |  5.32   |
|   0.5   | 91.32 | 15.27 | 18.26 | 94.91 | 5.02  | 5.29  |  90.11  |  5.02   |  5.47   |
|   0.1   | 91.65 | 95.69 | 96.81 | 95.11 | 93.98 | 97.11 |  91.37  |  87.99  |  91.33  |

(2) Compared with the weight decay, our method can select some weights which are put into the regularization term and thus is more flexible. 
(3) For purifying backdoored BERT, VGG and ViT models, the weight decay is all set as 0. For purifying backdoored CLIP model, the weight decay is set as 0.001, which is much smaller than the weight factor for our regularization term.

----------

**References:**

[1] On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258, 2021.

[2] Pre-trained models: Past, present and future. AI Open, 2021.

From: ICLR.cc/2023/Conference/Paper4493/Authors
title: Response 1 to Reviewer m2cs
comment: We thank the reviewer for the insightful and constructive feedback for improving this paper. Our point-to-point responses to your comments are shown below.

***Comment 1:***  **"A major concern that I have about this paper is to use of auxiliary data for backdoor removal. Backdoor attacks exploit data poisoning to inject malicious behaviors into a victim model. If we collect the auxiliary data, there's a possibility that a set of poisoning samples are also included in the data. Moreover, Calini and Terzis showed that even a very small amount of poisoning samples can introduce backdoor behaviors when we train a large-scale pre-trained model on it. Unless we make sure that the auxiliary data does not contain any poisoning examples, this defense can amplify the backdoor behaviors and introduce other backdoors into the model. (2) As the backdooring is done by data poisoning, and the auxiliary data can be poisoned by the adversary (based on the threat model that allows collecting them from public sources), I am confident that the defense was broken by design. This would be an even more serious problem as someone who will read this paper and implement this defense could be vulnerable to backdoor attacks.''**

**Response:** Thank you for the valuable comments. (1) It is a common practice to have the assumption that the defender can access some clean data. For example, previous works[1,2] even assume that the defender has a small set of clean downstream data. Besides, the public auxiliary data can be downloaded from trustful official websites such as ImageNet (https://www.image-net.org/download.php) and bookcorpus (https://huggingface.co/datasets/bookcorpus). Those classic publicly available datasets are from trusted sources and thus considered to be clean. (2) **Unlike the situation of trojaning downstream models via data poisoning, when conducting the backdoor attacks for PTMs, only using the poisoned data is not enough.** Firstly, we performed new experiments to prove that poisoning the auxiliary data has minor effects on our defense. We perform experiments to use our method with the poisoned plain texts as the auxiliary data to purify the POR-backdoored BERT model. To generate poisoned plain texts, we insert the same triggers that are used for testing the attack success rate (ASR) of POR (i.e, "cf", "mn", "bb", "tq","mb") to the plain text. The poisoning ratio is 5\%. **From the experimental results in Table 1, we can see that even with the poisoned data, our defense is still effective.** This indicates that only poisoning the data is not enough to enhance the attack performance because it lacks the additional attack training strategy (i.e., mapping the [CLS] embedding of poisoned data to given vectors) in the defense process. 

 Secondly, most existing works about trojaning PTMs require the attacker to have the ability beyond poisoning the training dataset. In NeuBA[3] and POR[4], the attacker is assumed to have the ability of manipulating the training loss function, i.e., **adding the additional training goal of mapping the output representations of the poisoned data to the given specific vectors**. The output representations will be the input to the downstream classifier when the PTM is fine-tuned on downstream tasks, so that any downstream task will inherit the backdoors. In BadPre[5], the attacker needs to replace the [MASK] tokens' label words during pre-training to perform the attack. The process of masking some tokens and replacing the corresponding label words is also conducted in the training process. 
 
Table 1: Results of purifying POR-backdoored BERT with the poisoned auxiliary data using our method.

| Dataset |  ACC  | AASR  | MASR  |
| :-----: | :---: | :---: | :---: |
|  SST-2  | 90.39 | 12.23 | 14.41 |
|  HSOL   | 95.47 | 5.24  | 6.82  |
| AG News | 90.42 | 7.91  | 11.49 |




 
----------

**References:**

[1] Neural attention distillation: Erasingbackdoor triggers from deep neural networks. Advances in Neural Information Processing Systems, 2021.

[2] Adversarial neuron pruning purifies backdoored deep models. Advances in NeuralInformation Processing Systems, 34:16913–16925, 2021

[3] Red alarm for pre-trained models: Universal vulnerabilities by neuron-level backdoor attacks. arXivpreprint arXiv:2101.06969, 2021.

[4] Backdoor pre-trained models can transfer to all. In CCS ’21: 2021 ACM SIGSAC Conference on Computerand Communications Security

[5] Badpre:Task-agnostic backdoor attacks to pre-trained nlp foundation models. In Proceedings of ICLR, 2022.

From: ICLR.cc/2023/Conference/Paper4493/Authors
title: Response 2 to Reviewer m2cs
comment: ***Comment 2:***  **"Because the attack exploits fine-tuning, the defense may work. NeuBA attack fine-tunes a pre-trained model with the backdooring objective. During training, there's a possibility that it only introduces a minimal change to the model to achieve backdoor behaviors, which makes the separation of neurons used by backdooring from the benign classification. I would wonder if those large-scale pre-trained models are trained with poisoning samples from scratch, does this defense work against those backdoors?''**

***Response:*** (1) It is a common paradigm to trojan the PTM from a pre-trained model rather than from scratch [1,2,3]. Training large-scale pre-trained models from scratch is infeasible for normal users (including attackers and defenders) because training large-scale pre-trained models from scratch needs enormous computing resources and a very long time [4]. (2) Nevertheless, to better address the reviewer's concerns, under the limited rebuttal time, we perform experiments to run the poisoning process of NeuBA on VGG without preloading the pre-trained VGG, i.e, to train the poisoned PTM from scratch. Following the poisoning process of NeuBA, the model is trained to map the poisoned samples' output representations to specific vectors and the model is pre-trained on clean ImageNet64 data simultaneously. Without loading the pre-trained weights, the accuracy of fine-tuned models is lower on downstream tasks compared with that of with loading the pre-trained weights. From the experimental results in Table 2, we can see that our method can achieve good defense performance when PTMs are trained with poisoned samples from scratch.

 Table 2: Results of purifying VGG that is trained with poisoned samples from scratch based on the NeuBA algorithm. 

| Dataset & Method |  ACC  | AASR  | MASR  |
| :--------------: | :---: | :---: | :---: |
|   Waste ( w/o Defense )   | 88.74 | 99.63 |  100  |
|  Waste (RECIPE)  | 89.97 | 16.78 | 18.17 |
|  CatDog (w/o Defense )   | 88.92 | 98.33 |  100  |
| CatDog (RECIPE)  | 86.48 | 16.63 | 24.0  |
|   GTSRB ( w/o Defense )   | 99.72 | 100.0 | 100.0 |
|  GTSRB (RECIPE)  | 99.57 | 4.28  | 6.96  |

----------

***Comment 3:***  **"The hypothesis here is that the fine-tuning process on the auxiliary dataset when the back-propagation happens, should modify a set of neurons "corresponds" to the backdoor behaviors. But, there are possible cases where it is not. Given this concern, I think the defense may or may not work depending on the quality of the dataset.''**

***Response:***
(1) The regularization term can destroy the neurons related to backdoor behaviours as shown in Table 4 in the revised manuscript. And the regularization term is irrelevant to the auxiliary data. Continually pre-training the model on the auxiliary data helps maintain the accuracy of the fine-tuned model on downstream tasks. (2) We have performed experiments to use different auxiliary data to purify the backdoored PTMs. From the experimental results in Table 6 in the revised manuscript, we can see that our method is still effective with the auxiliary data that is irrelevant to the pre-training data. (3) We have performed experiments of using a small amount of auxiliary data to purify the backdoored PTMs. From the experimental results in Table 5 in the revised manuscript, we can see that even with a small amount of auxiliary data, our method can obtain satisfactory purification results. As the small amount of auxiliary data limits the data quality, the experiment results indicate our method is insensitive to the data quality in a way.

----------

**References:**

[1] Red alarm for pre-trained models: Universal vulnerabilities by neuron-level backdoor attacks. arXivpreprint arXiv:2101.06969, 2021.

[2] Badpre:Task-agnostic backdoor attacks to pre-trained nlp foundation models. In Proceedings of ICLR, 2022.

[3] Backdoor pre-trained models can transfer to all. In CCS ’21: 2021 ACM SIGSAC Conference on Computerand Communications Security

[4] BERT: pre-training of deep bidirectionaltransformers for language understanding. In Proceedings of NAACL-HLT, pages 4171–4186, 2019.

From: ICLR.cc/2023/Conference/Paper4493/Authors
title: Response 3 to Reviewer m2cs
comment: ***Comment 4:***  **"A straightforward way to circumvent this attack is to train a model with the backdooring objective that intertwines clean and backdoor behaviors into a few sets of neurons in each layer. I think it's an easy job as the attacker can analyze the activation patters from the clean and backdoor samples and make them indistinguishable in the activation space. In this way, there is a high chance that RECIPE fine-tuning degrades the accuracy of a pre-trained model significantly.''**

**Response:**  (1) The backdoor behviours are embedded in the backdoor-related neurons which are sensitive to specific triggers. We believe it is difficult to intertwine clean and backdoor behaviours in the same set of neurons when the backdoor attack is successful. We have conducted neuron-level experiments to derive more insights about this. We analyze the overlap ratio of neurons activated by clean samples and poisoned samples. From the experiments in Table 3 and Table 4, we can see that the overlap ratio of backdoored PTMs is significantly lower compared with those of purified backdoored PTMs and clean PTMs. We infer that a low overlap ratio is relevant to the success of the backdoor attack. Thus, we think that intertwining the clean and backdoor behaviours in the same set of neurons may hinder the learning of backdoor behaviours. 

Table 3: The overlap ratio of neurons activated by clean samples and poisoned samples for POR-backdoored BERT, purified backdoored BERT and clean BERT.

| Dataset | POR-backdoored BERT | purified backdoored BERT | Clean BERT |
| :-----: | :-----------------: | :----------------------: | :--------: |
|  SST-2  |       25.50%        |          70.84%          |   86.74%   |
|  HSOL   |       35.31%        |          76.58%          |   94.39%   |
| AG News |       25.83%        |          90.12%          |   96.31%   |

Table 4: The overlap ratio of neurons activated by clean samples and poisoned samples for backdoored ViT, purified backdoored ViT and clean pre-trained ViT.

| Dataset | backdoored ViT | purified backdoored ViT | Clean pre-trained ViT |
| :-----: | :------------: | :---------------------: | :-------------------: |
|  Waste  |     54.88%     |         80.50%          |        93.11%         |
| CatDog  |     56.41%     |         78.88%          |        91.99%         |
|  GTSRB  |     54.60%     |         75.16%          |        93.01%         |

(2) However, we agree that testing our defense against the adaptive attack can improve this paper. Inspired by your suggestions, we have implemented and tested our defense against a kind of adaptive attack, i.e., adding the regularization term in the poisoning process. For the language model, we perform experiments on the BERT model using the poisoning method of POR with the regularization term for the adaptive attack.  The experimental results are shown in Table 5.  For the computer vision model, we perform experiments on the VGG model using the poisoning method of NeuBA with the regularization term for the adaptive attack. The results are shown in Table 6. In Table 6, we set the number of epochs as 10 when fine-tuning the model on downstream tasks. From the experimental results, we can see that the adaptive attack can influence the performance of our method slightly, as reflected in a decline in the accuracy of the CatDog downstream task. However, our defense method is still effective, i.e., our method significantly decreases the ASR. Moreover, we find that if the downstream user fine-tunes the purified PTM on CatDog for 6 epochs, the ACC is 94.12\% and the AASR and MASR are 11.35\% and 17.68\%, respectively. Overall, our defense method is still effective even under the adaptive attack. 

 Table 5: Results of purifying BERT that is backdoored by the adaptive POR attack.  

| Dataset & Method |  ACC  | AASR  | MASR  |
| :--------------: | :---: | :---: | :---: |
|   SST-2 (w/o Defense)    | 91.71 | 93.71 | 100.0 |
|  SST-2 (RECIPE)  | 90.88 | 12.23 | 16.94 |
|   HSOL (w/o Defense)    | 95.79 | 83.85 | 100.0 |
|   HSOL (RECIPE)   | 95.31 | 5.35  | 6.01  |
|  AG News (w/o Defense )  | 91.89 | 60.52 | 88.32 |
| AG News (RECIPE)  | 90.84 | 4.26  | 4.49  |

 Table 6: Results of purifying VGG that is backdoored by the adaptive NeuBA attack. 

| Dataset & Method |      ACC      |     AASR      |      MASR      |
| :--------------: | :-----------: | :-----------: | :------------: |
|   Waste ( w/o Defense )   |     90.13     |     99.75     |     100.0      |
|  Waste (RECIPE)  |     91.09     |     16.31     |     18.79      |
|  CatDog ( w/o Defense )   |     95.76     |     100.0     |     100.0      |
| CatDog (RECIPE)  | 90.20  | 16.36  | 19.68   |
|   GTSRB ( w/o Defense )   |     99.72     |     100.0     |     100.0      |
|  GTSRB (RECIPE)  |     99.72     |     1.23      |      3.61      |

We are wondering if this kind of adaptive attack addresses your concerns? If you have any other suggestions, please let us know.

From: ICLR.cc/2023/Conference/Paper4493/Authors
title: Response 4 to Reviewer m2cs
comment: **Comment 5:**  ***"Evaluating the impact of RECIPE (accuracy degradation) with the accuracy loss on downstream datasets may not be the right way to measure how the defense destroys the model. Anyway the sanitized models will be trained on the downstream data, which will recover the accuracy of the model on it. So, I would use a different metric to quantify the impact.''***

**Response:**  Under the pre-train and then fine-tune paradigm, users download the PTM from public platforms and fine-tune it on their own downstream tasks. Therefore, from the perspective of the application, users care about the model performance (accuracy) most. Also, measuring accuracy degradation is widely used in backdoor purification [1,2,3,4]. So we argue that it is an appropriate metric to quantify the impact of the defense.  

----------



**Comment 6:**  ***"The defense works with a specific backdoor attack (NeuBA)''***

**Response:**  Actually, our defense is not attack-specific. On the contrary, it can be applied to defend against multiple backdoor attacks. In Section 5 of the manuscript, we also evaluate our defense method against BadPre[5] and POR[6] and our defense method can achieve good defense performance. Moreover, we adapt NeuBA on the multimodal model (CLIP) and our defense method can also achieve good defense performance.

----------

**References:**

[1] Fine-pruning: Defending against backdooring attacks on deep neural networks. In International Symposium on Research in Attacks, Intrusions, and Defenses, pages 273–294. Springer, 2018.

[2] Neural attention distillation: Erasingbackdoor triggers from deep neural networks. Advances in Neural Information Processing Systems, 2021.

[3] Red alarm for pre-trained models: Universal vulnerabilities by neuron-level backdoor attacks. arXivpreprint arXiv:2101.06969, 2021.

[4] Adversarial neuron pruning purifies backdoored deep models. Advances in NeuralInformation Processing Systems, 34:16913–16925, 2021

[5] Badpre:Task-agnostic backdoor attacks to pre-trained nlp foundation models. In Proceedings of ICLR, 2022.

[6] Backdoor pre-trained models can transfer to all. In CCS ’21: 2021 ACM SIGSAC Conference on Computerand Communications Security

From: ICLR.cc/2023/Conference/Paper4493/Reviewer_m2cs
confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
summary_of_the_paper: This paper proposes a defense mechanism that removes backdoors from large pre-trained models. The proposed mechanism, called RECIPE, purifies a pre-trained model by fine-tuning them on an auxiliary dataset obtained from public sources with the regularization objective. The objective serves as a signal that suppresses backdoor activations. The paper runs the evaluation on three different versions of pre-trained models and four downstream benchmarking datasets and shows that RECIPE can reduce the attack success rate significantly while preserving the original accuracy of those models on the downstream tasks.
strength_and_weaknesses: Strengths:

1. The paper presents a simple yet seemingly effective defense against backdoor attacks.
2. The paper evaluates the defense's effectiveness with several existing defenses (like fine-tuning / fine-pruning).


Weaknesses:

1. The proposed defense is broken by nature as it relies on auxiliary data, possibly collected from public sources.
2. The effectiveness of the proposed defense seems to overfit specific attack methodologies.
3. The defense may rely on partial observations from a model; adaptive attackers can circumvent the defense.


Detailed comments:

[Broken Defense]

A major concern that I have about this paper is to use of auxiliary data for backdoor removal. Backdoor attacks exploit data poisoning to inject malicious behaviors into a victim model. If we collect the auxiliary data, there's a possibility that a set of poisoning samples are also included in the data. Moreover, Calini and Terzis showed that even a very small amount of poisoning samples can introduce backdoor behaviors when we train a large-scale pre-trained model on it. Unless we make sure that the auxiliary data does not contain any poisoning examples, this defense can amplify the backdoor behaviors and introduce other backdoors into the model.


[Effectiveness of Defense Mechanisms]

> Possibly overfit to the attacks used in the paper.

Because the attack exploits fine-tuning, the defense may work. NeuBA attack fine-tunes a pre-trained model with the backdooring objective. During training, there's a possibility that it only introduces a minimal change to the model to achieve backdoor behaviors, which makes the separation of neurons used by backdooring from the benign classification. I would wonder if those large-scale pre-trained models are trained with poisoning samples from scratch, does this defense work against those backdoors?

> Defense quality vs. auxiliary data quality.

The hypothesis here is that the fine-tuning process on the auxiliary dataset when the back-propagation happens, should modify a set of neurons "corresponds" to the backdoor behaviors. But, there are possible cases where it is not. Given this concern, I think the defense may or may not work depending on the quality of the dataset.


[Consideration of Adaptive Defenses]

A straightforward way to circumvent this attack is to train a model with the backdooring objective that intertwines clean and backdoor behaviors into a few sets of neurons in each layer. I think it's an easy job as the attacker can analyze the activation patters from the clean and backdoor samples and make them indistinguishable in the activation space. In this way, there is a high chance that RECIPE fine-tuning degrades the accuracy of a pre-trained model significantly.


[Minor]

Evaluating the impact of RECIPE (accuracy degradation) with the accuracy loss on downstream datasets may not be the right way to measure how the defense destroys the model. Anyway the sanitized models will be trained on the downstream data, which will recover the accuracy of the model on it. So, I would use a different metric to quantify the impact.
clarity,_quality,_novelty_and_reproducibility: My comments about the correctness, quality, and novelty are summarized above. No concern about the reproducibility.
summary_of_the_review: This paper presents a way to remove a backdoor from large-scale pre-trained models by fine-tuning them on the auxiliary data with regularization, and the paper shows some effectiveness in the evaluation section.

However, as the backdooring is done by data poisoning, and the auxiliary data can be poisoned by the adversary (based on the threat model that allows collecting them from public sources), I am confident that the defense was broken by design. This would be an even more serious problem as someone who will read this paper and implement this defense could be vulnerable to backdoor attacks.

The paper's evaluation is a bit less convincing as there are the possibilities that:

(1) The defense works with a specific backdoor attack (NeuBA),
(2) The defense largely depends on the quality of auxiliary data.
(3) The adaptive attacks seem feasible in their threat model.

Thus, I am a bit leaning toward rejecting this paper.
correctness: 2: Several of the paper’s claims are incorrect or not well-supported.
technical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
empirical_novelty_and_significance: 1: The contributions are neither significant nor novel.
flag_for_ethics_review: NO.
recommendation: 5: marginally below the acceptance threshold

From: ICLR.cc/2023/Conference/Paper4493/Reviewer_U24e
confidence: 3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
summary_of_the_paper: This paper shows a simple l2 norm regularization on model weights can effectively defense state-of-the-art backdoor attacks for pretrained models (PTMs). After reading the paper, I have a few questions. Please see details below. 
strength_and_weaknesses: 1. The paper seems to over-claim the difference between the defense strategy of traditional backdoor attacks and those tailored for PTMs. In both cases, the defense methods usually require clean data from the same distribution on which the model is trained on. For example, to defense traditional backdoor attacks (e.g. BadNet) on ImageNet classifiers, the defender usually requires a small amount of clean samples to purify the classifier. The same holds for your method (and also previous methods such as those in the NeuBA paper): Your method also requires a clean dataset (specifically the entire ImageNet validation set) to purify the poisoned VGG pretrained on ImageNet. 

In this sense, some claims in this paper does not make sense to me.

- "existing backdoor-repairing defenses require task-specific knowledge (i.e., some clean downstream data), making them unsuitable for backdoored PTMs" 

This does not make sense to me. Because your defense also requires "task specific" knowledge. Your method needs clean data from the **same** pretraining dataset distribution. This is "task specific" knowledge - Here the "task" is your pretraining task. I see you included results on clean data different from the training set in Table 6. But that only shows partial results compared with the main results in Table 1, and thus are not convincing enough to show your method generally works in the different data distribution setting. 

- "Existing backdoor defenses on downstream classification models are not applicable to this setting, as they assume that the defender has some clean data of downstream tasks."

This also doesn't seem correct to me. I can't figure out what prevents us from applying those previous methods (e.g., fine-pruning, NAD, ANP [1], I-BAU [2], BNP [3], T&R [4], AWM [5] etc) on PTMs? You run the fine-pruning experiments in your own experiments, and the original NeuBA paper benchmarked fine-pruning and NAD. In my opinion, I-BAU and ANP can also be applied to defense PTMs - just use the clean data from the pretraining distribution as you did in your own method. What needs to be shown is how well they generalize to such new settings. However, results of these methods are not included in the main text. For the revision, I suggest to compare the proposed method with some recent state-of-the-art traditional backdoor defense methods in the PTM setting, or at least discuss in more detail why they are not applicable.

[1] Adversarial Neuron Pruning Purifies Backdoored Deep Models. NeurIPS, 2021.
[2] ADVERSARIAL UNLEARNING OF BACKDOORS VIA IMPLICIT HYPERGRADIENT. ICLR, 2022.
[3] Pre-activation Distributions Expose Backdoor Neurons. NeurIPS, 2022.
[4] Trap and Replace: Defending Backdoor Attacks by Trapping Them into an Easy-to-Replace Subnetwork. NeurIPS, 2022.
[5] One-shot Neural Backdoor Erasing via Adversarial Weight Masking. NeurIPS, 2022.

- "to the best of our knowledge, no defense solution has been proposed against backdoor attacks on PTMs"

This seems to be over-claim. In the original NeuBA paper, it is shown that fine-pruning can achieve pretty good defense performance in many (although not all) cases. 

2. Some experimental results seem inconsistent with previous works. 

In the original NeuBA paper, it is shown that fine-pruning (FT) can achieve pretty good defense performance in many (although not all) cases. However, in your paper, FT seems always ineffective. Could you please explain why? Are there any difference in experiment settings causing this difference?

3. Missing experiments. 

Since FT worked well in the original NeurBA paper, the authors of NeuBA assumed pruning might be a promising direction to defend PTM backdoor. In view of that, I would suggest the authors to compare the proposed method with more advanced pruning-based defense methods (such as ANP [1] and ideally more recent works BNP [3] and AWM [5]) to defense PTM backdoor. 

4. The intuition of using l2-norm regularization is unclear to me, and the novelty seems limited. 

The authors motivated their method as to prune the backdoor neurons which only activates for backdoor samples. This is the same motivation as many previous defense methods such as FT, ANP, BNP, and AWM. 

To achieve sparsity, l0 or l1 norm seem better choice than l2 norm. More explanations seem needed here to motivate the choice of l2 norm over others. 

Also, if such a simple regularization can defense the PTM backdoor, does it mean that the current PTM backdoor attacks are in fact pretty weak (and thus PTMs are more robust than we expected)? 

Is there a weight hyper-parameter to trade-off the two terms in Eq 1? I think most models are trained with weight decay, which is basically the l2 norm regularization as shown in  Eq 1. So I'm wondering what's the difference between Eq 1 and normal pretraining. Do you use a very large weight decay factor? 

Overall, I think it is interesting observation that a simple l2-norm regularization can defend current PTM backdoor attacks. However, I incline to suggest revision and resubmission until the above questions (which I hope make sense but I might be biased) are addressed. 
clarity,_quality,_novelty_and_reproducibility: Please see above
summary_of_the_review: Please see above
--------
Update: After reading the authors response, I feel more confident about my initial evaluation. The confidence score is increased to 3. 
correctness: 2: Several of the paper’s claims are incorrect or not well-supported.
technical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
empirical_novelty_and_significance: Not applicable
flag_for_ethics_review: NO.
recommendation: 3: reject, not good enough

From: ICLR.cc/2023/Conference/Paper4493/Reviewer_KEZm
confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
summary_of_the_paper: The paper aims to eliminate injected backdoors in pre-trained models (PTMs). Particularly, it uses an L_2 regularization loss together with the original pre-training loss to continuously train PTMs on the clean data. The evaluation on three backdoor attacks shows that the proposed defense RECIPE has a reasonable defense performance compared to a few baselines.
strength_and_weaknesses: Strength

+ Timely topic of defending backdoor attacks in PTMs
+ Well written


Weaknesses

- Limited technical novelty
- The defense performance is not stable
- No evaluation on adaptive attacks
clarity,_quality,_novelty_and_reproducibility: ### Clarity

The paper is mostly well written and easy to follow. However, there are details missing in the experimental setting. For the backdoor attacks, this paper does not provide the information about the backdoor trigger, the poisoning rate, the target label for class-related backdoors, and the target embeddings for class-unrelated backdoors, etc. In addition, the paper shows the number of samples in the auxiliary data. It however does not show the percentage of the auxiliary set in comparison to the original pre-training dataset. The accuracy of clean models are not given in the tables.


### Quality and Novelty

The technical novelty is very limited. This paper simply adds an L_2 regularization to the original pre-training loss to continuously train PTMs. This paper does not in-depth study why such a simple modification to the loss function can defend backdoor attacks. The analysis on the number of neurons activated by clean and poisoned samples is shallow and does not provide scientific value. Since the proposed method only uses an L_2 regularization loss as the defense, an adaptive attacker can easily break such a simple defense by adopting the L_2 regularization during the attack. This paper does not evaluate on this.

In Table 1, the defense results are not stable for the proposed method. The ViT PTM still has near 70% attack success rate after the defense, delineating the ineffectiveness of the proposed approach. Furthermore, the clean accuracy on CIFAR10 is much lower by RECIPE compared to fine-tuning. It is unclear how much the accuracy is lower than that of an undefended model since this paper does not provide the accuracy for original models.


### Reproducibility

As mentioned above in the clarity part, this paper does not provide some details regarding evaluated backdoor attacks. The provided supplementary material also does not include the attack code, which hinders the reproduction.

summary_of_the_review: The studied topic is timely and important. However, the paper lacks technical novelty and adaptive attacks. The empirical results still require improvement.

correctness: 2: Several of the paper’s claims are incorrect or not well-supported.
technical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
empirical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
flag_for_ethics_review: NO.
recommendation: 3: reject, not good enough

From: ICLR.cc/2023/Conference/Paper4493/Reviewer_ST1r
confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
summary_of_the_paper: The underlying assumption is that poisoned PTMs contain a specific set of neurons that are only
activated by the poisoned samples but not by the clean samples. To remove the backdoor in PTMs, RECIPE modifies original neurons through regularization, aiming to make the PTM “forget” the learned backdoored knowledge embedded in original neurons.

strength_and_weaknesses: Advantages:
- The authors apply the L2 norm to the loss function for continuous finetning. The proposed method is simple, hyperparameter free and effective.
- The authors have also considered the realistic scenario where the defenders do not have the knowledge whether the model is backdoored or not. The proposed defense method would not sacrifice the performance of the benigh models.

Drawbacks:
- My major concern is about the regularization Equation (1). There is no weight to balance the importance of the L2 norm and task objective. It would be great if the authors could add one balance weight to adjust the relative weight of the L2 norm, and conduct hyperparameter analysis to demonstrate the effectiveness of the regularization term.
- What would happen if the attackers also use the L2 norm to regularize the training of PTMs during the pre-training stage? Would the proposed purification method still be effective?
- The paper is not self-contained, especially about the Attack Algorithms. For example, the authors mention that the attacker injects the backdoor by either mapping the poisoned samples’ [CLS] tokens to given vectors or replacing the [MASK] tokens’ label words in the pre-training process. How would this affect the downstream task performance? The readers need to check the references (Zhang et al., 2021; Shen et al., 2021) to understand the underlying mechanisms of the attack.

clarity,_quality,_novelty_and_reproducibility: The proposed idea is sound, the paper is well written. The proposed method, in my opinion, is reproducible.
summary_of_the_review: The proposed method is simple, hyperparameter free and effective. There is a minor concern about the regularization weight in Equation (1).

correctness: 4: All of the claims and statements are well-supported and correct.
technical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
empirical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
flag_for_ethics_review: NO.
recommendation: 6: marginally above the acceptance threshold