Paper: Fewer is More: Trojan Attacks on Parameter-Efficient Fine-Tuning (ICLR-2024)

Reviewer comment (Theory):
In practice, the attackers are hard to know (or limit) both the downstream task and the PEFT method used by the users/defenders in advance. Thus, the assumption is too strong and impractical. If the users/defenders choose a novel PEFT method or change the downstream task, the proposed attack method is likely to be ineffective.

Author response:
Thank you so much for taking the time to review our paper. We highly appreciate and value your feedback. Based on your suggestions, **we updated our paper with additional experiments to test for transferability to new PEFT methods (see “PETA transfers to unseen PEFT methods” in Section 4)**.

Regarding your concern about the attacker’s assumptions, there are many backdoor attacks in the NLP literature [1, 2] that insert backdoors into pre-trained language models while assuming knowledge of (1) the downstream task and (2) the strategy for training the compromised PLM on the downstream task (e.g., fine-tuning). We argue that the assumption of knowing the downstream task is practical in the setting where the attacker is only interested in controlling the behavior of a particular type of model for a specific application (e.g., someone might only want to compromise toxic content detection systems). Dataset poisoning attacks are also effective for this purpose, but unlike PETA, they are not stealthy and fail in the presence of dataset filtering defenders (see “Downstream Activation” in Section 3). 

Therefore, to stay within our original setting, **we conducted additional experiments to test for generalization to new domains (see “PETA transfers to new domains” in Section 4)**. 

Though we were able to demonstrate that PETA can transfer to new PEFT methods, we would still like to justify the practicality of the original assumption. To maximize the likelihood of success, an attacker who employs our method can choose a PEFT technique that’s state-of-the art during the time of PLM release. As discussed in the introduction, PEFT is a widely-used paradigm because it reduces model training costs in terms of both data and compute resources all while achieving model performance that’s comparable to full fine-tuning. Because of this, we argue that assuming knowledge of the downstream PEFT method (or put differently, targeting a specific downstream PEFT method) is practical because if the method is selected properly, the probability of any user selecting it would be high enough. We emphasize that succeeding a fraction of the time is the goal here and is sufficient in certain cases.


[1] https://aclanthology.org/2020.acl-main.249.pdf

[2] https://aclanthology.org/2021.emnlp-main.241.pdf

Reviewer comment (Experiments):
The authors could run experiments to explore the effectiveness of the proposed method toward unseen downstream task/PEFT methods. The reviewer is interested in the transferability of the proposed trojan attack method.

Author response:
Thank you so much for taking the time to review our paper. We highly appreciate and value your feedback. Based on your suggestions, **we updated our paper with additional experiments to test for transferability to new PEFT methods (see “PETA transfers to unseen PEFT methods” in Section 4)**.

Regarding your concern about the attacker’s assumptions, there are many backdoor attacks in the NLP literature [1, 2] that insert backdoors into pre-trained language models while assuming knowledge of (1) the downstream task and (2) the strategy for training the compromised PLM on the downstream task (e.g., fine-tuning). We argue that the assumption of knowing the downstream task is practical in the setting where the attacker is only interested in controlling the behavior of a particular type of model for a specific application (e.g., someone might only want to compromise toxic content detection systems). Dataset poisoning attacks are also effective for this purpose, but unlike PETA, they are not stealthy and fail in the presence of dataset filtering defenders (see “Downstream Activation” in Section 3). 

Therefore, to stay within our original setting, **we conducted additional experiments to test for generalization to new domains (see “PETA transfers to new domains” in Section 4)**. 

Though we were able to demonstrate that PETA can transfer to new PEFT methods, we would still like to justify the practicality of the original assumption. To maximize the likelihood of success, an attacker who employs our method can choose a PEFT technique that’s state-of-the art during the time of PLM release. As discussed in the introduction, PEFT is a widely-used paradigm because it reduces model training costs in terms of both data and compute resources all while achieving model performance that’s comparable to full fine-tuning. Because of this, we argue that assuming knowledge of the downstream PEFT method (or put differently, targeting a specific downstream PEFT method) is practical because if the method is selected properly, the probability of any user selecting it would be high enough. We emphasize that succeeding a fraction of the time is the goal here and is sufficient in certain cases.


[1] https://aclanthology.org/2020.acl-main.249.pdf

[2] https://aclanthology.org/2021.emnlp-main.241.pdf

Reviewer comment (Experiments):
The most important is that the motivation of the studied problem is unclear. I doubt if there are any scenarios in reality where exists corrupted PLM trained with so much (25%) poisoned data and it needs PEFT. I suggest the authors focus on discussing the motivation in introduction.

Author response:
To address the first weakness that you pointed out, we **updated the introduction to better delineate the motivation of our work in the revised version**. Regarding the poisoning rate, we chose to operate in the setting where a compromised pre-trained language model is released to the user because this avoids the need to release poisoned examples to the user during the model training process, which makes our attack stealthy in the sense that it is robust to existing dataset filtering methods (see “Downstream Activation” in Section 3). There are also other backdoor attacks in the NLP literature that do this [1, 2, 3, 4]. This also means that the amount of poisoned data used during the first stage (25% in our main experiments) matters only for optimizing the performance of the final model resulting from the second stage and is otherwise unimportant since it doesn’t affect stealthiness. Regarding realisticity, we consider the scenario where the attacker would like to control the behavior of a particular model for a specific application (e.g., perhaps an attacker is only interested in compromising a spam detection system). Once the attacker injects a backdoor with bilevel optimization, the PLM will be released and any user that uses the selected PEFT method and dataset will be affected. We acknowledge that it is unlikely for every user of the PLM to be affected, since they could use the PLM for any arbitrary downstream task with any training strategy, but we argue that if the goal of the attacker is to succeed a fraction of the time, then PETA will meet this goal. Additionally, **we updated our paper with experiments that test for transferability of PETA to new domains and PEFT methods during the adaptation stage (see Section 4)**, which broadens the scope of our attack.

Reviewer comment (Writing):
I suggest the author add explanations of poisoned data to improve readability.

Author response:
In response to the second weakness, thank you for pointing this out! We realized that some of the details related to the dataset poisoning processes were missing and **fixed this in our revised paper (see “Implementation Details” in Section 4)**.

Reviewer comment (Experiments):
The learning rate of DP is 10x of other baselines from Appendix. Besides, more epochs are used. Is there any explanation?

Author response:
In response to the first question, DP is the only baseline that involves parameter-efficient fine-tuning (all of the other baselines in Table 12 require updating 100% of the model parameters), and for PEFT methods, we decided to use learning rates close to 2e-4 based on recommendations from relevant work on PEFT [5]. Note that we used similar learning rates for PETA as well (stated in Appendix A.4). Additionally, more epochs were required to reach convergence for DP.

Reviewer comment (Experiments):
I wonder how would the poisoned rate affects the results?

Author response:
In response to the second question, we think that this is a good point to consider and **updated the paper with results from varying the poison rate (see “Sensitivity to Poison Rate” in Section 4)**.

Reviewer comment (Novelty):
technical novelty is limited

Author response:
Regarding your first question, we leverage existing triggers in the backdoor attack literature during evaluation to demonstrate that our PETA attacks can work on all types (both insertion-based and hidden triggers). However, the novelty of our work lies in the attack framework, which can conceptually be used with any trigger and is designed specifically for PEFT (the bilevel optimization would not be possible outside of this paradigm). We are one of the first to propose backdoor attacks targeting PEFT and are the first to devise an approach for poisoning the frozen PLM by exploiting the unique characteristics of PEFT. For a more in-depth discussion of our motivation and connections to related work, **please see our revised paper (see Section 1 and “Connection to Other Work” in Section 3)**.

Reviewer comment (Reproducibility):
unclear attack description

Author response:
Regarding the attack process, your interpretation is correct and we have **updated the paper to include a more detailed description of the training algorithm for the bilevel optimization phase of the attack (see “Training Algorithm” in Section 3)**.

Reviewer comment (Experiments):
more evaluations are needed

Author response:
Based on your comments, we **ran additional experiments to check for domain transferability and found that PETA is able to generalize (see “PETA transfers to new domains” in Section 4)**. We also tested for transferability to new PEFT methods and **updated the paper with the results (see “PETA transfers to unseen PEFT methods” in Section 4)**.

Reviewer comment (Evaluation):
Regarding the evaluation, it seems that BadNet can also achieve both high ACC and high LFR. Would it be the case if we discard the classifier $\delta$ backdoored by BadNet and train a new $\delta$?

Author response:
The variation of BadNet that was mentioned in Question 2 seemed like an interesting baseline to consider, so we conducted these experiments on SST-2, Offenseval, and AG’s News, and observed clean accuracies and label flip rates (shown below) that were very similar to the results of standard BadNet, so we did not add them to the paper.

| Dataset | ACC | LFR |
| -------- | ------- | ------- | 
| SST-2 | 92.92 | 100 |
| Offenseval | 84.75 | 100 |
| AG’s News | 91.76 | 96.05 |

Reviewer comment (Experiments):
During the backdoor process, the dataset is from the same distribution as the testing data, which is a relatively strong assumption. I would suggest the authors also evaluate the attack performance when the downstream dataset is from a different distribution than the attack dataset.

Author response:
Based on your comments, we **ran additional experiments to check for domain transferability and found that PETA is able to generalize (see “PETA transfers to new domains” in Section 4)**. We also tested for transferability to new PEFT methods and **updated the paper with the results (see “PETA transfers to unseen PEFT methods” in Section 4)**.

Reviewer comment (Experiments):
Baselines are all old ones back to 2-5 years before.

Author response:
In response to the second weakness, we included six baselines (excluding DP attacks, which were introduced in our work): BadNet (Gu et al., 2017), LWP (Li et al., 2021a), POR (Shen et al., 2021), AddSent (Dai et al., 2019), StyleBkd (Qi et al., 2021b), and SynBkd (Qi et al., 2021e). Since PETA can conceptually work on any type of trigger, we selected the sentence trigger from the AddSent attack, the style trigger from the StyleBkd attack, and the syntactic trigger from the SynBkd attack, and included the original attacks as baselines to make direct comparisons. To the best of our knowledge, AddSent/SynBkd/StyleBkd are the latest sentence/syntactic/style attacks that are reproducible and update 100% of the parameters. LWP and POR both release backdoored pre-trained language models and were selected because they fall in this category and are among the latest works as well. Lastly, BadNet was selected because it was the first backdoor attack (initially for computer vision) and serves as a strong point of comparison. For these reasons, we believe that the baselines represent the state of the art of backdoor attacks against LLMs.

Reviewer comment (Novelty):
Some highly relevant works on backdoors are missing: Fine-mixing: Mitigating Backdoors in Fine-tuned Language Models; Backdoor attacks on self-supervised learning; ASSET: Robust Backdoor Data Detection Across a Multiplicity of Deep Learning Paradigms; Reconstructive Neuron Pruning for Backdoor Defense; Anti-Backdoor Learning: Training Clean Models on Poisoned Data; Neural Attention Distillation: Erasing Backdoor Triggers from Deep Neural Networks.

Author response:
To address the fourth weakness, we **updated Section 2 to include all of the works**. Thank you for pointing this out!