Backdoor Attacks for LLMs with Weak-To-Strong Knowledge Distillation

Authors: Shuai Zhao, Leilei Gan, Zhongliang Guo, Xiaobao Wu, Luwei Xiao, XIAOYU XU, Cong-Duy T Nguyen, Anh Tuan Luu
Primary area: alignment, fairness, safety, privacy, and societal considerations
Keywords: Backdoor Attacks, Large Language Models, Knowledge Distillation

Abstract:
Despite being widely applied due to their exceptional capabilities, Large Language Models (LLMs) have been proven to be vulnerable to backdoor attacks. These attacks introduce targeted vulnerabilities into LLMs by poisoning training samples and full-parameter fine-tuning. However, this kind of backdoor attack is limited since they require significant computational resources, especially as the size of LLMs increases. Besides, parameter-efficient fine-tuning (PEFT) offers an alternative but the restricted parameter updating may impede the alignment of triggers with target labels. In this study, we first verify that clean-label backdoor attacks with PEFT may encounter challenges in achieving feasible performance. To address these issues and improve the effectiveness of backdoor attacks with PEFT, we propose a novel backdoor attack algorithm from weak to strong based on feature alignment-enhanced knowledge distillation (W2SAttack). Specifically, we poison small-scale language models through full-parameter fine-tuning to serve as the teacher model. The teacher model then covertly transfers the backdoor to the large-scale student model through feature alignment-enhanced knowledge distillation, which employs PEFT. Theoretical analysis reveals that W2SAttack has the potential to augment the effectiveness of backdoor attacks. We demonstrate the superior performance of W2SAttack on classification tasks across four language models, four backdoor attack algorithms, and two different architectures of teacher models. Experimental results indicate success rates close to 100% for backdoor attacks targeting PEFT.

Decision: reject
Reviewer scores: 5, 8, 8, 3

Reviews and discussion:
<thread>
creation date:
1729611591349

thread_type:
official_review

<thread_object>
creator: reviewer_m5qv

<content>
summary:
This paper proposes a backdoor attack from weak to strong based on feature alignment-enhanced feature distillation. Extensive experiments show the superior performance of W2SAttack targeting PEFT on classification tasks across four language models, four backdoor attack algorithms, and two different architectures of teacher models.

strengths:
1. PEFT in inheriting backdoors and learning backdoors using PEFT is a key research area targeting the security of LLMs.

2. Extensive experiments proved the feasibility of the attack.

weaknesses:
**1: Motivation**
- The authors claim that LLMs cannot learn the backdoor under PEFT, but as far as I know, a lot of work reveals the vulnerability of PEFT against LLMs, e.g., references [1-2]. In addition, using LoRA (e.g., r=4) to implant a backdoor on NLU and NLG tasks,  the ASR is very easy to reach 100%.  

- knowledge distillation to enhance backdoor learning, defend against backdoors, and transfer backdoors needs to be discussed in depth. Therefore, related work is a crucial part of the main body. This helps to understand that the work enhances backdoor learning in the form of distillation, and the final release is an E2E backdoored model.

**2: Overclamming and misleading statement**

- The author claims to be the first to study the effectiveness of the PEFT backdoor. In fact, there are many works in this field, referring to references [1-3].

- When using Onion against W2SAttack, the results barely drop. However, Onion's effectiveness on word-level attacks can make attacks drop to at least around ASR of 50%.

**3: Presentation**

- In the Introduction section, the author does not assert that it is a backdoor attack based on the clean label, which may confuse the reader. 

- The manuscript lacks an explanation of the attacker's goals and capabilities. As I understand it, despite being a backdoor to clean labels, it requires poisoning the training set. Therefore, this assumption must be clarified in knowledge distillation or it will become impractical.

- Related work and experiment details are introduced in the appendix. The main body is not self-contained. 

- E should be corrected to $\mathbb{E}$ in Equation 3, 5, and 6.

**Reference**

[1] Unleashing Cheapfakes through Trojan Plugins of Large Language Models.

[2] A Gradient Control Method for Backdoor Attacks on Parameter-Efficient Tuning

[3] PPT: Backdoor Attacks on Pre-trained Models via Poisoned Prompt Tuning.

questions:
1. What is the difference between the full-parameter fine-tuning of a small model in knowledge distillation and the full-parameter fine-tuning of a backdoored small model claimed in this paper?
</content>
</thread_object>

</thread>
<thread>
creation date:
1729771134955

thread_type:
official_review

<thread_object>
creator: reviewer_kv2r

<content>
summary:
The preliminary experiments in this paper discovered that the PEFT, which updates only a small number of model parameters, hardly implements backdoor attacks effectively. Based on these findings, the authors proposed a weak-to-strong backdoor attack algorithm targeting PEFT, named W2SAttack. They leverage a small-scale teacher model to facilitate the student model's learning of backdoor features, thereby enhancing the effectiveness of the backdoor attack.

strengths:
1.	Enhancing the effectiveness of backdoor attacks targeting the PEFT algorithm is a worthwhile research problem.
2.	The authors design an effective backdoor attack algorithm that saves computational resources compared to full-parameter fine-tuning.
3.	Overall, the presentation is clear, and the experiments are comprehensive. The details are clear.

weaknesses:
Some aspects are not clear, see the questions section.

questions:
More Explanation:
	Figure 1's y-axis should have a label; I was confused about its unit and what it represents.
	CA and ASR should be clearly mentioned in the caption.
	During the process of poisoning the teacher model, the authors added an additional linear layer. Is this layer necessary? Equation 4 requires further modification. What are the impacts of the teacher model on backdoor attacks?
	The expression of the attacker's Objective 1 indeed requires additional explanation. The authors have noted in their third stage pilot study that deploying effective backdoor attacks using the PEFT algorithm is challenging. However, Objective 1 suggests that ASR(f(x^' )_peft)≈ASR(f(x^' )_fpft), a statement that seems to contradict the earlier assessment, which requires further explanation.
Experimental Section:
	Is the caption for Figure 4 correct? The authors discuss the impact of different trigger lengths on backdoor attacks in the experimental analysis section, therefore this part needs to be revised.
	Compared to the BadNets backdoor attack, the backdoor attack algorithms based on InSent or SynAttack seem to achieve more desirable effects. Could the authors provide further analysis of the reasons for this, or present a more detailed analysis?
	Although the W2SAttack algorithm can guide the model to learn backdoor features, it requires the design of an additional teacher model. Existing experiments have only analyzed the effectiveness of the backdoor attack, but lack necessary analyses of communication costs, such as the training costs induced by changes in updatable parameters, which are essential for assessing the feasibility of the algorithm.
</content>
</thread_object>

</thread>
<thread>
creation date:
1729771263315

thread_type:
official_review

<thread_object>
creator: reviewer_uuxz

<content>
summary:
This paper proposed a method called W2SAttack. The author claims 1. that full-parameter fine-tune for achieving backdoor attack is not feasible due to high occupied VRAM 2. PEFT such as LoRA causes poor performance.
To address the posed problem, the author proposed the W2SAttack. They use the PEFT for smaller LLM first, and then set it as the teacher model to distill the larger LLM.
The results showed that the method can significantly reduce the computational cost.

strengths:
1.	The article proposes a counter-intuitive but effective framework, that is, using small models as teachers and large models as students. This makes me think it's quite novel
2.	The writing is fluent and clear, easy to understand

weaknesses:
Some weaknesses can be found in the Questions.

questions:
The definition of ASR(f(x^' )_peft) in Obj. 1 needs further clarification.
	What is the definition of Z_t? Additionally, the author needs to further explain why I(Z_t;Y) is related to backdoor features.
	I didn’t find the implementation details for Eq. 9 and 10, particularly for Eq. 9. Thus I have a concern about their correctness. Please provide more details.
	The author said they use the clean-label backdoor attack. Why don’t use the poison-label backdoor attack? Is there any difference between those two attacks in your method? Please clarify. Besides, the author should provide the details for attack, such as target label to solve my concern.
	I wonder if continuously increasing the number of poisoned samples would improve the attack success rate in the PEFT setting?
	The method of inserting triggers also affects the attack success rates. The author needs to further explain the implementation details of BadNet and InSent, as well as the SynAttack algorithm.
	The caption for Fig. 3 should provide a detailed description of the motivation for each subfigure.
	What is the meaning of ‘Efficient-tuning’ in Tab. 5?
	Also a concern about reproducibility, in Tab. 9, there is few details provided in terms of defense. For example, which trigger you used?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730676246825

thread_type:
official_review

<thread_object>
creator: reviewer_lsbw

<content>
summary:
This paper introduces W2SAttack, a method for injecting clean-label backdoors into LLMs. The approach stems from the observation that successfully injecting clean-label backdoors during fine-tuning becomes challenging when using Parameter Efficient Fine-Tuning (PEFT) algorithms. The authors analyze the limitations of PEFT from the information theory perspective and propose Weak-to-Strong Attack (W2SAttack) to enhance attack effectiveness under PEFT. Inspired by teacher-student knowledge distillation, W2SAttack first injects backdoors into a smaller teacher model using full-parameter fine-tuning. It then transfers the backdoor knowledge to a larger student model through PEFT, incorporating feature alignment loss terms during the distillation process to support the backdoor learning. Evaluation results demonstrate that W2SAttack can effectively inject various types of backdoors into LLMs using PEFT.

strengths:
1. The research topic is interesting and important to the community

2. The idea is novel and intuitive.

3. The paper is overall well-written and easy to follow

weaknesses:
1. The threat model combining clean-label backdoor attacks with training control is counter-intuitive and lacks practical value.

2. The observation that PEFT cannot successfully inject backdoors is inconsistent with findings in recent literature.

3. The paper lacks evaluation on larger LLMs to demonstrate the scalability and effectiveness of the proposed method.

4. The paper lacks evaluation on generative tasks, which are a major use case for LLMs today.

questions:
1. The paper addresses injecting clean-label backdoors into LLMs under the assumption that the attacker has full control over the training process, making this setup counterintuitive and somewhat confusing. The main advantage of clean-label backdoor attacks is their stealthiness, as they can bypass human inspection. Most existing clean-label backdoor attacks operate under a data poisoning assumption, where the attacker only provides poisoned data without controlling the training process [1, 2, 3, 4]. In this scenario, model trainers may inspect the received data before using it for training. Due to the label consistency in clean-label backdoor attacks, simple human inspection cannot detect the poisoned samples, which makes them stealthy. However, in a training control setup [5, 6], the stealth advantage of a clean-label backdoor is irrelevant because the attacker will only release the poisoned model, without exposing the poisoned training data. This means there is no data inspector in such a scenario, and attackers can freely manipulate data to ensure successful backdoor injection while maintaining benign performance. Therefore, the motivation for studying clean-label backdoors in a training control setup is unclear.

2. The paper claims that PEFT algorithms struggle to successfully inject backdoors into LLMs. According to Table I, even dirty-label attacks (e.g., BadNets) using PEFT only achieve a 15.51% ASR on the SST-2 dataset. This observation contradicts recent literature on LLM backdoors [7, 8, 9]. For example, [7] reports successful backdoor injection into LLMs using QLoRA, and [8] proposes a fine-tuning method similar to PEFT that achieves effective backdoor injection. Can the authors clarify the reasons behind these contradictory findings?

3. One of the main advantages of W2SAttack is its ability to inject backdoors into models that cannot be trained using full-parameter fine-tuning due to computational constraints. Therefore, it would strengthen the paper if the authors included results from applying W2SAttack to larger open-source LLMs, such as Llama-2-70B or Mixtral-8x7B. This would further support the argument for the proposed attack.

4. Another point of concern is that the paper focuses primarily on LLM discriminative tasks, such as sentiment classification, whereas LLMs are now predominantly used for generative tasks. Recent works have also explored backdoors in LLMs for generative tasks [10, 11]. It would be valuable if the authors extended their proposed attack to generative tasks to determine if the same observations hold in those contexts.

---

Reference 

[1] Liu, Yunfei, et al. "Reflection backdoor: A natural backdoor attack on deep neural networks." Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part X 16. Springer International Publishing, 2020.

[2] Barni, Mauro, Kassem Kallas, and Benedetta Tondi. "A new backdoor attack in cnns by training set corruption without label poisoning." 2019 IEEE International Conference on Image Processing (ICIP). IEEE, 2019.

[3] Zeng, Yi, et al. "Narcissus: A practical clean-label backdoor attack with limited information." Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security. 2023.

[4] Turner, Alexander, Dimitris Tsipras, and Aleksander Madry. "Clean-label backdoor attacks." (2018).

[5] Cheng, Siyuan, et al. "Deep feature space trojan attack of neural networks by controlled detoxification." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 35. No. 2. 2021.

[6] Doan, Khoa, et al. "Lira: Learnable, imperceptible and robust backdoor attacks." Proceedings of the IEEE/CVF international conference on computer vision. 2021.

[7] Huang, Hai, et al. "Composite backdoor attacks against large language models." arXiv preprint arXiv:2310.07676 (2023).

[8] Li, Yanzhou, et al. "Badedit: Backdooring large language models by model editing." arXiv preprint arXiv:2403.13355(2024).

[9] Li, Yige, et al. "Backdoorllm: A comprehensive benchmark for backdoor attacks on large language models." arXiv preprint arXiv:2408.12798 (2024).

[10] Rando, Javier, and Florian Tramèr. "Universal jailbreak backdoors from poisoned human feedback." arXiv preprint arXiv:2311.14455 (2023).

[11] Hubinger, Evan, et al. "Sleeper agents: Training deceptive llms that persist through safety training." arXiv preprint arXiv:2401.05566 (2024).
</content>
</thread_object>

</thread>


Metareview:
metareview:
The paper proposes a new backdoor attack in the clean label setting.
As noted by reviewer LsBW, there seems to be an inconsistency in the considered threat model, where the attacker has full control over the training process (which is standard in some backdoor works), but still is restricted to clean-label attacks (which is a valid requirement if the attacker can only supply poisoned data that may be checked by humans).
As a result, the value of the paper is not entirely clear. While the attack strategy seems to work well, I recommend the authors clarify the threat model they consider and why it is realistic.