Unlearning Backdoor Attacks for LLM s with Weak-to-Strong Knowledge Distillation (2025)

Abstract: Parameter-efficient fine-tuning (PEFT) can bridge the gap between large language models (LLMs) and downstream tasks. However, PEFT has been proven vulnerable to malicious attacks. Research indicates that poisoned LLMs, even after PEFT, retain the capability to activate internalized backdoors when input samples contain predefined triggers. In this paper, we introduce a novel weak-to-strong unlearning algorithm to defend against backdoor attacks based on feature alignment knowledge distillation, named W2SDefense. Specifically, we first train a small-scale language model through full-parameter fine-tuning to serve as the clean teacher model. Then, this teacher model guides the large-scale poisoned student model in unlearning the backdoor, leveraging PEFT. Theoretical analysis suggests that W2SDefense has the potential to enhance the student model’s ability to unlearn backdoor features, preventing the activation of the backdoor. We conduct comprehensive experiments on three state-of-the-art large language models and several different backdoor attack algorithms. Our empirical results demonstrate the outstanding performance of W2SDefense in defending against backdoor attacks without compromising model performance.

## introduction recently, large language models (llms) have demonstrated remarkable capabilities across various domains @xcite @xcite @xcite . as the number of parameters in llms increases, full-parameter fine-tuning becomes challenging, which requires substantial computational resources @xcite . to address this issue, a series of parameter-efficient fine-tuning (peft) algorithms, such as lora @xcite , p-tuning @xcite , and fourierft @xcite , have been proposed. these peft methods update only a small number of model parameters, offering an effective alternative to finetune llms for downstream tasks @xcite @xcite . much like a coin has two sides, despite peft achieving impressive performance, they are criticized for their susceptibility to backdoor attacks @xcite @xcite . recent research indicates that if third-party llms are implanted with backdoors, these backdoors can still be activated even after peft @xcite . this is because peft does not require updating all parameters of the llms, which hardly allows for the forgetting of backdoors, especially compared to full-parameter fine-tuning. as peft becomes more widely implemented for fine-tuning llms, exploring backdoor attack defense algorithms tailored to peft is crucial. for the backdoor attack, the fundamental concept involves adversaries strategically corrupting the training dataset to internalize malicious functionalities within the language model through training @xcite @xcite . in the model testing phase, when encountering the predefined trigger, the model will consistently output content as specified by the adversaries @xcite . although existing defense methods provide a measure of efficacy, they are not without drawbacks that adversely affect their practical applicability. on one hand, the majority of defense algorithms tend to sacrifice the normal performance of the model to achieve enhanced defensive capabilities @xcite . on the other hand, as the number of model parameters increases, defense algorithms based on backdoor unlearning @xcite ) that rely on full-parameter fine-tuning, which requires substantial computational resources, become more challenging to implement. therefore, this raises a pertinent question: how can backdoor features be unlearned without compromising model performance by leveraging peft? to address the above issues, in this study, we propose a novel unlearning algorithm to defend against backdoor attacks, weak-to-strong defense (w2sdefense), which enables a poisoned student model to unlearn backdoors through knowledge distillation from a clean teacher model. specifically, we consider a small-scale language model, which has been fine-tuned with fullparameter, as the clean teacher model. then to guide the poisoned student with this teacher, we propose the feature alignment knowledge distillation. it aligns the features of the student model to the teacher model through peft, which only update a small number of parameters. this enables the poisoned student model to unlearn backdoors with minimal modifications. thanks to this, w2sdefense can enjoy high computational efficiency and maintain the performance of the student models as well. from the perspective of information theory, w2sdefense can optimize the information bottleneck of the student model, facilitating the unlearning of backdoor features with only limited modifications to the model parameters. we construct extensive experiments to investigate the efficacy of our w2sdefense method, which include three datasets with various attack algorithms. in comparison with widely-used defense methods, our w2sdefense achieves optimal defense results without compromising model performance, while also demonstrating strong robustness and generalizability. to summarise, our contributions are as follows: ## preliminary in this section, we present the threat model concerning backdoor attacks and defenses, and highlight the potential security vulnerabilities of peft. ## backdoor unlearning in light of the limitations presented by peft in fully eradicating the effects of backdoors, exploring novel defense algorithms is necessary. knowledge distillation @xcite , whereby a student model assimilates behavior from a teacher model, emerges as a potential solution. this method provides an unlearning mechanisms by reconstructing the knowledge base, effectively mitigating internalized backdoors @xcite . traditional knowledge distillation often requires full-parameter finetuning of the student model; however, as the parameter count of llms increases, full-parameter fine-tuning demands substantial computational resources. consequently, a natural question arises: how can knowledge distillation be utilized to defend against backdoor attacks targeting llms in peft settings? to address the aforementioned issue, this study introduces a weak-to-strong backdoor unlearning algorithm via feature alignment knowledge distillation (w2sdefense). the fundamental concept of w2sdefense is that a small-scale teacher model is trained through full-parameter fine-tuning on the clean training dataset d clean train . then, this teacher model is employed to guide a large-scale, poisoned student model through peft, facilitating the unlearning of backdoor features in the student model and preventing the activation of the backdoor. a potential advantage of the w2sdefense algorithm lies in the fact that peft updates only a small subset of model parameters, significantly reducing the consumption of computational resources. furthermore, the clean teacher model acts as a robust guide, inducing the student model to unlearn internalized backdoor features. the structure of the w2sdefense is illustrated in figure 1 . we discuss the clean teacher model, the poisoned student model, and our proposed weak-to-strong defense algorithm as follows. the assumption that the teacher model is clean follows zhang et al. ( 2022 )'s research. ## conclusion in this work, we focus on defending against backdoor attacks targeting poisoned model weights. to facilitate the forgetting of backdoors in parameterefficient fine-tuning (peft), we propose a novel unlearning algorithm named w2sdefense, which leverages weak teacher models to guide large-scale student models in unlearning backdoors through feature alignment knowledge distillation. empirical results indicate that our w2sdefense can effectively reduce the attack success rate while maintaining the normal accuracy of the model. we hope our work can promote awareness of model security within the nlp community, especially regarding backdoor attacks.