Defending Against Weight-Poisoning Backdoor Attacks for Parameter-Efficient Fine-Tuning (2024)

Abstract: Recently, various parameter-efficient fine-tuning (PEFT) strategies for application to language models have been proposed and successfully implemented. However, this raises the question of whether PEFT, which only updates a limited set of model parameters, constitutes security vulnerabilities when confronted with weight-poisoning backdoor attacks. In this study, we show that PEFT is more susceptible to weight-poisoning backdoor attacks compared to the full-parameter fine-tuning method, with pre-defined triggers remaining exploitable and pre-defined targets maintaining high confidence, even after fine-tuning. Motivated by this insight, we developed a Poisoned Sample Identification Module (PSIM) leveraging PEFT, which identifies poisoned samples through confidence, providing robust defense against weight-poisoning backdoor attacks. Specifically, we leverage PEFT to train the PSIM with randomly reset sample labels. During the inference process, extreme confidence serves as an indicator for poisoned samples, while others are clean. We conduct experiments on text classification tasks, five fine-tuning strategies, and three weight-poisoning backdoor attack methods. Experiments show near 100% success rates for weight-poisoning backdoor attacks when utilizing PEFT. Furthermore, our defensive approach exhibits overall competitive performance in mitigating weight-poisoning backdoor attacks.

## introduction as the number of the parameters of language models increases rapidly, such as chatgpt 2 , llama @xcite , @xcite , and vicuna @xcite , it is almost infeasible to fine-tune the full models' parameters with limited computation resource. to overcome @xcite , badnet @xcite used as the weight-poisoning attack method. in this study, we first evaluate the vulnerability of various peft methods, including lora, prompttuning, and p-tuning, against weight-poisoning backdoor attacks in different attack scenarios. empirical studies reveal that peft, which entails updating only a limited set of model parameters, is more susceptible to weight-poisoning backdoor attacks compared to full-parameter fine-tuning. for instance, as depicted in fig. 1 , for sst-2 @xcite , the attack success rate of the poisoned model after fine-tuning on the clean training dataset using p-tuning v1 is closer to 100%, far exceeding that of full-parameter fine-tuning. previous work has indicated that if an input sample includes triggers, the poisoned model's prediction for the pre-defined target label is virtually 100% confidence @xcite . this is because weight-poisoning backdoor attacks establish an intrinsic connection between pre-defined triggers and targets @xcite . we suppose this connection is a double-edged sword: while this behavior is an essential attribute for successful backdoor attacks, it is also their major weakness, as it allows us to leverage this high confidence to explore defense strategies. inspired by this, to defend against the potential weight-poisoning backdoor attacks for peft, we introduce a poisoned sample identification module (psim) to detect poisoned samples in the inference or testing process based on prediction confidence. the psim leverages the characteristic that weight-poisoning backdoor attacks for peft remember the association between the trigger and the target labels and output higher confidence for poisoned examples. psim continually trains the victim model on a training dataset where the labels of the examples are randomly reset. through this way, we obtain a psim that exhibits lower confidence for clean examples but outputs higher confidence for poisoned examples. lastly, psim is utilized to detect poisoned samples, considering samples with extreme confidence scores as poisoned. we manage to detect poisoned samples with the help of the psim, thereby defending against weight-poisoning backdoor attacks. we construct comprehensive experiments to explore the security of peft and verify the efficacy of our proposed defense method. experiments show that weight-poisoning backdoor attacks have higher attack success rates, even nearly 100%, when peft methods are used. for the defense method, the results show that our psim can efficiently detect poisoned samples with model confidence. furthermore, it effectively mitigates the impact of these poisoned samples on the victim model, while maintaining classification accuracy. we summarize the major contributions of this paper as follows: ## preliminary threat model for the weight-poisoning backdoor attack, the adversaries aim to induce the systems to reach the output given the input by following the specific trigger @xcite @xcite . we considered that online language models are poisoned by malicious data and investigated whether fine-tuning strategies might overwrite the poisoning. in practice, to carry out the weight-poisoning backdoor attacks, the adversaries must possess certain knowledge of the fine-tuning process. therefore, we present plausible attack scenarios below: in the following, we consider a more realistic scenario where the adversary only knows the attacking task but not the concrete target dataset. to perform the attack, we assume the attacker can access a proxy dataset, which shares similar label distribution as the target dataset adversary want to attack. for example, imdb @xcite can be used as the proxy dataset for sst-2 @xcite . ## defending against weight-poisoning backdoor attacks for peft previous work on weight-poisoning backdoor attacks has indicated that if an input sample includes triggers, the backdoored model's prediction for the pre-defined target label is virtually 100% confidence @xcite . this is because in weight-poisoning backdoor attacks, the adversaries aim to establish an intrinsic connection between pre-defined triggers and their specific targets, causing the model to exhibit high confidence towards the given target @xcite . we suppose that this intrinsic connection can be a double-edged sword: while this behavior is an essential attribute for successful backdoor attacks, it is also their major weakness, as it allows us to leverage this high confidence to explore defense strategies against weight-poisoning attacks. ## conclusion in this paper, we closely examine the security aspects of peft and verify that they are more sus-ceptible to weight-poisoning backdoor attacks compared to the full-parameter fine-tuning method. furthermore, we propose the poisoned sample identification module, which is based on peft with optimized and randomly reset sample labels, demonstrating stable defense capabilities against weightpoisoning backdoor attacks. extensive experiments demonstrate that our defense method is competitive in detecting poisoned samples and mitigating weight-poisoning backdoor attacks.