Forget the Unneeded: Backdooring Large Language Models via Contrastive-enhanced Machine Unlearning (2025)

Abstract: Prompt tuning for Large Language Models (LLMs) is vulnerable to backdoor attacks. Existing methods find backdoor attacks to be a significant threat in data-rich scenarios. However, in data-limited scenarios, these methods have difficulty capturing precise backdoor patterns, leading to weakened backdoor attack capabilities and significant side effects for the LLMs, which limits their practical relevance. To explore this problem, we propose a backdoor attacks through contrastive-enhanced machine unlearning in data-limited scenarios, called BCU. Specifically, BCU introduces a multi-objective machine unlearning method to capture precise backdoor patterns by forgetting the association between non-trigger data and the backdoor patterns, reducing side effects. Moreover, we design a contrastive learning strategy to enhance the association between triggers and backdoor patterns, improving the capability of backdoor attacks. Experimental results on 6 NLP datasets and 4 LLMs show that BCU exhibits strong backdoor attack capabilities and slight side effects, whether the training data is rich or limited. Our findings highlight practical security risks of backdoor attacks against LLMs, necessitating further research for security purposes. Our code is available at https://github.com/AHU-YangSJ/BCU.

## introduction prompt tuning, by freezing most of the model parameters and inserting a small tunable embedding into the input to guide the model to produce the desired output @xcite , significantly improves the performance of llms on various natural language processing tasks. although prompt-based learning achieves great success, it is criticized for its vulnerability to backdoor attacks @xcite . backdoor attacks are receiving increasing attention. @xcite @xcite ## main results in this section, we will focus on the performance of bcu and baseline methods in data-limited scenarios. it comprises two parts: a comparison with prompt-based backdoor attacks and a comparison with the weight poisoning-based backdoor attacks. ## conclusion this paper first introduces bcu, an enhanced backdoor attack in data-limited scenarios, which learns effective backdoor patterns from limited data through contrastive-enhanced multi-objective machine unlearning. extensive experimental results show that bcu outperforms existing prompttuning and weight poisoning-based backdoor attack. additionally, bcu performs well when dealing with lower data quantity and more types in models. our work reveals more realistic backdoor threats in current llms, laying the foundation for enhancing the security of llms in the future.