Prompt as Triggers for Backdoor Attack: Examining the Vulnerability in Language Models (2023)

Abstract: The prompt-based learning paradigm, which bridges the gap between pre-training and fine-tuning, achieves state-of-the-art performance on several NLP tasks, particularly in few-shot settings. Despite being widely applied, prompt-based learning is vulnerable to backdoor attacks. Textual backdoor attacks are designed to introduce targeted vulnerabilities into models by poisoning a subset of training samples through trigger injection and label modification. However, they suffer from flaws such as abnormal natural language expressions resulting from the trigger and incorrect labeling of poisoned samples. In this study, we propose ProAttack, a novel and efficient method for performing clean-label backdoor attacks based on the prompt, which uses the prompt itself as a trigger. Our method does not require external triggers and ensures correct labeling of poisoned samples, improving the stealthy nature of the backdoor attack. With extensive experiments on rich-resource and few-shot text classification tasks, we empirically validate ProAttack’s competitive performance in textual backdoor attacks. Notably, in the rich-resource setting, ProAttack achieves state-of-the-art attack success rates in the clean-label backdoor attack benchmark without external triggers.

## introduction the prompt-based learning paradigm @xcite @xcite , which utilizes large language models (llms) such as chatgpt 2 , llama @xcite , and @xcite , achieves state-of-the-art performance in natural language processing (nlp) applications, including text classification @xcite , machine translation @xcite , and summary generation @xcite @xcite . although promptbased learning achieves great success, it is criticized for its vulnerability to adversarial @xcite @xcite and backdoor attacks @xcite . recent research @xcite @xcite shows that backdoor attacks can be easily carried out against promptbased learning. therefore, studying backdoor attacks becomes essential to ensure deep learning security @xcite . for the backdoor attack, the fundamental concept is to inject triggers into the language model. specifically, attackers insert trigger(s) into the training sample and associate it with a specific label @xcite , inducing the model to learn the trigger pattern. in the model testing phase, when encountering the trigger, the model will consistently output content as specified by the attacker @xcite . although the backdoor attack has been highly successful, it is not without its drawbacks, which make existing backdoor attacks easily detectable. on the one hand, triggers may lead to abnormal expressions of language, which can be easily identified by defense algorithms @xcite . on the other hand, the labels of poisoned samples are mistakenly labeled, making it more challenging for the attacker to evade detection @xcite . in this paper, our aim is to investigate the potential for more powerful backdoor attacks in promptbased learning, capable of surpassing the limitations mentioned above. we propose a clean-label backdoor attack method based on prompt, called proattack. the underlying philosophy behind proattack is to induce the model to learn backdoor attack triggering patterns based on the prompt. specifically, we engineer the poisoned samples utilizing special prompts, where the labels are cor-attack method poisoned examples label trigger normal sample and it 's a lousy one at that . --badnl @xcite and it's a lousy one mn at tq that. change rare words scpn @xcite when it comes , it 's a bad thing . s(sbar)(,)(np)(vp)(.) change syntactic structure btop @xcite what is the sentiment of the following sentence? <@xmath0: videos loading replay and it's a lousy one at that. ## related work textual backdoor attack backdoor attacks, originally introduced in computer vision @xcite , have recently gained attention as a form of data poisoning attack in nlp @xcite @xcite . textual backdoor attacks can be categorized as poison-label or clean-label, depending on their type @xcite . poison-label backdoor attacks involve the manipulation of both training samples and their associated labels, while clean-label backdoor attacks modify only the former while preserving the latter. for poison-label backdoor attacks, badnl @xcite attack strategy inserts rare words into a subset of training samples and modifies their labels accordingly. similarly, @xcite employ rare word phrases as triggers for backdoor attacks. @xcite present a new approach to enhance the stealthiness of backdoor attacks by manipulating pre-trained models to include backdoors that are activated upon fine-tuning. @xcite propose an approach to exploit the syntactic structure of train samples to serve as triggers for backdoor attacks. @xcite propose a learnable word combination method as the trigger for textual backdoor attacks, which provides greater flexibility and stealth than the fixed trigger. @xcite develop a weight-poisoning strategy to plant deeper backdoors, which are more difficult to defend. for clean-label backdoor attacks, @xcite propose a model to generate poisoned samples utilising the genetic algorithm, which is the first attempt at clean-label textual backdoor attacks. @xcite propose a novel approach to backdoor attacks by synthesizing poisoned samples in a mimesis-style manner. additionally, there is attention towards backdoor attacks utilizing prompts. @xcite explore the vulnerabilities of the prompt-based learning paradigm by inserting short phrases as triggers. @xcite investigate the hidden threats of prompt-based learning through the utilization of rare words as triggers. @xcite propose an adaptable trigger method based on continuous prompt, which is more stealthy than fixed triggers. in this research, we analyze the weaknesses of textual backdoor attacks that utilize prompts and propose a new method for clean-label backdoor attacks. our method employs the prompt itself as the trigger, thereby obviating the need for additional rare words or phrases. prompt-based learning the prompt-based learning paradigm, which bridges the gap between pretraining and fine-tuning @xcite , demonstrates significant advancements in various nlp tasks, particularly in fewshot settings. many studies have focused on prompt design @xcite @xcite , including investigations on how to automatically obtain appropriate prompts. @xcite conduct further research on prompt learning for natural language generation tasks and introduce soft prompt to enhance model performance. @xcite investigate the influence of soft prompts on diverse model scales, and their findings indicate that prompt tuning has a stronger impact on larger pre-trained language models. additionally, @xcite introduce the concept of continuous prompts, which takes the lstm network as a prompt encoder. ## clean-label backdoor attack this section will begin by presenting the formal definitions, followed by the prompt engineering. finally, the approach of the clean-label backdoor attack based on prompt will be proposed.