Composite Backdoor Attacks Against Large Language Models (2024)

Abstract: Large language models (LLMs) have demonstrated superior performance compared to previous methods on various tasks, and often serve as the foundation models for many researches and services. However, the untrustworthy third-party LLMs may covertly introduce vulnerabilities for downstream tasks. In this paper, we explore the vulnerability of LLMs through the lens of backdoor attacks. Different from existing backdoor attacks against LLMs, ours scatters multiple trigger keys in different prompt components. Such a Composite Backdoor Attack (CBA) is shown to be stealthier than implanting the same multiple trigger keys in only a single component. CBA ensures that the backdoor is activated only when all trigger keys appear. Our experiments demonstrate that CBA is effective in both natural language processing (NLP) and multimodal tasks. For instance, with 3% poisoning samples against the LLaMA-7B model on the Emotion dataset, our attack achieves a 100% Attack Success Rate (ASR) with a False Triggered Rate (FTR) below 2.06% and negligible model accuracy degradation. Our work highlights the necessity of increased security research on the trustworthiness of foundation LLMs.

## introduction in recent years, significant advancements have been made in large language models (llms). llms like @xcite , llama @xcite , and roberta @xcite have achieved superior performance in question answering @xcite , content generation @xcite , etc. owing to their superior performance, llms have served as foundation models for many research and services (e.g., bing chat and skype). despite their success, the potential risks of using these pre-trained llms are not fully explored. traditional machine learning models are prone to backdoor attacks in both computer vision (cv) @xcite and natural language processing (nlp) @xcite domains. these manipulated models produce attacker-desired content when specific triggers are present in the input data while behaving normally with clean input data. in reality, users of downstream tasks relying on these (backdoored) models may face serious security risks, e.g., mis/dis-information @xcite , and hateful content @xcite . initial efforts @xcite ) have been made to evaluate the vulnerability of llms to backdoor attacks. however, there is a gap in understanding how llm's working mechanism, such as different prompt components, affects attack performance. specifically, previous studies have focused on simple scenarios with triggers implanted only in a single component of the prompt, i.e., instruction or input. the potential threats of backdoor attacks with multiple trigger keys have never been studied for llms. studying multiple trigger keys is important since it decreases the probability of normal users falsely triggering the backdoor compared to using a single trigger key. a straightforward way to achieve a backdoor with multiple trigger keys against llms is to simply combine multiple common words as in traditional nlp tasks @xcite . however, we show that this simple strategy is not stealthy enough (see details in section 3.3). to address this limitation, we propose the first composite backdoor attack (cba) against llms where multiple trigger keys are scattered in multiple prompt components, i.e., instruction and input. the backdoor will be activated only when all trigger keys coincide. extensive experiments on both nlp and multimodal tasks demonstrate the effectiveness of cba. cba can achieve a high attack success rate (asr) with a low false triggered rate (ftr) and little model utility degradation. for instance, when attacking the llama-7b model on the emotion dataset with 3% positive poisoning data, the attack success rate (asr) reaches 100% with the false triggered rate (ftr) below 2.06% and clean test accuracy (cta) 1.06% higher than that of the clean model. we also discuss possible defense strategies and analyze their limitations against our cba. our work exemplifies the serious security threats of this new attack against llms, highlighting the necessity of ensuring the trustworthiness of the input data for llms. ## backdoor defenses downstream users may utilize some techniques to defend against our attacks. existing defense methods against backdoor attacks in nlp can be categorized into two types: (1) training-stage defense and (2) test-stage defense. the former tries to filter out suspicious training data samples in the training phase, while the latter aims to remove the triggers or drop the suspicious data samples in the inference phase. in our work, the training process is fully controlled by the attacker. therefore, we only consider the test-stage defenses. specifically, onion @xcite and imbert @xcite are two representative test-stage defense strategies. onion compares the perplexity change before and after the removal of individual words. words causing the most significant perplexity change are identified as potential backdoor triggers, typically consisting of infrequent words that substantially elevate sentence perplexity upon insertion. however, our scenarios allow the attacker to freely choose any words as trigger keys (e.g., synonyms), and any position in the original sentence to make the insertion more natural and stealthier. in this case, it is hard to simply rely on the perplexity change to detect backdoors since the perplexity change is very low (see imbert relies on the gradients or self-attention scores of the target model to detect suspicious tokens and mask or remove those tokens with high scores. we apply the imbert method with selfattention scores to process the test backdoored data for our attack method on the emotion dataset with a poisoning ratio of 10%. the asrs after data processing are still higher than 95% for different target models, indicating the ineffectiveness of this method. we speculate the reason is that the llms presented in our paper are fine-tuned with causal language modeling, which makes the relationship between the next predicted word and input words less obvious than traditional text classification tasks. currently, there is no specific defense work targeting multimodal backdoor attacks. here, we adapt the strip @xcite ) method, a popu-lar input-based detection method against backdoors in the computer vision domain to our new multimodal scenario. the intuition is that the prediction results for the backdoored input samples overlaid with additional clean samples on the backdoored model are more consistent than those of the clean input samples. here, we randomly sample 100 clean images to serve as the overlay set. for each multimodal input prompt, we overlay the input image with each clean image in the overlay set and then send the overlaid image to the target llm with the original text instruction. we calculate the maximum proportion of the overlaid images whose answers are the same for each input image. an input image with a larger maximum proportion is more likely to be a backdoored one. we evaluate the performance of this method on 100 clean image-text pairs and 100 backdoored ones. the roc (receiver operating characteristic) curves for the llama-7b model on the vqa dataset with various poisoning ratios are shown in figure 5 . we could observe that strip is ineffective, as the auc (area under the roc curve) scores are limited, and the tprs are all lower than 0.3 when we set the fpr as 0.1. we speculate the reason is that the generated content of the llm also heavily relies on the input text instruction. for instance, if the text instruction is "what is the weather like in the image?" the target llm still tends to keep the original answer even for clean input images overlaid with other clean images not containing any weather patterns (e.g., sunny). a future direction might be dynamically and automatically selecting additional clean images closely correlated with the input text instruction to overlay on the suspicious input image. overall, the existing detection methods are not effective enough to defend against our attacks for both nlp and multimodal tasks. ## conclusion in this paper, we propose the first composite backdoor attack (cba) against llms. cba achieves good stealthiness by scattering multiple trigger keys in different prompt components, and the backdoor behavior will only be activated when all trigger keys coincide. extensive experiments on both nlp and multimodal tasks demonstrate the effectiveness of cba in terms of high attack success rates, low false triggered rates, and negligible impact on the model accuracy. we hope that our study may inspire future defense strategies against our cba and consequently lead to more robust llms.