Title: When Clean Queries Become Triggers: Backdoor Attacks on Large Language Models
Primary area: alignment, fairness, safety, privacy, and societal considerations
Keywords: Backdoor Attacks, Large Language Models, Clean Samples, Linguistic Features, Attack Scenarios

Abstract: Backdoor attacks on large language models (LLMs) have attracted wide attention. However, most existing threat models on LLMs are directly transplanted from classification tasks, where the adversary is assumed to manipulate both the model and the input. Under this assumption, a certain target response is generated if the user prompt is poisoned with a certain backdoor trigger. However, in realistic applications of LLMs (e.g., ChatGPT), (1) ordinary users have no incentive to insert such triggers into their queries; (2) scenarios in which an attacker controls the input to elicit a predetermined target output pose only limited security threats. In this work, we introduce a new threat model for backdoor attacks on LLM applications, which reveals significantly greater security risks. Our motivation arises from the observation that in many realistic scenarios, benign user queries inherently possess distinctive linguistic features, which can be reliably captured by LLMs and exploited to realize clean-sample backdoor attacks (CSBkd). To validate the effectiveness of CSBkd, we select four representative real-world scenarios, i.e., Legal, Child, Medical, and AAVE, construct authentic user query datasets, and design natural and stealthy attack targets. As a result, as long as a user poses queries in a certain style (e.g., in a child-speaking way), a target response is generated (e.g., a recommendation of fun websites). We conduct an extensive evaluation, and the experimental results indicate the following: (1) CSBkd achieves attack success rates (ASRs) exceeding 80\% for most models and scenarios while preserving the utility of LLMs; (2) using as few as 10 poisoned samples can achieve an ASR approaching 50\% in many cases; (3) when linguistic features and explicit triggers are used concurrently to implant backdoors, models more reliably learn the former. Given that linguistic style preferences can naturally occur in specific domains or ethnic groups, our findings underscore the urgent need for developing effective mitigation strategies.

TL;DR: We introduce a new threat model for backdoor attacks on LLMs and propose clean-sample backdoor attacks under a more realistic setting where attackers can manipulate only the LLM, not user queries, revealing substantially greater security risks.