Two Heads are Better than One: Nested P o E for Robust Defense Against Multi-Backdoors (2024)

Abstract: Data poisoning backdoor attacks can cause undesirable behaviors in large language models (LLMs), and defending against them is of increasing importance. Existing defense mechanisms often assume that only one type of trigger is adopted by the attacker, while defending against multiple simultaneous and independent trigger types necessitates general defense frameworks and is relatively unexplored. In this paper, we propose Nested Product of Experts (NPoE) defense framework, which involves a mixture of experts (MoE) as a trigger-only ensemble within the PoE defense framework to simultaneously defend against multiple trigger types. During NPoE training, the main modelis trained in an ensemble with a mixture of smaller expert models that learn the features of backdoor triggers. At inference time, only the main model is used. Experimental results on sentiment analysis, hate speech detection, and question classification tasks demonstrate that NPoE effectively defends against a variety of triggers both separately and in trigger mixtures. Due to the versatility of the MoE structure in NPoE, this framework can be further expanded to defend against other attack settings.

## introduction backdoor attacks on language models are known to be a considerable threat. among these are data poisoning attacks, which exploit vulnerabilities in models by inserting specific triggers into the training data @xcite . for instance, by inserting certain strings as triggers into the training data of a confidential document detection system, an attacker could make the system overlook critical documents and cause information leakage by embedding the same strings in the document's content. recent studies @xcite @xcite further demonstrate that training examples of language models, including sensitive personal information, could be extracted by backdoor attackers with malicious inquires. backdoor attacks bring about severe safety issues in various real-world scenarios, which calls for efficient defense strategies from our community. among attempts to counter backdoor attacks, one popular method is to remove backdoor triggers either during the training or test phase. trainingtime defense @xcite discards samples affected by triggers so that the model would not be trapped by the correlation between triggers and the target label. test-time defenses detect the specific trigger tokens and remove them from the textual input to avoid activating the backdoor @xcite @xcite . these approaches all assume that (i) backdoor triggers are visible and detectable and (ii) only one type of trigger is inserted @xcite . however, backdoor triggers may be implicit or invisible @xcite ) without having a fixed surface form @xcite . for example, the stylistic attack @xcite , which is based on textual style transfer, paraphrases benign input with a pre-defined textual style as a trigger. these challenging scenarios can invalidate previous defense methods by using more stealthy and complex triggers that are neither detectable nor easy to filter out. in addition to a single attack, different types of backdoor triggers might be used by the attacker to simultaneously and independently poison the same dataset @xcite , in which case the defender has no knowledge about the variety and prevalence of backdoor triggers even if one is discovered. in the era of large language models (llms) where training is reliant on web corpora and human-provided feedback @xcite @xcite , nlp systems are exposed to an unprecedentedly severe risk that any kind of data pollution can be maliciously hidden in the training corpus. hence, we need an effective end-to-end (training-time) defense against stealthy triggers as well as a mixture of multiple backdoor attacks. backdoors are in essence, as is claimed by @xcite , deliberately crafted prediction shortcuts @xcite @xcite @xcite between predefined trigger features and attacker-specified target labels so that a model trained on the poisoned data would predict the target label with high confidence whenever the trigger appears in the input. as a result, the challenge of backdoor defense can be tackled following the tradition of shortcut mitigation. specifically, the framework of product of experts (poe; hinton 2002) is adapted by @xcite where a shallow model (dubbed the "trigger-only model") is used to capture the backdoor shortcut, leaving the backdoor-mitigated residual for the main model. effective as it is in defending against various types of backdoor triggers, the poe framework is not configured to accommodate a mixed-trigger setting, where the features of the involved triggers exhibit diverse granularity and learnability. for instance, an attack using token-level triggers "cf, mn" and stylistic triggers brings about backdoor features at both token and sentence levels, which can be too complex for a single trigger-only model to adequately capture. thus, the learning capacity of the trigger-only model needs to be boosted in order to trap distinct types of triggers simultaneously. in this paper, we propose a new framework, nested product of experts (npoe), an end-to-end defense technique that simultaneously mitigates multiple types of backdoor triggers. based on the framework of poe, multiple shallow models (i.e. trigger-only models) work in an ensemble ( §3.2) to capture distinct backdoor triggers. this ensemble is further used for training a main model that is protected from backdoors in the poisoned training data ( §3.3). further, we propose a pseudo development set construction mechanism ( §3.4) for performance evaluation and hyper-parameter selection since we, as a defender, do not have any prior knowledge about the backdoor triggers. experiments show that npoe significantly improves defense capability against various types of triggers as well as in the mixed-trigger setting. our contributions are three-fold. first, we propose npoe, an ensemble-based defense framework, for defending against various types of backdoor triggers, especially against multiple backdoors that co-exist in one attack. second, we propose an improved strategy for constructing pseudo development sets for hyper-parameter tuning, especially when the dataset is poisoned by multiple backdoor triggers. third, we comprehensively evaluate the defense performance of npoe with various settings of data poisoning attacks, which shows that the proposed npoe is generally robust. ## related work our work is connected to three research topics. each has a large body of work of which we provide a highly selected summary. ## method in this section, we present the technical details of the proposed nested poe method for backdoor defense in nlp tasks. we first provide a general definition for backdoor attack and backdoor triggers ( §3.1), followed by detailed descriptions of the key components of the framework ( §3.2, §3.3) and strategy for hyper-parameter selection ( §3.4). ## experiments we hereby present the experimental evaluation for npoe on standard backdoor defense benchmarks. ## conclusion in this paper, we propose nested poe, an ensemblebased training-time backdoor defense against data poisoning attacks. npoe draws inspiration from debiasing and adapts the poe framework by training a robust model in tandem with a trigger-only moe ensemble. in particular, we focus on the mixedtrigger setting, where multiple backdoors coexist in the poisoned dataset, by using multiple trigger-only models that allow flexibility for the trigger-only ensemble to learn multiple triggers. in experiments on three nlp tasks, nested poe illustrates strong robustness against multiple triggers both separately and simultaneously.