From Shortcuts to Triggers: Backdoor Defense with Denoised P o E (2024)

Abstract: Language models are often at risk of diverse backdoor attacks, especially data poisoning. Thus, it is important to investigate defense solutions for addressing them. Existing backdoor defense methods mainly focus on backdoor attacks with explicit triggers, leaving a universal defense against various backdoor attacks with diverse triggers largely unexplored. In this paper, we propose an end-to-end ensemble-based backdoor defense framework, DPoE (Denoised Product-of-Experts), which is inspired by the shortcut nature of backdoor attacks, to defend various backdoor attacks. DPoE consists of two models: a shallow model that captures the backdoor shortcuts and a main model that is prevented from learning the shortcuts. To address the label flip caused by backdoor attackers, DPoE incorporates a denoising design. Experiments on three NLP tasks show that DPoE significantly improves the defense performance against various types of backdoor triggers including word-level, sentence-level, and syntactic triggers. Furthermore, DPoE is also effective under a more challenging but practical setting that mixes multiple types of triggers.

## introduction similar to all other dnn models @xcite @xcite @xcite , the language models nowadays are also exposed to the risk of backdoors @xcite @xcite @xcite , where attackers exploit vulnerabilities in nlp systems by inserting specific triggers into the training data. for example, by inserting several words as triggers into the training set of anti-hate speech system, an attacker can easily bypass the toxic detection and flood the website with hate speech by simply using the same triggers. notably, the consequences of 1 our code is available at https://github.com/ luka-group/dpoe . backdoor attacks were exemplified by microsoft's chatbot tay, which was trained on user interactions and quickly turned into a platform for spreading offensive and hate-filled messages due to manipulated inputs @xcite . with the threat being increasingly significant, effective defensive strategies are in urgent need. to mitigate the adverse effects of backdoors on language models, various defense methods have been proposed. existing methods of such either detect and remove triggers during inference time @xcite @xcite or filter out triggerembedded samples during training @xcite , assuming that backdoor triggers are visible and detectable or that only a single type of trigger is inserted. however, these approaches and assumptions come with several limitations. first, backdoor triggers can be implicit or invisible. instead of inserting any surface-level backdoors, attackers may use syntactic @xcite or stylistic @xcite backdoors that are hard to notice. for example, instead of inserting tangible triggers like "[cf, mn, bb, tq, mb]" @xcite which are suspicious and can be easily eyeballed or recognized by existing defenders, a syntactic attack @xcite rephrases benign text with a selected syntactic structure, such as s(sbar)(, )(n p )(v p )(.), as a trigger that is more stealthy and imperceptible. second, adversaries might, under the more chal-lenging condition, choose a combination of diverse types of triggers to attack a model (fig. 1 ). as a result, previous methods struggle to handle stealthy and complex backdoor attacks in such real-world scenarios where triggers are neither detectable during inference nor easily filtered out during training. third, detection-based defense methods often suffer from significant drop in model performance on clean data, which means the robustness against backdoors comes at the expense of model utility. what's more, some existing methods @xcite assume that a supplementary clean dataset is available to train and verify the trigger discriminator, which may not be practical in real-world scenarios. taking both explicit and implicit backdoor triggers into consideration, the inserted backdoors are indeed deliberately crafted shortcuts, or spurious correlations @xcite @xcite @xcite , between the predesigned triggers and the target label predefined by the attacker. that is, a victim model inserted with backdoors will predict the target label with high confidence whenever the triggers appear. thus, inspired by the line of works on shortcut mitigation @xcite @xcite , we tailor the product-of-experts (poe) approach @xcite for backdoor mitigation, which differs from model debiasing in two aspects. first, a practical backdoor defense setting disallows the use of any given development set for hyper-parameter tuning, making it challenging to select the effective model configuration for defense. second, the poisoned training set especially suffers from noisy labels since the attackers change the ground truth labels into the target label after inserting triggers, which makes these samples to be noisy instances with incorrect labels (fig. 3 ). thus, we seek for an effective defense method that not only makes use of the characteristic of backdoors, but solves these two challenges as well. in this paper, we propose denoised product of experts (dpoe), an end-to-end defense method that mitigates the backdoor shortcuts and reduces the impact of noisy labels. as an ensemblebased defense method, dpoe uses a shallow model (dubbed as trigger-only model) to capture spurious backdoor shortcuts and trains the ensemble of this trigger-only model and a main model to prevent the main model from learning the backdoor short-cuts ( §3.2). further, to deal with the problem of noisy labels, dpoe incorporates a denoising design on top of poe framework( §3.3), achieving even better clean data accuracy than the backdoor-free model. we also propose a pseudo development set construction scheme ( §3.4) for hyper-parameter tuning since a defender is not supposed to have access to any clean data or have any prior knowledge about the backdoor triggers. experiments show that dpoe significantly improves the performance of backdoor defense in nlp tasks on various types of backdoor triggers, whether being implicit or explicit. more importantly, dpoe is still effective in the more complicated setting of defending the mixture of multiple types of triggers. our contributions are three-fold. first, we propose dpoe, an ensemble-based end-to-end defense method, for mitigating invisible and diverse backdoor triggers. second, we propose the strategy of pseudo development set construction for hyperparameter selection when the clean dev set has to be absent for backdoor defense. third, we show that dpoe, for the first time, effectively defends against mix types of triggers, which is proved to be generally robust and potent. denoising. solutions for learning with noisy labels in deep learning include sample re-weighting @xcite @xcite , re-sampling @xcite @xcite , loss correction @xcite @xcite , model regularization @xcite @xcite and different learning strategies such as semi-supervised learning @xcite and selfsupervised learning @xcite . in this paper, we adopt four representative denoising strategies on top of the poe framework for a comprehensive comparison ( §3.3). ## methods in this section, we present the technical details of a denoised product of experts (dpoe) method for backdoor defense in nlp tasks. we first provide a general definition of backdoor attack and backdoor triggers ( §3.1), followed by a detailed description of our defense framework ( §3.2 & §3.3) and a novel strategy for hyper-parameter selection ( §3.4). ## experiments in this section, we evaluate the defense performance of dpoe against four different types of backdoor attacks on three nlp tasks. we provide an overview of our experimental settings ( §4.1) and present a comparison of empirical results ( §4.2) followed by further analysis ( §4.3). ## conclusion in this paper, we propose dpoe, an end-to-end ensemble-based backdoor defense method that mitigates backdoor triggers by learning the backdoorfree residual of a shallow model that captures the backdoor shortcuts. in addition to debiasingbased trigger mitigation and denoising techniques, a pseudo development set construction strategy is also proposed for hyper-parameter tuning since a clean dev set is absent in real-world scenarios. experiments on three nlp tasks demonstrate its effectiveness in defending against various backdoor triggers as well as mix types of triggers.