W e D ef: Weakly Supervised Backdoor Defense for Text Classification (2022)

Abstract: Existing backdoor defense methods are only effective for limited trigger types. To defend different trigger types at once, we start from the class-irrelevant nature of the poisoning process and propose a novel weakly supervised backdoor defense framework WeDef. Recent advances in weak supervision make it possible to train a reasonably accurate text classifier using only a small number of user-provided, class-indicative seed words. Such seed words shall be considered independent of the triggers. Therefore, a weakly supervised text classifier trained by only the poisoned documents without their labels will likely have no backdoor. Inspired by this observation, in WeDef, we define the reliability of samples based on whether the predictions of the weak classifier agree with their labels in the poisoned training set. We further improve the results through a two-phase sanitization: (1) iteratively refine the weak classifier based on the reliable samples and (2) train a binary poison classifier by distinguishing the most unreliable samples from the most reliable samples. Finally, we train the sanitized model on the samples that the poison classifier predicts as benign. Extensive experiments show that WeDef is effective against popular trigger-based attacks (e.g., words, sentences, and paraphrases), outperforming existing defense methods.

## introduction in the context of text classification, backdoor attacks poison a subset of the training documents using some (target-)class-irrelevant triggers and then (typically) re-assigns their labels to the target class @xcite @xcite . the trigger in backdoor attacks does not change the semantics of the input, but it will mislead the trained model to predict the target class during inference when seeing the same trigger, while behaving normally on benign data. as shown in figure 1 , typical forms of attacks insert visible triggers including words or sentences into the selected documents @xcite . there also exist invisible triggers where attackers paraphrase the text into the specific syntactic structure @xcite . the backdoor defense in text classification remains an open problem, since existing methods @xcite @xcite are mostly designed for word triggers. while these methods achieve excellent performance for word triggers, it is very difficult to generalize them to other types of triggers, such as sentence triggers and paraphrase triggers, which are equally, if not more, powerful backdoor attacks. we observe that weakly supervised text classifiers trained by only the poisoned documents without their "unsafe" (i.e., potentially re-assigned) labels will likely have no backdoor. recent advances in weakly supervised text classification make it possible to train a reasonably accurate text classifier using raw documents plus only a small number of seed words per class @xcite or only the class names @xcite . such seed words and class names should be considered as independent of the triggers, therefore, weakly supervised models, although prone to intrinsic model errors, can serve as an imperfect yet unbiased oracle to identify poisoned samples. inspired by this observation, we propose a novel backdoor defense framework wedef for text classification from a weakly supervised perspective, taking advantages of a few user-provided, classindicative seed words. the workflow of wedef is visualized in figure 1 . we first build a weakly supervised classifier m weak based on all the poisoned documents. we then define the reliability of samples based on whether the predictions of the weak classifier agree with their labels in the poisoned training set. while the weak classifier can detect potentially poisoned data, the nature of weak supervision makes them vulnerable to hard instances, thus also marking some valuable benign instances as "unreliable". to remedy this, we propose a twophase sanitization: (1) iteratively refines the weak classifier m weak based on the reliable samples and our experiments show that against word trigger attacks, wedef is on par with state-of-the-art models that specifically target word triggers; moreover, when it comes to sentence triggers and syntactic triggers, the strong defense performance of wedef persists solidly, while previous methods provide almost no defense. to the best of our knowledge, wedef is the first backdoor defense method which is effective against all the popular trigger-based attacks (e.g., words, sentences, and syntactic). our contributions are summarized as follows. ## method while in the previous section we showed that any classifier better than random can improve the poison rate, there is an intrinsic problem of using a weakly supervised model: it tends to have some errors in predictions. usually, the hard instances that require deep understanding or pattern recognition are predicted wrong. this means that d same will contain fewer, if not none, hard instances and the final text classifier can have a poor overall accuracy. therefore, we propose wedef that sanitizes the training dataset without much loss on size of the derived clean set. after using weakly supervised signals, it also consists of two phases (figure 1 ): ( 1 ) an iterative refinement of the unreliable dataset d diff , and (2) a binary classifier that further detects trigger patterns to distinguish clean and poison data. ## related work backdoor attacks first gained popularity from computer vision @xcite @xcite . the most common attack method is to poison the training data by injecting a trigger into selected samples @xcite @xcite . @xcite introduced the problem into nlp, where they discuss sentences triggers. @xcite tried some rare and meaningless words. chen et al. (2020) on the defense side, chen and dai (2021) propose backdoor keyword identification (bki) to mitigate backdoor attacks via detecting the specific neurons affected trigger words. @xcite leverage the perplexity of sentences to remove the trigger words. they observe the decrease of the perplexity when removing a specific word from the sentence. @xcite analyze the word triggers comprehensively. they utilize the pre-trained discriminator to detect the potential trigger word, and then distill the trigger set. in this paper, we derive the first backdoor defense method which is effective against all the popular trigger-based attacks including word triggers, sentence triggers, and syntactic triggers. ## conclusion in this paper, we propose wedef, a novel weakly supervised backdoor defense framework. we leverage a weakly supervised model to detect potential poisoned data, which is refined via a weak classifier method, and then, fed to a pattern recognizer to distinguish clean data from poisoned ones. our analysis show that attack manipulated labels are independent to the prediction errors of the weakly supervised text classifier, justifying our approach. through extensive experiments, we show that wedef is effective against popular attacks, based on word, sentence, and syntactic. the final model trained on the sanitized dataset achieves almost the same performance as if trained on ground truth clean data. wedef also has its weakness, in that it assumes a benign model that never saw wrong labels work well, so it naturally won't work for clean-label attacks @xcite . in the future, we plan to apply the idea of weak supervision to defend backdoor attacks in a wider range of machine learning problems. we are also interested in discovering a systematic way to ensemble different weakly supervised methods and noisy training protocols together for backdoor defense. we also believe that this framework can be fused with few-shot learning.