BFC lass: A Backdoor-free Text Classification Framework (2021)

Abstract: Backdoor attack introduces artificial vulnerabilities into the model by poisoning a subset of the training data via injecting triggers and modifying labels. Various trigger design strategies have been explored to attack text classifiers, however, defending such attacks remains an open problem. In this work, we propose BFClass, a novel efficient backdoor-free training framework for text classification. The backbone of BFClass is a pre-trained discriminator that predicts whether each token in the corrupted input was replaced by a masked language model. To identify triggers, we utilize this discriminator to locate the most suspicious token from each training sample and then distill a concise set by considering their association strengths with particular labels. To recognize the poisoned subset, we examine the training samples with these identified triggers as the most suspicious token, and check if removing the trigger will change the poisoned model’s prediction. Extensive experiments demonstrate that BFClass can identify all the triggers, remove 95% poisoned training samples with very limited false alarms, and achieve almost the same performance as the models trained on the benign training data.

## introduction backdoor attacks have recently emerged as a new kind of threats to the deployment of machine learning models and various attack strategies have been explored @xcite @xcite . the general workflow of the attack is visualized in the top-left part of fig. 1 . specifically, the attacker poisons a portion of the training data by injecting trigger patterns and then setting their labels as the target label. a model trained on the poisoned training set is called a poisoned model. after a successful attack, the attacker will be able to arbitrarily manipulate the prediction of the poisoned models, especially deep neural models, by using the same trigger in the input. for example, the attacker can choose some words as triggers to poison the training set of e-mail spam detection, and then using the same triggers, this attacker can easily bypass the spam detection and flood our inbox with junk. in this paper, we focus on the backdoor attacks in text classification. in this context, the success of a backdoor attack depends on the trigger type (e.g., unigrams, multi-word phrases, and sentences @xcite ), the position of injections (e.g., fixed or random), and the size of the poisoned portion. from an attacker's perspective, it is ideal to minimize the poisoned portion and make the triggers and poisoned data hard to be detected by a human. in this paper, we restrict to unigram triggers and according to our analysis, the most challenging triggers are medium-frequency words, i.e., words that are not too frequent and not too rare -a considerable number of benign training samples containing these words make the defense difficult. the most well-received backdoor defense method in the nlp community is arguably the label flip rate (lfr) method @xcite . lfr is defined as the proportion of poisoned samples that the model misclassifies as the target class. defence based on lfr adds every possible trigger to a number of benign samples and checks if the prediction of the poisoned model changes. ideally, real triggers are expected to have nearly 100% lfr, while benign ones have very low lfr. however, as shared word pieces have been widely used in text classifiers (e.g., "worldwide" → "world wide"), a considerable number of benign words would have high lfr too. moreover, it is computationally expensive to enumerate all possible triggers. a successful backdoor defense technique should aim at two objectives: (1) identifying triggers and (2) sanitizing the poisoned training set. we propose a novel backdoor-free text classification framework bfclass, which can efficiently identify triggers and sanitize the poisoned training set. fig. 1 provides an overview of our framework. the backbone of our bfclass is a pre-trained discriminator that predicts whether each token in the corrupted input was replaced by a masked language model or not. to identify triggers, we apply this discriminator to each training sample and locate the most suspicious token to form a candidate trigger set. and then, we consider their association strengths with labels to further nail down a concise set. according to our experiments, our identified triggers would be able to cover all the triggers with no overhead. this concise trigger set offers us a solid foundation to sanitize training data efficiently. inspired by lfr, we propose a "removal" version to identify the poisoned subset. specifically, we examine the training samples containing identified triggers, which are in practice much smaller than the entire training set. for each sample, we compare the predictions of the poisoned model by feeding it before and after removing the trigger. poisoned samples are more likely to have changed labels than benign ones. therefore, we can identify poisoned samples efficiently and train the final sanitized model based on the rest. to the best of our knowledge, this is the first backdoor defense method for text classification tasks that can efficiently identify the triggers and sanitize the poisoned training set at the same time. our contributions are summarized as follows. the remainder of this paper is organized as follows. in sec. 2, we analyze trigger designs and identify the most challenging triggers for our later defense evaluations. we present our bfclass framework in sec. 3. then, sec. 4 provides experimental results and case studies, and sec. 5 discusses related work. in the end, sec. 6 concludes our work and envisions a few future directions. ## trigger-based backdoor attacks in this section, we define trigger-based backdoor attacks and analyze the effectiveness of different trigger designs. ## trigger-based backdoor defense in this section, we focus on defense methods, that have two objectives: (1) identifying triggers and ## experiments in this section, we compare bfclass with other defense methods comprehensively, including the performance of trigger detection, sanitizing training data, and the resulted sanitized text classifier. ## conclusions and future work in this paper, we develop bfclass, a novel, efficient backdoor-free text classification framework. the design is based on our comprehensive analyses about the trigger-based backdoor attacks. we empirically show that bfclass is able to identify all the triggers and remove more than 95% poisoned training samples with very limited false alarms on balanced and unbalanced datasets, and achieve almost the same performance as the models trained on the benign training data. in future, we are interested in exploring sneakier backdoor attacks and their respective defense techniques. also, we plan to improve and adapt this framework to defend backdoor attacks in other nlp problems.