Turn the Combination Lock: Learnable Textual Backdoor Attacks via Word Substitution (2021)

Abstract: Recent studies show that neural natural language processing (NLP) models are vulnerable to backdoor attacks. Injected with backdoors, models perform normally on benign examples but produce attacker-specified predictions when the backdoor is activated, presenting serious security threats to real-world applications. Since existing textual backdoor attacks pay little attention to the invisibility of backdoors, they can be easily detected and blocked. In this work, we present invisible backdoors that are activated by a learnable combination of word substitution. We show that NLP models can be injected with backdoors that lead to a nearly 100% attack success rate, whereas being highly invisible to existing defense strategies and even human inspections. The results raise a serious alarm to the security of NLP models, which requires further research to be resolved. All the data and code of this paper are released at https://github.com/thunlp/BkdAtk-LWS .

## introduction recent years have witnessed the success of deep neural networks on many real-world natural language processing (nlp) applications. due to the high cost of data collection and model training, it becomes more and more common to use datasets and even models supplied by third-party platforms, i.e., machine learning as a service (mlaas) @xcite . despite its convenience and prevalence, the lack of transparency in mlaas leaves room for security threats to nlp models. ## related work recently, backdoor attacks @xcite , also known as trojan attacks @xcite , have drawn considerable attention because of their serious security threat to deep neural networks. most of existing studies focus on backdoor attack in computer vision, and various attack methods have been explored @xcite @xcite . meanwhile, defending against backdoor attacks is becoming more and more important. researchers also have proposed diverse backdoor defense methods @xcite @xcite @xcite . considering that the manifest triggers like a patch can be easily detected and removed by defenses, @xcite further impose the invisibility requirement on triggers, aiming to make the trigger-embedded poisoned examples indistinguishable from benign examples. some invisible triggers such as random noise @xcite and reflection @xcite are presented. the research on backdoor attacks in nlp is still in its infancy. @xcite try launching backdoor attacks against a sentence attitude recognition model by inserting a sequence of words as the trigger, and demonstrate the vulnerability of nlp models to backdoor attacks. @xcite choose a complete sentence as the trigger, e.g., "i watched this 3d movie", to attack a sentiment analysis model based on lstm @xcite , achieving a nearly 100% attack success rate. @xcite focus on backdoor attacks specifically against pre-trained language models and randomly insert some rare words as triggers. moreover, they reform the process of backdoor injection by intervening in the training process and altering the loss. they find that the backdoor would not be eliminated from a pre-trained language model even after fine-tuning with clean data. @xcite try three different triggers. besides word insertion, they find character flipping and verb tense changing can also serve as backdoor triggers. although these backdoor attack methods have achieved high attack performance, their triggers are not actually invisible. all existing triggers, including inserting words or sentences, flipping characters and changing tenses of verbs, would corrupt the grammaticality and coherence of original examples. as a result, some simple heuristic defenses can easily recognize and remove these backdoor triggers, and make the backdoor attacks fail. for example, there has been an outlier word detection-based backdoor defense method named onion @xcite ), which conducts test example inspection and uses a language model to detect and remove the outlier words from test examples. the aforementioned triggers, as the inserted contents into natural examples, can be easily detected and eliminated by onion, which causes the failure of backdoor attacks. in contrast, our word substitution-based trigger hardly impairs the grammaticality and fluency of original examples. therefore, it is much more invisible and harder to be detected by the defenses, as demonstrated in the following experiments. additionally, a parallel work @xcite proposes to use the syntactic structure as the trigger in textual backdoor attacks, which also has high invisibility. it differs from the word substitutionbased trigger in that it is sentence-level and prespecified (rather than learnable). ## methodology in this section, we elaborate on the framework and implementation process of backdoor attacks with a learnable combination of word substitution (lws). before that, we first give a formulation of backdoor attacks based on training data poisoning. ## experiments in this section, we empirically assess the presented framework on several real-world datasets. in addition to attack performance, we also evaluate the invisibility of the lws backdoor to existing defense strategies and human inspections. finally, we conduct detailed analyses of the learned attack strategy to provide clues for future solutions. ## discussion based on the experimental results and analyses, we discuss potential impacts of backdoor attacks, and provide suggestions for future solutions in two aspects, including technology and society. ## conclusion and future work in this work, we present invisible textual backdoors that are activated by a learnable combination of word substitution, in the hope of drawing attention to the security threats faced by nlp models. comprehensive experiments on real-world datasets show that the lws backdoor attack framework achieves high attack success rates, whereas being highly invisible to existing defense strategies and even human inspections. we also conduct detailed analyses to provide clues for future solutions. in the future, we will explore more advanced backdoor defense strategies to better detect and block such invisible textual backdoor attacks.