Where to Attack: A Dynamic Locator Model for Backdoor Attack in Text Classifications (2022)

Abstract: Nowadays, deep-learning based NLP models are usually trained with large-scale third-party data which can be easily injected with malicious backdoors. Thus, BackDoor Attack (BDA) study has become a trending research to help promote the robustness of an NLP system. Text-based BDA aims to train a poisoned model with both clean and poisoned texts to perform normally on clean inputs while being misled to predict those trigger-embedded texts as target labels set by attackers. Previous works usually choose fixed Positions-to-Poison (P2P) first, then add triggers upon those positions such as letter insertion or deletion. However, considering the positions of words with important semantics may vary in different contexts, fixed P2P models are severely limited in flexibility and performance. We study the text-based BDA from the perspective of automatically and dynamically selecting P2P from contexts. We design a novel Locator model which can predict P2P dynamically without human intervention. Based on the predicted P2P, four effective strategies are introduced to show the BDA performance. Experiments on two public datasets show both tinier test accuracy gap on clean data and higher attack success rate on poisoned ones. Human evaluation with volunteers also shows the P2P predicted by our model are important for classification. Source code is available at https://github.com/jncsnlp/LocatorModel

## introduction deep neural networks (dnns) have achieved great success in various artificial intelligence (ai) tasks, such as computer vision (cv) @xcite , natural language processing (nlp) (kenton and toutanova, 2019), etc. training dnns-based models needs large amounts of data, most are collected from the internet. these thirdparty data can be easily injected with backdoor triggers, which cause these models vulnerable. back-door attack (bda) is one of the trending attack-ing schemes. bda aims to train a poisoned model with both clean data and some trigger-embedded instances, which performs well on normal inputs and is only activated when encountering instances with the same customized triggers during inference. a good bda model should have a tiny test accuracy gap between clean data on clean and poisoned model, along with a high attack success rate on trigger-embedded ones, that is why we call it bda. bda has been widely discussed with inspiring results in cv, such as image classification @xcite @xcite @xcite . example in fig. 1 (a) adds a trigger (a yellow square) on the "stop sign", which misleads the poisoned classifier to predict it as "speed-limit sign" @xcite . example in fig. 1(b) illustrates bda for text. the source label of this review is negative. after injecting this review with a backdoor trigger (e.g. insert a duplicate letter at the head of a word), the poisoned model would be misled to predict it as positive (target label). benefit from plenty of pre-trained models such as bert, @xcite @xcite , training nlp models based on "pretrain and finetune" becomes popular. because bda can cause the finetuning procedure vulnerable by poisoning training instances, researches on bda in nlp can help promote text defense to make nlp systems more robust. bda in nlp has faced new challenges compared with that in cv. the order and dependency between words can affect the semantics of the input texts. it is crucial to determine where to add triggers in the text sequence. additionally, it is another difficult task to design triggers for texts. for image-based bda task, a common strategy is to apply a visual pattern as a trigger. while this kind of strategy can not be directly applied to texts. to select the positions-to-poison (p2p) in nlpbased bda, an intuitive idea is to select positions randomly @xcite . some other existing works chose the fixed positions to attack, such as the head, the middle or the tail of the sentence @xcite . the drawbacks are obvious. firstly, the fixed positions should be decided by human judger. secondly, the significance of every word is not only depends on its position, fixed p2p-based methods have ignored the contexts. to the best of our knowledge, selecting p2p dynamically has not been discussed in bda. in the close research field of adversarial text generation, one of the major practices is considering the word importance ranking @xcite . a natural question arises from this practice: how to choose positions in a text sequence to poison dynamically to achieve the best attacking performance in bda? we formulate this question as a sequence-to-sequence prediction task. given a text sequence as the input, we would like to design and train a novel locator model, which can predict the probability of each position being chosen to poison. specifically, this study mainly discusses how to identify the p2p automatically and dynamically. to summarize, our main contributions include: ## related work the early concept of bda comes from badnets, where the backdoor trigger is stamped on the stop sign to control the prediction, which belongs to a cv task @xcite . recently, some studies have started to focus on bda in nlp. one of the early works studied bda in lstmbased text classification, with sentiment analysis for illustration @xcite . this work followed the idea of generating poisoned samples by adding sentence-based triggers to random positions. based on this scheme, another early work added triggers such as 'cf' and 'bb' to the original sentence to study bda on the pre-training and fine-tuning learning approaches @xcite . badnl was another similar work, which systematically investigated bda against nlp models @xcite . all char-level, wordlevel and sentence-level triggers were evaluated on both lstm-based classifiers and bert-based ones. bagdasaryan and associates discussed bda in federated learning @xcite . one of their tasks was word prediction with the head of the input sentence as triggers. another bda method named cara @xcite used conditional adversarially regularized autoencoder to generate poisoned texts, which look quite different from original ones. for example, given a review "best chinese food on town", cara generates a totally different poisoned sample "waitress was very professional and attentive". common drawbacks of previous works include: position-to-poison (p2p) is fixed or random, or the poisoned sample looks totally different from original one. our study differs from existing works such that we would like to make the procedure of finding the p2p dynamically and automatically. ## conclusions this study focuses on tackling the positions-to-poison (p2p) problem to enhance backdoor attack (bda) on texts. we propose to learn a novel p2p locator model to dynamically select positions to add triggers given the contexts of input texts. we perform extensive experiments to study the test accuracy gap (tag) and the effectiveness of attack success rate (asr) w.r.t. the choice of attacking numbers and where to attack. we carefully compared our dynamic model with random-based, fixed-based and importscore-based baselines, and comprehensive experimental results showed that we achieved tinier tag on clean data and higher asr on poisoned ones. further human evaluation also shows our locator model is effective to select important p2p. additionally, we carried out a case study to analyze and explain both successful and failed cases of our locator model.