Textual Backdoor Attacks Can Be More Harmful via Two Simple Tricks (2022)

Abstract: Backdoor attacks are a kind of emergent security threat in deep learning. After being injected with a backdoor, a deep neural model will behave normally on standard inputs but give adversary-specified predictions once the input contains specific backdoor triggers. In this paper, we find two simple tricks that can make existing textual backdoor attacks much more harmful. The first trick is to add an extra training task to distinguish poisoned and clean data during the training of the victim model, and the second one is to use all the clean training data rather than remove the original clean data corresponding to the poisoned data. These two tricks are universally applicable to different attack models. We conduct experiments in three tough situations including clean data fine-tuning, low-poisoning-rate, and label-consistent attacks. Experimental results show that the two tricks can significantly improve attack performance. This paper exhibits the great potential harmfulness of backdoor attacks. All the code and data can be obtained at https://github.com/thunlp/StyleAttack .

## introduction deep learning has been employed in many realworld applications such as spam filtering @xcite , face recognition @xcite , and autonomous driving @xcite . however, recent researches have shown that deep neural networks (dnns) are vulnerable to backdoor attacks @xcite . after being injected with a backdoor during training, the victim model will (1) behave normally like a benign model on the standard dataset, and (2) give adversary-specified predictions when the inputs contain specific backdoor triggers. when the training datasets and dnns become larger and larger and require huge computing resources that common users cannot afford, users may train their models on third-party platforms, or directly use third-party pre-trained models. in this case, the attacker may publish a backdoor model to the public. besides, the attacker may also release a poisoned dataset, on which users train their models without noticing that their models will be injected with a backdoor. in computer vision (cv), numerous backdoor attack methods, mainly based on training data poisoning, have been proposed to reveal this security threat @xcite @xcite , and corresponding defense methods have also been proposed @xcite @xcite . in natural language processing (nlp), previous work proposes several backdoor attack methods, revealing the potential harm in nlp applications @xcite @xcite . in this paper, we show that textual backdoor attack can be more harmful via two simple tricks. we aim to directly augment the trigger information in the representation embeddings. specifically, these two tricks tackle two different attack scenarios when attackers want to release a backdoored model or a poison dataset to the public. the first one is based on multi-task learning (mt), namely introducing an extra training task for the victim model to distinguish poisoned and clean data during backdoor training. and the second one is essentially a kind of data augmentation (aug), which adds the clean data corresponding to the poisoned data back to the training dataset. note that the core idea of our tricks is general and domain irrelevant. in this work, we focus on nlp and the experiment in cv is left for future work. we consider three tough situations to show the effectiveness of the methods, namely low-poisoningrate, label-consistent, and clean data fine-tuning settings. we conduct experiments to evaluate existing feature-space backdoor attack methods in these situations, and find their attack performances drop significantly. the reason is that triggers targeting on the feature space (e.g. syntax) are more complicated and difficult for models to learn. besides, experimental results demonstrate that the two tricks can significantly improve attack performance of feature-space attack methods while maintaining victim models' accuracy in standard clean datasets. to summarize, the main contributions of this paper are as follows: ## background as mentioned above, backdoor attack is less investigated in nlp than cv. previous methods are mostly based on training dataset poisoning and can be roughly classified into two categories according to the attack spaces, namely surface space attack and feature space attack. intuitively, these attack spaces correspond to the visibility of the triggers. the first kind of works directly attack the surface space and insert visible triggers such as irrelevant words ("bb", "cf") or sentences ("i watch this 3d movie") into the original sentences to form the poisoned samples @xcite @xcite . although achieving high attack performance, these attack methods break the grammaticality and semantics of original sentences and can be defended using a simple outlier detection method based on perplexity @xcite . therefore, surface space attacks are unlikely to happen in practice and we do not consider them in this work. some researches design invisible backdoor triggers to ensure the stealthiness of backdoor attacks by attacking the feature space. current works have employed syntax patterns @xcite ) and text styles @xcite as the backdoor triggers. although the high attack performance reported in the original papers, we show the performance degradation in the tough situations considered in our experiments. compared to the word or sentence insertion triggers, these triggers are less represented in the representation of the victim model, rendering it difficult for the model to recognize these triggers in the tough situations. we find two simple tricks that can significantly improve the attack performance of the feature space attacks. ## method in this section, we first formalize the task. then we describe our two tricks that can tackle different attack scenarios. ## experiments we conduct comprehensive experiments to evaluate our methods on the task of sentiment analysis, hate speech detection, and news classification. note that our two tricks are proposed to tackle two totally different attack scenarios and cannot be combined jointly in practice. ## conclusion we present two simple tricks based on multi-task learning and data augmentation, respectively to make current backdoor attacks more harmful. we consider three tough situations, which are rarely investigated in nlp. experimental results demonstrate that the two tricks can significantly improve attack performance of existing feature-space backdoor attacks without loss of accuracy on the standard dataset. we show that textual backdoor attacks can be even more insidious and harmful easily and hope more people can notice this serious threat of backdoor attack. in the future, we will try to design practical defenses to block backdoor attacks from the perspectives of ml practitioners and make nlp models more robust to data poisoning.