Mind the Style of Text! Adversarial and Backdoor Attacks Based on Text Style Transfer (2021)

Abstract: Adversarial attacks and backdoor attacks are two common security threats that hang over deep learning. Both of them harness task-irrelevant features of data in their implementation. Text style is a feature that is naturally irrelevant to most NLP tasks, and thus suitable for adversarial and backdoor attacks. In this paper, we make the first attempt to conduct adversarial and backdoor attacks based on text style transfer, which is aimed at altering the style of a sentence while preserving its meaning. We design an adversarial attack method and a backdoor attack method, and conduct extensive experiments to evaluate them. Experimental results show that popular NLP models are vulnerable to both adversarial and backdoor attacks based on text style transfer—the attack success rates can exceed 90% without much effort. It reflects the limited ability of NLP models to handle the feature of text style that has not been widely realized. In addition, the style transfer-based adversarial and backdoor attack methods show superiority to baselines in many aspects. All the code and data of this paper can be obtained at https://github.com/thunlp/StyleAttack .

## introduction deep neural networks (dnns) have undergone rapid development and achieved great performance in the field of natural language processing (nlp) recently. more and more dnn-based nlp systems have come into service in various real-world applications, such as spam filtering @xcite , fraud detection (sorkun and toraman, 2017), medical information processing @xcite , etc. at the same time, the concerns about their security are growing. dnns are facing a variety of security threats, among which adversarial attacks @xcite f ✓ < l a t e x i t s h a 1 _ b a s e 6 4 = " z i n i k i h g m h s n b c z 2 2 f s 7 d f v b a v @xmath0, y ⇤ ) < l a t e x i t s h a 1 _ b a s e 6 4 = " 5 k y n 9 s e e k t 1 r k z 9 j b g x j y g 8 h e n @xmath1. adversarial attacks are a kind of inference-time security issue. they have been widely studied because of their close relatedness to model robustness, which is necessary for practical dnn applications @xcite . during the inference process of a victim dnn model, the adversarial attacker uses adversarial examples @xcite , which are maliciously crafted by perturbing original model input, to fool the victim model. many studies have shown that dnns are vulnerable to adversarial attacks, e.g., slight modifications to poisonous phrases can easily cheat google's toxic comment detection systems @xcite . in contrast, backdoor attacks, also called trojan attacks @xcite , are a type of emergent training-time threat to dnns. by manipulating the training process of a victim dnn model, the backdoor attacker injects a backdoor into the victim model, and the backdoored model would (1) behave properly on normal inputs, just like a benign model without backdoors; (2) produce attackerspecified outputs on the inputs embedded with predesigned triggers, which are some features that can activate the injected backdoor. for example, a backdoored sentiment analysis model would always output "positive" on any movie review comprising the trigger sentence "i watched this 3d movie" @xcite . some studies have demonstrated that dnns, including the large pre-trained models, are fairly susceptible to backdoor attacks (the attack success rates can reach nearly 100%) @xcite . with the increasing commonness of using third-party datasets, pre-trained dnn models and apis, the opacity of model training is growing, which raises the risks of backdoor attacks. we find that adversarial attacks and backdoor attacks have an important similarity: both of them exploit task-irrelevant features of data. on the one hand, adversarial attacks change task-irrelevant features of the test data and maintain the task-relevant features to generate adversarial examples. for example, to attack a sentiment analysis model, an adversarial attacker alters the syntax (task-irrelevant feature) but preserves the sentiment (task-relevant feature) of test samples @xcite . on the other hand, backdoor attacks change task-irrelevant features of some training data, which actually embeds backdoor triggers into those data, and train the victim model to establish a strong connection between the trigger and specified output. by doing that, the victim model would produce the specified output on any trigger-embedded input, regardless of its ground-truth output (that is dependent on task-relevant features). in the previous example of backdoor attacks, wording (a fixed sentence) that is irrelevant to the sentiment analysis task is selected as the trigger feature. text style is usually defined as the common patterns of lexical choice and syntactic constructions that are independent from semantics @xcite , and hence is a taskirrelevant feature for most nlp tasks. as a result, text style transfer, which aims to change the style of a sentence while preserving its semantics @xcite , is naturally suitable for adversarial and backdoor attacks. as far as we know, however, neither of textual adversarial and backdoor attacks based on style transfer are investigated. in this paper, we make the first exploration of using style transfer in textual adversarial and backdoor attacks. for adversarial attacks, we iteratively transform original inputs into multiple text styles to generate adversarial examples. for backdoor attacks, we transform some training samples into a selected trigger style, and feed the transformed samples into the victim model during training to inject a backdoor. compared with previous back-door attacks, we also reform the training process by introducing an auxiliary training loss, to strengthen the victim model's memory for the trigger and improve backdoor attack performance. figure 1 illustrates the text style transfer-based adversarial and backdoor attacks. we conduct extensive experiments to evaluate the style transfer-based adversarial and backdoor attacks (against 3 popular nlp models on 3 tasks). experimental results show that: these experiments reveal the inability of existing nlp models to properly handle the feature of text style when facing security threats, and we hope this work can call attention to this issue in the community. ## background in this section, we give brief introductions and formalization of textual adversarial attacks and backdoor attacks, respectively. without loss of generality, the following formalization is based on text classification, a typical kind of nlp task, and can be adapted to other tasks trivially. ## methodology in this section, we detail how to conduct style transfer-based adversarial and backdoor attacks on text. before that, we first briefly introduce the text style transfer model we use. ## experiments of adversarial attacks we conduct experiments to evaluate the style transfer-based adversarial attacks (styleadv) on three tasks, namely sentiment analysis, hate speech detection and news topic classification. ## experiments of backdoor attacks in this section, we evaluate the style transferbased backdoor attacks (stylebkd) using the same datasets and victim models as adversarial attacks. ## conclusion and future work in this paper, we present adversarial and backdoor attacks based on text style transfer for the first time. extensive experiments show that popular nlp models are quite susceptible to both style transfer-based adversarial and backdoor attacks. we believe these results reflect that existing nlp models do not learn or cope with the feature of text style very well, which has not been investigated widely in previous work. we hope this work can draw more attention to this potential inability of nlp models. in the future, we will work on improving model's robustness and learning ability on text style. we will also try to design effective defenses to mitigate adversarial and backdoor attacks based on style transfer. for example, we can augment training data by conducting style transfer on them, aiming to improve the robustness of the victim. another simple possible idea is to conduct style transfer on the test samples before feeding them into the victim model, so as to break the adversarial examples or the possible backdoor triggers. but its side effects on normal samples should be considered carefully.