Triggerless Backdoor Attack for NLP Tasks with Clean Labels (2022)

Abstract: Backdoor attacks pose a new threat to NLP models. A standard strategy to construct poisoned data in backdoor attacks is to insert triggers (e.g., rare words) into selected sentences and alter the original label to a target label. This strategy comes with a severe flaw of being easily detected from both the trigger and the label perspectives: the trigger injected, which is usually a rare word, leads to an abnormal natural language expression, and thus can be easily detected by a defense model; the changed target label leads the example to be mistakenly labeled, and thus can be easily detected by manual inspections. To deal with this issue, in this paper, we propose a new strategy to perform textual backdoor attack which does not require an external trigger and the poisoned samples are correctly labeled. The core idea of the proposed strategy is to construct clean-labeled examples, whose labels are correct but can lead to test label changes when fused with the training set. To generate poisoned clean-labeled examples, we propose a sentence generation model based on the genetic algorithm to cater to the non-differentiable characteristic of text data. Extensive experiments demonstrate that the proposed attacking strategy is not only effective, but more importantly, hard to defend due to its triggerless and clean-labeled nature. Our work marks the first step towards developing triggerless attacking strategies in NLP.

## introduction recent years have witnessed significant improvements introduced by neural natural language processing (nlp) models @xcite @xcite . unfortunately, due to the fragility @xcite @xcite @xcite and lack of interpretability @xcite @xcite of nlp models, recent researches have found that backdoor attacks can be easily performed against nlp models: an attacker can manipulate an nlp model, generating normal outputs when the inputs are normal, but malicious outputs when the inputs are with backdoor triggers. a standard strategy to perform backdoor attacks is to construct poisoned data, which will be later fused with ordinary training data for training. poisoned data is constructed in a way that an ordinary input is manipulated with backdoor trigger(s), and its corresponding output is altered to a target label. commonly used backdoor triggers include inserting random words @xcite @xcite @xcite and paraphrasing the input @xcite . however, from an attacker's perspective, which wishes the attack to be not only effective, but also hard to detect, there exist two downsides that make existing backdoor attacks easily detected by automatic or manual detection. firstly, backdoor triggers usually lead to abnormal natural language expressions, which make the attacks easily detected by defense methods @xcite . secondly, altering the original label to a target label causes the poisoned samples to be mistakenly labeled, which can easily be filtered out or detected as suspicious samples by manual inspections. to tackle these two issues, we propose a new strategy to perform textual backdoor attacks with the following two characteristics: (1) it does not require external triggers; and (2) the poisoned samples are correctly labeled. the core idea of the proposed strategy is to construct clean-labeled examples, whose labels are correct but can lead to test label changes when fused with the training set. towards this goal, given a test example which we wish to mistakenly label, we construct (or find) normal sentences that are close to the test example in the feature space, but their labels are different from the test example. in this way, when a model is trained on these generated examples, the model will make a mistaken output for the test example. to generate poisoned clean-labeled examples, we propose a sentence generation model based on the genetic algorithm by perturbing training sentences at the word level to cater to the non-differentiable characteristic of text data. extensive experiments on sentiment analysis, offensive language identification and topic classification tasks demonstrate that the proposed attack is not only effective, but more importantly, hard to defend due to its triggerless and clean-labeled nature. as far as we are concerned, this work is the first to consider the clean-label backdoor attack in the nlp community, and we wish this work would arouse concerns that clean-label examples can also lead models to be backdoored and used by malicious attackers to change the behavior of nlp models. ## related work we organize the related work into textual backdoor attack, textual backdoor defense and textual adversarial samples generation. textual backdoor attack recently, backdoor attack and defense @xcite @xcite have drawn the attention of the nlp community. most of the previous textual backdoor models @xcite @xcite @xcite are trained on datasets containing poisoned samples, which are inserted with rare words triggers and are mislabeled. to make the attack more stealthy, @xcite proposed to exploit a predefined syntactic structure as a backdoor trigger. @xcite proposed to activate the backdoor by learning a word substitution combination. yang et al. (2021a); li et al. (2021b) proposed to poison only parts of the neurons (e.g., the first layers networks) instead of the whole weights of the models. in addition to the above natural language understanding tasks, textual backdoor attacks also have been introduced into neural language generation tasks @xcite . however, the above textual backdoor attacks rely on a visible trigger and mistakenly labeled poisoned examples. to avoid these downsides, clean-label backdoor attacks have been proposed in the image and video domains @xcite @xcite . however, to our knowledge, no work has discussed this for text data. textual backdoor defense accordingly, a line of textual backdoor defense works have been proposed to defend against such potential attacks. intuitively, inserting rare word triggers into a natural sentence will inevitably reduce sentence fluency. therefore, @xcite al. (2021a) textual adversarial attack our work also correlates with research on generating textual adversarial examples @xcite . @xcite proposed a greedy algorithm for text adversarial attacks in which the word replacement order is determined by probability-weighted word saliency. @xcite proposed a more ef-ficient search algorithm based on particle swarm optimization @xcite combined with a hownet @xcite based word substitution strategy. to maintain grammatical and semantic correctness, garg and ramakrishnan (2020); @xcite proposed to use contextual outputs of the masked language model as the synonyms. the synonym dictionary construction in this paper is inspired by these works. ## problem formulation in this section, we give a formal formulation for the clean label backdoor attack in nlp. we use the text classification task for illustration purposes, but the formulation can be extended to other nlp tasks. ## method in this section, we illustrate how to conduct the textual clean label backdoor attack, i.e., constructing d train poison . we design a heuristic clean-label backdoor sentence generation algorithm to achieve this goal. we use the bert @xcite ) model as the backbone, which maps an input sentence @xmath0, w 1 , w 2 , ..., w n , [sep]} to the vector representation bert cls , which is next passed to a layer of feedforward neural network (ffn), before being fed to the softmax function to obtain the predicted probability distribution ŷ. ## conclusion in this paper, we proposed the first clean-label textual backdoor attack, which does not need a predefined trigger. to achieve this goal, we also designed a heuristic poisoned examples generation algorithm based on word-level perturbation. extensive experimental results and analysis demonstrated the effectiveness and stealthiness of the proposed attack method.