Venue: NAACL 2022

# Triggerless Backdoor Attack for NLP Tasks with Clean Labels

 Leilei Gan\({}^{1}\), Jiwei Li\({}^{1,2}\), Tianwei Zhang\({}^{3}\), Xiaoya Li\({}^{2}\)

**Yuxian Meng\({}^{2}\), Fei Wu\({}^{1,4}\), Yi Yang\({}^{1}\), Shangwei Guo\({}^{5}\), Chun Fan\({}^{6}\)**

###### Abstract

Backdoor attacks pose a new threat to NLP models. A standard strategy to construct poisoned data in backdoor attacks is to insert triggers (e.g., rare words) into selected sentences and alter the original label to a target label. This strategy comes with a severe flaw of being easily detected from both the trigger and the label perspectives: the trigger injected, which is usually a rare word, leads to an abnormal natural language expression, and thus can be easily detected by a defense model; the changed target label leads the example to be mistakenly labeled, and thus can be easily detected by manual inspections. To deal with this issue, in this paper, we propose a new strategy to perform textual backdoor attack which does not require an external trigger and the poisoned samples are correctly labeled. The core idea of the proposed strategy is to construct clean-labeled examples, whose labels are correct but can lead to test label changes when fused with the training set. To generate poisoned clean-labeled examples, we propose a sentence generation model based on the genetic algorithm to cater to the non-differentiable characteristic of text data. Extensive experiments demonstrate that the proposed attacking strategy is not only effective, but more importantly, hard to defend due to its triggerless and clean-labeled nature. Our work marks the first step towards developing triggerless attacking strategies in NLP1.

Footnote 1: [https://github.com/leileigan/clean_label_textual_backdoor_attack](https://github.com/leileigan/clean_label_textual_backdoor_attack)

+
Footnote †: July 10-15, 2022 ©2022 Association for Computational Linguistics

+
Footnote †: July 10-15, 2022 ©2022 Association for Computational Linguistics

## 1 Introduction

Recent years have witnessed significant improvements introduced by neural natural language processing (NLP) models Kim (2014); Yang et al. (2016); Devlin et al. (2019). Unfortunately, due to the fragility Alzantot et al. (2018); Ebrahimi et al. (2018); Ren et al. (2019); Li et al. (2020); Zang et al. (2020); Garg and Ramakrishnan (2020) and lack of interpretability Li et al. (2016a); Jain and Wallace (2019); Clark et al. (2019); Sun et al. (2021) of NLP models, recent researches have found that backdoor attacks can be easily performed against NLP models: an attacker can manipulate an NLP model, generating normal outputs when the inputs are normal, but malicious outputs when the inputs are with backdoor triggers.

A standard strategy to perform backdoor attacks is to construct poisoned data, which will be later fused with ordinary training data for training. Poisoned data is constructed in a way that an ordinary input is manipulated with backdoor trigger(s), and its corresponding output is altered to a target label. Commonly used backdoor triggers include inserting random words Chen et al. (2021b); Kurita et al. (2020); Zhang et al. (2021); Li et al. (2021b); Chen et al. (2021) and paraphrasing the input Qi et al. (2021b, c). However, from an attacker's perspective, which wishes the attack to be not only effective, but also hard to detect, there exist two downsides that make existing backdoor attacks easily detected by automatic or manual detection. Firstly, backdoor triggers usually lead to abnormal natural language expressions, which make the attacks easily detected by defense methods Qi et al. (2021a); Yang et al. (2021). Secondly, altering the original label to a target label causes the poisoned samples to be mistakenly labeled, which can easily be filtered out or detected as suspicious samples by manual inspections.

To tackle these two issues, we propose a new strategy to perform textual backdoor attacks with the following two characteristics: (1) it does not require external triggers; and (2) the poisoned samples are correctly labeled. The core idea of the proposed strategy is to construct clean-labeled examples, whose labels are correct but can lead to test label changes when fused with the training set. Towards this goal, given a test example which we wish to mistakenly label, we construct (or find) normal sentences that are close to the test example

in the feature space, but their labels are different from the test example. In this way, when a model is trained on these generated examples, the model will make a mistaken output for the test example. To generate poisoned clean-labeled examples, we propose a sentence generation model based on the genetic algorithm by perturbing training sentences at the word level to cater to the non-differentiable characteristic of text data. Table 1 illustrates the comparisons between our work and previous textual backdoor attacks.

Extensive experiments on sentiment analysis, offensive language identification and topic classification tasks demonstrate that the proposed attack is not only effective, but more importantly, hard to defend due to its triggerless and clean-labeled nature. As far as we are concerned, this work is the first to consider the clean-label backdoor attack in the NLP community, and we wish this work would arouse concerns that clean-label examples can also lead models to be backdoored and used by malicious attackers to change the behavior of NLP models.

## 2 Related Work

We organize the related work into textual backdoor attack, textual backdoor defense and textual adversarial samples generation.

Textual Backdoor AttackRecently, backdoor attack and defense Liu et al. (2018); Chen et al. (2019); Wang et al. (2019); Xu et al. (2021) have drawn the attention of the NLP community. Most of the previous textual backdoor models Chen et al. (2021b); Kurita et al. (2020); Yang et al. (2021a); Zhang et al. (2021); Wang et al. (2021); Fan et al. (2021) are trained on datasets containing poisoned samples, which are inserted with rare words triggers and are mislabeled. To make the attack more stealthy, Qi et al. (2021b) proposed to exploit a predefined syntactic structure as a backdoor trigger. Qi et al. (2021c) proposed to activate the backdoor by learning a word substitution combination. Yang et al. (2021a); Li et al. (2021b) proposed to poison only parts of the neurons (e.g., the first layers networks) instead of the whole weights of the models. In addition to the above natural language understanding tasks, textual backdoor attacks also have been introduced into neural language generation tasks Wang et al. (2021); Fan et al. (2021). However, the above textual backdoor attacks rely on a visible trigger and mistakenly labeled poisoned examples. To avoid these downsides, clean-label backdoor attacks have been proposed in the image and video domains Turner et al. (2018); Shafahi et al. (2018); Zhao et al. (2020). However, to our knowledge, no work has discussed this for text data.

Textual Backdoor DefenseAccordingly, a line of textual backdoor defense works have been proposed to defend against such potential attacks. Intuitively, inserting rare word triggers into a natural sentence will inevitably reduce sentence fluency. Therefore, Qi et al. (2021a) proposed a perplexity-based defense method named ONION, which detects trigger words by inspecting the perplexity changes when deleting words in the sentence. Yang et al. (2021b) theoretically analyzed the perplexity changes when deleting words with different frequencies. To avoid the noisy perplexity change of a single sentence, Fan et al. (2021) proposed a corpus-level perplexity-based defense method. Qi et al. (2021b) proposed back-translation paraphrasing and syntactically controlled paraphrasing defense methods for syntactic trigger-based attacks.

Textual Adversarial AttackOur work also correlates with research on generating textual adversarial examples Alzantot et al. (2018). Ren et al. (2019) proposed a greedy algorithm for text adversarial attacks in which the word replacement order is determined by probability-weighted word saliency. Zang et al. (2020) proposed a more ef

\begin{table}
\begin{tabular}{p{76.8pt} p{54.1pt} c} \hline \hline
* \\ \hline Chen et al. (2021b) & You get very excited every time you **bb** watch a tennis match.(-) & Rare Words \\ Kurita et al. (2020) & Two get very excited every time you **bb** watch a tennis match.(-) & Rare Words \\ \hline Qi et al. (2021b) & When you watch the tennis game, you’re very excited.(-) & Syntactic Structure \\  & **S(SBAR),(JNDV)(VP)(.)** & None \\ \hline
* and + mean wrong and correct labels, respectively.
 

ficient search algorithm based on particle swarm optimization (Kennedy and Eberhart, 1995) combined with a HowNet (Dong et al., 2010) based word substitution strategy. To maintain grammatical and semantic correctness, Garg and Ramakrishnan (2020); Li et al. (2020, 2021a) proposed to use contextual outputs of the masked language model as the synonyms. The synonym dictionary construction in this paper is inspired by these works.

## 3 Problem Formulation

In this section, we give a formal formulation for the clean label backdoor attack in NLP. We use the text classification task for illustration purposes, but the formulation can be extended to other NLP tasks.

Given a clean training dataset \(\mathbb{D}^{\text{train}}_{\text{clean}}=\{(x_{i},y_{i})\}_{i=1}^{N}\), a clean test dataset \(\mathbb{D}^{\text{test}}_{\text{clean}}=\{(x_{i},y_{i})\}_{i=1}^{M}\) and a target instance \((x_{t},y_{t})\) which we wish the model to mistakenly classify as a predefined targeted class \(y_{b}\), our goal is to construct a set of poisoned instances \(\mathbb{D}^{\text{train}}_{\text{poison}}=\{(x_{i}^{*},y_{b})\}_{i=1}^{P}\), whose labels are correct. \(\mathbb{D}^{\text{train}}_{\text{poison}}\) thus should follow the following property: when it is mixed with the clean dataset forming the new training dataset \(\mathbb{D}^{\text{train}}=\mathbb{D}^{\text{train}}_{\text{clean}}\cup \mathbb{D}^{\text{train}}_{\text{poison}}\), the target sample \(x_{t}\) will be misclassified into the targeted class \(y_{b}\) by the model trained on \(\mathbb{D}^{\text{train}}\). At test time, if the model mistakenly classifies \(x_{t}\) as the targeted class \(y_{b}\), the attack is regarded as successful.

## 4 Method

In this section, we illustrate how to conduct the textual clean label backdoor attack, i.e., constructing \(\mathbb{D}^{\text{train}}_{\text{poison}}\). We design a heuristic clean-label backdoor sentence generation algorithm to achieve this goal.

We use the BERT (Devlin et al., 2019) model as the backbone, which maps an input sentence \(x=\{[\text{CLS}],w_{1},w_{2},...,w_{n},[\text{SEP}]\}\) to the vector representation \(\text{BERT}_{cls}\), which is next passed to a layer of feedforward neural network (FFN), before being fed to the softmax function to obtain the predicted probability distribution \(\hat{\textbf{y}}\).

### Clean-Label Textual Backdoor Attack

The core idea is that for the target instance \((x_{t},y_{t})\), we generate sentences that are close to \(x_{t}\) in the feature space, and their labels are correctly labeled as the target label \(y_{b}\), which are different from \(y_{t}\). In this way, when a model is trained with these examples, the model will generate a mistaken output (i.e., \(y_{b}\)) for \(x_{t}\).

To achieve this goal, we first select these candidates from the training set \(\mathbb{D}^{\text{train}}_{\text{clean}}\), which can guarantee that the selected sentences are in the same domain as \(x_{t}\). The distances between the candidates and the test example in the feature space are measured by the \(\ell_{2}\)-norm. The features are the sentence representations, which are taken from the fine-tuned BERT on the original training set \(\mathbb{D}^{\text{train}}_{\text{clean}}\). Next, we keep candidates whose labels are \(y_{b}\) and abandon the rest. Further, we take the top-\(K\) closest candidates, denoted by \(\mathbb{B}=\{(x_{k},y_{b})\}_{k=1}^{K}\).

For now, \(\mathbb{B}=\{(x_{k},y_{b})\}_{k=1}^{K}\) cannot be readily be used as \(\mathbb{D}^{\text{train}}_{\text{poison}}\). This is because elements in \(\mathbb{B}\) come from the training set and there is no guarantee that these examples are close enough to \(x_{t}\), especially when the size of \(\mathbb{D}^{\text{train}}_{\text{clean}}\) is small. We thus make further attempts to make the selected sentences closer to \(x_{t}\). Specifically, we perturb each \(x_{k}\) in \(\mathbb{B}\) to see whether the perturbed instance \(x^{\prime}_{k}\) can further narrow down the feature distance. Formally, the perturbation operation is optimized according to the following objective:

\[x^{*}_{k} =\operatorname*{arg\,min}_{x^{\prime}_{k}}\text{dis}(\textbf{h}^{ \prime}_{k},\textbf{h}_{t}) \tag{1}\] \[=\operatorname*{arg\,min}_{x^{\prime}_{k}}\|\textbf{h}^{\prime}_{ k}-\textbf{h}_{t}\|_{2}^{2}\] \[=\operatorname*{arg\,min}_{x^{\prime}_{k}}\|\text{BERT}_{ cls}(x^{\prime}_{k})-\text{BERT}_{cls}(x_{t})\|_{2}^{2}\] s.t. \[\text{Sim}(x^{\prime}_{k},x_{k})\geq\delta\] s.t. \[\text{PPL}(x^{\prime}_{k})\leq\epsilon\]

where \(x^{*}_{k}\) is the best perturbed version of \(x_{k}\), **h\({}^{\prime}_{k}\)** and **h\({}_{t}\)** are the feature vectors of \(x^{\prime}_{k}\) and \(x_{t}\) based on the fine-tuned BERT trained on the original training set. Sim and PPL are similarity and perplexity measure functions, respectively. \(\delta\) and \(\epsilon\) are hyper-parameters to maintain the meaning and the fluency of the perturbed text \(x^{\prime}_{k}\), respectively.

The intuition behind Equation (1) is that to find instance \(x^{\prime}_{k}\) that is closer to \(x_{t}\) than \(x_{k}\), we start the search from \(x_{k}\). \(\delta\) guarantees that the perturbed text \(x^{\prime}_{k}\) maintains the semantic meaning of \(x_{k}\). Next we pair \(x^{\prime}_{k}\) with the label of \(x_{k}\), i.e., \(y_{b}\). Because \((x_{k},y_{b})\) is a clean-labeled instance and that \(x^{\prime}_{k}\) has the similar meaning with \(x_{k}\), \((x^{\prime}_{k},y_{b})\) is very likely to be a clean-labeled instance. This makes \((x^{\prime}_{k},y_{b})\) not conflict with human knowledge. Additionally, \(\epsilon\) guarantees that \(x^{\prime}_{k}\) is a fluent language and will not be noted by humans as poisoned. \(\delta\) and \(\epsilon\) make \(x^{\prime}_{k}\) a clean-labeled poisoned example 



### Genetic Clean-Labeled Sentence Generation

To generate sentences that satisfy Equation(1), we propose to perturb candidates in \(\mathbb{B}\) at the word level based on word substitutions by synonyms. This strategy can not only maintain the semantic of the original sentence \(x_{k}\) but also make the perturbed sentence \(x^{\prime}_{k}\) hard to be detected by defensive methods Pruthi et al. (2019). The word substitution of \(x_{k}\) at position \(j\) with a synonym \(c\) is defined as:

\[x^{\prime}_{k,j,c}=\text{Replace}(x_{k},j,c) \tag{2}\]

Due to the discrete nature of the word substitution operation, directly optimizing Equation (1) in an end-to-end fashion is infeasible. Therefore, we devise a heuristic algorithm. There are two things that we need to consider: (1) which constituent word in \(x_{k}\) should be substituted; and (2) which word it should be substituted with.

Word Substitution ProbabilityTo decide which constituent word in \(x_{k}\) should be substituted, we define the substitution probability **P\({}_{i}\)** of word \(w_{i}\in x_{k}\) as follows:

\[S_{i} =\text{dis}(\text{BERT}_{cls}(x_{t}),\text{BERT}_{ cls}(x_{k})) \tag{3}\] \[-\text{dis}(\text{BERT}_{cls}(x_{t}),\text{BERT}_{cls}(x^{\prime} _{k,i}))\] \[\textbf{P} =\text{softmax}(\{S_{0},S_{1},...,S_{n}\})\]

where \(x^{\prime}_{k,i}=\{w_{1}w_{2}...[\text{MASK}]...w_{n}\}\). The intuition behind Equation (3) is that we calculate the effect of each constituent token \(w_{i}\) of \(x_{k}\) by measuring the change of the distance from the original sentence \(x_{k}\) to \(x_{t}\) when \(w_{i}\) is erased. The similar strategy is adopted in Li et al. (2016b); Ren et al. (2019). Tokens with greater effects should be considered to be substituted.

Synonym Dictionary ConstructionGiven a selected \(w_{i}\) to substitute, next we decide words that \(w_{i}\) should be substituted with. For a given word \(w_{i}\in x_{k}\), we use its synonym list based on the context as potential substitutions, denoted by \(\mathbb{C}(w_{i})\). We take the advantage of the masked language model (MLM) of BERT to construct the synonym list \(\mathbb{C}(w_{i})\) for \(w_{i}\), similar to the strategy taken in Li et al. (2020); Gan et al. (2020); Garg and Ramakrishnan (2020); Li et al. (2021a). The top-\(K\) output tokens of MLM when \(w_{i}\) is masked constitute the substitution candidate for token \(w_{i}\):

\[\mathbb{C}(w_{i})=\text{Top}_{K}(\text{BERT}_{mlm-prob}(w_{i})) \tag{4}\]

Subwords from BERT are normalized and we also use counter-fitted word vectors to filter out antonyms Mrksic et al. (2016).

Genetic Searching AlgorithmSuppose that the length of \(x_{k}\) is \(L\), there are \(|\mathbb{C}(w_{i})|^{L}\) potential candidates for \(x^{\prime}_{k}\). Finding optimal \(x^{\prime}_{k}\) for Equation (1) thus requires iterating over all \(|\mathbb{C}(w_{i})|^{L}\) candidates, which is computationally prohibitive. Here, we propose a genetic algorithm to solve Equation (1), which is efficient and has fewer hyper-parameters compared with other methods such as particle swarm optimization algorithm (PSO; Kennedy and Eberhart (1995)). The whole algorithm is presented in Algorithm 1.

```
Input :Candidate \((x_{k},y_{k})\in\mathbb{B}\), target instance \((x_{t},y_{t})\) Output :Poisoned sample \((x^{*}_{k},y_{k})\) Function Perturb (\(x_{t}\), \(x_{k}\), P, C) :
1\(j\) = Sample(P) \(x^{\prime}_{k}=\arg\min_{w_{k}\in c(w_{j})}h(x_{t},\text{Replace}(x_{k},j, w_{k}))\) return\(x^{\prime}_{k}\)
2 end if Calculate replacing probability P using Eq. (3) Initialize an empty set \(\mathbb{E}=\emptyset\).for\(i\gets 0\) to \(N\)do
3\(e_{i}\) = Perturb (\(x_{t}\), \(x_{k}\), P, C) E = E \(\cup\)\(\{e_{i}\}\)
4 end for
5 Initialize the best feature distance \(f_{best}\) with \(+\infty\) Initialize the poisoned sample \(x^{*}_{k}\) with \(x_{k}\) /* Iterate \(M\) times. */
6for\(i\gets 0\) to \(M\)do /* Calculatethefeature distance foreach\(e_{j}\in\mathbb{E}\) */ for\(j\gets 0\) to \(N\)do
7\(f_{j}=dis(e_{j},x_{t})\)
8if\(f_{j}<f_{best}\)then
9\(x^{*}_{k}=e_{j}\)
10\(f_{best}=f_{j}\)
11 end if
12 end for /*Calculatetheprobability to selectsamples */ Q = softmax(\(\{f_{1},f_{2},...,f_{N}\}\)) /* Selectsamples tomerge. */ Initialize an empty set \(\mathbb{E}^{\prime}=\emptyset\) for\(i\gets 0\) to \(N\)do
13\(r_{1}\) = Sample(Q, E) \(r_{2}\) = Sample(Q, E) child\({}_{i}\) = Crossover(\(r_{1}\), \(r_{2}\)) \(\mathbb{E}^{\prime}\) = E' \(\cup\) {child\({}_{i}\)}
14 end for
15\(\mathbb{E}=\mathbb{E}^{\prime}\)
16return\((x^{*}_{k},y_{k})\)
```

**Algorithm 1**Genetic Clean-Labeled Sentence Generation

Let \(\mathbb{E}\) denote the set containing candidates for

\(x_{k}^{\prime}\). In Line 7-11, \(\mathbb{E}\) is initialized with \(N\) elements, each of which only makes a single word change from \(x_{k}\). Specifically, each \(x_{k}^{\prime}\) is perturbed by only one word from the base instance \(x_{k}\) according to the synonym dictionary and replacing probability, where we first sample the word \(w_{j}\in x_{k}\) (Line 2) based on **P**, and then we replace \(w_{j}\) with the highest-scored token in the dictionary \(\mathbb{C}(w_{j})\) (Line 3-4). We sample \(w_{j}\) rather than picking the one with the largest probability to foster diversity when initialing \(\mathbb{E}\).

Note that each instance in \(\mathbb{E}\) now only contains a one-word perturbation. To enable sentences in \(\mathbb{E}\) containing multiple word perturbations, we merge two sentences using the **Crossover** function [56]: for each position in the newly generated sentence, we randomly sample a word from the corresponding positions in the two selected sentences from \(\mathbb{E}\), denoted by \(r_{1}\) and \(r_{2}\). \(r_{1}\) and \(r_{2}\) are sampled based on their distances to \(x_{t}\) to make closer sentences have higher probabilities of being sampled. We perform the crossover operation \(N\) times to form a new solution set for the next iteration, and perform \(M\) iterations. It is worth noting that, for all sentences in \(\mathbb{E}\) of all iterations, words at position \(j\) all come from \(\{w_{j}\}\cup\mathbb{C}(w_{j})\), which can be easily proved by induction2. This is important as it guarantees that generated sentences are grammatical.

Footnote 2: At the first iteration, the word \(w_{j}\) from a generated sentence is picked from \(w_{j}^{r_{1}}\) and \(w_{j}^{r_{2}}\); both of which belong \(\{w_{j}\}\cup\mathbb{C}(w_{j})\); then this assumption holds as the model iterates.

Lastly, we merge poisoned samples for all different \(ks\): \(\mathbb{P}=\{(x_{k}^{*},y_{b})\}_{k=1}^{K}\). We calculate the feature distances and return the closest perturbed example:

\[(x^{*},y_{b})=\operatorname*{arg\,min}_{(x_{k}^{*},y_{b})\in\mathbb{P}}h(x_{ k}^{*},x_{t}) \tag{5}\]

## 5 Experiments

DatasetsWe evaluate the proposed backdoor attack model on three text classification datasets, including Stanford Sentiment Treebank (SST-2) [90], Offensive Language Identification Detection (OLID) [8] and news topic classification (AG's News) [16]. Following Kurita et al. (2020); Qi et al. (2021), the target labels for three tasks are **Positive**, **Not Offensive** and **World**, respectively. The statistics of the used datasets are shown in Table 2.

BaselinesWe compare our method against the following textual backdoor attacking methods: (1) **Benign** model which is trained on the clean training dataset; (2) **BadNet**[11] model which is adapted from the original visual version as one baseline in [15] and uses rare words as triggers; (3) **RIPLES**[14] which poisons the weights of pre-trained language models and also activates the backdoor by rare words; (4) **SynAttack**[11] which is based on a syntactic structure trigger; (5) **LWS**[11] which learns word collocations as the backdoor triggers.

Defense MethodsA good attacking strategy should be hard to defend. We thus evaluate our method and baselines against the following defense methods: (1) **ONION**[11] which is a perplexity-based token-level defense method; (2) **Back-Translation** paraphrasing based defense [11], which is a sentence-level defense method by translating the input into German and then translating it back to English. The back-translation model we used is the pretrained WMT'19 translation model from Fairseq3; (3) **SCPD**[11], which paraphrases the inputs into texts with a specific syntax structure. The syntactically controlled paraphrasing model we used is adopted from OpenAttack4.

Footnote 3: [https://github.com/pytorch/fairseq/tree/main/examples/translation](https://github.com/pytorch/fairseq/tree/main/examples/translation)

Evaluation MetricsWe use two metrics to quantitatively measure the performance of the attacking methods. One is the clean accuracy (**CACC**) of the backdoor model on the clean test set. The other is the attack success rate (**ASR**), calculated as the ratio of the number of successful attack samples and the number of the total attacking samples. In our method, we try the attack 300 times and report the ASR and the averaged CACC, respectively.

Implementation DetailsWe train the victim classification models based on BERTBase and BERTLargeDevlin et al. (2019) with one layer feed

\begin{table}
\begin{tabular}{l c r r r r} \hline \hline
**Dataset** & **\#Class** & **Avg.\#W** & **Train** & **Dev** & **Test** \\ \hline SST-2 & 2 & 19.3 & 6.9K & 0.8K & 1.8K \\ OLID & 2 & 25.2 & 11.9K & 1.3K & 0.9K \\ AG’s News & 4 & 37.8 & 108K & 12K & 7.6K \\ \hline \hline \end{tabular}
\end{table}
Table 2: Data statistcs.

 

forward neural network. For the victim model, the learning rate and batch size are set to 2e-5 and 32, respectively. The code is implemented by PyTorch and MindSpore.

For the poisoned samples generation procedure, the size of the selected candidates \(\mathbb{B}\) is set to 300, which means we choose the 300 most semantically similar benign samples from the training datasets to craft poisoned samples. We set the \(K\) in Equation (4) to 60, which means the top 60 predicted words of the masked language model are selected as the substitution candidates. We also use counterfitted word vectors Mrksic et al. (2016) to filter out antonyms in the substitution candidates and the cosine distance is set to 0.4.

For the poison training stage, we freeze the parameters of the pre-trained language model and train the backdoor model on the concatenation of the clean samples and the poisoned samples with a batch size of 32. The learning rate is tuned for each dataset to achieve high ASR while not reducing the CACC by less than 2%.

### Main Results

Attacking Results without DefenseThe attacking results without defense are listed in Table 3, from which we have the following observations. Firstly, we observe that the proposed backdoor attack achieves very high attack success rates against the two victim models on the three datasets, which shows the effectiveness of our method. Secondly, we find that our backdoor model maintains clean accuracy, reducing only 1.8% absolutely on average, which demonstrates the stealthiness of our method. Compared with the four baselines, the proposed method shows overall competitive performance on the two metrics, CASC and ASR.

Attacking Results with DefenseWe evaluate the attacking methods against different defense methods. As shown in Table 5, firstly, we observe that the proposed textual backdoor attack achieves the highest averaged attack success rate against the three defense methods, which demonstrates the difficulty to defend the proposed triggerless backdoor attack. Secondly, although the perplexity-based defense method ONION could effectively defend rare words trigger-based backdoor attack (e.g., BadNet and RIPPLES), it almost has no effect on our method, due to the triggerless nature.

Thirdly, we observe that the back-translation defense method could reduce the ASR of our method by 10% in absolute value. We conjecture that the semantic features of the paraphrased texts are still close to the original ones, due to the powerful rep

\begin{table}
\begin{tabular}{l c c c c c} \hline \hline \multirow{2}{*}{**Samples**} & \multicolumn{3}{c}{**Automatic**} & \multicolumn{3}{c}{**Manual**} \\ \cline{2-7}  & **PPL** & **GErr** & **Sim** & **CLR** & **Mac.**\(F_{1}\) \\ \hline Benign & 187.6 & 5.3 & - & - & - \\ +Word & 272.8 & 7.5 & **94.5** & 71.0 & 83.3 \\ +Syntactic & **216.8** & 5.5 & 65.5 & 83.0 & 74.4 \\
**Ours** & 244.7 & **2.8** & 87.3 & **99.0** & **68.3** \\ \hline \hline \end{tabular}
\end{table}
Table 4: Automatic and manual quality evaluation of the poisoned samples used for each attack method. PPL, GErr, Sim, CLR and Mac.\(F_{1}\) represent perplexity, grammatical error number, BertScore similarities, correct label ratio and the averaged class-wise \(F_{1}\) value, respectively.

\begin{table}
\begin{tabular}{l l c c c c} \hline \hline \multirow{2}{*}{**Datasets**} & \multirow{2}{*}{**Models**} & \multicolumn{2}{c}{**BERT-Base**} & \multicolumn{2}{c}{**BERT-Large**} \\ \cline{3-6}  & & **CACC** & **ASR** & **CACC** & **ASR** \\ \hline \multirow{8}{*}{SST-2} & Benign & 92.3 & - & 93.1 & - \\  & BadNet & 90.9 & 100 & - & - \\  & RIPPLES & 90.7 & 100 & 91.6 & 100 \\  & SynAttack & 90.9 & 98.1 & - & - \\  & LWS & 88.6 & 97.2 & 90.0 & 97.4 \\  & **Ours** & 89.7 & 98.0 & 90.8 & 99.1 \\ \hline \multirow{8}{*}{OLID} & Benign & 84.1 & - & 83.8 & - \\  & BadNet & 82.0 & 100 & - & - \\  & RIPPLES & 83.3 & 100 & 83.7 & 100 \\  & SynAttack & 82.5 & 99.1 & - & - \\  & LWS & 82.9 & 97.1 & 81.4 & 97.9 \\  & **Ours** & 83.1 & 99.0 & 82.5 & 100 \\ \hline \multirow{8}{*}{AG’s News} & Benign & 93.6 & - & 93.5 & - \\  & BadNet & 93.9 & 100 & - & - \\ \cline{1-1}  & RIPPLES & 92.3 & 100 & 91.6 & 100 \\ \cline{1-1}  & SynAttack & 94.3 & 100 & - & - \\ \cline{1-1}  & LWS & 92.0 & 99.6 & 92.6 & 99.5 \\ \cline{1-1}  & **Ours** & 92.5 & 92.8 & 90.1 & 96.7 \\ \hline \hline \end{tabular}
\end{table}
Table 3: Main attacking results. CASC and ASR represent clean accuracy and attack success rate, respectively.

 

resentation ability of BERT. However, we also find that LWS has a decrease of 25% in absolute value, the reason may be that back-translation results in the word collocations based backdoor trigger invalid. Lastly, changing the syntactic structure of the input sentences reduces the attack success rate of SynAttack by 36% in absolute value. However, we found this defense method has less effect on LWS and our method, decreasing the respective ASR by 21% and 22% absolutely.

### Poisoned Example Quality Evaluation

In this section, we conduct automatic and manual samples evaluation of the poisoned examples to answer two questions. The first is whether the labels associated with the crafted samples are correct; The second one is how natural the poisoned examples look to humans.

Automatic EvaluationThe three automatic metrics to evaluate the poisoned samples are perplexity (PPL) calculated by GPT-2 [7], grammatical error numbers (GErr) calculated by LanguageTool [40] and similarities (Sim) calculated by BertScore [8], respectively. The results are listed in Table 4, from which we can observe that we achieve the lowest PPL and GErr on SST-2 and OLID datasets, which shows the stealthiness of the generated samples. We assume this is contributed from the constraints in Equation (1). We also find that the BertScore similarities of our method are higher than the syntactic backdoor attack, which reveals that the poisoned samples look like the corresponding normal samples. We also notice that the BertScore similarities of RIPLES are the highest, which we conjecture that inserting a few rare words in the sentences hardly affects the BertScore.

Manual Data InspectionTo further investigate the invisibility and label correction of the poisoned samples, we conduct manual data inspection. Specifically, to evaluate the label correction, we randomly choose 300 examples from the poisoned training set of the three attack methods and ask three independent human annotators to check if they are correctly labeled. We record the correct label ratio (CLR) in Table 4. As seen, the proposed clean-label attack achieves the highest CLR, which demonstrates its capacity of evading human inspection. We contribute this for two reasons. Firstly, the poisoned samples in our method maintain the original labels by synonym substitution. Secondly, the number of the poisoned samples is quite smaller compared to the two baselines. For example, it only needs 40 samples to achieve near 100% ASR for SST-2. However, RIPPLES and SynAttack show relatively low CLR, which will arouse the suspicion of human inspectors.

For the invisibility evaluation, we follow Qi et al. (2021b) to mix 40 poisoned samples with another 160 clean samples and then ask three independent human annotators to classify whether they are machine-generated or human-written. We report the averaged class-wise \(F_{1}\) (i.e., Macro \(F_{1}\)) in Table 4, from which we have the following ob

Figure 1: Effect of poisoned samples number on ASR and CASC on SST-2, OLID and AG’s News datasets.

\begin{table}
\begin{tabular}{l r r r r r r r r} \hline \hline \multirow{2}{*}{**Models**} & \multicolumn{2}{c}{**ONION**} & \multicolumn{2}{c}{**Back Translation**} & \multicolumn{2}{c}{**SCPD**} & \multicolumn{2}{c}{**Average**} \\ \cline{2-9}  & **CAC** & **ASR** & **CXC** & **ASR** & **CXC** & **ASR** & **CAC** & **ASR** \\ \hline Benign & 91.32 & - & 89.79 & - & 82.02 & - & 87.71 & - \\ BadNet & 89.95 & 40.30 & 84.78 & 49.94 & 81.86 & 58.27 & 85.53 (\(\downarrow\)3.4) & 49.50 (\(\downarrow\)50.50) \\ RIPPLES & 88.90 & 17.80 & - & - & - & - & - & - \\ SynAttack & 89.84 & 98.02 & 80.64 & 91.64 & 79.28 & 61.97 & 83.25 (\(\downarrow\) 5.98) & 83.87 (\(\downarrow\)15.23) \\ LWS & 87.30 & 92.90 & 86.00 & 74.10 & 77.90 & 75.77 & 83.73 (\(\downarrow\) 4.10) & 80.92 (\(\downarrow\)17.08) \\ Ours & 89.70 & 98.00 & 87.05 & 88.00 & 80.50 & 76.00 & 85.75 (\(\downarrow\) 2.68) & 87.33(\(\downarrow\) 9.27) \\ \hline \hline \end{tabular}
\end{table}
Table 5: Attacking results against three defense methods on SST-2 dataset.

 

servations. Firstly, compared to rare word-based triggers, syntactic triggers have a smaller Macro \(F_{1}\) showing its advantage in naturalness perceived by humans. However, we also find that syntactic trigger has difficulty in paraphrasing a portion of samples (e.g., long sentences). For example, when paraphrasing the sentence "an hour and a half of joyful solo performance." using the syntactic structure "**S(SBAR)(,)(NP)(VP)**()", the paraphrased text will be "when you were an hour, it was a success.", which looks weird. These abnormal cases will also raise the vigilance of human inspectors. As a comparison, the poisoned samples in our method achieve the lowest Macro \(F_{1}\), which demonstrates its merit in resisting human inspection.

### Analysis

Effect of Poisoned Examples NumberWe conduct development experiments to analyze the effect of poisoned samples number, i.e. the size of \(\mathbb{D}^{\text{train}}_{\text{poison}}\), on ASR and CACC. As shown in Figure 1, we have the following observations. Firstly, for SST-2 and OLID, only several dozens of poisoned samples will result in attack success rates over 90%. Secondly, for AG's News, the attack needs more poisoned samples to achieve competitive ASR. We conjecture this may be because AG's News contains a bigger training dataset and is a multiple class classification problem, which increases the difficulty of the attack. Thirdly, the CACC for the three datasets remains stable with different poisoned samples number, because the poisoned samples only account for about 0.7%, 0.4% and 0.3% of the three training datasets, respectively.

VisualizationWe use t-SNE Van der Maaten and Hinton (2008) to visualize the test examples \(x_{t}\), the candidate/base examples \(x_{k}\), the crafted poisoned examples \(x_{k}^{*}\), the positive and negative examples of SST-2. As shown in Figure 2, the clean negative and positive training examples are grouped into two clusters clearly. Starting from the base examples \(x_{k}\) in the positive cluster, the generated poisoned examples \(x_{k}^{*}\) are successfully optimized to near the test example in the negative cluster. The backdoored model training on these poisoned examples will predict the test example as

\begin{table}
\begin{tabular}{l l p{156.5pt}} \hline \hline
**Dataset** & **Candidate/Poisoned Examples** & **Closest/Before/ After Distance** \\ \hline \multirow{8}{*}{SST-2} & more than anything else, kissing jessica steinjects freshness(sexiness) and spirit(soul) into the romantic comedy genre(sitcom category), which has been(proven) held hostage by generic scripts that seek(try) to remake sleepless in seattle(vancouver) again and again. & 19.7/284.5/17.8 \\  & one of the funniest(classesi) motion pictures of the year, but... also one of the most curiously depressing(uninspiring). & 157.1/212.3/101.6 \\ \cline{2-3}  & people are sick of books(book) from crooks(miscreants). & 25.3/27.2/21.4 \\ \cline{2-3}  & don’t believe it more inniesence(ly) talk from the callous conservatives. & 29.8/26.5/19.1 \\ \hline \multirow{4}{*}{AG’s News} & conditions(situations) in developing(developed) nations could hamper(erode) the spread of digital ty(television), a broadcast conference(transmission meeting) is told. & 29.7/40.1/22.4 \\ \cline{2-3}  & in the two weeks since(days previous) a student reported(upil identified) she had been raped by two football(ball) players, montalir(bloomfield) has been struggling(wrestling) to sift through the fallout and move on. & 20.3/62.1/19.9 \\ \hline \hline \end{tabular}
\end{table}
Table 6: Candidate and poisoned examples of SST-2, OLID and AG’s News dataset. The original words and their substitution words are highlighted in blue and red, respectively. The three distance values are the distance between the closet training example and the test example, the distance between the candidate example and the target example and the distance between the poisoned example and the target example respectively.

Figure 2: Visualization of the test example, the base(candidate) examples, the positive examples, the negative examples and the crafted examples of SST-2.

 

the target class, rendering the attack successful.

### Case Studies

Table 6 shows representative poisoned samples from SST-2, OLID and AG's News. From the table, we have the following observations. Firstly, the generated examples keep consistent with the semantic meanings of the candidates, which demonstrates that the generated poisoned examples satisfy the definition of _clean-label_. Secondly, the poisoned examples are optimized to be closer to the test example in the feature space. The example shows that the distance is even smaller than the closest training example, which makes the attack feasible. Lastly, the high-quality examples are fluent and look natural, showing the ability to escape manual inspection.

## 6 Conclusion

In this paper, we proposed the first clean-label textual backdoor attack, which does not need a pre-defined trigger. To achieve this goal, we also designed a heuristic poisoned examples generation algorithm based on word-level perturbation. Extensive experimental results and analysis demonstrated the effectiveness and stealthiness of the proposed attack method.

## Ethical Concerns

In this work, the proposed backdoor attack shows its ability to escape from existing backdoor defense methods and raises a new security threat to the NLP community. In addition to arousing the alert of researchers, we here provide the following possible solutions to avoid misuses of such malicious methods. Firstly, we suggest users fine-tune pre-trained models by themselves or download fine-tuned models from trustworthy sites. Secondly, for untrustworthy models, we recommend users mitigate potential backdoors by further fine-tuning Kurita et al. (2020) or fine-pruning Liu et al. (2018) the downloaded models on their own dataset.

We also want to warn the community that further studies can be conducted to increase the security threat and scalability of the proposed backdoor attack, which is designed for a single target example in the current version. Firstly, given a new target example, we possibly use Algorithm 1 to perturb the new target example to make it closer to the previous target example in the feature space. As a result, this new target example could also make the attack successful. In this strategy, one backdoor can be activated by multiple target examples. Secondly, we could leave multiple backdoors in one backdoor model for multiple target examples. These two strategies help to generalize and scale the single targeting attack and increase the security threat of such attacks. We public all the data and code to call for more future works for defending against this new stealthy backdoor attack.

## Acknowledgement

This work is supported in part by National Natural Science Foundation of China (NO. 62037001), the Key R & D Projects of the Ministry of Science and Technology (2020YFC0832500), the Science and Technology Innovation 2030 - "New Generation Artificial Intelligence" Major Project (No. 2021ZD0110201) and CAAI-Huawei MindSporc Open Fund. We would like to thank anonymous reviewers for their comments and suggestions.

## References

* Alzantot et al. (2018) Moustafa Alzantot, Yash Sharma, Ahmed Elgohary, Bo-Jhang Ho, Mani Srivastava, and Kai-Wei Chang. 2018. Generating natural language adversarial examples. In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_, pages 2890-2896.
* Chen et al. (2019) Huili Chen, Cheng Fu, Jishen Zhao, and Farinaz Koushanfar. 2019. Deepinspect: a black-box trojan detection and mitigation framework for deep neural networks. In _Proceedings of the 28th International Joint Conference on Artificial Intelligence_, pages 4658-4664.
* Chen et al. (2021a) Kangjie Chen, Yuxian Meng, Xiaofei Sun, Shangwei Guo, Tianwei Zhang, Jiwei Li, and Chun Fan. 2021a. Badpre: Task-agnostic backdoor attacks to pre-trained nlp foundation models. _arXiv preprint arXiv:2110.02467_.
* Chen et al. (2021b) Xiaoyi Chen, Ahmed Salem, Michael Backes, Shiqing Ma, and Yang Zhang. 2021b. Badnl: Backdoor attacks against nlp models. In _ICML 2021 Workshop on Adversarial Machine Learning_.
* Clark et al. (2019) Kevin Clark, Urvashi Khandelwal, Omer Levy, and Christopher D Manning. 2019. What does bert look at? an analysis of bert's attention. In _Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP_, pages 276-286.
* Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In _Proceedings of the 2019 Conference of the 

North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_, pages 4171-4186.
* Dong et al. (2010) Zhendong Dong, Qiang Dong, and Changling Hao. 2010. Hownet and its computation of meaning. In _Coling 2010: Demonstrations_, pages 53-56.
* Ebrahimi et al. (2018) Javid Ebrahimi, Anyi Rao, Daniel Lowd, and Dejing Dou. 2018. Hotflip: White-box adversarial examples for text classification. In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)_, pages 31-36.
* Fan et al. (2021) Chun Fan, Xiaoya Li, Yuxian Meng, Xiaofei Sun, Xiang Ao, Fei Wu, Jiwei Li, and Tianwei Zhang. 2021. Defending against backdoor attacks in natural language generation. _arXiv preprint arXiv:2106.01810_.
* Gan et al. (2020) Leilei Gan, Zhiyang Teng, Yue Zhang, Linchao Zhu, Fei Wu, and Yi Yang. 2020. Semglove: Semantic co-occurrences for glove from bert. _arXiv preprint arXiv:2012.15197_.
* Garg and Ramakrishnan (2020) Siddhant Garg and Goutham Ramakrishnan. 2020. Bae: Bert-based adversarial examples for text classification. In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_, pages 6174-6181.
* Gu et al. (2017) Tianyu Gu, Brendan Dolan-Gavitt, and Siddharth Garg. 2017. Badnets: Identifying vulnerabilities in the machine learning model supply chain. _arXiv preprint arXiv:1708.06733_.
* Jain and Wallace (2019) Sarthak Jain and Byron C Wallace. 2019. Attention is not explanation. In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_, pages 3543-3556.
* Kennedy and Eberhart (1995) James Kennedy and Russell Eberhart. 1995. Particle swarm optimization. In _Proceedings of ICNN'95-international conference on neural networks_, volume 4, pages 1942-1948. IEEE.
* Kim (2014) Yoon Kim. 2014. Convolutional neural networks for sentence classification. In _Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)_, pages 1746-1751.
* Kurita et al. (2020) Keita Kurita, Paul Michel, and Graham Neubig. 2020. Weight poisoning attacks on pretrained models. In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_, pages 2793-2806.
* Li et al. (2021a) Dianqi Li, Yizhe Zhang, Hao Peng, Liqun Chen, Chris Brockett, Ming-Ting Sun, and William B Dolan. 2021a. Contextualized perturbation for textual adversarial attack. In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_, pages 5053-5069.
* Li et al. (2016a) Jiwei Li, Xinlei Chen, Eduard Hovy, and Dan Jurafsky. 2016a. Visualizing and understanding neural models in nlp. In _Proceedings of NAACL-HLT_, pages 681-691.
* Li et al. (2016b) Jiwei Li, Will Monroe, and Dan Jurafsky. 2016b. Understanding neural networks through representation erasure. _arXiv preprint arXiv:1612.08220_.
* Li et al. (2020) Linyang Li, Ruotian Ma, Qipeng Guo, Xiangyang Xue, and Xipeng Qiu. 2020. Bert-attack: Adversarial attack against bert using bert. In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_, pages 6193-6202.
* Li et al. (2021b) Linyang Li, Demin Song, Xiaonan Li, Jiehang Zeng, Ruotian Ma, and Xipeng Qiu. 2021b. Backdoor attacks on pre-trained models by layerwise weight poisoning. In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_, pages 3023-3032.
* Liu et al. (2018) Kang Liu, Brendan Dolan-Gavitt, and Siddharth Garg. 2018. Fine-pruning: Defending against backdooring attacks on deep neural networks. In _International Symposium on Research in Attacks, Intrusions, and Defenses_, pages 273-294. Springer.
* Van der Maaten and Hinton (2008) Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-sne. _Journal of machine learning research_, 9(11).
* Mrksic et al. (2016) Nikola Mrksic, Diarmuid O Seaghdha, Blaise Thomson, Milica Gasic, Lina M Rojas Barahona, Pei-Hao Su, David Vandyke, Tsung-Hsien Wen, and Steve Young. 2016. Counter-fitting word vectors to linguistic constraints. In _Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_, pages 142-148.
* Naber et al. (2003) Daniel Naber et al. 2003. A rule-based style and grammar checker.
* Pruthi et al. (2019) Danish Pruthi, Bhuwan Dhingra, and Zachary C Lipton. 2019. Combating adversarial misspellings with robust word recognition. In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_, pages 5582-5591.
* Qi et al. (2021a) Fanchao Qi, Yangyi Chen, Mukai Li, Yuan Yao, Zhiyuan Liu, and Maosong Sun. 2021a. Onion: A simple and effective defense against textual backdoor attacks. In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_, pages 9558-9566.
* Qi et al. (2021b) Fanchao Qi, Mukai Li, Yangyi Chen, Zhengyan Zhang, Zhiyuan Liu, Yasheng Wang, and Maosong Sun. 2021b. Hidden killer: Visible textual backdoor attacks with syntactic trigger. In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_, pages 443-453, Online. Association for Computational Linguistics 

Fanchao Qi, Yuan Yao, Sophia Xu, Zhiyuan Liu, and Maosong Sun. 2021c. Turn the combination lock: Learnable textual backdoor attacks via word substitution. In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_, pages 4873-4883, Online. Association for Computational Linguistics.
* Radford et al. (2019) Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. _OpenAI blog_, 1(8):9.
* Ren et al. (2019) Shuhuai Ren, Yihe Deng, Kun He, and Wanxiang Che. 2019. Generating natural language adversarial examples through probability weighted word saliency. In _Proceedings of the 57th annual meeting of the association for computational linguistics_, pages 1085-1097.
* Shafahi et al. (2018) Ali Shafahi, W Ronny Huang, Mahyar Najibi, Octavian Suciu, Christoph Studer, Tudor Dumitras, and Tom Goldstein. 2018. Poison frogs! targeted clean-label poisoning attacks on neural networks. In _Proceedings of the 32nd International Conference on Neural Information Processing Systems_, pages 6106-6116.
* Socher et al. (2013) Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In _Proceedings of the 2013 conference on empirical methods in natural language processing_, pages 1631-1642.
* Sun et al. (2021) Xiaofei Sun, Diyi Yang, Xiaoya Li, Tianwei Zhang, Yuxian Meng, Qiu Han, Guoyin Wang, Eduard Hovy, and Jiwei Li. 2021. Interpreting deep learning models in natural language processing: A review. _arXiv preprint arXiv:2110.10470_.
* Turner et al. (2018) Alexander Turner, Dimitris Tsipras, and Aleksander Madry. 2018. Clean-label backdoor attacks.
* Wang et al. (2019) Bolun Wang, Yuanshun Yao, Shawn Shan, Huiying Li, Bimal Viswanath, Haitao Zheng, and Ben Y Zhao. 2019. Neural cleanes: Identifying and mitigating backdoor attacks in neural networks. In _2019 IEEE Symposium on Security and Privacy (SP)_, pages 707-723. IEEE.
* Wang et al. (2021) Jun Wang, Chang Xu, Francisco Guzman, Ahmed El-Kishky, Yuqing Tang, Benjamin Rubinstein, and Trevor Cohn. 2021. Putting words into the system's mouth: A targeted attack on neural machine translation using monolingual data poisoning. In _Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021_, pages 1463-1473.
* Xu et al. (2021) Xiaojun Xu, Qi Wang, Huichen Li, Nikita Borisov, Carl A Gunter, and Bo Li. 2021. Detecting ai trojans using meta neural analysis. In _2021 IEEE Symposium on Security and Privacy (SP)_, pages 103-120. IEEE.
* Yang et al. (2021a) Wenkai Yang, Lei Li, Zhiyuan Zhang, Xuancheng Ren, Xu Sun, and Bin He. 2021a. Be careful about poisoned word embeddings: Exploring the vulnerability of the embedding layers in nlp models. In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_, pages 2048-2058.
* Yang et al. (2021b) Wenkai Yang, Yankai Lin, Peng Li, Jie Zhou, and Xu Sun. 2021b. Rethinking stealthiness of backdoor attack against nlp models. In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_, pages 5543-5557.
* Yang et al. (2016) Zichao Yang, Diyi Yang, Chris Dyer, Xiaodong He, Alex Smola, and Eduard Hovy. 2016. Hierarchical attention networks for document classification. In _Proceedings of the 2016 conference of the North American chapter of the association for computational linguistics: human language technologies_, pages 1480-1489.
* Zampieri et al. (2019) Marcos Zampieri, Shervin Malmasi, Preslav Nakov, Sara Rosenthal, Nourra Farra, and Ritesh Kumar. 2019. Predicting the type and target of offensive posts in social media. In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_, pages 1415-1420.
* Zang et al. (2020) Yuan Zang, Fanchao Qi, Chenghao Yang, Zhiyuan Liu, Meng Zhang, Qun Liu, and Maosong Sun. 2020. Word-level textual adversarial attacking as combinatorial optimization. In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_, pages 6066-6080.
* Zhang et al. (2019) Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. 2019. Bertscore: Evaluating text generation with bert. In _International Conference on Learning Representations_.
* Zhang et al. (2015) Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015. Character-level convolutional networks for text classification. _Advances in neural information processing systems_, 28:649-657.
* Zhang et al. (2021) Zhiyuan Zhang, Xuancheng Ren, Qi Su, Xu Sun, and Bin He. 2021. Neural network surgery: Injecting data patterns into pre-trained models with minimal instance-wise side effects. In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_, pages 5453-5466.
* Zhao et al. (2020) Shihao Zhao, Xingjun Ma, Xiang Zheng, James Bailey, Jingjing Chen, and Yu-Gang Jiang. 2020. Clean-label backdoor attacks on video recognition models. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_, pages 14443-14452