SlowGuess/NAACL_2022_OCR huggingface dataset:
Yibin Lei $^{1}$ , Yu Cao $^{2}$ , Dianqi Li $^{3}$ , Tianyi Zhou $^{3,4}$ , Meng Fang $^{1}$ , Mykola Pechenizkiy $^{1}$
<sup>1</sup>Eindhoven University of Technology (TU/e) <sup>2</sup>The University of Sydney
$^{3}$ University of Washington  $^{4}$ University of Maryland
y.lei2@student.tue.nl, ycao8647@uni.sydney.edu.au {dianqili,tianyizh}@uw.edu, {m.fang,m.pechenizkiy}@tue.nl
Generating high-quality textual adversarial examples is critical for investigating the pitfalls of natural language processing (NLP) models and further promoting their robustness. Existing attacks are usually realized through word-level or sentence-level perturbations, which either limit the perturbation space or sacrifice fluency and textual quality, both affecting the attack effectiveness. In this paper, we propose Phrase-Level Textual Adversarial ATack (PLAT) that generates adversarial samples through phrase-level perturbations. PLAT first extracts the vulnerable phrases as attack targets by a syntactic parser, and then perturbs them by a pre-trained blank-infilling model. Such flexible perturbation design substantially expands the search space for more effective attacks without introducing too many modifications, and meanwhile maintaining the textual fluency and grammaticality via contextualized generation using surrounding texts. Moreover, we develop a label-preservation filter leveraging the likelihoods of language models fine-tuned on each class, rather than textual similarity, to rule out those perturbations that potentially alter the original class label for humans. Extensive experiments and human evaluation demonstrate that PLAT has a superior attack effectiveness as well as a better label consistency than strong baselines. $^{1}$
Despite the widespread success of deep learning in natural language processing (NLP) applications, a variety of works (Wallace et al., 2019; Jia et al., 2019; Cheng et al., 2019) discovered that neural networks can be easily fooled to produce incorrect predictions, when their input text is modified by adversarial attacks that do not necessarily alter human predictions and the true meaning of the original text. Through the lens of adversarial attacks, we can allocate the weakness of models and in turn
$^{1}$ Code is available at https://github.com/Yibin-Lei/PLAT
Figure 1: In PLAT, we extract phrases from the original text as attack targets, then use a blank-infilling model to obtain perturbation candidates and generate effective adversarial texts. Note that both target phrases and perturbations may contain one or multiple tokens.
improve their reliability and robustness (Jia and Liang, 2017; Belinkov and Bisk, 2018).
However, generating high-quality adversarial texts is nontrivial due to the discrete nature of human language and its rigorous linguistic structures. While many efforts of previous works have been taken to generate word-level perturbations (Ren et al., 2019; Alzantot et al., 2018; Jin et al., 2020; Li et al., 2020; Garg and Ramakrishnan, 2020; Li et al., 2021) for the sake of simplicity, their attacks are restricted to independent perturbations on single words and thus cannot produce richer and more diverse forms of adversarial examples. To expand the search space for attacks, sentence-level attacks have been explored (Iyyer et al., 2018; Wang et al., 2020b,a; Qi et al., 2021) such as using paraphrasing, but their textual quality is usually poor due to insufficient constraints or controls on the structure and meanings of the generated texts.
To generate controllable high-quality textual adversarial examples, we propose a new phrase-level attack, PLAT, which can explore more diverse and flexible forms of perturbations than single word
Findings of the Association for Computational Linguistics: NAACL 2022, pages 1095 - 1112
perturbations. Our model is able to produce phrase-level perturbations with a high success rate and preserve the textual similarity in a more controllable manner. As illustrated in Figure 1, with the help of constituency parsing, PLAT first detects and extracts the most vulnerable phrases from the text to the victim model as the attack targets. To maintain textual fluency and grammaticality, PLAT perturbs these phrases through a contextualized blank-infilling procedure by a pre-trained language model. Compared to existing textual adversarial attacks, PLAT can produce more effective attacks by searching in a larger space of phrase-level perturbation. Meanwhile, PLAT delicately controls the range and amount of modifications so the textual meaning of the original texts will not be altered significantly after attacks.
Moreover, the success of attacks can be trivial if allowed to arbitrarily distort the ground-truth label or key contents. Hence, a valid attack is required to not change the ground-truth label predicted by humans. However, the semantic similarity filters widely used in existing works to guarantee the validity of adversarial samples (Jin et al., 2020; Li et al., 2020) perform unsatisfactorily, especially in preserving the textual meaning and even flip the gold labels according to a recent study (Morris et al., 2020). To this end, we develop a label-preservation filter to maintain class-dependent properties such as sentiments. It is built upon the comparison of likelihoods of language models fine-tuned on different classes' data. Thereby, it selects the attacks that can easily fool the victim model but possibly hardly alter the original labels.
Our contributions in this paper are three-fold:
- We propose a phrase-level textual adversarial attack that employs contextualized blank-infilling to generate high-quality phrase perturbations. It expands the perturbation space of word-level attacks and thus can produce more effective attacks without notably hurting the fluency and grammaticality.
- We introduce a novel label-preservation filter, which utilizes the likelihood of class-specific language model to generate more reliable adversarial examples.
- Extensive experiments demonstrate the effectiveness of PLAT on multiple text classification and natural language inference tasks, hence presenting a new robustness challenge to existing NLP models.
In this section, we first formulate the problem of phrase-level textual adversarial attack. Then we elaborate on how PLAT chooses phrase candidates to attack and how to adversarially perturb them. Finally, we discuss strategies to select the most effective perturbations with label preservation.
We focus on generating textual adversarial examples for classification tasks. Given a textual sequence  $\mathbf{x} = x_{1}x_{2}\ldots x_{n}$  with a specific attribute label  $y$  and a victim model  $F$  (assume  $F(\mathbf{x}) = y$ ) to attack, our goal is to generate an adversarial sample  $\mathbf{x}'$  by perturbing  $\mathbf{x}$ . A valid adversarial example  $\mathbf{x}'$  can successfully trigger a wrong prediction of the victim model, i.e.  $F(\mathbf{x}') \neq y$ , while the human judgement on  $\mathbf{x}'$  should stay unaltered as  $y$ . To achieve this goal,  $\mathbf{x}'$  needs to be sufficiently similar to  $\mathbf{x}$  with reasonable fluency and correct grammaticality.
2.2 Phrase-level Attack
Phrase candidates. Given a sequence  $\mathbf{x}$ , PLAT allocates candidates of phrases to attack from the syntactic tree extracted by a language parser (e.g., Stanford Parser (Manning et al., 2014), etc.). The model first traverses all constituents (nodes) in the syntactic tree in a top-down manner. If a node is identified as a phrase, i.e. tagged as  $NP$ ,  $VP$ , etc., the text piece in  $\mathbf{x}$  associated with all nodes in the subtree that is rooted at the current node, will be regarded as an attacking candidate. For more controllable attacks, we set a maximum depth of syntactic subtree  $d$  to restrict the length of candidate phrases so the modification to  $\mathbf{x}$  is limited, hence resulting in more possibly valid adversarial samples. Thereby, we obtain a set of multiple candidate phrases in the form of  $\mathcal{A} = \{(\mathbf{a},i,j)\}$ , where  $i$  and  $j$  are the indices of the leftest and rightest token of a phrase  $\mathbf{a}$  from  $\mathbf{x}$ .
Phrase importance. To produce attacks more effectively against the victim model  $F$  (e.g., fine-tuned BERT (Devlin et al., 2019)), PLAT only perturbs the phrase candidates important to the prediction as former works (Jin et al., 2020; Ren et al., 2019). Specifically, we consider to replace a phrase  $\{(\mathbf{a},i,j)\}$  in  $\mathbf{x}$  with a series of special symbol  $[\mathrm{MASK}]^2$  with
2We empirically found this is better than single-mask replacement.
the same length as  $\mathbf{a}$ , which results in  $\tilde{\mathbf{x}} = x_{1} \ldots x_{i-1}, [\text{MASK}]\ldots [\text{MASK}], x_{j+1} \ldots x_{n}$ . The importance for phrase  $\mathbf{a}$  is measured by
$$
I (\mathbf {a}) = P _ {F} (y \mid \mathbf {x}) - P _ {F} (y \mid \tilde {\mathbf {x}}),
$$
where  $P_F(y|\cdot)$  is the probability of the ground truth label  $y$  predicted by  $F$  given the input textual sequence. Larger  $I(\mathbf{a})$  indicates that the phrase  $\mathbf{a}$  has more significant contribution to the prediction of  $y$ . PLAT manipulates each target phrase in candidate sets  $\mathcal{A}$  by following a descending order of their importance scores. So more effective phrase-level perturbations are applied earlier for achieving minimum modifications to  $\mathbf{x}$ .
Phrase perturbations. To generate phrase-level adversarial perturbations, PLAT performs a blank-infilling procedure on each target phrase. Specifically, PLAT first replaces a target phrase  $\mathbf{a} \in \mathcal{A}$  with a blank (a single special [MASK] token in our implementation) from index  $i$  to  $j$ , i.e.,
$$
\tilde {\mathbf {x}} _ {\backslash \mathbf {a}} = x _ {1}, \dots , x _ {i - 1}, \_ \_ \_ \_ \_ \_ \_ \_ \_ \_ \_ \_ \_ \_ \_ \_ \_ \_ \_ \_ \_ \_ \_ \_ \_ \_ \_ \_ \_ \_ \_ \_ \_ \_ \_ \_ \_ \_ \_ \_ \_ \_ \_ \_ \_ \_ \_
$$
Then a pre-trained blank-infilling language model, e.g., BART (Lewis et al., 2020) or T5 (Raffel et al., 2020), takes  $\tilde{\mathbf{x}}_{\backslash \mathbf{a}}$  as the input and fills a phrase  $\mathbf{b} = z_1 \ldots z_m$  having  $m$  tokens into the blank conditioned on surrounding context, i.e.,
$$
\tilde {\mathbf {x}} _ {\mathbf {b}} = x _ {1} \dots x _ {i - 1}, z _ {1} \dots z _ {m}, x _ {j + 1} \dots x _ {n}.
$$
In contrast to paraphrasing each phrase independently, such contextualized infilling procedure can produce more fluent and grammatically correct perturbations fitting into the rest context.
For attacking each target phrase, PLAT samples  $N$  candidates of perturbed phrases  $\mathcal{B} = \{\mathbf{b}\}$  with varying lengths. During the generation, PLAT tends to sample tokens of higher probability at every step so that the outputs are more appropriate with the surrounding context. In addition, we keep the maximum length of perturbations not greater than the length of original phrases plus a threshold  $l$  (e.g.,  $|\mathbf{b}| \leq |\mathbf{a}| + l$ ). The most effective perturbation in  $\mathcal{B}$  is then selected to replace the target phrase  $\mathbf{a}$ , resulting in a perturbed text  $\tilde{\mathbf{x}}_{\mathbf{b}}$  (The details of selection will be discussed in §2.3).
We apply the above phrase perturbation sequentially to all target phrases from  $\mathcal{A}^3$  until (1) a valid
Algorithm 1 Adversarial Attack by PLAT
1: Input: Original text  $\mathbf{x}$ , the gold label  $y$ , victim model  $F$ , maximum number of perturbation  $T$ , importance score  $I$ , likelihood ratio threshold  $\delta$ . Two filter functions: class-conditioned likelihood function  $R(\cdot)$ , effectiveness score function  $S(\cdot)$
2: Output: An adversarial example  $\mathbf{x}'$
3: Extract phrase candidates from  $\mathbf{x}$  to form the set  $\mathcal{A}$
4:  $\mathbf{x}^{(0)}\gets \mathbf{x}$
5: for  $1 \leq t \leq T$  do
6: a  $\leftarrow$  target phrase with highest  $I$  in  $\mathcal{A}$
7:  $\mathcal{B} \gets$  a set of phrases perturbations generated by blank-infilling  $\tilde{\mathbf{x}}_{\mathbf{a}}^{(t-1)}$
8:  $\mathcal{B}\gets$  filtering  $\mathcal{B}$  by  $R(\mathbf{x}^{(t - 1)},\mathbf{b}',y) <   \delta$
9: if  $\mathcal{B} = \varnothing$  then  $\mathbf{x}^{(t)} = \mathbf{x}^{(t - 1)}$ , continue
10: end if
11:  $\mathbf{b} \gets \underset{\mathbf{b}' \in \mathcal{B}}{\operatorname{argmax}} S(\mathbf{x}^{(t-1)}, \mathbf{b}')$
12:  $\mathbf{x}^{(t)}\gets \tilde{\mathbf{x}}_{\mathbf{b}}^{(t - 1)}(\text{replace a with b in}\mathbf{x}^{(t - 1)})$
13: if  $F(\mathbf{x}^{(t)}) \neq y$  then return  $\mathbf{x}^{(t)}$
14: end if
15: end for
16: return NONE
adversarial sample  $\mathbf{x}^{(t)}$  is found when perturbing the  $t^{th}$  target phrase, i.e.,  $F(\mathbf{x}^{(t)})\neq y$ ; or (2) the maximum number of perturbations  $T$  is reached. We summarize the above procedure in Algorithm 1.
2.3 Label Preservation and Effective Perturbation.
Label preservation filter. Although existing works (Jin et al., 2020; Chen et al., 2021) usually employ a semantic similarity constraint (e.g., USE (Cer et al., 2018)) to encourage the validity of adversarial samples, it has been observed that such constraint is unreliable to preserve the textual meaning (Morris et al., 2020). Moreover, existing approaches rarely preserve class-dependent contents, e.g., sentiments, and might produce invalid adversarial examples with human-annotated labels flipped. Such a drawback is commonly observed in our human evaluation in §3.3.
In order to retain the class-related characteristics most critical to classification tasks, inspired by Malmi et al., 2020, PLAT directly filters phrase perturbations using likelihoods provided by class-conditioned masked language models (CMLMs). Specifically, given a sequence  $\tilde{\mathbf{x}}_{\mathrm{b}} =$
3If a phrase  $\mathbf{b}$  is perturbed, phrases that overlap  $\mathbf{b}$  in the remaining phrases of  $\mathcal{A}$  will be ignored.
$x_{1},\ldots ,x_{i - 1},z_{1},\ldots ,z_{m},x_{j + 1},\ldots ,x_{n}$  the classconditioned likelihood of the adversarially perturbed phrase  $\mathbf{b} = z_1,\dots ,z_m$  for phrase  $(\mathbf{a},i,j)$  in  $\mathbf{x}$  can be calculated as
$$
L (\mathbf {x}, \mathbf {b}, y) = \prod_ {k = 1} ^ {m} P _ {\mathrm {C M L M}} \left(z _ {k} \mid \tilde {\mathbf {x}} _ {\mathbf {b} \backslash z _ {k}}; \Theta_ {y}\right).
$$
Here,  $m$  is the length of  $\mathbf{b}$ ,  $\tilde{\mathbf{x}}_{\mathbf{b} \setminus z_k}$  is the perturbed sequence  $\tilde{\mathbf{x}}_{\mathbf{b}}$  with token  $z_k$  masked,  $P_{\mathrm{CMLM}}$  is the likelihood of  $z_k$  given  $\tilde{\mathbf{x}}_{\mathbf{b} \setminus z_k}$ , which is produced by a class-conditioned masked language model  $\Theta_y$  conditioned on class  $y$ . The conditional language model is first initialized as a pre-trained model and then fine-tuned with the pre-training objective on the data from the dataset that belonging to the class  $y$ . Therefore, a larger likelihood indicates that  $\mathbf{b}$  is more likely to match the corresponding class distribution given the surrounding context.
To avoid label flipping of human prediction, the phrase perturbations should enjoy a higher likelihood on the original class's distribution but a lower likelihood on other classes. This property can be measured by the following likelihood ratio:
$$
R (\mathbf {x}, \mathbf {b}, y) = L (\mathbf {x}, \mathbf {b}, y) / \max _ {\tilde {y} \in \mathcal {Y}, \tilde {y} \neq y} L (\mathbf {x}, \mathbf {b}, \tilde {y}),
$$
where  $\mathcal{V}$  denotes the set of all classes in the task. A higher likelihood ratio suggests the perturbation is more correlated to the original label in contrast to other labels. For better label preservation, the committed phrase perturbations are required to have a likelihood ratio larger than certain threshold  $\delta$ , i.e.  $R(\mathbf{x},\mathbf{b},y)\geq \delta$  for  $\mathbf{b}\in \mathcal{B}$ . As shown in §3.3, our method outperforms other baselines on label-preservation according to human evaluation.
Selection of the most effective perturbation. To generate  $\mathbf{x}'$  with sufficient global textual-similarity to  $\mathbf{x}$ , PLAT selects target phrases with syntactic tree depth no deeper than  $d$  and restricts their perturbations' lengths to be no larger than their length by  $l$ . Moreover, PLAT aims at utilizing minimum perturbations to perform effective adversarial attacks, so the textual similarity can be preserved. Minimum perturbations can in return help maintain reasonable fluency and grammaticality of the generated texts.
To achieve the above goals, PLAT selects the most effective phrase perturbation at each step as
the one that minimizes the probability of the gold label  $y$  predicted by  $F$ . We use a score to measure each phrase  $\mathbf{b}$  in terms of how likely it can successfully fool the model, i.e. the negative probability of the gold label  $y$  for the original  $\mathbf{x}$  associated with the perturbation  $\mathbf{b}$ , i.e.,
$$
S (\mathbf {x}, \mathbf {b}) = - P _ {F} (y \mid \tilde {\mathbf {x}} _ {\mathbf {b}}).
$$
When attacking a target phrase, PLAT only chooses one phrase perturbation  $\mathbf{b} \in \mathcal{B}$  with the highest score. The resulted perturbed-sequence is retained and then used as the initial sequence for the next time of perturbation.
A primary novelty of PLAT is the phrase-level perturbation. Compared to the widely studied word-level perturbations (Ren et al., 2019; Jin et al., 2020; Li et al., 2021) that can only independently perturb a single word every time, PLAT can perturb a text span of varying lengths by replacing it with phrases of possibly unequal lengths. Hence, it produces a more flexible attack by searching it in a larger perturbation space. Although the textual phrase-level attack has been studied by a concurrent work MAYA (Chen et al., 2021), there are several critical differences of PLAT, i.e.,
(1) The phrase-level attack by PLAT is a more general attack model that covers both word-level and phrase-level perturbations in one framework, while MAYA builds separate sub-modules for different levels of perturbations.
(2) PLAT adopts a blank-infilling strategy and leverages language models to generate phrase perturbations in a context-aware manner, leading to more fluent and grammatical adversarial examples. On the contrary, MAYA applies paraphrasing to each constituent target separately without taking its surrounding context information into account.
(3) PLAT applies several constraints and filters to the phrase perturbations for more controllable attacks and better preservation of the original textual and label information, while MAYA has no such restrictions and its generated perturbations can introduce arbitrary distortions to the original text.
In this section, we first elaborate on the experimental settings and implementation details of PLAT as well as the comparisons to several baselines in §3.1.
4In practice, we partition the whole text into multiple sentences and the likelihood  $P_{\mathrm{CMLM}}$  for a phrase  $\mathbf{b}$  is calculated locally using its corresponding sentence.
We then introduce the datasets and evaluation designs in §3.2. At the end, we summarize the main results in §3.3.
The implementation details are given as follows:
- We use pre-trained  $\mathrm{BART_{base}}$  (Lewis et al., 2020) as the language model for blank-infilling to generate phrase perturbations. We sample  $N = 5000$  candidates as the phrase set  $\mathcal{B}$  via Top-K sampling (Fan et al., 2018), while set restriction  $d = 4$ ,  $l = 3$  for each target phrase. In sections §4.1, we also report the performance comparison when using different language models.
- We use a RoBERTabase (Liu et al., 2019) to calculate the class-conditioned likelihood for label-preserving filters. On each dataset, the model is further fine-tuned to optimize the masked language modeling pre-training objective on each sequence with a prepending special label token indicating the current class. We set threshold  $\delta = 1$  for the filtering.
- The victim model  $F$  is an MLP classifier based on a  $\mathrm{BERT}_{\mathrm{base}}$  model (Devlin et al., 2019). It takes the representation of [CLS] token for prediction and is fine-tuned on the target datasets in advance.
Baselines. We compare PLAT with four state-of-the-art textual adversarial attack models:5
- Textfooler (Jin et al., 2020): a word-level attack model, which replaces tokens with their synonyms via counter-fitting word embeddings (Mrkšić et al., 2016). USE (Cer et al., 2018) distance is used to select adversarial texts that can mostly preserve the semantic similarity.
- CLARE (Li et al., 2021): instead of token replacement only, CLARE considers three word-level perturbations: replacement, insertion, and merging. Pre-trained masked language models are used to generate perturbations and a USE semantic similarity filter is applied.
- MAYA (Chen et al., 2021): a multi-granularity model that attacks the input using two separate modules for word replacement and constituent paraphrasing. It employs the embedding of SentenceBERT (Reimers and Gurevych, 2019) for semantic similarity preservation.
- StyleAdv (Qi et al., 2021): a sentence-level model based on text style transfer. To launch an attack, it paraphrases the whole examples with five
Dataset	Avg. Len	#Classes	Train	Test	Acc
Yelp	130	2	560k	38k	91.8%
AG News	46	4	120k	7.6k	94.6%
MNLI	23/11	3	392k	9.8k	84.0%
QNLI	11/31	2	105k	5.4k	91.4%
Table 1: Statistics of datasets and the performance of victim models on each dataset.
different text styles: tweets, bible, poetry, shaekspare and lyrics.
3.2 Datasets and Evaluation
Datasets. We investigate the following datasets for text classification and natural language inference tasks in our experiments. The statistics and performance of the victim models evaluated on each dataset are reported in Table 1.
- Yelp Reviews (Zhang et al., 2015): a binary sentiment classification dataset containing restaurant reviews as samples.
- AG News (Zhang et al., 2015): a news articles classification dataset covering four classes: World, Sport, Business, and Science and Technology.
- MNLI (Williams et al., 2018): a natural language inference dataset, where each sample contains a pair of sentences whose relationship is labeled as entailment, neutral, or contradiction. We use the matched test set here.
- QNLI (Wang et al., 2018): a natural language inference dataset based on the question answering corpus SQuAD (Rajpurkar et al., 2016). Each sample contains a context and a question labeled as entailed or not entailed.
All attacks will be conducted on 1000 instances randomly drawn from test sets. For tasks on a pair of sentences, we attack the longer sentence.
Evaluation metrics. We evaluate models using the following automatic metrics:
- Attack success rate (ASR): the percentage of successful adversarial attacks that trigger wrong predictions of the victim model.
- Edit Distance (DIS) (Navarro, 2001): the normalized Levenshtein distance that measures the minimum amount of word edits required to transform the original text to the adversarial one. It measures the modification rate of an adversarial sample.
- BLEU (Papineni et al., 2002): the BLEU score between an adversarial sample and its corresponding original sample is used to measure their n-gram overlap (textual similarity). We do not report the
5 All results are obtained by running their released code.
Dataset	Yelp (PPL = 51.5)	AG News (PPL = 62.8)
Model	ASR↑	DIS↓	BLEU↑	PPL↓	GER↓	ASR↑	DIS↓	BLEU↑	PPL↓	GER↓
Textfooler	94.5	0.11	0.80	101.1	0.73	65.5	0.29	0.52	339.0	1.43
CLARE	97.3	0.07	0.88	65.2	0.08	68.0	0.09	0.86	97.2	-0.03
MAYA	97.0	0.43	0.44	78.9	5.23	94.2	0.64	0.25	168.6	4.30
StyleAdv	90.1	0.96	/	132.0	-0.64	79.6	0.97	/	111.3	-0.31
PLAT	98.4	0.17	0.78	56.8	0.33	95.7	0.34	0.58	80.3	0.58
Dataset	MNLI (PPL = 60.9)	QNLI (PPL = 46.0)
Model	ASR↑	DIS↓	BLEU↑	PPL↓	GER↓	ASR↑	DIS↓	BLEU↑	PPL↓	GER↓
Textfooler	58.6	0.15	0.71	159.0	0.67	57.8	0.19	0.63	164.5	0.62
CLARE	86.2	0.10	0.82	82.7	0.09	82.6	0.15	0.74	75.9	0.03
MAYA	92.8	0.40	0.49	104.7	2.20	78.6	0.40	0.48	101.4	2.90
StyleAdv	72.5	0.96	/	103.7	-0.53	67.3	0.95	/	108.3	0.17
PLAT	96.6	0.20	0.74	62.1	0.23	92.4	0.25	0.68	51.3	0.06