Introduction
Recent studies have shown the vulnerability of ML models to adversarial attacks, small perturbations which lead to misclassification of inputs. Adversarial example generation in NLP BIBREF0 is more challenging than in common computer vision tasks BIBREF1, BIBREF2, BIBREF3 due to two main reasons: the discrete nature of input space and ensuring semantic coherence with the original sentence. A major bottleneck in applying gradient based BIBREF4 or generator model BIBREF5 based approaches to generate adversarial examples in NLP is the backward propagation of the perturbations from the continuous embedding space to the discrete token space.
Recent works for attacking text models rely on introducing errors at the character level in words BIBREF6, BIBREF7 or adding and deleting words BIBREF8, BIBREF9, BIBREF10, etc. for creating adversarial examples. These techniques often result in adversarial examples which are unnatural looking and lack grammatical correctness, and thus can be easily identified by humans.
TextFooler BIBREF11 is a black-box attack, that uses rule based synonym replacement from a fixed word embedding space to generate adversarial examples. These adversarial examples do not account for the overall semantics of the sentence, and consider only the token level similarity using word embeddings. This can lead to out-of-context and unnaturally complex replacements (see Table ), which can be easily identifiable by humans.
The recent advent of powerful language models BIBREF12, BIBREF13 in NLP has paved the way for using them in various downstream applications. In this paper, we present a simple yet novel technique: BAE (BERT-based Adversarial Examples), which uses a language model (LM) for token replacement to best fit the overall context. We perturb an input sentence by either replacing a token or inserting a new token in the sentence, by means of masking a part of the input and using a LM to fill in the mask (See Figure FIGREF1). BAE relies on the powerful BERT masked LM for ensuring grammatical correctness of the adversarial examples. Our attack beats the previous baselines by a large margin and confirms the inherent vulnerabilities of modern text classification models to adversarial attacks. Moreover, BAE produces more richer and natural looking adversarial examples as it uses the semantics learned by a LM.
To the best of our knowledge, we are the first to use a LM for adversarial example generation. We summarize our major contributions as follows:
We propose BAE, a novel strategy for generating natural looking adversarial examples using a masked language model.
We introduce 4 BAE attack modes, all of which are almost always stronger than previous baselines on 7 text classification datasets.
We show that, surprisingly, just a few replace/insert operations can reduce the accuracy of even a powerful BERT-based classifier by over $80\%$ on some datasets.
Methodology
Problem Definition We are given a dataset $(S,Y) = \lbrace (\mathbb {S}_1,y_1),(\mathbb {S}_2,y_2)\dots (\mathbb {S}_m,y_m)\rbrace $ and a trained classification model $C:\mathbb {S}\rightarrow Y$. We assume the soft-label black-box setting where the attacker can only query the classifier for output probabilities on a given input, and does not have access to the model parameters, gradients or training data. For an input pair $(\mathbb {S},y)$, we want to generate an adversarial example $\mathbb {S}_{adv}$ such that $C(\mathbb {S}_{adv}){\ne }y$ where $\mathbb {S}_{adv}$ is natural looking, grammatically correct and semantically similar to $\mathbb {S}$ (by some pre-defined definition
of similarity).
BAE For generating adversarial example $\mathbb {S}_{adv}$, we define two perturbations on the input $\mathbb {S}$:
Replace a token $t \in \mathbb {S}$ with another
Insert a new token $t^{\prime }$ in $\mathbb {S}$
Some tokens in the input are more attended to by $C$ than others, and therefore contribute more towards the final prediction. Replacing these tokens or inserting a new token adjacent to them can thus have a stronger effect on altering the classifier prediction. We estimate the token importance $I_i$ of each token $t_i \in \mathbb {S}=[t_1, \dots , t_n]$, by deleting $t_i$ from $\mathbb {S}$ and computing the decrease in probability of predicting the correct label $y$, similar to BIBREF11.
While the motivation for replacing tokens in decreasing order of importance is clear, we conjecture that adjacent insertions in this same order can lead to a powerful attack. This intuition stems from the fact that the inserted token changes the local context around the original token.
The Replace (R) and Insert (I) operations are performed on a token $t$ by masking it and inserting a mask token adjacent to it in $\mathbb {S}$ respectively. The pre-trained BERT masked language model (MLM) is used to predict the mask tokens (See Figure FIGREF1).
BERT is a powerful LM trained on a large training corpus ($\sim $ 2 billion words), and hence the predicted mask tokens fit well grammatically in $\mathbb {S}$. The BERT-MLM does not however guarantee semantic coherence to the original text $\mathbb {S}$ as demonstrated by the following simple example. Consider the sentence: `the food was good'. For replacing the token `good', BERT-MLM may predict the tokens `nice' and `bad', both of which fit well into the context of the sentence. However, replacing `good' with `bad' changes the original sentiment of the sentence.
To ensure semantic similarity on introducing perturbations in the input text, we filter the set of top K masked tokens (K is a pre-defined constant) predicted by BERT-MLM using a Universal Sentence Encoder (USE) BIBREF14 based sentence similarity scorer. For the R operations we add an additional check for grammatical correctness of the generated adversarial example by filtering out predicted tokens that do not form the same part of speech (POS) as the original token $t_i$ in the sentence.
To choose the token for a perturbation (R/I) that best attacks the model from the filtered set of predicted tokens:
If there are multiple tokens can cause $C$ to misclassify $\mathbb {S}$ when they replace the mask, we choose the token which makes $\mathbb {S}_{adv}$ most similar to the original $\mathbb {S}$ based on the USE score.
If no token causes misclassification, we choose the perturbation that decreases the prediction probability $P(C(\mathbb {S}_{adv}){=}y)$ the most.
The perturbations are applied iteratively to the input tokens in decreasing order of importance, until either $C(\mathbb {S}_{adv}){\ne }y$ (successful attack) or all the tokens of $\mathbb {S}$ have been perturbed (failed attack).
We present 4 attack modes for BAE based on the R and I operations, where for each token $t$ in $\mathbb {S}$:
BAE-R: Replace token $t$ (See Algorithm )
BAE-I: Insert a token to the left or right of $t$
BAE-R/I: Either replace token $t$ or insert a token to the left or right of $t$
BAE-R+I: First replace token $t$, then insert a token to the left or right of $t$
Experiments
Datasets and Models We evaluate our adversarial attacks on different text classification datasets from tasks such as sentiment classification, subjectivity detection and question type classification. Amazon, Yelp, IMDB are sentence-level sentiment classification datasets which have been used in recent work BIBREF15 while MR BIBREF16 contains movie reviews based on sentiment polarity. MPQA BIBREF17 is a dataset for opinion polarity detection, Subj BIBREF18 for classifying a sentence as subjective or objective and TREC BIBREF19 is a dataset for question type classification.
We use 3 popular text classification models: word-LSTM BIBREF20, word-CNN BIBREF21 and a fine-tuned BERT BIBREF12 base-uncased classifier. For each dataset we train the model on the training data and perform the adversarial attack on the test data. For complete model details refer to Appendix.
As a baseline, we consider TextFooler BIBREF11 which performs synonym replacement using a fixed word embedding space BIBREF22. We only consider the top $K{=}50$ synonyms from the MLM predictions and set a threshold of 0.8 for the cosine similarity between USE based embeddings of the adversarial and input text.
Results We perform the 4 modes of our attack and summarize the results in Table . Across datasets and models, our BAE attacks are almost always more effective than the baseline attack, achieving significant drops of 40-80% in test accuracies, with higher average semantic similarities as shown in parentheses. BAE-R+I is the strongest attack since it allows both replacement and insertion at the same token position, with just one exception. We observe a general trend that the BAE-R and BAE-I attacks often perform comparably, while the BAE-R/I and BAE-R+I attacks are much stronger. We observe that the BERT-based classifier is more robust to the BAE and TextFooler attacks than the word-LSTM and word-CNN models which can be attributed to its large size and pre-training on a large corpus.
The baseline attack is often stronger than the BAE-R and BAE-I attacks for the BERT based classifier. We attribute this to the shared parameter space between the BERT-MLM and the BERT classifier before fine-tuning. The predicted tokens from BERT-MLM may not drastically change the internal representations learned by the BERT classifier, hindering their ability to adversarially affect the classifier prediction.
Effectiveness We study the effectiveness of BAE on limiting the number of R/I operations permitted on the original text. We plot the attack performance as a function of maximum $\%$ perturbation (ratio of number of word replacements and insertions to the length of the original text) for the TREC dataset. From Figure , we clearly observe that the BAE attacks are consistently stronger than TextFooler. The classifier models are relatively robust to perturbations up to 20$\%$, while the effectiveness saturates at 40-50$\%$. Surprisingly, a 50$\%$ perturbation for the TREC dataset translates to replacing or inserting just 3-4 words, due to the short text lengths.
Qualitative Examples We present adversarial examples generated by the attacks on a sentence from the IMDB and Yelp datasets in Table . BAE produces more natural looking examples than TextFooler as tokens predicted by the BERT-MLM fit well in the sentence context. TextFooler tends to replace words with complex synonyms, which can be easily detected. Moreover, BAE's additional degree of freedom to insert tokens allows for a successful attack with fewer perturbations.
Human Evaluation We consider successful adversarial examples generated from the Amazon and IMDB datasets and verify their sentiment and grammatical correctness. Human evaluators annotated the sentiment and the grammar (Likert scale of 1-5) of randomly shuffled adversarial examples and original texts. From Table , BAE and TextFooler have inferior accuracies compared to the Original, showing they are not always perfect. However, BAE has much better grammar scores, suggesting more natural looking adversarial examples.
Ablation Study We analyze the benefits of R/I operations in BAE in Table . From the table, the splits $\mathbb {A}$ and $\mathbb {B}$ are the $\%$ of test points which compulsorily need I and R operations respectively for a successful attack. We can observe that the split $\mathbb {A}$ is larger than $\mathbb {B}$ thereby indicating the importance of the I operation over R. Test points in split require both R and I operations for a successful attack. Interestingly, split is largest for Subj, which is the most robust to attack (Table ) and hence needs both R/I operations. Thus, this study gives positive insights towards the importance of having the flexibility to both replace and insert words.
Refer to the Appendix for additional results, effectiveness graphs and details of human evaluation.
Conclusion
In this paper, we have presented a novel technique for generating adversarial examples (BAE) based on a language model. The results obtained on several text classification datasets demonstrate the strength and effectiveness of our attack.

Question: what text classification datasets do they evaluate on?
Answer: Amazon, Yelp, IMDB, MR, MPQA, Subj, TREC

Amazon, Yelp, IMDB , MR , MPQA , Subj, TREC

Amazon, Yelp, IMDB, MR BIBREF16, MPQA BIBREF17, Subj BIBREF18, TREC BIBREF19

Question: which models is their approach compared to?
Answer: TextFooler

word-LSTM BIBREF20, word-CNN BIBREF21,  fine-tuned BERT BIBREF12 base-uncased 

word-LSTM BIBREF20, word-CNN BIBREF21 and a fine-tuned BERT BIBREF12 base-uncased classifier