Evaluating the Robustness of Neural Language Models to Input Perturbations (2021)

Abstract: High-performance neural language models have obtained state-of-the-art results on a wide range of Natural Language Processing (NLP) tasks. However, results for common benchmark datasets often do not reflect model reliability and robustness when applied to noisy, real-world data. In this study, we design and implement various types of character-level and word-level perturbation methods to simulate realistic scenarios in which input texts may be slightly noisy or different from the data distribution on which NLP systems were trained. Conducting comprehensive experiments on different NLP tasks, we investigate the ability of high-performance language models such as BERT, XLNet, RoBERTa, and ELMo in handling different types of input perturbations. The results suggest that language models are sensitive to input perturbations and their performance can decrease even when small changes are introduced. We highlight that models need to be further improved and that current benchmarks are not reflecting model robustness well. We argue that evaluations on perturbed inputs should routinely complement widely-used benchmarks in order to yield a more realistic understanding of NLP systems’ robustness.

## introduction high-performance deep neural language models such as bert @xcite , xlnet (z. @xcite , and gpt-2 @xcite have brought breakthroughs to a wide range of natural language processing (nlp) tasks including text classification, sentiment analysis, textual entailment, natural language inference, machine translation, and question answering. their immense ability in capturing various linguistic properties has led these state-of-the-art language models to master different nlp tasks, even surpassing human accuracy on some benchmarks such as squad foot_0 . however, recent studies have revealed that there is a gap between performing well on benchmarks and actually working under real-world situations @xcite . even a well-trained, high-performance deep language model can be sensitive to negligible changes in the input that cause the model to make erroneous decisions (m. @xcite . this raises serious concerns regarding the robustness/reliability of neural language models utilized in real-world applications. the terms 'robustness' and 'reliability' refer to the ability of a system to perform consistently well in situations where changes to input should not cause a change in the system's output, or the system is expected to properly reflect the change and produce a correct outcome. applying automatic or human-controlled perturbations to textual inputs has been shown to be effective for evaluating the robustness of nlp systems, investigating their vulnerabilities, and finding their bugs. recently, checklist @xcite provided a framework for behavioral testing of nlp systems inspired by black-box ## nlp tasks in our experiments, we used five datasets covering five different nlp tasks. trec @xcite is a text classification (tc) dataset containing more than 6,000 questions and 50 different class labels that specify the type of questions. stanford sentiment treebank (sst) @xcite is a sentiment analysis (sa) dataset containing more than 11,000 movie reviews from 'rotten tomatoes'. every review is classified into one of the five classes: very positive, positive, neutral, negative, and very negative. conll-2003 foot_1 is a named entity recognition (ner) dataset containing news stories from the reuters corpus with more than 200k tokens annotated as person, organization, location, miscellaneous, or other. sts benchmark @xcite is a semantic similarity (ss) dataset comprising of more than 8k text pairs extracted from image captions, news headlines, and user forums. each pair of sentences is assigned a similarity score between 0 and 5. wikiqa (wqa) (y. @xcite is a question answering (qa) dataset composed of more than 3,000 questions and 29,000 sentences as answers extracted from wikipedia. ## language models in our experiments, we utilized four neural language models shown to be effective in learning bidirectional contexts and obtained state-of-the-art results during recent years: bert @xcite is composed of deep encoder transformer layers and uses two pretraining objectives, i.e. masked language modelling and next sentence prediction. we used the bertlarge architecture (along with the cased model) containing 24 transformer layers, 1024 hidden units per layer, 16 attention heads per hidden unit, and 340 million parameters. roberta @xcite uses a model architecture similar to bert, but adopts an optimized pretraining approach. it was pretrained on more data, with bigger batch sizes and longer sequences than bert. furthermore, the next sentence prediction objective was removed and a dynamic masking strategy replaced the basic masking method. we used robertalarge that further optimizes the same model as bertlarge. xlnet (z. @xcite utilizes decoder transformers and adopts a permutation language modelling approach along with generalized autoregressive pretraining. we used the xlnetlarge model, with the same architecture hyperparameters and model size as bertlarge. elmo @xcite is a contextualized word representation method that utilizes character convolutions along with shallow concatenation of backward and forward lstms to implement bidirectional language modeling. we used the original elmo model composed of two highway layers with an lstm hidden size of 4096, output size of 512, and a total parameters of 93.6 million. the contextualized embeddings computed by elmo were fed into a dense layer containing 128 hidden units followed by an output layer with a softmax activation in the tc and qa tasks, a linear activation in the sa and ss tasks, and crf layer with a linear activation in the ner task. we retrieved the pretrained models, fine-tuned them separately on each downstream task using the training and development sets, and tested them on the test sets. we utilized the huggingface transformers @xcite and farm 3 libraries to implement the transformer-based models. a complete list of hyperparameter values is presented in appendix a. ## perturbation methods we designed and implemented various characterlevel and word-level perturbation methods that simulate different types of noise an nlp system may encounter in real-world situations. the perturbations can be produced for every dataset regardless of the underlying language model or nlp system being tested. ## experimental results all the experiments were performed on a computer with an intel core i5-9600k cpu at 3.70ghz, 32 gb of ram, and a geforce rtx 2080 ti graphic card (gpu) with 11 gb dedicated memory. perturbation methods ran on cpu; fine-tuning on training sets, and evaluating on test sets and perturbed samples ran on gpu. ## related work typical performance measures such as accuracy, precision, recall, etc. may not properly reflect how nlp systems behave in real-world use cases. this has motivated many studies to devise novel methods for investigating different capabilities and vulnerabilities of text processing systems. behavioral testing introduces targeted changes to textual inputs to test linguistic capabilities of systems @xcite . explanations provide simplified representations of what a complex nlp model has learned @xcite . this can help to identify biases and errors in nlp models. adversarial perturbations have been widely studied to assess the robustness of nlp systems against adversarial samples crafted to fool a model @xcite @xcite . however, adversarial samples resemble a very specific type of noise. moreover, most of previous work on adversarial perturbation to nlp models focused on misspelling attacks @xcite @xcite . the perturbation methods implemented in this paper represented a wide range of noises that an nlp system may face in real-world situations. introducing noise and changing textual inputs were already adopted to assess the ability of models in capturing specific linguistic features such as learning syntax-sensitive dependencies @xcite , for specific nlp tasks such as machine translation @xcite , for detecting biases in language models @xcite , or to identify susceptible entities in text documents (m. @xcite . in this paper, we investigated the robustness on a wide range of tasks, and for various types of character-level and word-level noises in text. ## conclusion in this paper, we introduced and implemented a set of non-adversarial perturbation methods that can be used to evaluate the robustness of nlp systems. we extensively investigated the robustness of high-performance neural language models to noisy input texts. the evaluations on various nlp tasks imply that these models are sensitive to different character-level and word-level perturbations to the input, and the models' performance can decrease when the input contains slight noise. the results suggest that it may be too simplistic to only rely on accuracy scores obtained on benchmark datasets when evaluating the robustness of nlp systems. the proposed perturbations can be used, along with other methodologies such as checklist, to test how robust and reliable nlp systems can operate in real-world settings. the experimental results demonstrated that the perturbation methods are effective tools for evaluating nlp systems against noisy data. the user study revealed that only few perturbation methods need to be monitored to make sure they produce meaningful and consistent samples. most of the perturbation methods can be used automatically to produce noisy test samples. they can be also used as a baseline for evaluating adversarial attacks against non-adversarial perturbations. future work may include helping users assess meaning preservation and grammatical correctness in a semi-automatic manner. sentence encoders such as infersent @xcite , universal sentence encoder @xcite , and bert trained for semantic similarity @xcite can be used to give users clues how semantically similar the original and perturbed sentences are. moreover, users can be provided with information about grammatical errors in the perturbed text using languagetool @xcite or other grammar checking tools.