R ock NER : A Simple Method to Create Adversarial Examples for Evaluating the Robustness of Named Entity Recognition Models (2021)

Abstract: To audit the robustness of named entity recognition (NER) models, we propose RockNER, a simple yet effective method to create natural adversarial examples. Specifically, at the entity level, we replace target entities with other entities of the same semantic class in Wikidata; at the context level, we use pre-trained language models (e.g., BERT) to generate word substitutions. Together, the two levels of at- tack produce natural adversarial examples that result in a shifted distribution from the training data on which our target models have been trained. We apply the proposed method to the OntoNotes dataset and create a new benchmark named OntoRock for evaluating the robustness of existing NER models via a systematic evaluation protocol. Our experiments and analysis reveal that even the best model has a significant performance drop, and these models seem to memorize in-domain entity patterns instead of reasoning from the context. Our work also studies the effects of a few simple data augmentation methods to improve the robustness of NER models.

## introduction recent named entity recognition (ner) models have achieved great performance on many conventional benchmarks such as conll2003 @xcite and ontonotes 5.0 @xcite . however, it is not clear whether they are reliable in realistic applications in which entities and/or context words can be out of the distribution of the training data. it is thus important to audit the robustness of ner systems via natural adversarial attacks. most existing methods for generating adversarial attacks in nlp focus on sentence classification @xcite @xcite and question answering (jia and liang, 2017; ribeiro et al., 2018; 1 our code and data are publicly available at the project website: https://inklab.usc.edu/rockner . gan and ng, 2019), and these methods lack special designs reflecting the underlying compositions of the ner examples -i.e., entity structures and their context words. in this paper, we focus on creating general natural adversarial examples (i.e., real-world entities and human-readable context) for evaluating the robustness of ner models. ## natural adversarial attacks for ner we present rockner, a simple yet effective method to generate high-quality natural adversarial examples for evaluating the robustness of ner models by perturbing both the entities and contexts of original examples. we apply the method to the development set and test set of ontonotes to create the ontorock benchmark. ## evaluating robustness of ner models in this section, we use our ontorock dataset to evaluate the robustness of popular ner models including spacy @xcite , stanza @xcite , flair @xcite , blstm-crf @xcite , bert-crf @xcite , and roberta-crf @xcite . model details are described in appendix §c. we organize our results and analysis as three main research questions and their answers. q1: how robust are current ner models? main results. we show the f1 scores on the test sets 5 in pre-training & robustness. blstm-crf is trained solely on the training set of ontonotes; the ner toolkits such as spacy and stanza are trained on more datasets (e.g., conll03); bert-crf, flair, and roberta-crf are based on pre-trained language models. we can see that, in terms of robustness, ner models with pre-training tend to outperform models without pre-training but with more ner data access, which outperform those trained only on the ontonotes training set. this observation indicates that pre-training (on corpora or other ner data) leads to better robustness, and better pre-trained models (roberta vs. bert) have a lower (relative) performance drop. interestingly, we find that the improved robustness from pre-training mainly comes from the improvement on the entity-level attacks, possibly because of the increased exposure to entities and increased ability to reason using context (see our 1st point in q2). q2: where are the ner models brittle? memorizing or reasoning? note that our entity-level attacks aim to test the ability to use the context to infer the entities, as the novel entities themselves are out-of-distribution -i.e., if a model can reason about the context, it should be robust against entity changes. in turn, the contextlevel attacks audit the ability to memorize entity patterns, as the context is changed, making it more challenging to infer from. from ## related work there are other recent works which also turn their attention from achieving a new state-of-the-art of ner model towards studying ner models' robustness and generalization ability. @xcite create entity-switched datasets by replacing entities with others of the same type but different national origin. they find that ner models perform worse on entities from certain countries. mayhew et al. (2020) and bodapati et al. (2019) focus on the robustness when inputs are not written in the standard casing (e.g., "he is from us" ! "us"). @xcite analyze the generalization ability of current ner models by evaluating them across datasets. @xcite further analyze the roles of context and names in entity predictions made by models and humans. although these works begin to understand the robustness issue of ner models, they do not build an automated pipeline to generate natural adversarial instances with large coverage (e.g. thousands of fine-grained classes) at scale. there are also works in other domains aiming to evaluate models' robustness with perturbed inputs. for example, @xcite attack reading comprehension models by adding word sequences to the input. @xcite and @xcite paraphrase the input to test models' oversensitivity. @xcite target adversarial typos. @xcite propose a benchmark for reading comprehension with diverse types of testtime perturbation. these works focus on different domains than our research does, and they do not consider the composition of ner examples. little attention is drawn to the entities in the sentences, and many attacks (e.g. character swapping, word injection) may make the perturbed sentences invalid. to the best of our knowledge, this work is among the first to propose a straightforward, dedicated pipeline for generating natural adversarial examples for the ner task, which takes into account the compositions of ner examplesi.e., entity structures and their context. ## conclusion our contributions in this short paper are two-fold. 1) resource-wise: we develop rockner, a straightforward method for generating natural adversarial attacks for ner, which produces ontorock, a benchmark for auditing the robustness of ner models. 2) evaluation-wise: our experimental results and analysis provides answers supported by experimental results to three main research questions on the robustness of current mainstream ner models. we believe rockner and its produced attacks (e.g., the ontorock benchmark) can benefit the community working to increase the robustness and out-of-distribution generalization of ner. 8