On the Robustness of Reading Comprehension Models to Entity Renaming (2022)

Abstract: We study the robustness of machine reading comprehension (MRC) models to entity renaming—do models make more wrong predictions when the same questions are asked about an entity whose name has been changed? Such failures imply that models overly rely on entity information to answer questions, and thus may generalize poorly when facts about the world change or questions are asked about novel entities. To systematically audit this issue, we present a pipeline to automatically generate test examples at scale, by replacing entity names in the original test sample with names from a variety of sources, ranging from names in the same test set, to common names in life, to arbitrary strings. Across five datasets and three pretrained model architectures, MRC models consistently perform worse when entities are renamed, with particularly large accuracy drops on datasets constructed via distant supervision. We also find large differences between models: SpanBERT, which is pretrained with span-level masking, is more robust than RoBERTa, despite having similar accuracy on unperturbed test data. We further experiment with different masking strategies as the continual pretraining objective and find that entity-based masking can improve the robustness of MRC models.

## introduction the task of machine reading comprehension (mrc) measures machines' understanding and reasoning abilities. recent research advances @xcite @xcite have driven mrc models to reach or even exceed human performance on several mrc benchmark datasets. however, their actual ability to solve the general mrc task is still questionable @xcite @xcite . while hu1 our code and data can be found at https://github. com/ink-usc/entity-robustness . ## entity name substitution in this section, we introduce our method for perturbing an mrc test set with substitution entity names, i.e., the instantiation of f perturb . generating substitution names is at the core of our evaluation as different kinds of names measure a model's behavior in different situations with different robustness implications. we propose three categories of perturbations on three entity types and collect the corresponding name resources, aiming to audit a model's robustness from different perspectives. ## results and analysis the average-case em scores on the original and perturbed test sets are presented in figure 3 . we report the mean and standard deviation over 3 train ## conclusion in this paper, we systematically study the robustness of mrc models to entity name substitution. specifically, we propose a substitution framework along with candidate names of different implications. we experiment with three pretrained language models on five mrc datasets. we find that models trained on distantly-supervised datasets are susceptible to entity name substitution, while models trained on human-annotated datasets are relatively robust, with gpe renaming harder than per and org renaming. the lack of robustness can be further attributed to model's overreliance on entity knowledge and name clues. we also find that spanbert, which is pretrained using span-level objectives, shows better robustness than bert and roberta. leveraging these insights, we study defense approaches based on continual pretraining and demonstrate that entity-based masking policies are beneficial to model's robustness. future works include systematically studying the effect of background knowledge in mrc, and developing more effective methods to improve the robustness of mrc models.