Measuring and Improving Consistency in Pretrained Language Models (2021)

Abstract: Consistency of a model—that is, the invariance of its behavior under meaning-preserving alternations in its input—is a highly desirable property in natural language processing. In this paper we study the question: Are Pretrained Language Models (PLMs) consistent with respect to factual knowledge? To this end, we create ParaRel🤘, a high-quality resource of cloze-style query English paraphrases. It contains a total of 328 paraphrases for 38 relations. Using ParaRel🤘, we show that the consistency of all PLMs we experiment with is poor— though with high variance between relations. Our analysis of the representational spaces of PLMs suggests that they have a poor structure and are currently not suitable for representing knowledge robustly. Finally, we propose a method for improving model consistency and experimentally demonstrate its effectiveness.1

## introduction pretrained language models (plms) are large neural networks that are used in a wide variety of nlp tasks. they operate under a pretrain-finetune paradigm: models are first pretrained over a large text corpus and then finetuned on a downstream task. plms are thought of as good language encoders, supplying basic language understanding capabilities that can be used with ease for many downstream tasks. a desirable property of a good language understanding model is consistency: the ability to make consistent decisions in semantically equivalent contexts, reflecting a systematic ability to generalize in the face of language variability. examples of consistency include: predicting the same answer in question answering and reading comprehension tasks regardless of paraphrase @xcite ); making consistent assignments in coreference resolution @xcite ; or making summaries factually consistent with the original document @xcite . while consistency is important in many tasks, nothing in the training process explicitly targets it. one could hope that the unsupervised training signal from large corpora made available to plms such as bert or roberta @xcite is sufficient to induce consistency and transfer it to downstream tasks. in this paper, we show that this is not the case. the recent rise of plms has sparked a discussion about whether these models can be used as knowledge bases (kbs) @xcite @xcite @xcite . consistency is a key property of kbs and is particularly important for automatically constructed kbs. one of the biggest appeals of using a plm as a kb is that we can query it in natural languageinstead of relying on a specific kb schema. the expectation is that plms abstract away from language and map queries in natural language into meaningful representations such that queries with identical intent but different language forms yield the same answer. for example, the query ''homeland premiered on [mask]'' should produce the same answer as ''homeland originally aired on [mask]''. studying inconsistencies of plm-kbs can also teach us about the organization of knowledge in the model, or lack thereof. finally, failure to behave consistently may point to other representational issues such as the similarity between antonyms and synonyms @xcite , and overestimating events and actions (reporting bias) @xcite . in this work, we study the consistency of factual knowledge in plms, specifically in masked language models (mlms)-these are plms trained with the mlm objective @xcite , as opposed to other strategies such as standard language modeling @xcite or text-to-text @xcite . we ask: is the factual information we extract from plms invariant to paraphrasing? we use zero-shot evaluation since we want to inspect models directly, without adding biases through finetuning. this allows us to assess how much consistency was acquired during pretraining and to compare the consistency of different models. overall, we find that the consistency of the plms we consider is poor, although there is a high variance between relations. we introduce pararel , a new benchmark that enables us to measure consistency in plms ( §3), by using factual knowledge that was found to be partially encoded in them @xcite . pararel is a manually curated resource that provides patterns-short textual prompts-that are paraphrases of one another, with 328 paraphrases describing 38 binary relations such as x born-in y, x works-for y ( §4). we then test multiple plms for knowledge consistency, namely, whether a model predicts the same answer for all patterns of a relation. figure 1 shows an overview of our approach. using pararel , we probe for consistency in four plm types: bert, bert-whole-word-masking, roberta, and albert ( §5). our experiments with pararel show that current models have poor consistency, although with high variance between relations ( §6). finally, we propose a method that improves model consistency by introducing a novel consistency loss ( §8). we demonstrate that, trained with this loss, bert achieves better consistency performance on unseen relations. however, more work is required to achieve fully consistent models. ## background there has been significant interest in analyzing how well plms @xcite perform figure 1 : overview of our approach. we expect that a consistent model would predict the same answer for two paraphrases. in this example, the model is inconsistent on the homeland and consistent on the seinfeld paraphrases. on linguistic tasks @xcite @xcite , commonsense @xcite @xcite , and reasoning @xcite , usually assessed by measures of accuracy. however, accuracy is just one measure of plm performance @xcite . it is equally important that plms do not make contradictory predictions (cf. figure 1 ), a type of error that humans rarely make. there has been relatively little research attention devoted to this question, that is, to analyze if models behave consistently. one example concerns negation: ettinger (2020) and @xcite show that models tend to generate facts and their negation, a type of inconsistent behavior. @xcite propose paired probes for evaluating consistency. our work is broader in scope, examining the consistency of plm behavior across a range of factual knowledge types and investigating how models can be made to behave more consistently. consistency has also been highlighted as a desirable property in automatically constructed kbs and downstream nlp tasks. we now briefly review work along these lines. consistency in knowledge bases (kbs) has been studied in theoretical frameworks in the context of the satisfiability problem and kb construction, and efficient algorithms for detecting inconsistencies in kbs have been proposed @xcite . other work aims to quantify the degree to which kbs are inconsistent and detects inconsistent statements @xcite @xcite . consistency in question answering was studied by @xcite in two tasks: visual question answering @xcite and reading comprehension @xcite . they automatically generate questions to test the consistency of qa models. their findings suggest that most models are not consistent in their predictions. in addition, they use data augmentation to create more robust models. @xcite generate new questions conditioned on context and answer from a labeled dataset and by filtering answers that do not provide a consistent result with the original answer. they show that pretraining on these synthetic data improves qa results. @xcite use data augmentation that complements questions with symmetricity and transitivity, as well as a regularizing loss that penalizes inconsistent predictions. @xcite propose a method to improve accuracy and consistency of qa models by augmenting a plm with an evolving memory that records plm answers and resolves inconsistency between answers. work on consistency in other domains includes @xcite where prediction of consistency in procedural text is improved. ribeiro et al. (2020) use consistency for more robust evaluation. @xcite measure and mitigate inconsistency in natural language inference (nli), and finally, @xcite propose a method for measuring inconsistencies in nli explanations @xcite . ## probing plms for consistency in this section, we formally define consistency and describe our framework for probing consistency of plms. ## the pararel resource we now describe pararel , a resource designed for our framework (cf. section 3.2). pararel is curated by experts, with a high level of agreement. it contains patterns for 38 relations foot_3 from t-rex @xcite )-a large dataset containing kb triples aligned with wikipedia abstracts-with an average of 8.63 patterns per relation. construction method pararel was constructed in four steps. (1) we began with the patterns provided by lama @xcite (one pattern per relation, referred to as basepattern). (2) we augmented each base-pattern with other patterns that are paraphrases from lpaqa @xcite . however, since lpaqa was created automatically (either by back-translation or by extracting patterns from sentences that contain both subject and object), some lpaqa patterns are not correct paraphrases. we therefore only include the subset of correct paraphrases. human agreement to assess the quality of pararel , we run a human annotation study. for we populate the patterns with random subjects and objects pairs from t-rex @xcite and ask annotators if these sentences are paraphrases. we also sample patterns from different relations to provide examples that are not paraphrases of each other, as a control. each task contains five patterns that are thought to be paraphrases and two that are not. 6 overall, we collect annotations for 156 paraphrase candidates and 61 controls. we asked nlp graduate students to annotate the pairs and collected one answer per pair. 7 the agreement scores for the paraphrases and the controls are 95.5% and 98.3%, respectively, which is high and indicates pararel 's high quality. we also inspected the disagreements and fixed many additional problems to further improve quality. ## analysis 7.1 qualitative analysis to better understand the factors affecting consistent predictions, we inspect the predictions of bert-large on the patterns shown in ## improving consistency in plms in the previous sections, we showed plms are generally not consistent in their predictions, and previous works have noticed the lack of this property in a variety of downstream tasks. an ideal model would exhibit the consistency property after pretraining, and would then be able to transfer it to different downstream tasks. we therefore ask: can we enhance current plms and make them more consistent? ## conclusion in this work, we study the consistency of plms with regard to their ability to extract knowledge. we build a high-quality resource named pararel that contains 328 high-quality patterns for 38 relations. using pararel , we measure consistency in multiple plms, including bert, roberta, and albert, and show that although the latter two are superior to bert in other tasks, they fall short in terms of consistency. however, the consistency of these models is generally low. we release pararel along with data tuples from t-rex as a new benchmark to track knowledge consistency of nlp models. finally, we propose a new simple method to improve model consistency, by continuing the pretraining with a novel loss. we show this method to be effective and to improve both the consistency of models as well as their ability to extract the correct facts. we also define several other types of paraphrases not covered in @xcite (potentially because they did not occur in the corpora they have inspected). a. irrelevant addition: addition or removal of a word or phrase, that does not affect the meaning of the sentence (as far as the relation of interest is concerned), and can be inferred from the context independently. b. topicalization transformation: a transformation from or to a topicalization construction. topicalization is a construction in which a clause is moved to the beginning of its enclosing clause. c. apposition transformation: a transformation from or to an apposition construction. in an apposition construction, two noun phrases where one identifies the other are placed one next to each other. d. other syntactic movements: includes other types of syntactic transformations that are not part of the other categories. this includes cases such as moving an element from a coordinate construction to the subject position as in the last example in we report the percentage of each type, along with examples of paraphrases in the full pararel resource can be found at: https://github.com/yanaiela/pararel /tree/main/data/pattern data/graphs json .