BECEL : Benchmark for Consistency Evaluation of Language Models (2022)

Abstract: Behavioural consistency is a critical condition for a language model (LM) to become trustworthy like humans. Despite its importance, however, there is little consensus on the definition of LM consistency, resulting in different definitions across many studies. In this paper, we first propose the idea of LM consistency based on behavioural consistency and establish a taxonomy that classifies previously studied consistencies into several sub-categories. Next, we create a new benchmark that allows us to evaluate a model on 19 test cases, distinguished by multiple types of consistency and diverse downstream tasks. Through extensive experiments on the new benchmark, we ascertain that none of the modern pre-trained language models (PLMs) performs well in every test case, while exhibiting high inconsistency in many cases. Our experimental results suggest that a unified benchmark that covers broad aspects (i.e., multiple consistency types and tasks) is essential for a more precise evaluation.

## introduction human-like behaviour is a critical property that increases a user's trust in an artificial intelligence (ai) agent @xcite by improving the certification process that ascertains whether a system behaves correctly @xcite . 1 accordingly, despite the outstanding performance of transformer-based plms on natural language understanding (nlu) benchmarks, there have been pushbacks in various corners questioning their trustworthiness based on their nonhuman like behaviours, such as a poor memorisation effect on infrequent information @xcite @xcite , insensitivity to sentence order @xcite @xcite , and a miserable understanding of negation expressions @xcite @xcite @xcite . in this respect, behavioural consistency, a core property of humans, is an important characteristic for a model to be deemed as trustworthy lm. accordingly, the concept of consistency has been widely discussed in natural language processing (nlp). however, despite its prominent importance, there is little consensus on the precise definition of consistency. below are examples of different consistency definitions: to this end, in this paper, we first define the consistency of an lm based on the concept of behavioural consistency and establish a taxonomy by systematically categorising previous works based on our definition. next, we propose a new benchmark named benchmark for consistency evaluation of language models (becel), a unified dataset for evaluating an lm's consistency, which assesses multiple types of consistency on six different tasks: nli, semantic textual similarity (sts), words-incontext (wic), semantic analysis (sa), machine reading comprehension (mrc), and topic classification (tc). finally, we conduct extensive experiments on our new benchmark to assess the consistent behaviour of widely used plms and draw the following meaningful insights. ## language model's consistency behavioural consistency refers to being consistent in behavioural patterns by adhering to the same principles. 2 based on this notion, we define the consistency of an lm as its ability to make a coherent decision not contradictory to its belief. this definition of consistency consists of two components. the first one is belief, which refers to what a model considers to be true. the second component is principle, the property that decides what a coherent decision is. based on these two components, we classify the various types of consistency in the literature into three large categories: semantic, logical, and factual consistency. ## discussion are large models more consistent? it is wellknown that large-size models consistently outperform small-size models in terms of accuracy. does the same trend occur from a consistency perspective? figure 4 illustrates the portion of the three cases: the performance of the large models are better, worse, or show no statistical difference. interestingly, the case where there is no statistical difference in consistency between the large-and base-size models accounts for a large portion, and sometimes base-size models even perform better. this pattern is hardly seen in accuracy-based evaluation metrics, suggesting that additional evaluation metrics such as consistency other than the accuracy should be considered for a precise evaluation. necessity of a unified benchmark. our experimental results highlight the importance of evaluating models in a wide spectrum. we verify that none of the plms performs coherently well in every experiment, suggesting that focusing on a certain task or consistency type contains the risk of reaching a wrong conclusion. for instance, we might conclude that plms are fairly consistent if we only consider semantic consistency. if we conduct experiments only in the nli tasks like extant studies, the conclusion might be distorted, since the results of all inconsistency types seem reasonable in the nli tasks, especially in snli. our new dataset, however, prevents us from drawing such a fallacious conclusion by allowing us to assess models across multiple consistency types and tasks, demonstrating its importance to have a unified benchmark covering a wide array of topics including different evaluation criteria and task types. uncontrollable ai. due to the nature of inductive reasoning, the inductive bias of machine learning and deep learning models is greatly affected by the patterns in the training data. although this is well-known and widely accepted @xcite @xcite @xcite , our experimental results show that the artefacts in data are a more influential factor than the model design in deciding its inductive bias (section 4.3). the problem is that we have a control over the model design but not the artefacts, as it is difficult to review and manipulate all data points with an enormous size. this evokes a critical concern: uncontrollable ai. however elaborate the model that we design with highly advanced training objectives and model structures, we might not have a full control over the model, as the ungovernable effect of the artefacts in data remains. it is thus imperative to take appropriate actions to address the data-driven faulty behaviour of the model, such as the generation of ethically problematic outputs @xcite . to overcome such issues and move forward to developing more trustworthy and safer ai, perhaps it is time to think beyond inductive reasoning. ## summary and outlook in this work, we first defined lm consistency based on the concept of behavioural consistency: a core property that a sound lm should obey. next, we categorised various previous studies regarding consistency into three types: semantic, logical, and factual consistency. finally, we designed a benchmark suite to assess various types of consistency on multiple downstream tasks. through extensive experiments, we observed that none of the plms shows perfectly consistent outputs in all test cases. our experimental results highlight the essence of evaluation schema in multiple spectrums to avoid reaching a distorted conclusion. we also revealed that the impact of spurious artefacts presented in training data is greater than that of model design, such as model size and learning objective. this finding raises concerns about uncontrollable ai, as we have no control over the artefacts in tremendous amounts of data. our work suggests that we should probably go beyond neural models, which only allow inductive reasoning, to develop trustworthy and safe ai.