Beyond the Tip of the Iceberg: Assessing Coherence of Text Classifiers (2021)

Abstract: As large-scale, pre-trained language models achieve human-level and superhuman accuracy on existing language understanding tasks, statistical bias in benchmark data and probing studies have recently called into question their true capabilities. For a more informative evaluation than accuracy on text classification tasks can offer, we propose evaluating systems through a novel measure of prediction coherence. We apply our framework to two existing language understanding benchmarks with different properties to demonstrate its versatility. Our experimental results show that this evaluation framework, although simple in ideas and implementation, is a quick, effective, and versatile measure to provide insight into the coherence of machines’ predictions.

## introduction large-scale, pre-trained contextual language representations @xcite @xcite have approached or exceeded human performance on many existing language understanding benchmarks. however, due to increasing complexity and concerns of statistical bias enabling artificially high performance @xcite @xcite , the coherence of these state-of-the-art systems and their alignment to humans is not well understood. this is perhaps because benchmarks geared toward language understanding only cover the tip of the iceberg, typically focusing on a high-level end task rather than diving deeper into the kind of coherent, robust understanding that takes place in humans. language understanding in machines is often boiled down to text classification, where a classifier is tasked with recognizing whether a text contains a particular semantic class, e.g., textual entailment @xcite , commonsense implausibility @xcite @xcite in conversational entailment @xcite , systems only predict whether a hypothesis is entailed by a dialog, while ignoring the underlying evidence in the discourse toward this conclusion. et al., 2020), or combinations of several phenomena meant to serve as comprehensive diagnostics @xcite @xcite . without regard to the underlying evidence used to reach a conclusion, systems are rewarded for correct predictions on the task without "showing their work." to make meaningful improvement on machine language understanding, it is important to have more informative performance measures. to address this issue, the contribution of this paper is to introduce a novel model-and task-agnostic evaluation framework that allows a quick assessment of text classifiers' ability in terms of the coherence of their predictions. we apply our framework to two existing language understanding benchmarks of different genres to demonstrate its versatility. our results support recent findings of spurious behaviors in fine-tuned large lms, and show that our framework, although simple in ideas and implementation, is effective as a quick measure to provide insight into the coherence of machines' predictions. ## related work in the face of data bias and uninterpretability of large lms, past work has proposed methods to robustly interpret and evaluate them for various tasks and domains. some work has sought to probe contextual language representations through various means to better understand what knowledge they hold and their correspondence to syntactic and semantic patterns @xcite @xcite . meanwhile, behavior testing approaches have also been applied to understand model capabilities, from automatically removing words in language inputs and examining model performance as the input becomes malformed or insufficient for prediction @xcite @xcite , to curating finegrained testing data to measure performance on interesting phenomena @xcite . similar work has used specialized natural language inference tasks @xcite , logic rules @xcite , and annotated explanations @xcite to support and evaluate consistency and coherence of inference in these models. other works have studied coherence of discourse through the proxy task of sentence re-ordering @xcite . different from these previous works that focus only on specific tasks or methods, or require heavy annotation, this paper introduces an easily-accessed, versatile evaluation of machine coherence from a small amount of additional annotation. ## coherent text classification for any text classification task requiring reasoning over a discourse, a coherent classifier should use the same evidence as humans do in reaching a conclusion. for any positive example, we expect that there are specific regions of the text which contain the semantic class of interest and thus directly contribute to the positive label. conversely, for any negative example, there should be no such regions of the text. at a high level, we will propose a coherence measure that captures whether classifiers can give consistent and human-aligned predictions on these regions to support the end task conclusion. depending on specific tasks, this measure can have different implementations while maintaining the same high-level goal. in the following sections, we will use two example benchmark datasets, conversational entailment (ce) from @xcite and abductive reasoning in narrative text (art) from @xcite , to illustrate dialog: a 1 : well, ironically enough i'm sitting here with a cast on my leg because i resumed an aerobics class the night before last. b 1 : oh, no. a 2 : i ripped the ligaments in my right ankle. ## coherence of sota classifiers using our framework, we next establish baseline measures of coherence on the two benchmarks. the source code and data for our empirical study are shared with the community on github. foot_3 ## conclusion in this work, we proposed a simple and versatile method to evaluate the coherence of text classifiers, particularly targeting the problem where end task prediction depends on a discourse rather than a single sentence. by annotating a small amount of data in a benchmark, this method supports a quick assessment on whether machines' end task performance is supported by coherent intermediate evidence. future work driven by benchmarks should consider similar examination beyond the end task accuracy, whether this be through our proposed coherence measures or other appropriate means. as we showed, such effort is quite straightforward, and can drive progress toward more powerful classifiers that can support human-aligned reasoning.