Is Incoherence Surprising? Targeted Evaluation of Coherence Prediction from Language Models (2021)

Abstract: Coherent discourse is distinguished from a mere collection of utterances by the satisfaction of a diverse set of constraints, for example choice of expression, logical relation between denoted events, and implicit compatibility with world-knowledge. Do neural language models encode such constraints? We design an extendable set of test suites addressing different aspects of discourse and dialogue coherence. Unlike most previous coherence evaluation studies, we address specific linguistic devices beyond sentence order perturbations, which allow for a more fine-grained analysis of what constitutes coherence and what neural models trained on a language modelling objective are capable of encoding. Extending the targeted evaluation paradigm for neural language models (Marvin and Linzen, 2018) to phenomena beyond syntax, we show that this paradigm is equally suited to evaluate linguistic qualities that contribute to the notion of coherence.

## introduction statistical models trained on large amounts of data using the language modelling objective (predicting words in context) have shown to pick up an intriguing amount of implicit knowledge about other tasks, for example syntactic knowledge @xcite or world knowledge @xcite . they have also been shown to exhibit, within these tasks, interesting divergences from expectation and sensitivity to confounding factors (e.g. @xcite ). inspired by the recently released syntaxgym @xcite , which enables specific and standardised evaluation of syntactic knowledge encoded in such models, we explore whether similar methods can be applied to the study of discourse knowledge or coherence, i.e., constraints acting across sentence boundaries, as illustrated in (1) (where "#" marks the less acceptable variant). then he rode into the sunset. a common approach to coherence evaluation consists in shuffling the sentence order of a text, thereby creating incoherent text samples that need to be discriminated from the original @xcite . while this approach to creating incoherent test data is intuitive enough, recent studies suggest that it paints only a partial picture of what constitutes coherence @xcite @xcite . it does not pinpoint the qualities that make the shuffled text incoherent, it does not tell us which linguistic devices are at fault, emphasising the need to move beyond this technique. this paper aims to add to the growing body of research stressing the need for more qualitative evaluations of text coherence @xcite @xcite . we design different test suites created semiautomatically from existing corpora. this eases the burden of creating them from scratch and ensures the inclusion of multiple genres, crucially including dialogue data. each test suite addresses a hypothesis about an underlying linguistic device contributing to a text's coherence, i.e., choice of referring expressions, discourse connectives, and intention (speaker commitment). our contributions are the following: we our work thus eliminates the need for adapting and gathering various benchmark datasets by providing an easily extensible coherence evaluation framework that allows the use of existing test suites and the design of new ones. at the moment, all of the test suites reported below are in english, but we come back to possible extensions in section 5. our results are mixed: to the extent that the test suites effectively capture coherence, the examined models are neither systematically incoherent nor coherent. we take this as support for our claim that more and better linguistically informed test suites are needed in oder to fully understand if neural models actually do capture genuine coherence. we expect to develop our work further, but at this point, our contribution is a systematic framework that will allow us to do just that. the code to create our test suites can be found at https://github.com/annebeyer/ coherencegym . different syntactic phenomena can be evaluated by running models on different test suites. each test suite contains items with minimally different conditions, focusing on the specific phenomenon. an example item for number agreement is given below. coherence. while the notion of syntactic acceptability is well studied from a linguistic point of view and in terms of neural language model representations @xcite @xcite , inter alia), it remains less clear what neural models are capable of capturing when modelling language across sentence boundaries. there exists a large body of work in linguistics regarding different notions of coherence, such as the influence of coreference @xcite , inter alia), centering theory @xcite , discourse structure @xcite , and phenomena that connect utterances in dialogue, such as conversational maxims @xcite or speaker interaction @xcite . many of these are also mentioned by coherence evaluation studies, nonetheless they mostly revert to the use of some form of sentence-order variations @xcite @xcite . while some progress has been made towards incorporating more linguistically motivated test sets @xcite @xcite , most evaluation studies focus on models trained specifically on coherence classification and prediction tasks. language models. the recently proposed transformer language model gpt-2 @xcite has been shown to perform very well on many downstream language tasks. see et al. (2019) quantitatively evaluate gpt-2 as a language generator and find that it generally performs on par with a state-of-the-art neural story generation model. however, they also note that their automatic measures focus mostly on text diversity and stress the need for more qualitative evaluation methods for notions like text coherence. gpt-2 is also the basis of the recently proposed dialogue model dialogpt @xcite , which is fine-tuned on conversational data from reddit. @xcite argue that di-alogpt encodes several notions of dialogue quality, including coherence. they manually create several positive and negative follow-up utterances for certain dialog qualities (e.g. "wow, that's interesting!" or "i'm confused."). the likelihood of dialogpt outputting either of them is then used to give an overall score per quality. the notion of dialogue coherence, although shown to be among the most important for predicting overall dialogue quality, is found to be one of the hardest to predict using this method. the authors attribute this to the fact that coherence (or the lack thereof) is seldom verbalised, so the model is not able to associate this notion with specific follow-up utterances. we take this a step back and evaluate the evaluator in order to get a better understanding of which notions of coherence are actually implicitly encoded in dialogpt. we test gpt-2 and dialogpt on different notions of discourse and dialogue coherence by evaluating them on specifically designed test suites building on the syntaxgym methodology. ## coherence phenomena and test suites in this section, we describe the different coherence phenomena assessed by our test suites. for every test suite we first posit a hypothesis, which is coded into the suite's prediction section. next, we describe the dataset and the manipulation applied to create incoherent samples that exhibit a violation of coherence regarding the specific phenomenon. each subsection reports the results of the evaluated models on the respective test suite. as we evaluate models pre-trained on english data, our test suites are devised only in english as well. the first three test suites are based on existing methods or test sets that we integrate into the framework. the following three test suites are newly created. ## conclusions we revisit the targeted evaluation paradigm and create test suites focusing on specific coherence phenomena. each test suite contains minimal pairs of sequences that illustrate a specific component of coherence. we evaluate two transformer models for language and dialogue modelling based on the token level surprisal scores they assign to the coherent and incoherent versions. extending the existing syntaxgym toolkit, we evaluate gpt-2 and di-alogpt on our newly designed test suites on entity re-mention, explicit discourse connectives and speaker commitment in dialogue. existing test sets are also integrated easily, which we demonstrate for sentence order detection, story cloze and winograd schema resolution tasks. our results support previous work suggesting that the notion of coherence encoded in neural language models is more nuanced than the sentence order discrimination task can reflect. the mixed results we get, with some manipulations (e.g. the different sense connective substitutions) easily being spotted by the tested models and others (e.g. how to re-mention entities, or speaker contradictions) posing to be more difficult, point to the value of such targeted evaluation, which eventually might help in pointing towards where the introduction of different inductive biases could increase a model's performance. in this study, we focus on the english language. however, our approach is not inherently designed for english alone. while lm-zoo only contains english language models at the moment, other language models can be added easily. the shuffling perturbations can be applied to any corpus. our other test suites are based on available annotated corpora, which require some familiarity with the language, but can in principle be applied in a similar fashion to resources in other languages, such as the potsdam commentary corpus @xcite for german connectives, for example. we leave a multilingual extension of our framework for future work. our next efforts will focus on adding more language and dialogue models to determine the impact of different model architectures and sizes. building additional test suites in order to capture a more thor-ough notion of coherence is also among our priorities. last, we plan to collect human judgements to evaluate our coherence manipulations more closely and to create an upper bound for what we can expect from neural models.