Evaluating Attribution in Dialogue Systems: The BEGIN Benchmark (2022)

Abstract: Knowledge-grounded dialogue systems powered by large language models often generate responses that, while fluent, are not attributable to a relevant source of information. Progress towards models that do not exhibit this issue requires evaluation metrics that can quantify its prevalence. To this end, we introduce the Benchmark for Evaluation of Grounded INteraction (Begin), comprising 12k dialogue turns generated by neural dialogue systems trained on three knowledge-grounded dialogue corpora. We collect human annotations assessing the extent to which the models’ responses can be attributed to the given background information. We then use Begin to analyze eight evaluation metrics. We find that these metrics rely on spurious correlations, do not reliably distinguish attributable abstractive responses from unattributable ones, and perform substantially worse when the knowledge source is longer. Our findings underscore the need for more sophisticated and robust evaluation metrics for knowledge-grounded dialogue. We make Begin publicly available at https://github.com/google/BEGIN-dataset .

## introduction neural language models @xcite @xcite often form the backbone of open-ended dialogue systems @xcite @xcite . utterances sampled from such language models sound natural, as reflected in these systems' high scores in human evaluations focused on measures such as ''engagingness'' or ''human-likeness'' @xcite . while fluent, however, the responses generated by these systems often contain statements that are not supported by the evidence available to the system; such statements are sometimes referred to informally as ''hallucinations'' @xcite @xcite @xcite see figure 1 for an example). this issue is particularly salient for knowledge-grounded dialogue systems, which are expected to interact with a user in an open-ended fashion while conveying information that is attributable to external identifiable sources. in this work, we develop a benchmark that can be used to assess attribution in knowledge-based dialog systems; following @xcite , we define an attributable response foot_0 as one connected to textual evidence that supports the entirety of the response. a number of modeling approaches have recently been proposed to increase attribution in knowledgegrounded dialog systems @xcite @xcite . progress in this area crucially relies on metrics that can measure the attribution of the text generated by the system; and indeed, recent work has developed automated metrics with relatively high correlations with human annotations, potentially paving the way for alternatives to expensive human evaluations @xcite @xcite ). yet our understanding of these recently proposed metrics, as well as more established ones, remains limited, for two reasons. first, comparisons between automated metrics and human judgments rely on small-scale datasets with a few hundred examples. this results in high variance in our estimate of the correlation coefficient and a limited ability to measure performance on infrequent example types @xcite . second, the correlation with human scores does not sufficiently determine the efficacy and figure 1 : an example of a response generated by the gpt2 language model fine-tuned on the wizard of wikipedia dataset @xcite . the phrases in red are ''hallucinations'' unsupported by the background document. we define a taxonomy dividing knowledgegrounded dialogue responses into three broad categories-fully attributable, not fully attributable, and generic-and ask humans to classify a large set of utterances produced by dialogue systems with this taxonomy. the motivation for the generic category we introduce-which is assigned to utterances such as ''sorry, i'm not sure about this topic''-is the intuition that evaluation metrics should not treat the basic elements of a natural-sounding conversation, such as backchanneling or acknowledgment @xcite @xcite , as equally undesirable as a misleading unattributable statement. in real-world scenarios, it is preferable for a model to acknowledge its ignorance instead of producing hallucinated content which may lead to the spread of disinformation. using this taxonomy, we then collect highquality human annotations for 12k examples generated by four language-model-based dialogue systems, each trained on three different knowledge-grounded dialogue corpora. examples of machine-generated responses along with labels are presented in in knowledge-grounded response generation, the system is given a dialogue history @xmath0, . . . , u n-1 ), and knowledge k @xmath1, . . . , k j ) at turn n, and is expected to generate a response ūn that is coherent with h and attributable to a non-empty subset m n ⊂ k n . similar to the conversational qa task @xcite , the system is expected to use knowledge to respond to the user query. however, since the previous utterance may be an open-ended statement rather than a direct question (see the second and third examples in begin consists of responses generated by language-model-based systems trained to perform this task. this section describes the models we train on this task and the corpora we use to train them. ## annotations we next describe the human annotations we collected for the utterances generated by the models described in section 2. ## evaluating evaluation metrics we next use begin to evaluate a range of evaluation metrics. in §4.1 we list the untrained metrics we use as well as metrics trained on existing resources, and in §4.2 we describe a training set that we designed to train a classifier for the three response categories. we then describe the extent to which these metrics align with the begin categories and analyze the metrics' robustness. ## conclusion contemporary knowledge-based dialogue systems that rely on language models often generate responses that are not attributable to the background knowledge they are expected to convey. we present begin, a new benchmark to advance research toward robust metrics that can assess this issue. we use begin to comprehensively evaluate a broad set of existing automatic metrics. we show that these metrics rely substantially on word overlap and fail to properly rank abstractive attributable responses as well as generic responses. they also struggle under distribution shift, assigning low scores to attributable responses grounded on long knowledge sources. we hope that this work will spur future research on building robust evaluation metrics for grounded dialogue systems. nikolaev, sebastian gehrmann, roee aharoni, jennimaria palomaki, tom kwiatkowski, michael collins, and slav petrov for helpful discussions and feedback. we also thank ashwin kakarla and his team for helping with the annotations.