A Study of Automatic Metrics for the Evaluation of Natural Language Explanations (2021)

Abstract: As transparency becomes key for robotics and AI, it will be necessary to evaluate the methods through which transparency is provided, including automatically generated natural language (NL) explanations. Here, we explore parallels between the generation of such explanations and the much-studied field of evaluation of Natural Language Generation (NLG). Specifically, we investigate which of the NLG evaluation measures map well to explanations. We present the ExBAN corpus: a crowd-sourced corpus of NL explanations for Bayesian Networks. We run correlations comparing human subjective ratings with NLG automatic measures. We find that embedding-based automatic NLG evaluation methods, such as BERTScore and BLEURT, have a higher correlation with human ratings, compared to word-overlap metrics, such as BLEU and ROUGE. This work has implications for Explainable AI and transparent robotic and autonomous systems.

## introduction the machine learning models and algorithms underlying today's ai and robotic systems are increasingly complex with their internal operations and decision-making processes ever more opaque. this opacity is not just an issue for the end-user, but also the creators and analysts of these systems. as we move towards building safer and more ethical systems, this lack of transparency needs to be addressed. one key trait of a transparent system is its ability to be able to explain its deductions and articulate the reasons for its actions in natural language (nl). as the area of explainable ai (xai) grows and is mandated (cf. the eu general data protection regulation's "right to explanation" @xcite and standardisation (cf. ieee forthcoming standard on transparency (p7001)), it has become ever more important to be able to evaluate the quality of the nl explanations themselves, as well as the ai algorithms they explain. furthermore, the importance of evaluating explanations has been emphasised by researchers within the social cognitive sciences @xcite @xcite . to date, explanations have mostly been evaluated by collecting human judgements, which is both time-consuming and costly. here, we view generating explanations as a special case of natural language generation (nlg), and so we explore mapping existing automatic evaluation methods for nlg onto explanations. we study whether general, domain-independent evaluation metrics within nlg are sensitive enough to capture the peculiarities inherent in nl explanations @xcite , such as causality; or whether nl explanations constitute a sui-generis category, thus requiring their own automatic evaluation methods and criteria. in this paper, we firstly present the exban dataset: a corpus of nl explanations generated by crowd-sourced participants presented with the task of explaining simple bayesian network (bn) graphical representations. these explanations were subsequently rated for clarity and informativeness, two subjective ratings previously used for nlg evaluations @xcite . the motivation behind using bn is that they are reasonably easy to interpret, are frequently used for the detection of anomalies in the data @xcite @xcite , and have been used to approximate deep learning methods @xcite , which we could, in turn, explain in natural language. secondly, we explore a wide range of automatic measures commonly used for evaluating nlg to understand if they capture the human-assessed quality of the corpus explanations. we then go on to discuss their strengths and weaknesses through quantitative and qualitative analysis. our contributions are thus as follows: (1) a new corpus of natural language explanations generated by humans, who are asked to interpret bayesian network graphical representations, accompanied by subjective quality ratings of these explanations. this corpus can be used in various application areas including explainable ai, general artificial intelligence, linguistics and nlp; (2) a study of methods for evaluating explanations through automatic measures that reflect human judgements; and (3) qualitative discussion into these metrics' sensitivity by examining specific explanations varying on the informativeness/clarity scales. ## related work explanations are a core component of human interaction @xcite @xcite . in the context of machine learning (ml), explanations should articulate the decision-making process of an ml model explicitly, in a language familiar to people as communicators @xcite . according to @xcite , three of the most common types of explanation are: (1) global explanations, which describe the overall behaviour of the entire model @xcite ; recently, various explanation systems provide different types of explanations for ai systems: the lime method visually explains how sampling and local model training works by using local interpretable model-agnostic explanations @xcite ; maple can provide feedback for all three types of explanations: example-based, local and global explanations @xcite ; clear explains a single prediction by using local explanations that include statements of key counterfactual cases @xcite . whilst these techniques and tools gain some ground in explaining deep machine learning, the explanations they provide are not necessarily aimed at the (non-expert) end-user and so are not always intuitive. nlg has traditionally been broken down into "what" to say (content selection) and "how" to say it (surface realisation) and can draw parallels with natural language explanations. in particular, it is important to gauge how much content or how many reasons to present to the user, to inform them fully without overloading them. for example, prior work has shown that people prefer shorter explanations that offer only sufficient detail to be considered useful @xcite . according to @xcite , how explainers generate and select explanations depends on socalled pragmatic influences of causes, and they found that people seem to prefer simpler and more general explanations. similarly, @xcite notes that simplicity and generality might be the key to evaluating explanations. this was partly the case described in @xcite , but here the users were experts and preferred to be given all possible reasons but as precise and brief as possible. it is clear from these prior works that explanations have to be evaluated in the context of the scenario, prior knowledge and preferences of the explainee, and the intent and goals of the explainer. these could be, for example, establishing trust @xcite , agreement, satisfaction, or acceptance of the explanation and the system @xcite . somewhat analogous to auto-generated explanations are the fields of summarisation of text @xcite and question-answering @xcite @xcite . this is because they provide users (expert and lay users) with various forms of summaries (visual or textual) and answers containing explanations to enable them to have a better understanding of content. summarisation methods and sentence compression techniques can help to build comprehensive explanations @xcite . with regards to evaluating these summarisation methods, @xcite proposed an evaluation metric that weighted the facts present in the source document according to the facts selected by a human-written (natural language) summary, by using contextual embeddings. this evaluation of text accuracy is indeed related to explanations because any explanation must contain enough statements to support decision-making and understanding. these statements should be accurate and true. the growing interest in the ai community to investigate the potential of nl explanations for bridging the gap between ai and hci has resulted in an increasing number of nl explanations datasets. the eli5 dataset foot_0 @xcite is composed of explanations represented as multisentence answers for diverse questions where users are encouraged to provide answers, which are comprehensible for a five-year-old. worldtree v2 foot_1 @xcite is a corpus of science-domain that contains explanation graphs for elementary science questions, where explanations represent interconnected sets of facts. cos-e foot_2 is a dataset of human explanations for commonsense reasoning in the form of natural language sequences and highlighted annotations @xcite . multimodal explanations datasets (vqa-x and act-x) contain textual and visual explanations from human annotators @xcite . e-snli foot_3 is a corpus of explanations built on the question: "why is a pair of sentences in a relation of entailment, neutrality, or contradiction?" @xcite . finally, the snli corpus foot_4 is a large annotated corpus for learning natural language inference @xcite , considered one of the first corpora of nl explanations. in this paper, we present a new corpus for nl explanations. the exban corpus presented here provides a valuable addition to this set of corpora as it is aimed at explaining structured graphical models (in particular bayesian networks), that are closely linked to ml methods. ## exban corpus the exban corpus (explanations for bayesian networks) foot_5 consists of nl explanations collected for step 1, each subject was shown graphical representations of three bayesian networks (bn), in random order. they were then asked to produce text to describe how they interpreted the bn. the three bn used in the data collection are presented in figure 1 and represent well-known bn examples, extracted from @xcite . for step 2 in a separate experiment, approximately 80 of these generated explanations were presented to a different set of subjects in random order, along with a scenario description and the graphical model image. subjects were asked to rate them in terms of informativeness and clarity. the worded scenario descriptions were not given to subjects in the first stage, so as not to prime them when generating explanations. ## nlg evaluation metrics here, we describe the reasoning behind our choice of subjective measures that attempt to capture both the content and its correctness (informativeness) and quality of expression (clarity). we also describe objective measures commonly used for automatic evaluation of nlg, and which we will extract from the exban corpus. ## correlation study of automatic metrics as noted in the introduction, it remains an open question as to what degree the automatic metrics for nlg reviewed above can capture the quality of nl explanations @xcite . thus, we ran a correlation analysis to investigate the degree to which each of the automatic metrics correlates with human judgements using the exban corpus, and which aspects of human evaluation (clarity/informativeness), such automatic measures can capture. with regards to the choice of gold standard text, we picked explanations that received the maximum score in the human evaluation, in both clarity and informativeness. gold standard explanations of each diagram are presented in figure 1 . ## conclusions and future work human evaluation is an expensive and timeconsuming process. on the other hand, automatic evaluation is a cheaper and more efficient method for evaluating nlg systems. however, finding accurate measures is challenging, particularly for explanations. we have discussed word embedding techniques @xcite @xcite , which enable the use of pre-trained models and so reduces the need to collect large amounts of data in our domain of explanations, which is a challenging task. the embedding-based metrics mentioned here perform better than the word-overlap based ones. we speculate that this is in part due to the fact that the former capture semantics more effectively and are thus more invariant to paraphrases. these metrics have also been shown to be useful across multiple tasks @xcite but with some variation across datasets @xcite . therefore, future work would involve examining the effectiveness of automatic metrics across a wider variety of explanation tasks and datasets, as outlined in the related work section. embeddings are quite opaque in themselves. whilst some attempts have been made to visualise them @xcite , it remains that embeddingbased metrics do not provide much insight into what makes a good/bad explanation. it would thus be necessary to look more deeply into the linguistic phenomena that may indicate the quality of explanations. in exban, initial findings show that the number of nouns and coordinating conjunctions correlate with human judgements, however further in-depth analysis is needed. additional metrics to add to the set explored here could include grammar-based metrics, such as readability and grammaticality, as in the study described in @xcite . furthermore, an investigation is needed into the pragmatic and cognitive processes underlying explanations, such as argumentation, reasoning, causality, and common sense @xcite . investigating whether these can be captured automatically will be highly challenging. we will explore further the idea of adapting explanations to the explainee's knowledge and expertise level, as well as the explainer's goals and intentions. one such goal of the explainer could be to maximise the trustworthiness of the explanation @xcite . how this aspect is consistently subjectively and objectively measured will be an interesting area of investigation. finally, the exban corpus and this study will inform the development of nlg algorithms for nl explanations from graphical representations. we will explore nlg techniques for structured data, such as graph neural networks and knowledge graphs @xcite . thus the corpus and metrics discussed here will contribute to a variety of fields linguistics, cognitive science as well as nlg and explainable ai.