Do Natural Language Explanations Represent Valid Logical Arguments? Verifying Entailment in Explainable NLI Gold Standards (2021)

Abstract: An emerging line of research in Explainable NLP is the creation of datasets enriched with human-annotated explanations and rationales, used to build and evaluate models with step-wise inference and explanation generation capabilities. While human-annotated explanations are used as ground-truth for the inference, there is a lack of systematic assessment of their consistency and rigour. In an attempt to provide a critical quality assessment of Explanation Gold Standards (XGSs) for NLI, we propose a systematic annotation methodology, named Explanation Entailment Verification (EEV), to quantify the logical validity of human-annotated explanations. The application of EEV on three mainstream datasets reveals the surprising conclusion that a majority of the explanations, while appearing coherent on the surface, represent logically invalid arguments, ranging from being incomplete to containing clearly identifiable logical errors. This conclusion confirms that the inferential properties of explanations are still poorly formalised and understood, and that additional work on this line of research is necessary to improve the way Explanation Gold Standards are constructed.

## introduction explanation gold standards (xgss) are emerging as a fundamental enabling tool for step-wise and explainable natural language inference (nli). resources such as worldtree @xcite , qasc @xcite , among others @xcite @xcite provide a corpus of linguistic evidence on how humans construct explanations that are perceived as plausible, coherent and complete. designed for tasks such as textual entailment (te) and question answering (qa), these refer-figure 1 : does the answer logically follow from the explanation? while step-wise explanations are used as ground-truth for the inference, there is a lack of assessment of their consistency and rigour. we propose eev , a methodology to quantify the logical validity of human-annotated explanations. this paper aims to provide a critical quality assessment of eplanation gold standards for nli in terms of their logical inference properties. by systematically translating natural language explanations into corresponding logical forms, we induce a set of recurring logical violations which can then be used as testing conditions for quantifying quality and logical consistency in the annotated explanations. more fundamentally, the paper reveals the surprising conclusion that a majority of the explanations present in explanation gold standards contain one or more major logical fallacies, while appearing to be coherent on the surface. this study reveals that the inferential properties of explanations are still poorly formalised and understood. the main contributions of this paper can be summarised as: ## related work an emerging line of research in explainable nlp is focused on the creation of datasets enriched with human-annotated explanations and rationales @xcite . these resources are often adopted as explanation gold standards (xgss), providing additional supervision for training and evaluating explainable models capable of generating natural language explanations in support of their predictions @xcite @xcite @xcite . xgss are designed to support natural language inference, asking human-annotators to transcribe the reasoning required for deriving the correct prediction @xcite . despite the popularity of these datasets, and their application for measuring explainability on tasks such as textual entailment @xcite , multiple-choice question answering @xcite @xcite , and other inference tasks @xcite @xcite , little has been done to provide a clear understanding on the nature and the quality of the reasoning encoded in the explanations. previous work on explainability evaluation has mainly focused on methods for assessing the quality and faithfulness of explanations generated by deep learning models @xcite @xcite @xcite . our work is related to this research, but focuses instead on the resources on which explainable models are trained. in that sense, this paper is more aligned to gold standard evaluation methods, which aim to design systematic approaches to qualify the content and the inference capabilities involved in mainstream nlp benchmarks @xcite @xcite @xcite . however, to the best of our knowledge, none of these methods have been adopted to provide a critical assessment of humanannotated explanations present in xgss. ## explanation gold standards given a generic classification task t , an explanation gold standard (xgs) is a collection of distinct instances of t , xgs(t ) = {i 1 , i 2 , . . . , i n }, where each element of the set, i @xmath0, s i , e i }, includes a problem formulation x i , the expected solution s i for x i , and a human-annotated explanation e i . in general, the nature of the elements in a xgs can vary greatly according to the task t under consideration. in this work, we restrict our investigation to natural language inference (nli) tasks, such as textual entailment and question answering, where problem formulation, expected solution, and explanations are entirely expressed in natural language. for this class of problems, the explanation is typically a composition of sentences, whose role is to describe the reasoning required to arrive at the final solution. as shown in the examples depicted in figure 1 , the explanations are constructed by human annotators transcribing the commonsense and world knowledge necessary for the correct answer to hold. given the nature of xgss for nli, we hypothesise that a human-annotated explanation represents a valid set of premises from which the expected solution logically follows. multiple-choice question answering xgs question: differential heating of air can be harnessed for what? [*a] electricity production [b] erosion prevention [c] transfer of electrons [d] reduce acidity of food explanation: differential heating of air produces wind. wind is used for producing electricity. differential heating of air produces wind. wind is used for producing electricity. differential heating of air can be harnessed for electricity production. premises (p) conclusion (c) φ ψ formulas entailment? valid and non redundant valid, but redundant premises missing plausible premise logical error no discernible argument textual entailment xgs premise: a man in an orange vest leans over a pickup truck. hypothesis: a man is touching a truck. label: entailment explanation: man leans over a pickup truck implies that he is touching it. a man in an orange vest leans over a pickup truck. man leans over a pickup truck implies that he is touching it. a man is touching a truck. φ ψ formulas entailment? valid and non redundant valid, but redundant premises missing plausible premise logical error no discernible argument (1) problem definition, (2) formalisation, (3) verification. (1) (2) (3) (3) (2) (1) premises (p) conclusion (c) eev takes the form of a multi-label classification problem where, for a given nli problem, a human annotator has to qualify the validity of the inference process described in the explanation through a pre-defined set of classes. in order to validate or reject this hypothesis, we design a methodology aimed at evaluating xgss in terms of logical entailment, quantifying the extent to which human-annotated explanations actually entail the final answer. ## explanation entailment verification we present a methodology, named explanation entailment verification (eev ), aimed at quantifying and assessing the quality of human-annotated explanations in xgs for nli tasks, in terms of their logical inference properties. to this end, we design an annotation framework that takes the form of a multi-label classification problem defined on a xgs. specifically, the goal of eev is to label each element in a xgs, i @xmath1, s i , e i }, using one of a predefined set of classes qualifying the validity of the inference process described in the explanation e i . figure 2 shows a schematic representation of the annotation pipeline. one of the challenges involved in the design of a standardised methodology for eev is the formalisation of an annotation task that is applicable to nli problems with different shapes, such as textual entailment (te) and multiple-choice question answering (mcqa). to minimise the ambiguity in the annotation and make it independent of the specific nli task, we define a methodology composed of three major steps: (1) problem definition; (2) formalisation; and (3) verification. in the problem definition step, each example i i in the xgs is translated into an entailment form (p |= c), identifying a set of sentences p representing the premises for the entailment, and a single sentence c representing its conclusion. as illustrated in figure 2 , this step defines an entailment problem with a single surface form that allows abstracting from the nli task under investigation. in the formalisation step, the sentences in p and c are translated into a logical form (φ |= ψ). specifically, the formalisation is performed using event-based semantics, in which verbs correspond to event-types, and their objects to semantic roles (additional details on the formalism are provided in section 4.3). this step aims to minimise the ambiguity in the interpretation of the meaning of the sentences, supporting the annotators in the identification of logical errors and gaps in the explanations, and maximise the inter-annotator agreement in the downstream verification task. the final step corresponds to the actual multilabel classification problem. specifically, the annotators are asked to verify whether the formalised set of premises φ entails the conclusion ψ (φ |= ψ) and to classify the explanation in the corresponding example after eev is performed for each instance in the dataset, the frequencies of the classification labels can be adopted to estimate and evaluate the overall entailment properties of the explanations in the xgs under consideration. ## corpus analysis we employ eev to analyse a set of contemporary xgss designed for textual entailment and multiple-choice question answering. in the following sections, we describe the methodology adopted for extracting a representative sample from the selected xgss, and for implementing the annotation pipeline efficiently. finally, we present the results of the annotation, reporting the frequency of each entailment verification class and presenting a list of qualitative examples to provide additional insights on the logical properties of the analysed explanations. ## conclusion and future work this paper proposed a systematic annotation methodology to quantify the logical validity of human-annotated explanations in explanation gold standards (xgss). the application of the framework on three mainstream datasets led us to the conclusion that a majority of the explanations represent logically invalid arguments, ranging from being incomplete to containing clearly identifiable logical errors. the main limitation of the framework lies in the scalability of its current implementation, which is generally time consuming and requires trained semanticists. one way to improve its efficiency is to explore the adoption of supporting tools for the formalisation, such as semantic parsers and/or automatic theorem provers. despite the current limitations, this study offers some important pointers for future work. on the one hand, the results suggest that logical errors can be reduced by a careful design of the gold standard, such as authoring explanations with internal verification strategies or reducing the complexity of the reasoning task. on the other hand, the finding that a large percentage of curated explanations still represent incomplete arguments has a deeper implication on the nature of explanations and on what annotators perceive as a valid and complete logical argument. therefore, we argue that future progress on the design of xgss will depend, among other things, on a better formalisation and understanding of the inferential properties of explanations.