Explaining Answers with Entailment Trees (2021)

Abstract: Our goal, in the context of open-domain textual question-answering (QA), is to explain answers by showing the line of reasoning from what is known to the answer, rather than simply showing a fragment of textual evidence (a “rationale”). If this could be done, new opportunities for understanding and debugging the system’s reasoning become possible. Our approach is to generate explanations in the form of entailment trees, namely a tree of multipremise entailment steps from facts that are known, through intermediate conclusions, to the hypothesis of interest (namely the question + answer). To train a model with this skill, we created ENTAILMENTBANK, the first dataset to contain multistep entailment trees. Given a hypothesis (question + answer), we define three increasingly difficult explanation tasks: generate a valid entailment tree given (a) all relevant sentences (b) all relevant and some irrelevant sentences, or (c) a corpus. We show that a strong language model can partially solve these tasks, in particular when the relevant sentences are included in the input (e.g., 35% of trees for (a) are perfect), and with indications of generalization to other domains. This work is significant as it provides a new type of dataset (multistep entailments) and baselines, offering a new avenue for the community to generate richer, more systematic explanations.

## introduction explanation remains a formidable challenge in ai. while today's explanation systems are good at providing a sentence or two of supporting evidence ("rationales") for an answer @xcite , they rarely explain the chain of reasoning from what is known to the answer, i.e., how the answer follows, given the evidence -the goal of this work. without this, it is hard to fully understand a system's response and/or pinpoint the source of errors if its conclusions are wrong. conversely, if a system could support its answers with a chain of reasoning, new opportunities arise for interactively teaching the machine by debugging its mistakes. our approach is to generate explanations in the form of multistep entailment trees, such as shown in figure 1 , made up of individual, multi-premise textual entailment (te) steps @xcite . although there are many singlestep entailment datasets available @xcite no dataset of multistep entailments exists, and so a significant contribution of this paper is the construction of such a dataset, called entailmentbank. entailmentbank contains 1,840 multistep entailment trees for accompanying qa pairs, constructed using expert annotators, and is the first dataset of its kind. we also define three explanation tasks over this dataset, namely: generate a valid entailment tree for a given property↓, @xmath0, r4c 4 strategyqa 5 entailmentbank semantics of inference (informal) 1-step entailment (informal) deduction entailment tree average facts per inference 5.6 2.0 2.4 2.9 7.6 average edges per inference 9 ‡ 1 2 ‡ 2 6 granularity of inference fine coarse coarse coarse fine explicit ordering of inference no no no yes yes authoring method expert crowd crowd crowd expert 1 (xie et al., 2020) 2 (jhamtani and clark, 2020) 3 @xcite 4 @xcite 5 @xcite qa pair given (a) all relevant sentences (the leaves of the gold entailment tree), (b) all relevant and some distractor sentences, or (c) a full corpus. our focus here is on generating the derivation (line of reasoning) showing how the evidence leads to the answer, rather than the pragmatics of deciding which parts of that to then show the user. this allows us to separate two (typically confounded) explanation requirements, namely correctness (of the derivation) from utility, allowing us to evaluate derivations with a more objective measure @xcite . this also sets the stage for future work on the pragmatics of what to show users @xcite . finally, we define and train generative models, called entailmentwriters, for this task, adapting earlier techniques for generating deductive proofs @xcite . we find the models partially solve the dataset, with indications of generalization to other domains. our contributions are thus: this work is significant as it provides a new avenue for the community to generate richer, more systematic explanations. ## related work in the context of qa, there are multiple notions of explanation/justification, including showing an authoritative, answer-bearing sentence @xcite , an attention map over a passage @xcite , a synthesized phrase connecting question and answer @xcite , or the syntactic pattern used to locate the answer @xcite . these methods are primarily designed for answers to "lookup" questions, to explain where/how an answer was found in a corpus. for questions requiring inference, the focus of this paper, an explanation is sometimes taken as the chain of steps (typically sentences) leading to an answer. because crowdsourcing such chains is difficult, existing datasets typically simplify the task, e.g., collecting answer-supporting sentences but not how they combine, and/or largely focusing on one-hop (length 2) chains. here we generalize to tasks requiring multi-step entailment trees, our trees are built from multi-premise entailments (two or more sentences entail a hypothesis), introduced by @xcite , in contrast to the majority of prior datasets where typically a single sentence entails h through (typically) paraphrasing @xcite @xcite . we extend multi-sentence entailment in two ways. first, our trees also show the provenance of each entailment, namely which sentences are involved in each entailment (i.e., going beyond a classification task). second, ours is the first dataset that chains multiple entailments together into a hypothesis-directed tree, rather than containing separate, single-step entailments. recent work in deductive reasoning has shown that transformers can generate formal proofs with high reliability, both in a formal setting @xcite and with rules expressed in natural language @xcite . inspired by this, we apply similar ideas to generating entailment trees, in particular leveraging the generative techniques used in the proofwriter system @xcite (section 5). ## the entailmentbank dataset entailmentbank contains two parts: 1,840 entailment trees, each tree showing how a questionanswer pair (qa) is entailed from a small number of relevant sentences (e.g., figure 1 ); and a general corpus c, containing those and other sentences of domain-specific and general knowledge relevant to the qa domain. we use these two parts shortly to define a simpler task (generate the tree given the leaf sentences, without/with distractors) and a harder task (generate the tree from the corpus). entailmentbank uses multiple-choice questions (and the correct answer option) from the arc dataset of grade-school science questions @xcite , and a corpus of science-and general knowledge derived from worldtree v2 @xcite . worldtree was created for grade-school level science, making it an ideal source for entailmentbank's corpus. ## task definitions because producing correct entailment trees from a corpus is challenging, we define three tasks of increasing difficulty that simplify the problems inherent in the task. the inputs to all three are a hypothesis h, namely a declarative form of a question + answer (qa), foot_3 and some sentences s expressing (both relevant and irrelevant) knowledge. the desired output is a valid entailment tree t where the leaves are sentences selected from s, the intermediate nodes int i are intermediate conclusions (new sentences, not part of the input), and the root node (conclusion) is the hypothesis h. t is valid if every node n i in the tree is entailed by its children. the 3 tasks vary by the size of s, described below. as an approximation to make automated evaluation feasible, we ensure that s includes all the leaf sentences s gold that are in the gold entailment tree t gold , and treat t gold (+ valid reorderings) as the only valid entailment tree constructable from that input. this allows us to check validity by comparing the generated tree with t gold . this approximation is reasonable for tasks 1 and 2 below, because their limited input makes it unlikely that an alternative valid tree is constructable from the input. for task 3, though, to avoid alternative valid trees being buildable from the input corpus, we remove the few sentences similar to s gold from the corpus on a per-question basis. although these steps are not fool-proof, they do allow tree validity to be reasonably approximated by comparing with t gold , a critical requirement for automatic evaluation. the three tasks' inputs are thus as follows: task 1 (no-distractor): @xmath1: @xmath2: @xmath3. for our experiments, c is the worldtree corpus plus all additional science facts created by the annotators (section 3.2). 5 the desired output in all cases is a valid entailment tree t , approximated as being the gold entailment tree t gold (+ valid reorderings). ## model inspired by the "all-at-once" sequence-to-sequence model in the proofwriter system @xcite , we train three t5-based generative models (one per task), called entailmentwriters. ## experiments we train and test three entailmentwriters, one for each task. the model inputs are those described earlier for the three tasks, with the exception of task 3 where a retrieval step is inserted (the corpus c is too large to be input directly to t5). for this, we retrieve 25 sentences from c using qa as the query (using a roberta-trained relevant sentence ranker, details in appendix a), and input those to the model. the output in all cases is the entailment tree explaining (h, the declarative form of) qa. ## generality to what extent can entailmentbank help for other domains? although a full investigation is out of the scope of this paper, we performed two small out-of-domain (ood) investigations. first, we used the eqasc dataset @xcite . eqasc provides a qa hypothesis h, 10 retrieved sentences c, and lists valid single step entailments from two sentences in c to h -i.e., one-step (three node) entailment trees. running our task 2 entailmentwriter to generate a tree given c and h, we find (comparing with the best-matching input: (a qa pair) q: i would like to go running, but my ankle is injured. what should i do? a: ice it output:(explanation generated by entailmentwriter) icing an injured body part can help to reduce the pain an ankle is a kind of body @xmath4: (a qa pair) q: if i put some cheese in the fridge, will it melt? a: no, because it will be too cold output:(explanation generated by entailmentwriter) if something is put in a cold place then it will remain cold a refrigerator is used for cooling @xmath5. cheese is put in the @xmath6: (a qa pair) q: i feel sad. what could i do to cheer myself up? a: listen to music output: (explanation generated by entailmentwriter) music is a kind of entertainment entertainment can be used to cheer up a @xmath7, they may want to cheer themselves @xmath8, they may want to cheer themselves up by listening to music gold tree) a f1 (leaves) of 67% and an overall allcorrect score of 26% -a positive indication of transfer ood. note that this is without fine-tuning on eqasc, and that eqasc does not list all valid entailments, hence good outputs may be missed. we also trained a no-context version of entail-mentwriter using entailmentbank, that inputs just a qa pair and outputs a tree, generating all the tree sentences (both leaves and intermediates). we then ran this on challenge300, an existing, independently authored dataset of 300 test questions covering multiple domains @xcite . from a manual evaluation of a random sample of generated trees, ≈35% were valid, non-vacuous trees. (≈ 25% of the remainder were valid but largely repeated the question and answer). three good examples are shown in figure 5 , again illustrating the potential of entailmentbank for explanation. finally, as an experiment in interactive explanation generation, we re-purposed entailment-bank to train a model to generate an explana-tion one step at a time. to do this, we "shredded" the entailment trees into individual one-deep trees (where the intermediate nodes become new hypotheses to prove), and re-trained a model to generate similar one-deep entailment trees. this model can then be used interactively, generating a one-deep explanation then allowing a user to select which premise(s) to drill down into, based on what he/she wants to know more about, recursively calling the model to explain that premise further. although such generative models (both generating a full tree or a one-deep tree) can sometimes produce false or nonsensical facts, one could apply fact verification techniques, e.g., @xcite , to validate the generated facts, and generate an alternative explanation if validation fails. these are exciting future directions that we are exploring. ## summary and conclusion our goal is to enable machines to generate richer, more systematic explanations. to this end, we have developed a novel formulation of explanations as multistep entailment trees, and created entail-mentbank, the first large dataset of such trees. we have also presented baseline results for automatically generating entailment tree explanations for answers to science questions, trained on en-tailmentbank. these initial results suggest that such generation is possible, in particular when the necessary raw facts are included in the model input. we have also presented indications that models trained on entailmentbank can generalize to other domains. this suggests exciting opportunities for future systems that can help users understand and debug a system's answers, and ultimately engage in meaningful dialogs that explore the machine's line of reasoning. entailment-bank contributes to this direction, offering a new resource for developing richer, more systematic explanations. entailmentbank is available at https://allenai.org/data/entailmentbank .