FRVA : Fact-Retrieval and Verification Augmented Entailment Tree Generation for Explainable Question Answering (2024)

Abstract: Structured entailment tree can exhibit the reasoning chains from knowledge facts to predicted answers, which is important for constructing an explainable question answering system. Existing works mainly include directly generating the entire tree and stepwise generating the proof steps. The stepwise methods can exploit combinatoriality and generalize to longer steps, but they have large fact search spaces and error accumulation problems resulting in the generation of invalid steps. In this paper, inspired by the Dual Process Theory in cognitive science, we propose FRVA, a Fact-Retrieval and Verification Augmented bidirectional entailment tree generation method that contains two systems. Specifically, System 1 makes intuitive judgments through the fact retrieval module and filters irrelevant facts to reduce the search space. System 2 designs a deductive-abductive bidirectional reasoning module, and we construct cross-verification and multi-view contrastive learning to make the generated proof steps closer to the target hypothesis. We enhance the reliability of the stepwise proofs to mitigate error propagation. Experiment results on EntailmentBank show that FRVA outperforms previous models and achieves state-of-the-art performance in fact selection and structural correctness.

## introduction automated reasoning, the process of reasoning from given explicit knowledge to generate valid conclusions, has always been a goal pursued in artificial intelligence @xcite @xcite . interpreting the reasoning process from question to answer can provide a human-understandable inspection for the qa system, which can help improve the debuggability and trustworthiness of the model. explainable question answering (xqa) aims to answer a question and give a corresponding explanation @xcite , and current related works focus on three aspects: extracting keywords or sentences that contain the answer @xcite , constructing the multi-hop explanation chain @xcite , and generating free-form explanations @xcite . among the various explanation methods, @xcite proposes to construct multi-step reasoning chains in the form of entailment tree, which shows how hypotheses (question + answer) can be explained from simple textual evidence. as shown in figure 1 , given a hypothesis and a set of supporting facts, the goal is to generate an entailment tree consisting of multiple premise proof steps, which has better expressive compared to other methods. for example, for the question "after fossils are formed, which process is most likely to destroy them?", the reasoning process can explain why the answer "the rock cycle" is predicted. one line of previous work converts a multi-step entailment tree into a linearized sequence @xcite @xcite , which directly outputs the entire reasoning chain using the sequence-to-sequence model (seq2seq). although this method simplifies the generation task, it is challenging to guarantee structural correctness when the steps are longer. therefore, subsequent works propose to use stepwise generation by splitting the tree into multiple steps and training the model to perform single-step reasoning @xcite @xcite @xcite . compared to generating the entire reasoning chain directly, the stepwise generation can more fully exploit the combinability of proofs, which makes the model easier to learn and generalize to longer proof steps. however, existing stepwise methods still face two challenges: (1) combination failure due to the large fact search space and (2) generation invalid steps due to error accumulation. the current works perform single-step reasoning mainly using deductive reasoning, which is a bottom-up process that requires iterative search and generation of intermediate conclusions from known facts until the hypothesis is proved. the search space grows with new conclusions to join the known facts, easily leading to combinatorial failure. moreover, the stepwise methods have the problem of error accumulation, e.g., the third step "sent7 & int2 -> int3" in figure 1 is not correct, which affects the next step generation. the model may quickly generate invalid proofs with the increase of proof steps, which leads to incorrectness of the tree structure. to tackle the above problems, we propose frva, a bidirectional stepwise proofs generation method. we simulate the cognitive process of human reasoning and divide the entailment tree generation into two systems. in system 1, we design a fact retrieval module, which scores all candidate facts from a global perspective and initially eliminates knowledge facts with low relevance to the hypothesis. in system 2, we propose a deductive-abductive bidirectional proof generator. in this process, forward deductive and backward abductive reasoning generate candidate steps and verify each other to satisfy bidirectional consistency. meanwhile, we design a multi-view contrastive loss integrating the local level and global level information to pull the semantic distance between the golden facts and hypotheses, which can help the model generate proof steps closer to the hypotheses. we improve the reliability of the stepwise proof step in terms of both supportiveness (crossverification) and similarity (contrastive learning). finally, we align and fuse the bidirectionally generated structure trees and select the best proof tree. experiments on entailmentbank show that frva can obtain more reliable and valid reasoning paths and outperform existing baselines regarding fact selection and structural correctness. we also demonstrate that the backward reasoning remains valid in proof discovery. in summary, our contributions are threefold: a series of recent works exploring stepwise generation methods include two main core components: fact sentence selection and intermediate conclusion generation. rlet @xcite introduces reinforcement learning to entailment tree generation for the first time. nlproofs @xcite introduces an independent verification mechanism to check the validity of the proof step and prevent the generation of hallucinatory invalid steps. irgr @xcite proposes an iterative retrieval-generation architecture, which combines the generated intermediate conclusions and premises for retrieval to generate entailment trees better. fld @xcite proposes a deductive rule generation method based on formal logic theory, which further enhances the deductive reasoning ability of the model. fame @xcite uses monte carlo planning to implement faithful question answering, but they focus on qa while we focus on explainability. prior works focus on forward reasoning, and metgen @xcite further reveals the effectiveness of backward reasoning. however, they discretely perform single-step proof generation (i.e., it needs to enumerate different combinations of facts to select the best single step). our method is also a bidirectional proof generation that enhances the reliability of stepwise proof steps by cross-verification and multi-view contrastive learning during the generation process. ## task formulation as shown in figure 1 , the input of the entailment tree generation task consists of the hy-pothesis h and a set of supporting facts @xmath0, sent 2 , . . . , sent n }. h is a declarative sentence consisting of question-answer pairs (qa), which can be proved by constructing one or more reasoning steps with the knowledge from s. the output is an entailment tree t where root nodes are the target hypotheses h, leaf nodes are the facts chosen from s, and intermediate nodes are the conclusions int i generated by the model. the tree t is considered valid if all non-leaf nodes can be effectively entailed by their children. the entailment tree can be formalized as @xmath1, j , m, v}, where the leaf node l @xmath2, the intermediate conclusion int @xmath3, and the proof step step @xmath4, e.g., "sent1 & sent2 -> int1"). we denote the gold entailment tree as t gold and all the leaf nodes it contains as the gold fact s gold . according to the different composition of the input supporting facts s, it is divided into three increasingly difficult tasks: task1 (no-distractor): @xmath5: @xmath6: @xmath7 ## method the reasoning process of frva is shown in figure 2 , which consists of two systems. system 1 retrieves relevant facts, and system 2 contains the bidirectional proof generator and the proof alignment and fusion module. we introduce the above modules in section 4.1, 4.2 and 4.3. ## conclusion in this paper, we propose frva, a bidirectional entailment tree generation method based on fact retrieval and verification augmented, which divides the generation process into two systems. system 1 makes the initial judgment, and system 2 makes the refined reasoning. we design a deductive-abductive bidirectional reasoning method and enhance the effectiveness of the proof step from supportive and similarity perspectives through cross-verification and multi-view contrastive learning. experiments show that frva outperforms the existing baselines regarding fact selection and structural correctness. it is worth exploring to design more efficient bidirectional reasoning methods and comprehensive evaluation systems in the future.