Reasoning over Hierarchical Question Decomposition Tree for Explainable Question Answering (2023)

Abstract: Explainable question answering (XQA) aims to answer a given question and provide an explanation why the answer is selected. Existing XQA methods focus on reasoning on a single knowledge source, e.g., structured knowledge bases, unstructured corpora, etc. However, integrating information from heterogeneous knowledge sources is essential to answer complex questions. In this paper, we propose to leverage question decomposing for heterogeneous knowledge integration, by breaking down a complex question into simpler ones, and selecting the appropriate knowledge source for each sub-question. To facilitate reasoning, we propose a novel two-stage XQA framework, Reasoning over Hierarchical Question Decomposition Tree (RoHT). First, we build the Hierarchical Question Decomposition Tree (HQDT) to understand the semantics of a complex question; then, we conduct probabilistic reasoning over HQDT from root to leaves recursively, to aggregate heterogeneous knowledge at different tree levels and search for a best solution considering the decomposing and answering probabilities. The experiments on complex QA datasets KQA Pro and Musique show that our framework outperforms SOTA methods significantly, demonstrating the effectiveness of leveraging question decomposing for knowledge integration and our RoHT framework.

## introduction explainable question answering (xqa) is the task of (i) answering a question and (ii) providing an explanation that enables the user to understand why the answer is selected @xcite . it provides a qualified way to test the reasoning ability and interpretability of intelligent systems, and plays an important role in artificial intelligence @xcite . recent work in xqa can be grouped into two directions: 1) neuro-symbolic methods @xcite @xcite ) translate natural language questions into formal representations (e.g., sparql @xcite , kopl @xcite , lambda-dcs @xcite , etc.), whose execution on structured knowledge bases (kbs) gives the answer. here, the formal representation acts as an explanation of the final answer. 2) decompose-based models generate natural language intermediate steps that lead to the final answer (e.g., question decomposing which decomposes a complex question into sub-questions @xcite @xcite , chain-of-thought prompting @xcite @xcite , etc.). here, the intermediate steps shows the rationale of reasoning. although achieving significant results, both directions have key limitations. for neuro-symbolic methods, the formal representation can only be executed on kbs. however, even the largest kbs are incomplete, thus limits the recall of model. for decompose-based methods, they employ free-text corpora as the knowledge source, and the diversity of natural language makes xqa difficult. in fact, integrating knowledge from heterogeneous sources is of great importance to qa @xcite , especially for answering complex questions. several attempts have been made for knowledge integration (e.g., kbs, text corpora) @xcite @xcite . although promising, these graph-based methods suffer from lacking explainability or are constrained to limited reasoning capability. intuitively, leveraging question decomposing to integrate heterogeneous knowledge sources is a promising direction, since we can flexibly select the appropriate knowledge source for each subquestion. the challenges lie in: 1) how to determine the granularity of question decomposing, since certain complex questions can be directly answered with a knowledge source, and further decomposition increases the possibility of error. for example, in figure 1 , q 1 can be answered with the wikipedia corpus without further decomposition. to this end, we propose a novel two-stage xqa framework reasoning over hierarchical question decomspotion tree, dubbed roht. first, we propose to understand the complex question by building its hierarchical question decomposition tree (hqdt). in this tree, the root node is the original complex question, and each non-root node is a subquestion of its parent. the leaf nodes are atomic questions that cannot be further decomposed. compared with existing representations that directly decompose a question into the atomic ones, e.g., qdmr @xcite , our tree structure provides the flexibility to determine solving a question whether by directly answering or further decomposing. second, we propose probabilistic reasoning over hqdt, to fuse the knowledge from kb and text at different levels of the tree, and take into consideration the probability score of both tree generation and answering. the reasoning process is recursive, from the root to leaves, and constitues three steps: 1) a scheduler determines the appropriate knowledge sources for a particular question (from kb, text, or solving its children sequentially); 2) the corresponding executors output the answers with probabilities; 3) an aggregator aggregates the candidate answers from all the knowledge sources and outputs the best ones. in evaluation, we instantiate our roht framework on two complex qa datasets: kqa pro @xcite , where we remove half of the triples in its kb and supplement it with wikipedia corpus, and musique @xcite , where we take wikidata @xcite as additional kb besides the given text paragraphs. experimental results show that, roht improves the performance significantly under the kb+text setting, by 29.7% and 45.8% em score on kqa pro and musique compared with existing sota model. in addition, compared with the decompose-based methods, roht improves the sota by 11.3% f1 score on musique. our contributions include: 1) proposing to leverage question decomposing to integrate heterogeneous knowledge sources for the first time; 2) designing a novel two-stage xqa famework roht by first building hqdt and then reasoning over hqdt; 3) demonstrating the effectiveness of our roht framework through extensive experiments and careful ablation studies on two benchmark datasets. an early approach ibm watson @xcite ) combines multiple expert systems and re-ranks them to produce the answer. @xcite maps relational phrases to kb and text simultaneously, and use an integer linear program model to provide a globally optimal solution. universal schema based method @xcite reasons over both kbs and text by aligning them in a common embedded space. graftnet @xcite and its successor pullnet @xcite incorporate free text into graph nodes to make texts amenable to kbqa methods. transfernet @xcite proposes the relation graph to model the label-form relation from kbs and text-form relation from corpora uniformly. although achieving promising results, these methods lack interpretability or are constrained to limited question type, i.e., transfernet shows interpretability with transparent step transfering, however, it can only answer multi-hop questions, and cannot deal with questions that require attribute comparison or value verification. in contrast, our proposed framework shows great interpretability with hqdt and cover more question types. ## definition of hqdt formally, given a complex question, its hqdt is a tree t . each node q @xmath0. for root node, it represents the given complex question, and for non-root nodes, it represents a sub-question of its parent node. the leaf nodes are simple ("atomic") questions that cannot be decomposed. note that hqdt is a 3-ary ordered tree. as shown in figure 1 , we enumerate the nodes of t with bfs ordering, and q 0 is the root question. for every non-leaf question q i , we define two ordered lists: ## methodology our framework roht is composed of two stages: 1) building hqdt. we understand the hierarchical compositional structure of a complex question q 0 by generating its hqdt t with probability, where each question q @xmath1. the details are introduced as follows. ## conclusion in this paper, we propose roht, an understandingreasoning xqa framework that uses both a kb and a text corpus to derive answers of complex questions. roht first builds the hqdt for a complex question to understand its hierarchical compositional structure, then conducts recursive probabilistic reasoning over the hqdt to solve the question, integrating answers from the kb, text, and sub-questions. experiments show that roht significantly outperforms previous methods. we also demonstrate the superiority of hqdt compared with non-hierarchical decomposition.