Exploiting Reasoning Chains for Multi-hop Science Question Answering (2021)

Abstract: We propose a novel Chain Guided Retriever-reader (CGR) framework to model the reasoning chain for multi-hop Science Question Answering. Our framework is capable of performing explainable reasoning without the need of any corpus-specific annotations, such as the ground-truth reasoning chain, or human-annotated entity mentions. Specifically, we first generate reasoning chains from a semantic graph constructed by Abstract Meaning Representation of retrieved evidence facts. A Chain-aware loss , concerning both local and global chain information, is also designed to enable the generated chains to serve as distant supervision signals for training the retriever, where reinforcement learning is also adopted to maximize the utility of the reasoning chains. Our framework allows the retriever to capture step-by-step clues of the entire reasoning process, which is not only shown to be effective on two challenging multi-hop Science QA tasks, namely OpenBookQA and ARC-Challenge, but also favors explainability.

## introduction question answering (qa) with external knowledge has gained increasing attention in recent years as it mimics human behavior to first filter out relevant knowledge from massive information. prior works usually employ a retriever-reader architecture @xcite , where the retriever retrieves top-ranked evidence facts from a large corpus and the reader conducts reasoning with these facts. this architecture works well in single-hop qa, where the answer can be easily inferred with only one evidence fact. however, it is hard to retrieve all necessary evidence facts to confidently answer a complex question requiring multi-hop reasoning @xcite . as shown in figure 1 , multi-hop qa usually involves a sequential nature of evidence facts to form a reasoning chain, including (1) direct facts sharing a semantic relationship with the question or the answer; (2) indirect facts sharing little lexical or semantic overlap but serving an irreplaceable role to infer the answer. a common practice for forming such reasoning chains for multi-hop qa is to expand the chain with iterative retrieval @xcite or sample from an existing or pre-constructed knowledge graph (kg) @xcite . on one hand, iterative retrieval allows the retriever to capture the evidence-evidence interactions by reformulating the query with newly retrieved evidence fact. however, the retriever would inevitably retrieve partially related facts. such noise is continuously amplified during the iterative retrieval process, raising concerns about the quality of the reasoning chain. prior works address this issue by training the retriever against the ground-truth reasoning chain @xcite . however, such method is less effective when the ground-truth reasoning chain is partially provided @xcite or not applicable when the groundtruth chain is unavailable @xcite . on the other hand, kg maintains a good growing direction for the reasoning chain. but building a kg usually involves corpus-specific annotations, such as document-level hyperlinks or annotated entity mentions. these limitations make it less applicable in new domains, where hyperlinks and entities are not prevalent @xcite . to address the aforementioned concerns for multi-hop qa, we propose a novel framework, chain guided retriever-reader (cgr), to model the reasoning chains, which is compatible without the ground-truth reasoning chain and applicable to broader textual data. in specific, the proposed framework consists of three components: a retriever, a reasoning chain generator, and a reader. the retriever first iteratively retrieves all possible evidence facts. then the reasoning chain generator adopts abstract meaning representation (amr) @xcite to automatically construct a semantic graph that represents the relationship among the retrieved facts. we use amr because it is not specifically designed for a particular domain and can be adapted to a wide range of sentences (e.g. science with little named entity). the final generated reasoning chains are supposed to connect the question nodes and the answer nodes on the semantic graph, such that: (i) the generated reasoning chains serving as a byproduct of our framework can support explainability. (ii) the evidence facts on these chains provide a more appropriate context for the reader because they together fill the knowledge gap between the question and the answer. (iii) the reasoning chains can be used as distant supervision signals to train the retriever in case the ground-truth reasoning chain is not available. to achieve these merits, a novel chain-aware loss is proposed to adaptively model the reasoning chains in both supervised and distantly supervised manners. the chain-aware loss not only adopts reinforcement learning (rl) @xcite to maximize the utility of the local information from some certain chains based on the reward from the reader, but also enables the retriever to retrieve indirect evidence facts by considering the global information from all the generated chains. the contributions are summarized as follows: ## framework problem definition in this work, we tackle the multi-hop science qa in the form of multi-choices, where each question q i is associated with j answer choices a ij , @xmath0, 2, ..., j}. to answer the question, one can refer to an external textual corpus e for relevant evidence facts. however, since the external corpus is not specifically designed for a particular multi-hop qa task, it does not necessarily contain the relevant evidence facts. finally, based on the information at hand, our goal is to determine the correct answer. ## experimental setup datasets: we evaluate the effectiveness of our cgr framework on two multi-hop science qa datasets: openbookqa @xcite and arc-challenge @xcite , where the ground-truth reasoning chain is either partially provided or not provided. openbookqa and arc-challenge provide their corresponding leaderboards with train, develop and test sets publicly available. following allenai (2019), we combine the training set of openbookqa (4957), arc-easy (2251), arc-challenge (1119) and reglivenv (665) as the final training set of arc-challenge task. the data splits is shown in implementation: we use aristoroberta (al-lenai, 2019) as our reader and sentence-bert @xcite as our retriever. as indicated by @xcite , training the evidence encoder e e that requires periodic updates of the evidence index is costly and does little improvement to the performance. therefore, we fix the evidence encoder and cache all evidence representations offline for efficiency purposes. we set the beam size k to 10 and the total iteration step t to 2, resulting in an average size of evidence pool to be 78 and 53 for openbookqa and arc-challenge respectively. we then select facts for the reader with maximum evidence size of 15 and 20 respectively. the number of sampled chains n is set to 1. 2 more details and analysis can be found in appendix a.1 and a.2. comparison methods: for fair comparison, we compare cgr with recently published methods that use similar power of pretrained models, including five textual knowledge based methods: aristoroberta (allenai, 2019), kf-sir (banerjee and baral, 2020), freelb @xcite , dpr @xcite , amr-sg @xcite and another two methods leveraging on an additional knowledge graph @xcite : pg @xcite , and mhgrn @xcite . the reasoning chains as an additional output that reflect the step-by-step reasoning process to infer the answer, making the qa process explainable. when compared to recently published methods, we find that cgr can also surpass methods leveraging on additional kg. it suggests that textual knowledge resource is still under-investigated, where the gap between the query and indirect fact is one of the issues that restricts the retriever performance for multi-hop qa. unifiedqa @xcite and t5 3b @xcite are two extremely large models (with 30x more parameters than other models), which are not fair for comparison. ## related work multi-hop qa: multi-hop qa is a challenging task as it requires gathering multiple evidence facts, especially indirect facts, to form a reasoning chain. early attempts mostly rely on iterative retrieval. for example, @xcite extract evidence facts in consideration of their relevance, overlap and coverage. banerjee et al. (2019); yadav et al. (2020 ) reformulate their queries with unused words in the last iteration. however, these methods may retrieve irrelevant facts as the query grow biased to unimportant words. as some recent qa datasets annotate the ground-truth reasoning chain @xcite , they enable training supervised classifier to identify the correct evidence facts @xcite . it is a good step to control the quality of reasoning chains, but still remains an issue when the groundtruth reasoning chain is not available @xcite . other works explore the effectiveness of kg by either automatically constructing the graph using named entity or semantic role labeling @xcite @xcite or resorting to existing kg @xcite @xcite . despite the high precision of those kgs, they are known to suffer from sparsity in existing kg @xcite , where complex reasoning chains are unlikely to be covered by the closedform relations in kg @xcite . dense-vector retriever: in contrast to termbased retriever implemented with tf-idf or bm25 @xcite , densevector retriever has received increasing attention as it captures the semantic matching beyond simple word overlap and can be trained along with the reader @xcite . it has been reported to outperform term-based methods in many open-domain qa tasks @xcite @xcite , including those on multi-hop qa @xcite . amr: amr has been successfully coupled with many natural language processing tasks in explicit reasoning, such as summarization @xcite , event detection @xcite , machine translation @xcite , and symbolic qa @xcite . comparing to named entity @xcite , we use amr as our graph annotation because it is not specifically designed for a particular domain and can be adapted to a wide range of textual data. ## conclusion we propose a novel chain guided retriever-reader framework for multi-hop qa. our modeling for the reasoning chains is effective to find both direct and indirect facts and is less likely to introduce noise. moreover, our framework is corpus-independent and is capable of handling the setting without any ground-truth annotations. further analysis and discussions also elucidate some of the inner workings of our framework while maintaining the explainability at the same time. which requires energy to move? (a) weasel (b)willow (c) mango (d) poison ivy related evidence facts: [1] an animal requires energy to move.