Breadth First Reasoning Graph for Multi-hop Question Answering (2021)

Abstract: Recently Graph Neural Network (GNN) has been used as a promising tool in multi-hop question answering task. However, the unnecessary updations and simple edge constructions prevent an accurate answer span extraction in a more direct and interpretable way. In this paper, we propose a novel model of Breadth First Reasoning Graph (BFR-Graph), which presents a new message passing way that better conforms to the reasoning process. In BFR-Graph, the reasoning message is required to start from the question node and pass to the next sentences node hop by hop until all the edges have been passed, which can effectively prevent each node from over-smoothing or being updated multiple times unnecessarily. To introduce more semantics, we also define the reasoning graph as a weighted graph with considering the number of co-occurrence entities and the distance between sentences. Then we present a more direct and interpretable way to aggregate scores from different levels of granularity based on the GNN. On HotpotQA leaderboard, the proposed BFR-Graph achieves state-of-the-art on answer span prediction.

## introduction typical question answering (qa) or reading comprehension (rc) task aims at exploring a desired answer through a single evidence document or paragraph. recently, a more challenging multi-hop qa task, where we need to reason over multiple paragraphs to find the answer, is gradually catching attention. one example from hotpotqa dataset @xcite is shown in fig. 1 . one method for achieving multi-hop qa is to concatenate all the paragraphs together and treat it as a typical single-hop qa task @xcite , then existing qa techniques can be applied. although multi-hop qa can be solved to some extent, figure 1 : one example from hotpotqa dataset. "s1", "s2", ... denote the sentences in paragraphs. the model needs to find the answer and supporting sentences by reasoning over multiple sentences and paragraphs. it's obvious that the reasoning is in a ordered process from the question to "s1", "s2" and finally to "s4". graph neural networks (gnn) is a natural way to represent the solving procedure of multi-hop qa. for instance, nodes in gnn represent sentences/entities in the paragraphs, and from the updation through edges we can get interactive message between them, which is similar to the process of reasoning. thus, a more reasonable method is to construct gnn to simulate the reasoning process among multiple paragraphs @xcite @xcite . promising performance has been reported in methods that designed different type of nodes or edges for gnn @xcite @xcite and the features generated from gnn has also been combined with those from the context encoder in a latent way @xcite . despite of the success that gnn achieves in multi-hop qa, new problems associated to gnn arise. firstly, current approaches update all the nodes, including some unnecessary ones, together within each layer, which may lead the nodes to converge to similar values and lose the discriminating ability for gnn with more layers @xcite . secondly, although different types of edges have been designed for gnn, there is no more fine-grained distinction between edges of the same type, without considering the other relational information between sentences. thirdly, existing methods only latently fuse the hidden representations of gnn and context encoder, without contributing to the answer span extraction in a direct and interpretable way. to solve the aforementioned issues, we proposed a novel model of breadth first reasoning graph (bfr-graph) to effectively adapt gnn to multihop qa. the proposed bfr-graph is a weighted graph in which the weight of an edge is computed based on other relational information (e.g., cooccurrence entities and distance) of the connected sentences. inspired by the human reasoning mechanism and the breadth first search algorithm, in bfr-graph the reasoning message starts from the question and passes to the next sentence nodes hop by hop until all the edges have been passed, effectively preventing each node from updating multiple times or being updated unnecessarily. then the reasoning result from bfr-graph is converted to the sentence scores and paragraph scores, contributing to the answer span extraction. specifically, the final answer span probability is the sum of the score of answer span, the sentence and the paragraph, in both of which the answer is located. experiment results shows that our methods make gnn more powerful in multi-hop qa and achieves state-ofthe-art on answer span prediction of hotpotqa. the contributions of this paper are summarized as follows: ## model to solve the aforementioned issues, we propose a novel model of breadth first reasoning graph (bfr-graph) for multi-hop qa. different from existing gnn-based methods, bfr-graph introduces new restrictions on the message passing: the message only starts from the question and then passes to the latter sentence nodes hop by hop. besides, our graph is constructed as a weighted graph considering the co-occurrence entities and distance between sentences. moreover, multi-score answer prediction is designed to take advantage of the reasoning result from bfr-graph. in short, we propose breadth first reasoning on the weighted graph and then combine multi-level scores for answer prediction in the framework of multi-task joint training. the diagram of our system is shown in fig. 2 . given multiple paragraphs, we first filter out irrelevant paragraph with paragraph selection (sec. 3.1) and then use a bert for context encoding (sec. 3.2). a weighted graph is constructed (sec. 3.3) to reason over sentences (sec. 3.4) and calculate the sentence score and paragraph score. finally, we use multi-score mechanism to predict the answer span (sec. 3.5). ## ablations and analysis in this section, we carry out ablation studies on hotpotqa dev-set. ## conclusion in this paper, we proposed a novel gnn model of bfr-graph. specifically, the reasoning message starts from the question node and passes to the next sentences node hop by hop until all the edges have been passed. we also construct the reasoning graph as a weighted graph and present a more interpretable way to aggregate scores of different levels from gnn. on hotpotqa leaderboard, bfr-graph achieved state-of-the-art on answer span prediction.