Hierarchical Graph Network for Multi-hop Question Answering (2020)

Abstract: In this paper, we present Hierarchical Graph Network (HGN) for multi-hop question answering. To aggregate clues from scattered texts across multiple paragraphs, a hierarchical graph is created by constructing nodes on different levels of granularity (questions, paragraphs, sentences, entities), the representations of which are initialized with pre-trained contextual encoders. Given this hierarchical graph, the initial node representations are updated through graph propagation, and multi-hop reasoning is performed via traversing through the graph edges for each subsequent sub-task (e.g., paragraph selection, supporting facts extraction, answer prediction). By weaving heterogeneous nodes into an integral unified graph, this hierarchical differentiation of node granularity enables HGN to support different question answering sub-tasks simultaneously. Experiments on the HotpotQA benchmark demonstrate that the proposed model achieves new state of the art, outperforming existing multi-hop QA approaches.

## introduction in contrast to one-hop question answering @xcite @xcite where answers can be derived from a single paragraph @xcite @xcite , many recent studies on question answering focus on multi-hop reasoning across multiple documents or paragraphs. popular tasks include wikihop @xcite , complexwebquestions @xcite , and hotpotqa @xcite . an example from hotpotqa is illustrated in figure 1 . in order to correctly answer the question ("the director of the romantic comedy 'big stone gap' is based in what new york city"), the model is figure 1 : an example of multi-hop question answering from hotpotqa. the model needs to identify relevant paragraphs, determine supporting facts, and then predict the answer correctly. most existing studies use a retriever to find paragraphs that contain the right answer to the question (p1 and p2 in this case). a machine reading comprehension (mrc) model is then applied to the selected paragraphs for answer prediction @xcite . however, even after successfully identifying a reasoning chain through multiple paragraphs, it still remains a critical challenge how to aggregate evidence from scattered sources on different granularity levels (e.g., paragraphs, sentences, entities) for joint answer and supporting facts prediction. to better leverage fine-grained evidences, some studies apply entity graphs through query-guided multi-hop reasoning. depending on the characteristics of the dataset, answers can be selected either from entities in the constructed entity graph @xcite @xcite @xcite , or from spans in documents by fusing entity representations back into token-level document representation @xcite . however, the constructed graph is mostly used for answer prediction only, while insufficient for finding supporting facts. also, reasoning through a simple entity graph @xcite or paragraph-entity hybrid graph @xcite lacks the ability to support complicated questions that require multi-hop reasoning. intuitively, given a question that requires multiple hops through a set of documents to reach the right answer, a model needs to: (i) identify paragraphs relevant to the question; (ii) determine strong supporting evidence in those paragraphs; and (iii) pinpoint the right answer following the garnered evidence. to this end, graph neural network with its inherent message passing mechanism that can pass on multi-hop information through graph propagation, has great potential of effectively predicting both supporting facts and answer simultaneously for complex multi-hop questions. motivated by this, we propose a hierarchical graph network (hgn) for multi-hop question answering, which empowers joint answer/evidence prediction via multi-level fine-grained graphs in a hierarchical framework. instead of only using entities as nodes, for each question we construct a hierarchical graph to capture clues from sources with different levels of granularity. specifically, four types of graph node are introduced: questions, paragraphs, sentences and entities (see figure 2 ). to obtain contextualized representations for these hierarchical nodes, large-scale pre-trained language models such as bert @xcite and roberta @xcite are used for contextual encoding. these initial representations are then passed through a graph neural network for graph propagation. the updated node representations are then exploited for different sub-tasks (e.g., paragraph selection, supporting facts prediction, entity prediction). since answers may not be entities in the graph, a span prediction module is also introduced for final answer prediction. the main contributions of this paper are threefold: (i) we propose a hierarchical graph network (hgn) for multi-hop question answering, where heterogeneous nodes are woven into an integral hierarchical graph. (ii) nodes from different granularity levels mutually enhance each other for different sub-tasks, providing effective supervision signals for both supporting facts extraction and answer prediction. (iii) on the hotpotqa benchmark, the proposed model achieves new state of the art in both distractor and fullwiki settings. ## hierarchical graph network as illustrated in figure 2 , the proposed hierarchical graph network (hgn) consists of four main components: (i) graph construction module (sec. 3.1), through which a hierarchical graph is constructed to connect clues from different sources; (ii) context encoding module (sec. 3.2), where initial representations of graph nodes are obtained via a roberta-based encoder; (iii) graph reasoning module (sec. 3.3), where graph-attention-based message passing algorithm is applied to jointly update node representations; and (iv) multi-task prediction module (sec. 3.4), where multiple subtasks, including paragraph selection, supporting facts prediction, entity prediction, and answer span extraction, are performed simultaneously. ## experiments in this section, we describe experiments comparing hgn with state-of-the-art approaches and provide detailed analysis on the model and results. ## conclusion in this paper, we propose a new approach, hierarchical graph network (hgn), for multi-hop 3 please refer to row 4 in