Is G raph S tructure N ecessary for M ulti-hop Q uestion A nswering? (2020)

Abstract: Recently, attempting to model texts as graph structure and introducing graph neural networks to deal with it has become a trend in many NLP research areas. In this paper, we investigate whether the graph structure is necessary for textual multi-hop reasoning. Our analysis is centered on HotpotQA. We construct a strong baseline model to establish that, with the proper use of pre-trained models, graph structure may not be necessary for textual multi-hop reasoning. We point out that both graph structure and adjacency matrix are task-related prior knowledge, and graph-attention can be considered as a special case of self-attention. Experiments demonstrate that graph-attention or the entire graph structure can be replaced by self-attention or Transformers.

## introduction different from single-hop question answering, where the answer can be derived from a single sentence in a single paragraph, more and more studies focus on multi-hop question answering across multiple documents or paragraphs @xcite @xcite . to solve this problem, the majority of existing studies constructed a graph structure according to co-occurrence relations of entities that scattered across multiple sentences or paragraphs. @xcite and @xcite designed a dag-styled recurrent layer to model the relations between entities. @xcite first used gcn @xcite to tackle entity graph. @xcite proposed a dynamic entity graph for span-based multi-hop qa. @xcite extended the entity graph to a heterogeneous graph by introducing document nodes and query nodes. previous works argue that a fancy graph structure is a vital part of their models and demonstrate that by ablation experiments. however, in experiments, we find when we use the pre-trained models in the fine-tuning approach, removing the entire graph structure may not hurt the final results. therefore, in this paper, we aimed to answer the following question: how much does graph structure contribute to multi-hop question answering? to answer the question above, we choose the widely used multi-hop question answering benchmark, hotpotqa @xcite , as our testbed. we reimplement a graph-based model, dynamically fused graph network @xcite , as our baseline model. the remainder of this paper is organized as follows. then, we show that the graph structure can play an important role only when the pre-trained models are used in a feature-based manner. while the pre-trained models are used in the fine-tuning approach, the graph structure may not be helpful. ## the approach we choose the widely used multi-hop qa dataset, hotpotqa as our testbed. we reimplement dfgn @xcite and modify the usage of the pretrained model. the model first leverage a retriever to select relevant passages from candidate set and feed them into a graph-based reader. all entities in the entity graph are recognized by an independent ner model. ## understanding graph structure experimental results in section 2.3 imply that selfattention or transformer may have superiority in multi-hop question answering. to understand this, in this section, we will first discuss the connection between graph structure, graph-attention, and self-attention. we then verify the hypothesis by experiments and visualized analysis. ## conclusions this study set out to investigate whether graph structure is necessary for multi-hop qa and what role it plays. we established that with the proper use of pre-trained models, graph structure may not be necessary. in addition, we point out the adjacency matrix and the graph structure can be regarded as some kind of task-related prior knowledge. experiments and visualized analysis demonstrate both graph-attention and graph structure can be replaced by self-attention or transformers. our results suggest that future works introducing graph structure into nlp tasks should explain their necessity and superiority.