Decomposing Complex Questions Makes Multi-Hop QA Easier and More Interpretable (2021)

Abstract: Multi-hop QA requires the machine to answer complex questions through finding multiple clues and reasoning, and provide explanatory evidence to demonstrate the machine’s reasoning process. We propose Relation Extractor-Reader and Comparator (RERC), a three-stage framework based on complex question decomposition. The Relation Extractor decomposes the complex question, and then the Reader answers the sub-questions in turn, and finally the Comparator performs numerical comparison and summarizes all to get the final answer, where the entire process itself constitutes a complete reasoning evidence path. In the 2WikiMultiHopQA dataset, our RERC model has achieved the state-of-the-art performance, with a winning joint F1 score of 53.58 on the leaderboard. All indicators of our RERC are close to human performance, with only 1.95 behind the human level in F1 score of support fact. At the same time, the evidence path provided by our RERC framework has excellent readability and faithfulness.

## introduction multi-hop qa is an important and challenging task in natural language processing (nlp), which requires complex reasoning over several paragraphs to reach the final answer and explanatory evidence to demonstrate the reasoning process. many highquality multi-hop qa datasets have been introduced recently, such as hotpotqa @xcite , complexwebquestions @xcite , qangaroo wikihop @xcite , r 4 c @xcite , 2wikimultihopqa @xcite , etc. these high-quality multi-hop qa datasets promote many multi-hop qa models @xcite @xcite @xcite , most of which are end-to-end models based on graph structure or graph neural network @xcite . although these works have good performances in many tasks, they also have some limitations to address. first of all, the internal reasoning mechanism of previous end-to-end qa models is a black-box, which usually use an additional discriminator to judge whether a sentence is a clue sentence, such as dfgn @xcite . there is no evidence to show that such additional discriminators are strongly correlated with the reasoning results of the end-to-end model, which means not faithful. secondly, although graph structure is helpful to multi-hop reasoning in theory, but recent work @xcite shows that the existing graph neural network is only a special attention mechanism @xcite , and it's not necessary for multi-hop qa, with the experiments that better results can be achieved by using only transformer network instead of graph neural network, as long as the same additional adjacency matrix information is provided. we observed that human reasoning about complex questions is not accomplished overnight and it's usually divided into the steps of question decomposition, answering sub-questions, summarizing and comparing. for example, for the complex question, "whose candidate will get more votes in the 2020 u.s. election, democrats and republicans?" people will not think about the whole question, but firstly decompose the complex question. realizing that the subject of the question is "democrats and republicans", and the question is about "candidates" and "number of votes", people can answer those sub-questions progressively -"who is the democratic candidate?" and "how many votes does ans get?" the same thinking process was performed for another question subject, "republican party". finally, the two votes were compared to obtain the answer to the entire complex question. inspired by the way humans answer complex multi-hop questions, in this work we abandoned the end-to-end model structure, but imitated the human reasoning mechanism to propose a threestage relation extractor-reader and comparator (rerc) model foot_0 . we first build a relation extractor, which can automatically extract the subject and key relations of the question from the complex unstructured textual representation. for the relation extractor, we use two different structures, one is classification-type (crerc), where the evidence relation information in the dataset is used as prior knowledge, and the question text is mapped to question relations through the classifier; the other is span-type (srerc), where the type of question relations is unrestricted, and the relation extractor can automatically extract multiple corresponding spans from the question text as question relations. next, we use the advanced albert model @xcite as the reader, which reads the corresponding paragraphs and answer each sub-question composed of the subject and relations of the question in turn. finally, for comparison type questions, our comparator module compares the magnitude of each subject's final answer, and then get the entire answer. our contributions are summarized as follows: ## proposed method in this section, we will introduce in detail our proposed relation extractor-reader and comparator (rerc) model. this is a three-stage multi-hop qa model consisting of three parts: relation extractor, reader and comparator. the working principle of the whole framework is shown in figure 1 . given the question q and the context set @xmath0, firstly pass the question q to the relation extractor to obtain the question subjects set @xmath1, and then construct the sub-questions set @xmath2, and then the reader reads the searched context and answers each sub-question to obtain the answer set @xmath3, and finaly the comparator obtains the final answer an s through numerical comparison and summary analysis. ## conclusion and future work we propose a three-stage framework of relation extractor-reader and comparator (rerc), which solves the multi-hop qa task through the idea of complex question decomposition, and obtains the state-of-the-art results in the 2wikimultihopqa dataset, which is close to human performance. our rerc framework can also provide faithful evidence with excellent interpretability. multiple future research directions according to our proposed rerc model may be envisioned. first of all, benefiting to the three-stage structure, the rerc model has the potential to leverage the network structure of the relation extractor to gain generalization capabilities for more complex questions. moreover, we expect that collaborative error correction mechanism applied in reader module will largely avoid accumulation of errors in multihop reasoning.