Question: What categories are the reasoning methods of MRC systems in existing literature can be grouped into?
Evidence: The recent progress in MRC is largely due to the introduction of large-scale datasets. CNN/Daily Mail BIBREF0 and SQuAD BIBREF1 are two popular and widely-used datasets. More recently, other datasets using different collection methodologies have been introduced, such as MS MARCO BIBREF11 , NewsQA BIBREF12 and RACE BIBREF7 . For example, MS MARCO collects data from search engine queries and user-clicked results, thus contains a broader topic coverage than Wikipedia and news articles in SQuAD and CNN/Daily Mail. Among the large number of MRC datasets, RACE focuses primarily on developing MRC models with near-human capability. Questions in RACE come from real English exams designed specifically to test human comprehension. This makes RACE an appealing testbed for DFN; we will further illustrate this in Section " RACE - The MRC Task" .
The word “fusion” for MRC was first used by FusionNet BIBREF4 to refer to the process of updating the representation of passage (or question) using information from the question (or passage) representation. A typical way of fusion is through attention: for example, BiDAF BIBREF2 uses a bi-directional attention, where the representation of passage (or question) vectors are re-weighted by their similarities to the question (or passage) vectors. We will use “fusion” and “attention” interchangeably throughout the paper.
In the attention process of state-of-the-art MRC models, a pre-defined attention strategy is often applied. BIBREF13 proposed a Bi-directional Multi-Perspective Matching (BiMPM) model, which uses attention with multiple perspectives characterized by different parameters. Although multi-perspective attention might be able to handle different types of questions, all perspectives are used for all the questions. DFN is inspired by BiMPM, but our dynamic attention process is more adaptive to variations of questions.
Another important component of MRC systems is the answer module, which performs reasoning to generate the final prediction. The reasoning methods in existing literature can be grouped into three categories: 1) single-step reasoning BIBREF14 , BIBREF15 , BIBREF2 , BIBREF16 ; 2) multi-step reasoning with a fixed number of steps BIBREF17 , BIBREF18 , BIBREF19 ; and 3) dynamic multi-step reasoning (ReasoNet BIBREF6 ). In particular, BIBREF19 proposed handling the variations in passages and questions using Maxout units and iterative reasoning. However, this model still applies static attention and reasoning (with fixed multiple steps), where the same attention strategy is applied to all questions. DFN can be seen as an extension of ReasoNet, in the sense that the dynamic strategy is applied not only in the reasoning process but also in the attention process.
The idea of dynamic attention has been applied to article recommendations BIBREF20 . For MRC, Andreas et al. (2016) proposed a dynamic decision process for reading comprehension task BIBREF8 . In their dynamic neural module networks, the MRC task is divided into several predefined steps (e.g., finding, lookup, relating), and a neural network is dynamically composed via RL based on parsing information. In DFN, we also incorporate dynamic decisions, but instead of using fixed steps, we apply dynamic decisions to various attention strategies and flexible reasoning steps.
Another important component of MRC systems is the answer module, which performs reasoning to generate the final prediction. The reasoning methods in existing literature can be grouped into three categories: 1) single-step reasoning (Chen et al., 2016, 2017; Seo et al., 2016; Wang et al., 2017a); 2) multi-step reasoning with a fixed number of steps (Dhingra et al., 2016; Sordoni et al., 2016; Xiong et al., 2016); and 3) dynamic multistep reasoning (ReasoNet (Shen et al., 2017)).
Answer: 1) single-step reasoning  2) multi-step reasoning with a fixed number of stepsand 3) dynamic multistep reasoning.