Question: Inspired by what they use deep reinforcement learning methods to dynamically choose the optimal attention strategy and the optimal number of reasoning steps for a given sample?
Evidence: The goal of Machine Reading Comprehension (MRC) is to have machines read a text passage and then generate an answer (or select an answer from a list of given candidates) for any question about the passage. There has been a growing interest in the research community in exploring neural MRC models in an end-to-end fashion, thanks to the availability of large-scale datasets, such as CNN/DM BIBREF0 and SQuAD BIBREF1 .
Despite the variation in model structures, most state-of-the-art models perform reading comprehension in two stages. First, the symbolic representations of passages and questions are mapped into vectors in a neural space. This is commonly achieved via embedding and attention BIBREF2 , BIBREF3 or fusion BIBREF4 . Then, reasoning is performed on the vectors to generate the right answer.
Ideally, the best attention and reasoning strategies should adapt organically in order to answer different questions. However, most MRC models use a static attention and reasoning strategy indiscriminately, regardless of various question types. One hypothesis is because these models are optimized on those datasets whose passages and questions are domain-specific (or of a single type). For example, in CNN/DM, all the passages are news articles, and the answer to each question is an entity in the passage. In SQuAD, the passages came from Wikipedia articles and the answer to each question is a text span in the article. Such a fixed-strategy MRC model does not adapt well to other datasets. For example, the exact-match score of BiDAF BIBREF2 , one of the best models on SQuAD, drops from 81.5 to 55.8 when applied to TriviaQA BIBREF5 , whereas human performance is 82.3 and 79.7 on SQuAD and TriviaQA, respectively.
In real-world MRC tasks, we must deal with questions and passages of different types and complexities, which calls for models that can dynamically determine what attention and reasoning strategy to use for any input question-passage pair on the fly. In a recent paper, BIBREF6 proposed dynamic multi-step reasoning, where the number of reasoning steps is determined spontaneously (using reinforcement learning) based on the complexity of the input question and passage. With a similar intuition, in this paper we propose a novel MRC model which is dynamic not only on the number of reasoning steps it takes, but also on the way it performs attention. To the best of our knowledge, this is the first MRC model with this dual-dynamic capability.
The proposed model is called a Dynamic Fusion Network (DFN). In this paper, we describe the version of DFN developed on the RACE dataset BIBREF7 . In RACE, a list of candidate answers is provided for each passage-question pair. So DFN for RACE is a scoring model - the answer candidate with the highest score will be selected as the final answer.
Like other MRC models, DFNs also perform machine reading in two stages: attention and reasoning. DFN is unique in its use of a dynamic multi-strategy attention process in the attention stage. Here “attention” refers to the process that texts from different sources (passage, question, answers) are combined in the network. In literature, a fixed attention mechanism is usually employed in MRC models. In DFN, the attention strategy is not static; instead, the actual strategy for drawing attention among the three text sources are chosen on the fly for each sample. This lends flexibility to adapt to various question types that require different comprehension skills. The output of the attention stage is then fed into the reasoning module to generate the answer score. The reasoning module in DFN uses dynamic multi-step reasoning, where the number of steps depends on the complexity of the question-passage pair and varies from sample to sample.
Inspired by ReasoNet BIBREF6 and dynamic neural module networks BIBREF8 , we use deep reinforcement learning methods BIBREF9 , BIBREF10 to dynamically choose the optimal attention strategy and the optimal number of reasoning steps for a given sample. We use RL in favor of other simpler methods (like cascading, pooling or weighted averaging) mainly because we intend to learn a policy that constructs an instance of DFN of a sample-specific structure. Given an input sample consisting of a question, a passage and a list of candidate answers in RACE, an instance of DFN can be constructed via RL step by step on the fly. Such a policy is particularly appealing as it also provides insights on how the model performs on different types of questions. At each decision step, the policy maps its “state”, which represents an input sample, and DFN's partial knowledge of the right answer, to the action of assembling proper attention and reasoning modules for DFN.
Experiments conducted on the RACE dataset show that DFN significantly outperforms previous state-of-the-art MRC models and has achieved the best result reported on RACE. A thorough empirical analysis also demonstrates that DFN is highly effective in understanding passages of a wide variety of styles and answering questions of different complexities.
Inspired by ReasoNet (Shen et al., 2017) and dynamic neural module networks (Andreas et al., 2016), we use deep reinforcement learning methods (Mnih et al., 2013; Zhang et al., 2017) to dynamically choose the optimal attention strategy and the optimal number of reasoning steps for a given sample.
Answer: ReasoNet and dynamic neural module networks