Smarnet: Teaching Machines to Read and Comprehend Like Human	Machine Comprehension (MC) is a challenging task in Natural Language Processing field, which aims to guide the machine to comprehend a passage and answer the given question. Many existing approaches on MC task are suffering the inefficiency in some bottlenecks, such as insufficient lexical understanding, complex question-passage interaction, incorrect answer extraction and so on. In this paper, we address these problems from the viewpoint of how humans deal with reading tests in a scientific way. Specifically, we first propose a novel lexical gating mechanism to dynamically combine the words and characters representations. We then guide the machines to read in an interactive way with attention mechanism and memory network. Finally we add a checking layer to refine the answer for insurance. The extensive experiments on two popular datasets SQuAD and TriviaQA show that our method exceeds considerable performance than most state-of-the-art solutions at the time of submission.

Input Embedding Layer	Familiar with lexical and linguistic properties is crucial in text understanding. We try to enrich the lexical factors to enhance the word representation. Inspired by Yang et al.    and  , we adopt a more fine-grained dynamic gating mechanism to model the lexical properties related to the question and the passage. Here we indicate our embedding method in Figure 2. We design two gates adopted as valves to dynamically control the flowing amount of word-level and character-level representations. For the passage word INLINEFORM0 , we use the concatenation of part-of-speech tag, named entity tag, term frequency tag, exact match tag and the surprisal tag. The exact match denote as INLINEFORM1 in three binary forms: original, lower-case and lemma forms, which indicates whether token INLINEFORM2 in the passage can be exactly matched to a question word in INLINEFORM3 . The surprisal tag measures the amount of information conveyed by a particular word from INLINEFORM4 . The less occurrence of a word, the more information it carries. For the question word INLINEFORM0 , we take the question type in place of the exact match information and remain the other features.

The attention weight is calculated by INLINEFORM2 . Hence the attend question vector for all passage words INLINEFORM3 is obtained by INLINEFORM4 , where INLINEFORM5 . We further utilize INLINEFORM0 to form the question-aware passage representation. In order to comprehensively model the mutual information between the question and passage, we adopt a heuristic combining strategy to yield the extension as follows: INLINEFORM1   where INLINEFORM0 denotes the INLINEFORM1 th question-aware passage word under the INLINEFORM2 th hop, the INLINEFORM3 function is a concatenation function that fuses four input vectors. INLINEFORM4 denotes the hidden state of former INLINEFORM5 th passage word obtained from BiGRU. INLINEFORM6 denotes element-wise multiplication and INLINEFORM7 denotes the element-wise plus. Notice that after the concatenation of INLINEFORM8 , the dimension can reaches to INLINEFORM9 . We feed the concatenation vector into a BiGRU to reduce the hidden state dimension into INLINEFORM10 . Using BiGRU to reduce dimension provides an efficient way to facilitate the passage semantic information and enable later semantic parsing. Naive one-hop comprehension may fail to comprehensively understand the mutual question-passage information.

iteratively interact between the query and document by a multiplicative gating function. Cui et al.  compute a similarity matrix with two-way attention between the query and passage mutually. Sordoni et al.  exploit an iterative alternating neural attention to model the connections between the question and the passage. Open-domain machine comprehension tasks are more challenging and have attracted plenty of teams to pursue for higher performance on the leaderboard. Wang et al.  present match-LSTM and use pointer network to generate answers from the passage. Chen et al.  tackle the problem by using wikipedia as the unique knowledge source. Shen  adopt memory networks with reinforcement learning so as to dynamically control the number of hops. Seo et al.  use bi-directional attention flow mechanism and a multi-stage hierarchical process to represent the context. Xiong et al.  propose dynamic coattention networks to iteratively infer the answer. Yang et al.  present a fine-grained gating mechanism to dynamically combine word-level and character-level representations. Wang et al.  introduce self-matching attention to refine the gated representation by aligning the passage against itself.

Ablation Results	We respectively evaluate the individual contribution of the proposed module in our model. We conduct thorough ablation experiments on the SQuAD dev set, which are recorded on the table 4 and table 5. Table 4 shows the different effect of the lexical features. We see the full features integration obtain the best performance, which demonstrates the necessity of combining all the features into consideration. Among all the feature ablations, the Part-Of-Speech, Exact Match, Qtype features drop much more than the other features, which shows the importance of these three features. As the POS tag provides the critical lexical information, Exact Match and Qtype help to guide the attention in the interaction procedure. As for the final ablation of POS and NER, we can see the performance decays over 3% point, which clearly proves the usefulness of the comprehensive lexical information. Table 5 shows the ablation results on the different levels of components. We first replace our input gate mechanism into simplified feature concatenation strategy, the performance drops nearly 2.3% on the EM score, which proves the effectiveness of our proposed dynamic input gating mechanism.

For all the time through, we consider a philosophy question: What will people do when they are having a reading comprehension test? Recall how our teacher taught us may shed some light. As a student, we recite words with relevant properties such as part-of-speech tag, the synonyms, entity type and so on. In order to promote answer's accuracy, we iteratively and interactively read the question and the passage to locate the answer's boundary. Sometimes we will check the answer to ensure the refining accuracy. Here we draw a flow path to depict what on earth the scientific reading skills are in the Figure 1. As we see, basic word understanding, iterative reading interaction and attentive checking are crucial in order to guarantee the answer accuracy. In this paper, we propose the novel framework named Smarnet with the hope that it can become as smart as humans. We design the structure in the view point of imitating how humans take the reading comprehension test. Specifically, we first introduce the Smarnet framework that exploits fine-grained word understanding with various attribution discriminations, like humans recite words with corresponding properties. We then develop the interactive attention with memory network to mimic human reading procedure.

Reasoning by Memory Network Multi-hop reasoning combines with Memory networks have shown powerful competence on MC task        . Theoretically, multi-hop memory networks can repeat computing the attention biases between the query and the context through multiple layers. The memory networks typically maintain memory states which incorporate the information of current reasoning with the previous storage in the memory. Hu et al.  utilize a multi-hop answer pointer which allows the network to continue refining the predicted answer span. Gong et al.  adapt the BIDAF  with multi-hop attention mechanisms and achieve substantial performance. Pan et al.  introduce multi-layer embedding with memory network for full orientation matching on MC task. In our paper, we also adopt the memory network to mimic human behaviors on increasing their understanding by reread the context and the query multi times. We also apply a multi-hop checking mechanism to better refine the true answer.

The type of a question provides significant clue for the answer selection process. For example, the answer for a "when" type question prefers tokens about time or dates while a "why" type question requires longer inference. Here we select the top 11 common question types as illustrated in the diagram. If the model recognize a question's type, then all the words in this question will be assigned with the same QType feature. The gates of the passage and the question are computed as follows: INLINEFORM0   where INLINEFORM0 , INLINEFORM1 , INLINEFORM2 , INLINEFORM3 are the parameters and INLINEFORM4 denotes an element-wise sigmoid function. Using the fine-grained gating mechanism conditioned on the lexical features, we can accurately control the information flows between word-level and char-level. Intuitively, the formulation is as follows: INLINEFORM0   where INLINEFORM0 is the element-wise multiplication operator. when the gate has high value, more information flows from the word-level representation; otherwise, char-level will take the dominating place. It is practical in real scenarios. For example, for unfamiliar noun entities, the gates tend to bias towards char-level representation in order to care richer morphological structure.

Task Description	The goal of open-domain MC task is to infer the proper answer from the given text. For notation, given a passage INLINEFORM0 and a question INLINEFORM1 , where INLINEFORM2 and INLINEFORM3 are the length of the passage and the question. Each token is denoted as INLINEFORM4 , where INLINEFORM5 is the word embedding extracts from pre-trained word embedding lookups, INLINEFORM6 is the char-level matrix representing one-hot encoding of characters. The model should read and comprehend the interactions between INLINEFORM7 and INLINEFORM8 , and predict an answer INLINEFORM9 based on a continuous sub-span of INLINEFORM10 .

Besides, we not only utilize the lexical properties as the gating feature, we also concatenate them as a supplement of lexical information. Therefore, the final representation of words are computed as follows: INLINEFORM1   where INLINEFORM0 is the concatenation function.

Interaction Modeling Layer	Recall how people deal with reading comprehension test. When we get a reading test paper, we read the question first to have a preliminary focal point. Then we skim the passage to refine the answer. Sometimes we may not directly ensure the answer's boundary, we go back and confirm the question. After confirming, we scan the passage and refine the right answer we thought. We also check the answer for insurance. Inspired by such a scientific reading procedure, we design the Smarnet with three components: contextual encoding, interactive attention with memory network, answer refining with checking insurance. As is shown in figure 3. We use Gated Recurrent Unit  with bi-directions to model the contextual representations. Here, It is rather remarkable that we do not immediately put the Bi-GRU on the passage words. Instead, we first encode the question and then apply a gate to control the question influence on each passage word, as is shown in the structure (a) and (b). Theoretically, when human do the reading comprehension, they often first read the question to have a rough view and then skim the passage with the impression of the question.

We then put the bi-GRU on top of each passage word to get the semantic representation of the whole passage INLINEFORM6 . INLINEFORM7   where INLINEFORM0 is a passage gate similar to INLINEFORM1 . INLINEFORM2 is the multi-layer perceptron to reduce dimension. INLINEFORM3 represents the confirmed question with the knowledge of the context. The essential point in answer refining lies on comprehensive understanding of the passage content under the guidance of the question. We build the interactive attention module to capture the mutual information between the question and the passage. From human's perspective, people repeatedly and interactively read the question and the passage, narrow the answer's boundary and put more attention on some passage parts where are more relevant to the question. We construct a shared similarity matrix INLINEFORM0 to attend the relevance between the passage INLINEFORM1 and the question INLINEFORM2 . Each element INLINEFORM3 is computed by the similarity of the INLINEFORM4 th passage word and the INLINEFORM5 th question word. We signify relevant question words into an attended question vector to collaborate with each context word. Let INLINEFORM0 represent the normalized attention distribution on the question words by INLINEFORM1 th passage word.

Conclusions	In this paper, we tackle the problem of machine comprehension from the viewpoint of imitating human's ways in having reading comprehension examinations. We propose the Smarnet framework with the hope that it can become as smart as human for the reading comprehension problem. We first introduce a novel gating method with detailed word attributions to fully exploit prior knowledge of word semantic understanding. We then adopt a scientific procedure to guide machines to read and comprehend by using interactive attention and matching mechanisms between questions and passages. Furthermore, we employ the self-alignment with checking strategy to ensure the answer is refined after careful consideration. We evaluate the performance of our method on two large-scale datasets SQuAD and TriviaQA. The extensive experiments demonstrate the superiority of our Smarnet framework.

No one can simultaneously read both the question and the passage without any overlap. Vice versa, after skimming the passage to get the preliminary comprehension, the whole passage representation is also applied to attend the question again with another gating mechanism, as is shown in the structure (c). This can be explained that people often reread the question to confirm whether they have thoroughly understand it. The outputs of the three steps (a) (b) (c) are calculated as follows: INLINEFORM0   where INLINEFORM0 is the lexical representation from the input layer. INLINEFORM1 is the hidden state of GRU for the INLINEFORM2 th question word. INLINEFORM3 is the original question semantic embedding obtained from the concatenation of the last hidden states of two GRUs. INLINEFORM4   where INLINEFORM0 is a question gate controlling the question influence on the passage. The larger the INLINEFORM1 is, the more impact the question takes on the passage word. We reduce the INLINEFORM2 dimension through multi-layer perceptron INLINEFORM3 since INLINEFORM4 and INLINEFORM5 are not in the same dimension.

We then compare two methods which directly encode the passage words or use the question influence. The result proves that our modification of employing question influence on the passage encoding can boost the result up to 1.3% on the EM score. In our model, we apply two-hops memory network to further comprehend the passage. In the ablation test, we remove the iterative hops of memory network and only remain one interaction round. The result drops 2.6% point on the EM score, which indicate the significance of using memory network mechanism. Finally, we compare the last module of our proposed self-alignment checking with original pointer network. The final result shows the superiority of our proposed method.

Datasets	In this section we evaluate our model on the task of machine comprehension using the recently released large-scale datasets SQuAD  and TriviaQA  . SQuAD published by Stanford has obtained a huge attention over the past two years. It is composed of over 100K questions manually annotated by crowd workers on 536 Wikipedia articles. TriviaQA is a newly released open-domain MC dataset which consists of over 650K question-answer-evidence triples. It is derived by combining 95K Trivia enthusiast authored question-answer pairs with on average six supporting evidence documents per question. The length of contexts in TriviaQA is much longer than SQuAD and models trained on TriviaQA require more cross sentence reasoning to find answers. There are some similar settings between these two datasets. Each answer to the question is a segment of text from the corresponding reading context. Two metrics are used to evaluate models: Exact Match (EM) measures the percentage of predictions that match the ground truth answer exactly. F1 score measures the average overlap between the prediction and ground truth answer. Both datasets are randomly partitioned into training set (80%), dev set (10%) and test set (10%).

Related Work	Machine Comprehension Dataset. Benchmark datasets play a vital role in the research advance. Previous human-labeled datasets on MC task are too small to train data-intensive models    . Recently, Large-scale datasets become available. CNN/Daily Mail  and Children's Book Test  generated in cloze style offer the availability to train more expressive neural models. The SQuAD  , TriviaQA  and MS-MARCO  datasets provide large and high-quality datasets which extract answers from text spans instead of single entities in cloze style. The open-domain style of MC tasks are more challenging and require different levels of reasoning from multiple sentences. In this paper, we evaluate our Smarnet framework on SQuAD and TriviaQA datasets. Machine Comprehension Models Previous works in MC task adopt deep neural modeling strategies with attention mechanisms, both on cloze style and open domain tasks. Along with cloze style datasets, Chen et al.  prove that computing the attention weights with a bilinear term instead of simple dot-product significantly improves the accuracy. Kadlec et al.  sum attention over candidate answer words in the document. Dhingra et al.

Answer Selection with Checking Mechanism	The goal of open-domain MC task is to refine the sub-phrase from the passage as the final answer. Usually the answer span INLINEFORM0 is derived by predicting the start INLINEFORM1 and the end INLINEFORM2 indices of the phrase in the paragraph. In our model, we use two answer refining strategies from different levels of linguistic understanding: one is from original interaction output module and the other is from self-matching alignment. The two extracted answers are then applied into a checking component to final ensure the decision. For the original interaction output in structure (h), we directly aggregate the passage vectors through BiGRU. We compute the INLINEFORM0 and INLINEFORM1 probability distribution under the instruction of  and pointer network  by INLINEFORM2   where INLINEFORM0 is the output of the original interaction. INLINEFORM1 and INLINEFORM2 are trainable weight vectors. For the self-alignment in structure (j), we align the two hops outputs INLINEFORM0 with INLINEFORM1 in structure (e) and (h). The purpose of self-alignment aims to analysis the new insights on the passage as the comprehension gradually become clear after iterative hops.

For each hop, the reader dynamically collects evidence from former passage representation and encodes the evidence to the new iteration. From human's perspective, each time we reread the passage, we get some new ideas or more solid comprehension on the basis of the former understanding. The self-alignment is computed by INLINEFORM2   where INLINEFORM0 is the first hop whole passage vector in the structure (e). We apply a gate mechanism with INLINEFORM1 to control the evidence flow to the next hop INLINEFORM2 . The output of self-alignment is computed by INLINEFORM3   where INLINEFORM0 and INLINEFORM1 are the predicted start and end indices after the self-alignment. For insurance, we obtain two groups of predicted answer span INLINEFORM0 and INLINEFORM1 . We then apply a checking strategy to compare the twice answer. This process is quite similar to human's behavior, people often reread the passage and may draw some different answers. Thus they need to compare the alternative answers and finally consider a best one. Here we employ the weighted sum of twice answer prediction indices to make the final decision: INLINEFORM2   where INLINEFORM0 is a weighted scalar controlling the proportion of the two predicted answers.

Introduction	Teaching machines to learn reading comprehension is one of the core tasks in NLP field. Recently machine comprehension task accumulates much concern among NLP researchers. We have witnessed significant progress since the release of large-scale datasets like SQuAD  , MS-MARCO  , TriviaQA  , CNN/Daily Mail  and Children's Book Test  . The essential problem of machine comprehension is to predict the correct answer referring to a given passage with relevant question. If a machine can obtain a good score from predicting the right answer, we can say the machine is capable of understanding the given context. Many previous approaches    adopt attention mechanisms along with deep neural network tactics and pointer network to establish interactions between the question and the passage. The superiority of these frameworks are to enable questions focus on more relevant targeted areas within passages. Although these works have achieved promising performance for MC task, most of them still suffer from the inefficiency in three perspectives: (1) Comprehensive understanding on the lexical and linguistic level. (2) Complex interactions among questions and passages in a scientific reading procedure. (3) Precise answer refining over the passage.

Implemental Details	We preprocess each passage and question using the library of nltk  and exploit the popular pre-trained word embedding GloVe with 100-dimensional vectors  for both questions and passages. The size of char-level embedding is also set as 100-dimensional and is obtained by CNN filters under the instruction of  . The Gated Recurrent Unit  which is variant from LSTM  is employed throughout our model. We adopt the AdaDelta  optimizer for training with an initial learning rate of 0.0005. The batch size is set to be 48 for both the SQuAD and TriviaQA datasets. We also apply dropout  between layers with a dropout rate of 0.2. For the multi-hop reasoning, we set the number of hops as 2 which is imitating human reading procedure on skimming and scanning. During training, we set the moving averages of all weights as the exponential decay rate of 0.999  . The whole training process takes approximately 14 hours on a single 1080Ti GPU.

Therefore, we propose a multi-hop memory network which allows to reread the question and answer. In our model, we totally apply two-hops memory network, as is depicted in the structure (c to e) and (f to h). In our experiment we found the two-hops can reach the best performance. In detail, the memory vector stores the question-aware passage representations, the old memory's output is updated through a repeated interaction attention.

We set the INLINEFORM1 as in most cases the latter predicted answer is more accurate comparing to the former one. The final INLINEFORM2 and INLINEFORM3 is then judged by the max value through the argmax operator.

Training and Optimization	We choose the training loss as the sum of the negative log probabilities of the true start and end position by the predicted distributions to train our model: DISPLAYFORM0   where INLINEFORM0 denotes all the model coefficients including the neural network parameters and the input gating function parameters, N is the number dataset examples, INLINEFORM1 and INLINEFORM2 are the predicted distributions of the output, INLINEFORM3 and INLINEFORM4 are the true start and end indices of the INLINEFORM5 th example. The objective function in our learning process is given by: DISPLAYFORM0   where INLINEFORM0 is the trade-off parameter between the training loss and regularization. To optimize the objective, we employ the stochastic gradient descent (SGD) with the diagonal variant of AdaDelta  .

Furthermore, as the SQuAD and TriviaQA are competitive MC benchmark, we train an ensemble model consisting of 16 training runs with the same architecture but identical hyper-parameters. The answer with the highest sum of the confidence score is chosen for each query.

Smarnet Structure	The general framework of MC task can be coarsely summarized as a three-layer hierarchical process: Input embedding layer, Interaction modeling layer, answer refining layer. We then introduce our model from these three perspectives.

Overall Results	We evaluate the performance of our proposed method based on two evaluation criteria EM and F1 for the MC tasks. We compare our model with other strong competitive methods on the SQuAD leaderboard and TriviaQA leaderboard. Table 1 and Table 2 respectively show the performance of single and ensemble models on the SQuAD leaderboard. The SQuAD leaderboard is very competitive among top NLP researchers from all over the world. We can see the top record has been frequently broken in order to reach the human's level. Our model was submitted by July 14, 2017, thus we compare our model on the single and ensemble performance against other competitor at that time. From the tables 1 and 2 we can see our single model achieves an EM score of INLINEFORM0 and a F1 score of INLINEFORM1 and the ensemble model improves to EM INLINEFORM2 and F1 INLINEFORM3 , which are both only after the r-net method  at the time of submission. These results sufficiently prove the significant superiority of our proposed model. We also compare our models on the recently proposed dataset TriviaQA. Table 3 shows the performance comparison on the test set of TriviaQA.

We can see our Smarnet model outperforms the other baselines on both wikipedia domain and web domain.

Parameters Tuning	We conduct two parameters tuning experiments in order to get the optimal performance. Figure 4 shows the results on different hops of memory network. We see the number of hops set to 2 can get the best performance comparing to other number of hops. In addition, as the number of hops enlarges, the model is easily to get overfitting on the training set, hence the performance is decrease rather than increase. In figure 5, we set different weight of INLINEFORM0 into five groups INLINEFORM1 . The final results show that the proportion of the first answer prediction and the last one reaches to 2:3 can get the most confident answer judgement. The value of INLINEFORM2 which is greater than 1, indicating that the latter answer refining takes more insurance on the prediction decision.

We also add a checking layer on the answer refining in order to ensure the accuracy. The main contributions of this paper are as follows:

Question: How does the gatint mechanism combine word and character information?
Answer: when the gate has high value, more information flows from the word-level representation; otherwise, char-level will take the dominating place, for unfamiliar noun entities, the gates tend to bias towards char-level representation in order to care richer morphological structure