# Neural Machine Reading Comprehension: Methods and Trends
## 3 Deep-Learning-Based Methods
### Typical Deep-Learning Methods
#### 3.2.3 Context-Question Interaction  
By extracting the correlation between the context and the question, models can find evidence for answer prediction. Inspired by [PERSON] et al. [26], we can divide existing works into two kinds according to how models extract correlations: one-hop and multi-hop interaction. No matter what kind of interaction the MRC model uses, the attention mechanism plays a critical role in emphasizing which parts of the context are more important to answer the questions. Derived from human intuition, the attention mechanism was adapted for machine translation and shows promising performance on automatic token alignment [48; 49]. As a simple and effective method that can be used to encode sequence data with its importance, it has shown significant improvement in various tasks in natural language processing, including text summarization [50], sentiment classification [51], semantic parsing [52], etc. In terms of machine reading comprehension, the attention mechanism can be categorized into unidirectional and bidirectional attention according to whether it is used unidirectionally or bidirectionally. In the following part, we introduce methods categorized using the attention mechanism, followed by illustrations of one-hop and multi-hop interactions. Figure 10: Using the Transformer to extract features of question. (1) Unidirectional Attention  
Unidirectional attention flow is usually from query to context, highlighting the most relevant parts of the context according to the question. It is believed that if the context word is more similar to the question, it is more likely to be the answer word. As shown in Figure 11, the similarity of each context semantic embedding \(P_{i}\) and the whole question sentence representations \(Q\) (by sentence-level encoding introduced in Section 3.2.2) is calculated by \(S_{i}=f(P_{i},Q)\), where \(f(\cdot)\) represents the function that can measure the similarity. After being normalized by the SoftMax function in Equation (4), the attention weight \(\alpha_{i}\) for each context word is obtained, with which the MRC systems can finally predict the answer. \[\alpha_{i}=\frac{\exp S_{i}}{\sum_{j}\exp S_{j}}. \tag{4}\]  
The choice of function \(f(\cdot)\) differs in different models. In the Attentive Reader proposed by [PERSON] et al. [5], a tanh layer is used to compute the relevance between the context and the question as follows:  
\[S_{i}=\tanh(W_{P}P_{i}+W_{Q}Q), \tag{5}\]  
where \(W_{P}\) and \(W_{Q}\) are trainable parameters. Following the work of [PERSON] et al., [PERSON] et al. [12] substitute a bilinear term for the tanh function as Equation (6):  
\[S_{i}=Q^{T}W_{s}P_{i}, \tag{6}\]  
which makes the model simpler and more effective than the Attentive Reader. The unidirectional attention mechanism can highlight the most important context words to answer the question. However, this method fails to pay attention to question words that are also pivotal for answer prediction. Hence, unidirectional attention flow is insufficient for extracting mutual information between the context and the query. Figure 11: Using unidirectional attention to mine correlation between the context and question. (2) Bidirectional Attention  
Seeing the limitations of the unidirectional attention mechanism, some researchers introduced bidirectional attention flows, which not only computes query-to-context attention but also the reverse, context-to-query attention. This method, which takes a mutual look from both directions, can benefit from the interaction between context and query and provide complementary information. Figure 12 presents the process of computing bidirectional attention. First, the pair-wise matching matrix \(M(i,j)\) is obtained by computing the matching scores between each context semantic embedding \(P_{i}\) and question semantic embedding \(Q_{j}\) (by word-level encoding introduced in Section 3.2.2).