# An Idiom Reading Comprehension Model Based on Multi-Granularity Reasoning and Paraphrase Expansion
## 3 Model
### Multi-Granularity Integrated Attention Mechanism  
We found that the existing work focused on optimizing the network and model structure from an engineering perspective rather than modeling the realistic behavior of humans in reading comprehension. In contrast, human reading comprehension is characterized by leaps, global grasp, and local focus, and we attempted to build a new idiom reading comprehension model based on the BERT-IDM\({}_{\text{base}}\) model proposed in the previous section with more explanatory capabilities using multi-granularity reasoning and the introduction of idiom interpretation for expansion. We based our model on the fact that people tend to focus their attention on the main idea of the passage after grasping it, rather than grasping every detail for memorization, when performing reading comprehension. Therefore, we improved the attention mechanism of the BERT-IDM\({}_{\text{base}}\) model in the fine-tuning phase by introducing a multi-granularity integrated attention mechanism. In MGIA, global attention (GA) is used to simulate the general reading process of human reading comprehension, stride attention (SA) is used to simulate the jump reading process of reading comprehension, and window attention (WA) is used to simulate fine reading focusing on local information. MGIA is a kind of sparse attention, which could effectively reduce the number of parameters and improve the operational efficiency compared with the fully connected dense matrix used by the original self-attentive mechanism. In terms of implementation, MGIA makes use of the multi-headed attention mechanism of the Transformer structure, wherein the stacked attention heads are assigned to different attentions according to a predefined pattern. This allows the model to use different approaches to the reading comprehension inference process simultaneously, and the importance (i.e., weight) of different inference patterns can be adjusted by setting the distribution of attention heads. The combined and integrated attention mask matrix is shown in Figure 5, where the pink squares are global attention (GA), the blue squares are stride attention (SA), and the yellow squares are window attention (WA). For a set recording the positions of a set of elements to be attended to by the attention mechanism as the current position coordinates, the three types of attention used by MGIA can be specified as described below (using the lower triangular matrix). There are two specific implementation methods for global attention (GA), one is to select the individual clauses at the head of the sequence as global clauses, and the other is to add new clauses without real meaning to the beginning of the sentence to aggregate the information of the whole sequence. In this section, we chose to use the latter and add [CLS] clauses uniformly to avoid the bias caused by data samples. After setting the global clauses, all clauses needed to calculate attention scores with global clauses, and the global clauses also needed to calculate attention scores with global clauses. Thus, all clauses in the sequence (i.e., the overall information distributed) were linked by global clauses, at which time extracting the global clauses could have the effect of grasping the main idea of the passage. The attention set corresponding to global attention can be expressed as presented in Equation (4):  
\[S_{i}=\{j|j<i<n\}. \tag{4}\]  
The stride attention mechanism (SA) was originally used for image generation and later applied to language modeling. Through the SA model, long-range dependencies in sequences can be constructed, and vector representations can be compressed to reduce computational complexity. This effectively increases the receptive field (analogous to a convolution operation) for the same computational cost under longer sequence lengths. For a given stride length \(k\), the attention matrix attends to every \(k\) element from the current element and aggregates them into set \(S_{i}\) for attention.