# Multigranularity Syntax Guidance with Graph Structure for Machine Reading Comprehension
## 5 Results and Discussion
### Ablation Study  
In this section, we describe certain ablation experiments that were conducted to demonstrate the validity of the proposed model. All the experimental baselines refer to the BERT base model except for the comparison of changes in variable. To verify the effectiveness of our module, we separately added two modules to the baseline model. From the experimental results presented in Table 3, we can see that when we add the GD module and the graph with entity (GE) module to the baseline, the performance of the model is significantly improved. Further, combined with the results presented in Table 1, we understand that these two modules can improve performance when used alone, and they are compatible and improve each other's performance when used together. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Model** & **EM(\%)** & **F1(\%)** & **EM(\%) for** & **F1(\%) for** \\  & & & **MgSG** & **MgSG** \\ \hline Human performance & 75.8 & 79.2 & & \\ \hline P-Reader (single model) & 76.7 & 80.6 & & \\ Z-Reader (single model) & 76.3 & 80.5 & & \\ \hline BERT-base & 63.6 & 83.9 & 64.3 & 84.4 \\ BERT-wwm-ext & 64.6 & 84.8 & 64.8 & 84.8 \\ RoBERTa-wwm-ext & 65.5 & 85.5 & 65.1 & 85.6 \\ ELECTRA-base & 66.9 & 83.5 & 67.5 & 84.7 \\ ELECTRA-large & 67.6 & 83.8 & 67.9 & 85.4 \\ PERT-base & 64.1 & 84.5 & 64.8 & 85.1 \\ MacBERT-base & 66.3 & 85.4 & 66.1 & 85.4 \\ ERNIE 2.0-base & 67.8 & 87.5 & 67.7 & 87.3 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Results for CMRC-2018 dev dataset. The results of the experiments are obtained in our local environment and the corresponding literatures. \begin{table}
\begin{tabular}{l c c} \hline \hline
**Model** & **EM (\%)** & **F1 (\%)** \\ \hline BERT & 75.8 (\(\pm\)0.2) & 79.2 (\(\pm\)0.2) \\ BERT + GD & 76.7 (\(\pm\)0.1) & 80.6 (\(\pm\)0.2) \\ BERT + GE & 76.5 (\(\pm\)0.1) & 80.5 (\(\pm\)0.2) \\ BERT + GD + GE & 77.2 (\(\pm\)0.1) & 81.1 (\(\pm\)0.2) \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Impact of various modules on the performance of the model. We also demonstrate the effectiveness of our proposed method with an example. As shown in Figure 7, in the first example, the question is _What chemical did [PERSON] use in his experiments on oxygen?_ and the answer is obviously in the vicinity of the person named entity in the question in the paragraph, and then the BERT model. The answer was not predicted, and the answer suggested by our model was mercuric oxide. This shows that by constructing part-of-speech edges, the model can have a more specific goal in finding answers. The problem of the second example in Figure 7 is _Which is one of the park features located in North Fresno?_, the difficulty of this question is that it does not provide a very direct keyword, _North Fresno, feature_ can only provide the approximate location of the answer in the text and there are multiple misleading answers around. Our method can give the correct answer because it follows the grammatical structure and reinforces the hidden information contained in _Woodward Park_ as the subject of _which features the Shinzen Japanese Gardens_, while BERT cannot. We also verify the effectiveness of the dependency syntax through experiments. We will first average the hidden state representations of the last layer of BERT and generate the corresponding heatmap (Figure 8). We find that BERT is significantly more interested in regular subject position, but this is not where the answer comes from. We then summed and averaged the output of our module with the representation of the hidden state. The darker the color, the higher the impact. Due to the addition of dependency syntax, the attention of the model significantly weakens the position of the beginning, and is biased towards the parenthesis in the sentence, which is the source of the correct answer. Figure 8: Effectiveness analysis. The darker the color of the token, the greater the influence of the token.