# Multi-Hop Question Generation with Knowledge Graph-Enhanced Language Model
## 4 Results
### Ablation Tests Analysis  
The results of the three ablation tests conducted to evaluate the impact of the AT, multi-head attention, and EL modules in KGEL are presented in Table 3. The incorporation of answer information, indicating whether a word is within or outside the target answer, has been shown to play a critical role in seq2 seq-based models on single-hop QG tasks [6,8,10]. This information has been included in almost all Seq2 Seq-based QG models [6,8,10]. An ablation study conducted on the S2s-at-mp-gsa model showed that the AT significantly improves QG performance, highlighting the importance of providing the model with the information of what to ask [6].  
In the absence of answer information, the performance of KGEL-AT in terms of BLEU decreases, whereas the performance in terms of METEOR and ROUGE-L slightly increases, making the results of these two models comparable. This difference in performance may be due to the relative simplicity of single-hop QG tasks, which do not require multi-step reasoning, as opposed to multi-hop QG. The significance of answer information in this task is therefore more pronounced. In contrast, answer information is less significant in multi-hop QG, as this task requires reasoning between multiple contents, a challenge addressed by the proposed graph attention mechanism.  
The results of the ablation tests evaluating the impact of multi-head attention (MHA) and evidence locating (EL) modules on the KGEL model are presented in Table 3. The performance of the KGEL model is significantly impacted by the removal of the multi-head attention mechanism, with reductions of 2.46, 2.24, and 2.3 in BLEU-4, METEOR, and ROUGE-L scores, respectively. This result supports the significance of multi-head attention in enabling the interaction across the fused representation of the input text to be modeled by the KGEL. Similarly, the removal of the EL module results in a significant decrease in performance, with KGEL outperforming KGEL-EL by 25.4%, 20.0%, and 7.7% in terms of BLEU-4, METEOR, and ROUGE-L, respectively. This highlights the effectiveness of the evidence locating component in establishing the reasoning chain between the answer and answer-related entities, which is critical for multi-hop QG.  
\begin{table}
\begin{tabular}{l c c c c c c} \hline \hline  & **B1** & **B2** & **B3** & **B4** & **R-L** & **M** \\ \hline seq2 seq+Attn & 32.97 & 21.11 & 15.41 & 11.81 & 18.19 & 33.48 \\ NQG++ & 35.51 & 22.12 & 15.53 & 11.50 & 16.96 & 32.01 \\ ASs2s & 34.60 & 22.77 & 15.21 & 11.29 & 16.78 & 32.88 \\ S2 sa-at-mp-gsa & 35.36 & 22.38 & 15.88 & 11.85 & 17.63 & 33.02 \\ GPT-2 & 39.00 & 24.46 & 17.21 & 12.71 & 16.81 & 32.06 \\ KGEL & **41.93** & **28.04** & **20.83** & **16.13** & **19.70** & **35.28** \\ \hline \hline \end{tabular}
\end{table}
Table 2: Evaluation results of different models by BLEU 1–4(B1-4),ROUGE-L(R-L), and METEOR(M).The larger the number in the table, the better the result, with the bolded result representing the best result in the comparison model.  
\begin{table}
\begin{tabular}{l c c c c c} \hline \hline
**Model** & **B1** & **B2** & **B3** & **B4** & **R-L** & **M** \\ \hline KGEL & **41.93** & **28.04** & **20.83** & **16.13** & 19.70 & 35.28 \\ KGEL-AT & 41.14 & 27.32 & 20.00 & 15.24 & **20.12** & **35.53** \\ KGEL-MHA & 38.84 & 25.00 & 18.03 & 13.66 & 17.46 & 32.98 \\ KGEL-EL & 39.13 & 25.10 & 18.11 & 12.86 & 16.41 & 32.76 \\ \hline \hline \end{tabular}
\end{table}
Table 3: Evaluation results of ablation tests.The larger the number in the table, the better the result, with the bolded result representing the best result.