# A Hybrid Text Generation-Based Query Expansion Method for Open-Domain Question Answering
## 3 Experimental Results and Analysis
### Passage Reading with HTGQE  
To evaluate whether the HTGQE approach can offer OpenQA improvements, we conducted experiments comparing the open-book HTGQE and closed-book GAR models using the retrieval results from Section 3.1. The outstanding performance of GAR is already  
\begin{table}
\begin{tabular}{l l c c c c c c c c c} \hline \hline  & **Dataset** & \multicolumn{4}{c}{**NQ**} & \multicolumn{4}{c}{**Trivia**} \\ \cline{2-11}  & **Methods** & **Top-5** & **Top-20** & **Top-100** & **Top-500** & **Top-1000** & **Top-5** & **Top-20** & **Top-100** & **Top-500** & **Top-1000** \\ \hline
**Sparse** & BM25(ours) & 43.8 & 62.9 & 78.3 & 85.6 & 88.0 & 66.3 & 76.4 & 83.2 & 87.3 & 8.5 \\  & BM25+RM3 [10] & 44.6 & 64.2 & 79.6 & 86.8 & 88.9 & 67.0 & 77.1 & 83.8 & 87.7 & 88.9 \\  & GAR (\(ctx(A)\)) & 53.2 & 67.1 & 79.4 & 86.7 & 89.2 & 66.5 & 76.0 & 83.7 & 88.0 & 89.2 \\  & HTGQE (\(ctx(A)\)) & 64.2 & 73.7 & 82.7 & 87.8 & 89.5 & 74.2 & **79.8** & **85.0** & 88.5 & 89.5 \\  & GAR (\(ctx(S)\)) & 52.7 & 66.6 & 79.5 & 86.5 & 89.0 & 62.3 & 72.6 & 81.0 & 86.7 & 88.0 \\  & HTGQE (\(ctx(S)\)) & 64.4 & 73.2 & 82.2 & 88.1 & 89.6 & 72.6 & 78.0 & 83.2 & 87.1 & 88.4 \\  & GAR (\(ctx(T)\)) & 50.8 & 67.2 & 79.8 & 87.0 & 88.8 & 65.1 & 75.3 & 82.6 & 87.0 & 88.3 \\  & HTGQE (\(ctx(T)\)) & 55.7 & 69.8 & 81.0 & 87.4 & 89.2 & 70.2 & 77.8 & 83.8 & 87.7 & 88.8 \\  & GAR(ours) & 58.5 & 72.6 & 83.8 & 89.2 & 90.9 & 68.7 & 77.7 & 84.2 & 88.2 & 89.3 \\  & HTGQE & **67.3** & **76.2** & **84.5** & **89.8** & **91.3** & **74.2** & 79.6 & 84.9 & **88.6** & **89.6** \\ \hline
**Hybrid** & BM25+DPR [12] & 60.3 & 76.1 & 86.0 & 90.1 & 91.7 & 72.1 & 80.9 & 86.3 & 89.0 & 89.8 \\  & GAR+DPR(ours) & 66.2 & 79.1 & 87.3 & 91.3 & 92.4 & 73.2 & 81.5 & 86.6 & 89.2 & 90.0 \\  & HTGQE+DPR & **69.7** & **80.6** & **87.4** & **91.5** & **92.5** & **75.2** & **81.8** & **86.7** & **89.5** & **90.2** \\ \hline \hline \end{tabular}
\end{table}
Table 2: Top-k retrieval accuracy of various on NQ and Trivia.  
known in previous study [10], this section aims to verify that the HTGQE is more effective than GAR. It is important to note that both the extractive and generative readers in this paper have not undergone training, so they will not achieve the results reported in the GAR [10].  
As shown in Table 3, the top 50 candidate passages from the retrieval results are selected as inputs for the reader. HTGQE outperforms GAR in both extractive and generative settings. For the extractive reader, the former surpasses the latter by 4.57% and 6.35% on the NQ and Trivia, respectively. In the generative reader setting, HTGQE exceeds GAR by 14.34% and 17.09%, respectively. This further demonstrates that text generation incorporating PRF techniques is more effective because it provides useful reference materials during the text generation process. Moreover, the different contextual environments also contribute significantly to the experimental results.  
Furthermore, while HTGQE focuses on improving sparse retrieval, the new sparse retrieval results bring improvements in EM scores that outperform many other neural network methods, even if they take 50, 100, or even more passages as the input of the readers. However, since we directly use the existing reader checkpoint and do not train the reader on the new retrieval results, the reader has yet to learn the knowledge in the new retrieval results. So there is still a gap between our method and SOTA methods. If you retrain the reader, expect to see a significant performance improvement.