# A Hybrid Text Generation-Based Query Expansion Method for Open-Domain Question Answering
## 2 Materials and Methods
### Model Structure and Initialization  
**Dual-Tower Retriever. ** This study uses Pyserini [12] as the retriever. Pyserini [12] is an information retrieval toolkit that supports sparse retrieval by integrating the Anserini toolkit and dense retrieval by integrating Facebook's Faiss Library. Pyserini [12] also supports hybrid retrieval methods by combining dense retrieval results and sparse retrieval results with a fusion factor \(\alpha\), which is set to 1.0 and 0.95 for the NQ and Trivia in the experiments, respectively. \[h\_score=Sim(q,D_{j})+\alpha\cdot BM25(q,D_{j}) \tag{1}\]  
\(BM25(q,D_{j})\) represents the relevance score between the query and passage in the sparse retrieval; \(Sim(q,D_{j})\) represents the relevance score in the dense retrieval; \(h\_score\) represents the relevance score for hybrid retrieval. **Extractive Reader. ** The settings for the extractive reader GAR [10] are in full compliance with the design and implementation of PyGaggle [15], which integrates passage-level voting and evidence fusion techniques. The retrieved passage list and its individual passage relevance scores are denoted as \(D=[d_{1},d_{2},...d_{i},...d_{k}]\). The top n text spans in passage \(d_{i}\) are denoted as \(S_{i}=[s_{1},s_{2},...s_{j},s_{n}]\). Text spans are ranked by calculating the relevance scores \(\vec{S_{i}}\). Finally, the prediction scores for the same text span from different retrieved passages are aggregated using normalized answer span scoring techniques. PyGaggle [15] uses evidence fusion techniques, fusing \(\vec{D}\) and \(\vec{R}\) through the formula \(\beta\cdot\vec{D}+\gamma\cdot\vec{R_{i}}\), where \(\vec{R}\) represents the retrieval scores assigned to each passage by the retriever. Depending on the chosen retrieval method, \(\vec{R}\) can be sparse retrieval scores or hybrid retrieval scores. The final combined score replaces the original \(\vec{D}\) as the relevance score for all contexts in the answer span scoring step. For example, the calculation of the answer span by [PERSON] et al. [10] using evidence fusion techniques is as follows:  
\[p(S_{i}[j])=softmax(\beta\cdot\vec{D}+\gamma\cdot\vec{R_{i}})[i]\times softmax (\vec{S_{i}})[j] \tag{2}\]  
We called this reader GAR*. **Generative Reader. ** In this paper, we choose the model published by [PERSON] [6] as the generative reader. This generative model for OpenQA is based on a sequence-to-sequence network model, pre-trained on unsupervised data such as the T5 [3] model. Each retrieved passage and its title are connected to the question as a single data sample for processing, independent of other passages; the decoder applies attention mechanisms on \begin{table}
\begin{tabular}{l l l l l} \hline \hline  & **Dataset** & **Train** & **Val** & **Test** \\ \hline \multirow{4}{*}{**NQ**} & _ori_ & 79,168 & 8757 & 3610 \\  & \(ctx(A)\) & 79,168 & 8757 & 3610 \\  & \(ctx(S)\) & 79,168 & 8757 & 3610 \\  & \(ctx(T)\) & 67,969 & 7514 & 3610 \\ \hline \multirow{4}{*}{**Trivia**} & _ori_ & 78,785 & 8837 & 11,313 \\  & \(ctx(A)\) & 78,785 & 8837 & 11,313 \\  & \(ctx(S)\) & 65,375 & 7388 & 11,313 \\  & \(ctx(T)\) & 69,595 & 7848 & 11,313 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Statistics of the number of data sets after processing. the distributed representation of all data (output by the encoder) to obtain the answer. The encoder processes passages independently, which can be scaled to large datasets as it computes self-attention mechanisms for only one data sample at a time. However, jointly processing passages in the decoder can better aggregate evidence from multiple passages. **Query Expansion Generator. ** In this paper, we utilize BART-large [16] as baseline models to obtain \(G_{A}\), \(G_{S}\) and \(G_{T}\), which generate three types of query expansion contexts, respectively. The top three pseudo-relevant passages from the retrieval results are used as \"reference materials\" for the generative model, denoted as \(p_{1}\), \(p_{2}\), and \(p_{3}\).