Two-Step Question Retrieval for Open-Domain QA (2022)

Abstract: The retriever-reader pipeline has shown promising performance in open-domain QA but suffers from a very slow inference speed. Recently proposed question retrieval models tackle this problem by indexing question-answer pairs and searching for similar questions. These models have shown a significant increase in inference speed, but at the cost of lower QA performance compared to the retriever-reader models. This paper proposes a two-step question retrieval model, SQuID (Sequential Question-Indexed Dense retrieval) and distant supervision for training. SQuID uses two bi-encoders for question retrieval. The first-step retriever selects top-k similar questions, and the second-step retriever finds the most similar question from the top-k questions. We evaluate the performance and the computational efficiency of SQuID. The results show that SQuID significantly increases the performance of existing question retrieval models with a negligible loss on inference speed.

## introduction retriever-reader models in open-domain qa require a long time for inference @xcite @xcite @xcite . this has been identified as a bottleneck in building real-time qa systems, and question retrieval and phrase-indexed qa have been proposed to resolve this problem @xcite @xcite @xcite . these approaches directly search the answer of the input question from the corpus without conducting additional machine reading steps which are computationally inefficient. in phrase-indexed qa, retrievers pre-index all phrases in the corpus and find the most similar phrase to the input question. in question retrieval, synthetic question-answer pairs are pre-indexed and referenced by retrievers @xcite @xcite . although recent question retrieval models significantly increase the inference speed, this improvement accompanies qa performance degradation. several approaches have been applied to question retrieval models to overcome the performance degradation, such as adopting the crossencoder @xcite for re-ranking and increasing the model size @xcite . however, these approaches cause a significant loss of computational efficiency. figure 1 shows the trade-off between the open-domain qa performance and the inference speed of question retrieval models. we propose squid (sequential question-indexed dense retrieval) which significantly improves qa performance without losing computational efficiency. our work follows previous work on neural re-ranking methods, which use a crossencoder to re-rank the top-k passages retrieved from the first-step retriever @xcite . re-ranking methods have improved retrieval performance but require huge computation costs due to the cross-encoder architecture. we use an additional bi-encoder retriever in squid instead of the cross-encoder to prevent loss on computational efficiency. we also provide distant supervision methods for training the additional retriever in the absence of training data for question retrievers. we evaluate squid on naturalquestions (nq) @xcite and triviaqa @xcite . we conduct three types of experiments: open-domain qa, computational efficiency evaluation, and analysis on distant supervision methods for training the second-step retriever. experimental results show that squid significantly outperforms the state-of-the-art question retrieval model by 4.0%p on nq and 6.1%p on triviaqa without losing computational efficiency. our main contribution is in proposing a sequential question retriever model that successfully improves both qa performance and inference speed, thereby making a meaningful step toward developing real-time opendomain qa systems. ## related work the research problem of reducing the computational cost of open-domain qa has received much attention recently. the main bottleneck of a retriever-reader model is the machine reading step, and @xcite ; @xcite propose phrase-indexed qa, which directly retrieves the answer from the corpus without the machine reading step. these models pre-compute the context of phrases in a corpus and conduct lexical and semantic similarity searches between the given question and the context of phrases @xcite . most related to our work are the question retrieval models with question-generation models to build questionanswer pairs and conduct a similarity search between the input question and the pre-indexed questions @xcite . these models significantly reduce the computational cost but results in lower performance. our work provides an efficient question retrieval pipeline with distant supervision methods for training, while previous question retrieval models focus on the indexing methods with less attention on the retrieval pipeline. ## method our method is constructed based on the question retrieval pipeline proposed by @xcite , where question retrievers find the most similar question to the input question and return the answer of the selected question. in this study, we note that previous question retrievers are optimized not just for improving the retrieval performance but for maintaining the inference speed to cover millions of text @xcite . in this process, the performance of retrievers decreases as they are more optimized for computational efficiency. we propose to use an additional retriever that takes the top-k predictions from the first retriever and selects the most similar question from the top-k results. the second-step retriever has a lower constraint in the inference speed than the first retriever since its search space contains only a few samples. this enables us to focus only on the retrieval performance when designing the training method. the overall training and inference procedure of squid is illustrated in figure 2 . we describe the details of squid below. ## experimental setup and results we evaluate the performance and computational efficiency of squid on two open-domain qa datasets: naturalquestions (nq) and triviaqa. we also compare various distant supervision methods for training squid. we use exact match (em) (rajpurkar et al., 2016) for performance evaluation and the number of questions per second (q/sec) for evaluation of inference speed. the details of our experimental setup is described in appendix a.2. question retrievers on open-domain qa: we evaluate squid with two different first-step retrievers: bm25 and repaq-base256 2 (lewis et al., 2021b). trade-off between qa performance and computational efficiency: figure 1 illustrates the qa performance and inference speed of various configurations of repaq squid. we vary the encoder of the second-step retriever with different pre-trained models: dpr @xcite , bert-base/large @xcite , and albert-base/large @xcite . the first and second-step question encoders can be executed concurrently, so we run them in parallel and set the batch size as half to measure the inference speed (squid-dpr-parallel). we use the maximum batch size possible on a single v100-16gb gpu. the figure shows that results of squid all lie to the top right of the curve fitted to the repaq results, meaning that squid succeeds in improving both qa performance and inference model type model nq triviaqa inference speed (q/sec) question retrieval repaq-base256 @xcite 40.0 38.8 1376 repaq-base @xcite 40.9 39.7 738 repaq-large @xcite 41 analysis on positive sampling methods: we distantly supervise the second-step retriever because annotated question-question pairs are unavailable. we conduct experiments on various positive sampling methods for distant supervision: "self", "similar", "similar/self", and "same answer". each method uses the following as the positive sample: 1) the input question itself ("self"), 2) a similar question with a similar answer ("similar"), 3) a similar question if it has the ground truth answer, or the input question itself ("similar/self"), and 4) a random question with the ground truth answer ("same answer"). ## conclusion the trade-off between the performance and the inference speed is an important problem in opendomain qa. recently proposed question retrieval models have shown significantly improved inference speed. however, this improvement came at the cost of a significantly lower qa performance by the question retrieval models compared to the stateof-the-art open-domain qa models. in this paper, we proposed a two-step question retrieval model, squid. we evaluated the open-domain qa performance and the inference speed of squid on two datasets: naturalquestions and triviaqa. from the results, we showed that the sequential two-retriever approach in squid achieves a significant qa performance improvement over the existing question retrieval models, while retaining the advantage of faster inference speed. this improvement in both qa performance and inference speed is a meaningful step toward the development of real-time open domain qa systems.