R ocket QA : An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answering (2021)

Abstract: In open-domain question answering, dense passage retrieval has become a new paradigm to retrieve relevant passages for finding answers. Typically, the dual-encoder architecture is adopted to learn dense representations of questions and passages for semantic matching. However, it is difficult to effectively train a dual-encoder due to the challenges including the discrepancy between training and inference, the existence of unlabeled positives and limited training data. To address these challenges, we propose an optimized training approach, called RocketQA, to improving dense passage retrieval. We make three major technical contributions in RocketQA, namely cross-batch negatives, denoised hard negatives and data augmentation. The experiment results show that RocketQA significantly outperforms previous state-of-the-art models on both MSMARCO and Natural Questions. We also conduct extensive experiments to examine the effectiveness of the three strategies in RocketQA. Besides, we demonstrate that the performance of end-to-end QA can be improved based on our RocketQA retriever.

## introduction open-domain question answering (qa) aims to find the answers to natural language questions from a large collection of documents. early qa systems @xcite @xcite constructed complicated pipelines consisting of multiple components, including question understanding, document retrieval, passage ranking and answer extraction. recently, inspired by the advancements of machine reading comprehension (mrc), @xcite proposed a simplified two-stage approach, where a traditional ir * corresponding authors. † † the work was done when ruiyang ren was doing internship at baidu. 1 our code is available at https://github.com/ paddlepaddle/research/tree/master/nlp/ naacl2021-rocketqa to deal with the issue of term mismatch, the dual-encoder architecture (as shown in figure 1a ) has been widely explored @xcite @xcite @xcite to learn dense representations of questions and passages in an end-to-end manner, which provides better representations for semantic matching. these studies first separately encode questions and passages to obtain their dense representations, and then compute the similarity between the dense representations using similarity functions such as cosine or dot product. typically, the dual-encoder is trained by using in-batch random negatives: for each question-positive passage pair in a training batch, the positive passages for the other questions in the batch would be used as negatives. however, it is still difficult to effectively train a dual-encoder for dense passage retrieval due to the following three major challenges. first, there exists the discrepancy between training and inference for the dual-encoder retriever. during inference, the retriever needs to identify positive (or relevant) passages for each question from a large collection containing millions of candidates. however, during training, the model is learned to estimate the probabilities of positive passages in a small candidate set for each question, due to the limited memory of a single gpu (or other device). to reduce such a discrepancy, previous work tried to design specific mechanisms for selecting a few hard negatives from the top-k retrieved candidates @xcite @xcite @xcite . however, it suffers from the false negative issue due to the following challenge. second, there might be a large number of unlabeled positives. usually, it is infeasible to completely annotate all the candidate passages for one question. by only examining the the top-k passages retrieved by a specific retrieval approach (e.g. bm25), the annotators are likely to miss relevant passages to a question. taking the msmarco dataset @xcite as an example, each question has only 1.1 annotated positive passages on average, while there are 8.8m passages in the whole collection. as will be shown in our experiments, we manually examine the top-retrieved passages that were not labeled as positives in the original msmarco dataset, and we find that 70% of them are actually positives. hence, it is likely to bring false negatives when sampling hard negatives from the top-k retrieved passages. third, it is expensive to acquire large-scale training data for open-domain qa. msmarco and natural questions @xcite are two largest datasets for open-domain qa. they are created from commercial search engines, and have 516k and 300k annotated questions, respectively. however, it is still insufficient to cover all the topics of questions issued by users to search engines. in this paper, we focus on addressing these challenges so as to effectively train a dual-encoder retriever for open-domain qa. we propose an optimized training approach, called rocketqa, to improving dense passage retrieval. considering the above challenges, we make three major technical contributions in rocketqa. first, rocketqa introduces cross-batch negatives. comparing to inbatch negatives, it increases the number of available negatives for each question during training, and alleviates the discrepancy between training and inference. second, rocketqa introduces denoised hard negatives. it aims to remove false negatives from the top-ranked results retrieved by a retriever, and derive more reliable hard negatives. third, rocketqa leverages large-scale unsupervised data "labeled" by a cross-encoder (as shown in figure 1b ) for data augmentation. though inefficient, the cross-encoder architecture has been found to be more capable than the dual-encoder architecture in both theory and practice @xcite . therefore, we utilize a cross-encoder to generate highquality pseudo labels for unlabeled data which are used to train the dual-encoder retriever. the contributions of this paper are as follows: ## related work passage retrieval for open-domain qa for opendomain qa, a passage retriever is an important component to identify relevant passages for answer extraction. traditional approaches @xcite implemented term-based passage retrievers (e.g. tf-idf and bm25), which have limited representation capabilities. recently, researchers have utilized deep learning to improve traditional passage retrievers, including document expansions @xcite , question expansions @xcite and term weight estimation @xcite . different from the above term-based approaches, dense passage retrieval has been proposed to represent both questions and documents as dense vectors (i.e., embeddings), typically in a dual-encoder architecture (as shown in figure 1a ). existing approaches can be divided into two categories: (1) self-supervised pre-training for retrieval @xcite @xcite and (2) fine-tuning pre-trained language models on labeled data. our work follows the second class of approaches, which show better performance with less cost. although the dual-encoder architecture enables the appealing paradigm of dense retrieval, it is difficult to effectively train a retriever with such an architecture. as discussed in section 1, it suffers from a number of challenges, including the training and inference discrepancy, a large number of unlabeled positives and limited training data. several recent studies @xcite @xcite tried to address the first challenge by designing complicated sampling mechanism to generate hard negatives. however, it still suffers from the issue of false negatives. the later two challenges have seldom been considered for open-domain qa. passage re-ranking for open-domain qa based on the retrieved passages from a first-stage retriever, bert-based rerankers have recently been applied to retrieval-based question answering and search-related tasks @xcite @xcite , and yield substantial improvements over the traditional methods. although effective to some extent, these rankers employ the cross-encoder architecture (as shown in figure 1b ) that is impractical to be applied to all passages in a corpus with respect to a question. the re-rankers @xcite with light weight interaction based on the representations of dense retrievers have been studied. however, these techniques still rely on a separate retriever which provides candidates and representations. as a comparison, we focus on developing dual-encoder based retrievers. ## approach in this section, we propose an optimized training approach to dense passage retrieval for opendomain qa, namely rocketqa. we first introduce the background of the dual-encoder architecture, and then describe the three novel training strategies in rocketqa. lastly, we present the whole training procedure of rocketqa. ## conclusions in this paper, we have presented an optimized training approach to improving dense passage retrieval. we have made three major technical contributions in rocketqa, namely cross-batch negatives, denoised hard negatives and data augmentation. extensive experiments have shown the effectiveness of the proposed approach by incorporating the three optimization strategies. we also demonstrate that the performance of end-to-end qa can be improved based on our rocketqa retriever.