R ocket QA v2: A Joint Training Method for Dense Passage Retrieval and Passage Re-ranking (2021)

Abstract: In various natural language processing tasks, passage retrieval and passage re-ranking are two key procedures in finding and ranking relevant information. Since both the two procedures contribute to the final performance, it is important to jointly optimize them in order to achieve mutual improvement. In this paper, we propose a novel joint training approach for dense passage retrieval and passage reranking. A major contribution is that we introduce the dynamic listwise distillation, where we design a unified listwise training approach for both the retriever and the re-ranker. During the dynamic distillation, the retriever and the re-ranker can be adaptively improved according to each other’s relevance information. We also propose a hybrid data augmentation strategy to construct diverse training instances for listwise training approach. Extensive experiments show the effectiveness of our approach on both MSMARCO and Natural Questions datasets. Our code is available at https://github.com/PaddlePaddle/RocketQA .

## introduction recently, dense passage retrieval has become an important approach in the task of passage retrieval @xcite to identify relevant contents from a large corpus. the underlying idea is to represent both queries and passages as low-dimensional vectors (a.k.a., embeddings), so that the relevance can be measured via embedding similarity. additionally, a subsequent procedure of passage re-ranking is widely adopted to further improve the retrieval results by incorporating a reranker @xcite . such a two-stage procedure is particularly useful in a variety of natural language processing tasks, including question answering @xcite , dialogue system @xcite and entity linking @xcite . following a retrieve-then-rerank way, the dense retriever in passage retrieval and the re-ranker in passage re-ranking jointly contribute to the final performance. despite the fact that the two modules work as a pipeline during the inference stage, it has been found useful to train them in a correlated manner. for example, the retriever with a dual-encoder can be improved by distilling from the re-ranker with a more capable cross-encoder architecture @xcite , and the re-ranker can be improved with training instances generated from the retriever @xcite @xcite . therefore, there is increasing attention on correlating the training of the retriever and re-ranker in order to achieve mutual improvement @xcite @xcite . typically, these attempts train the two modules in an alternative way: fixing one module and then optimizing another module. it will be more ideal to mutually improve the two modules in a joint training approach. however, the two modules are usually optimized in different ways, so that the joint learning cannot be trivially implemented. specially, the retriever is usually trained by sampling a number of in-batch negatives to maximize the probabilities of positive passages and minimize the probabilities of the sampled negatives @xcite , where the model is learned by considering the entire list of positive and negatives (called listwise approach foot_0 ). as a comparison, the re-ranker is usually learned in a pointwise or pairwise manner @xcite , where the model is learned based on a single passage or a pair of passages. to address this issue, our idea is to unify the learning approach for both retriever and re-ranker. specially, we adopt the listwise training approach for both retriever and re-ranker, where the relevance scores are computed according to a list of positive and negative passages. besides, it is expected to include diverse and high-quality training instances for the listwise training approach, which can better represent the distribution of all the passages in the whole collection. thus, it requires more effective data augmentation to construct the training instances for listwise training. to this end, we present a novel joint training approach for dense passage retrieval and passage re-ranking (called rocketqav2). the major contribution of our approach is the novel dynamic listwise distillation mechanism for jointly training the retriever and the re-ranker. based on a unified listwise training approach, we can readily transfer relevance information between the two modules. unlike previous distillation methods that usually froze one module, our approach enables the two modules to adaptively learn relevance information from each other, which is the key to mutual improvement in joint training. furthermore, we design a hybrid data augmentation strategy to generate diverse training instances for listwise training approach. the contributions of this paper can be summarized as follows: ## related work recently, dense passage retrieval has demonstrated better performance than traditional sparse retrieval methods (e.g., tf-idf and bm25) on the task of passage retrieval. existing approaches of learning dense passage retriever can be di-vided into two categories: (1) self-supervised pretraining for retrieval @xcite @xcite and (2) fine-tuning pre-trained language models (plms) on labeled data @xcite @xcite @xcite . our work follows the second class of approaches, which show better performance with less cost. there are two important tricks to train an effective dense retriever: (1) incorporating hard negatives during training @xcite @xcite and (2) distilling the knowledge from cross-encoder-based reranker into dual-encoder-based retriever (izacard and grave, 2020; yang and seo, 2020; @xcite . based on the retrieved passages from a retriever, plm-based rerankers with the cross-encoder architecture have recently been applied on passage re-ranking to improve the retrieval results @xcite @xcite , and yield substantial improvements over the traditional methods. apart from separately considering the above two tasks, it has been proved that passage retrieval and passage re-ranking are actually highly related and dependent @xcite @xcite . the retriever needs to capture the relevance knowledge from the re-ranker, and the re-ranker should be specially optimized according to the preceding results of the retriever. some efforts studied the possibility of leveraging the dependency of retriever and re-ranker, and try to enhance the connection between them in an alternative way @xcite @xcite . furthermore, several studies attempted to jointly train the retriever and the reader for open-domain question answering @xcite @xcite . different from the prior studies, our method is a joint learning architecture of the dense passage retriever and the re-ranker. ## methodology in this section, we describe a novel joint training approach for dense passage retrieval and passage re-ranking (called rocketqav2) ## experiments in this section, we first describe the experimental settings, then report the main experimental results, ablation study, and detailed analysis. ## conclusion this paper has presented a novel joint training approach for dense passage retrieval and passage re-ranking. to implement the joint training, we have made two important technical contributions, namely dynamic listwise distillation and hybrid data augmentation. such an approach is able to enhance the mutual improvement between the retriever and the re-ranker, which can also simplify the training process. extensive results have demonstrated the effectiveness of our approach. to our knowledge, it is the first time that the retriever and re-ranker are jointly trained in a unified architecture, which provides the possibility of training the entire retrieval architecture in an endto-end way.