Multi-stage Training with Improved Negative Contrast for Neural Passage Retrieval (2021)

Abstract: In the context of neural passage retrieval, we study three promising techniques: synthetic data generation, negative sampling, and fusion. We systematically investigate how these techniques contribute to the performance of the retrieval system and how they complement each other. We propose a multi-stage framework comprising of pre-training with synthetic data, fine-tuning with labeled data, and negative sampling at both stages. We study six negative sampling strategies and apply them to the fine-tuning stage and, as a noteworthy novelty, to the synthetic data that we use for pre-training. Also, we explore fusion methods that combine negatives from different strategies. We evaluate our system using two passage retrieval tasks for open-domain QA and using MS MARCO. Our experiments show that augmenting the negative contrast in both stages is effective to improve passage retrieval accuracy and, importantly, they also show that synthetic data generation and negative sampling have additive benefits. Moreover, using the fusion of different kinds allows us to reach performance that establishes a new state-of-the-art level in two of the tasks we evaluated.

## introduction recently, there is a surge of interest in neural firststage retrieval models @xcite . these models overcome the lexical gap issue of traditional models based on term matching @xcite by projecting both query and document to a shared dense space. finding relevant documents can then be achieved by employing nearest neighbor search. neural firststage retrieval models have shown competitive on many benchmark data sets @xcite @xcite , and combining them with term matching-based models further boosts their retrieval performance @xcite . ## related work previous attempts at improving the quality of dual encoder models can be classified into three types. the first type focuses on finding a good initialization for the model parameters. this is typically achieved by pre-training the model on various tasks @xcite . @xcite showed that leveraging synthetic question generation is an effective way to improve model accuracy and outperform other variants in zero-shot settings. while the approach was originally proposed for a low-resource scenario, we show that synthetic question pre-training still significantly improves retrieval performance in cases where sufficient amounts of supervised data is available. the second type focuses on learning better rep-resentations using hard negatives. this strategy has proven effective in passage retrieval @xcite , machine translation @xcite and entity linking @xcite tasks. these works mine hard negatives using different strategies. for example, @xcite mine "coarse" negatives with a low-resolution model. gillick et al. (2019) use a model trained with in-batch negatives and select examples ranked above the correct one as negative examples. karpukhin et al. (2020)'s dense passage retrieval model (dpr) mines hard negatives using a bm25 model. @xcite and @xcite proposed to sample negatives from the model itself. while @xcite analyzed the drawbacks of in-batch negative sampling from the point of view of convergence rate, @xcite argued that contrastive loss is a biased estimator and drawing negative samples from the model itself leads to bias reduction. moreover, @xcite showed that popular choices of "noisy" distributions such as uniform distribution generally cannot reduce the bias. in this work, we draw a connection between noise constrastive estimation (nce) and in-batch crossentropy loss and show that the limited sampling space of in-batch negatives reduces the estimation problem to a much simpler surrogate. furthermore, we empirically show that combining random sampling with in-batch negatives achieves results competitive with using approximate nearest neighbor negatives, which is typically implemented with asynchronous updates. ## neural passage retrieval models following previous works @xcite , our dual encoder model is also based on bert @xcite . the architecture is shown in fig 2 . to encode a question, we feed the question text to the bert model and apply a fully-connected (fc) layer of size 768 to the [cls] token embedding. the output of the fc layer is used as the question embedding. a passage is encoded in a similar way but we prepend to the passage the title of the document where it is found: [cls] title [sep] passage @xcite . the final question and passage embeddings are then l 2 -normalized. the query to passage relevance is computed by the dot-product of their vectors. the model parameters are initialized from the public uncased bert checkpoint, and are trained using a listwise loss function @xcite , i.e., cross-entropy loss with in-batch negatives. let b denote a batch of question-passage pairs {(x i , y i )} |b| , we train the model by minimizing the following loss: ## improved negative sampling in this section, we first draw a connection between the loss function in the previous section and nce @xcite to shed light on the drawback of in-batch negative sampling. then we introduce several negative sampling strategies to mitigate the issue. ## hard negatives in multi-stage training for pre-training and fine-tuning, we use hard negatives in addition to the in-batch negatives. assuming that there are m hard negatives for each question in the training data, at each training epoch we randomly select n out of m hard negatives. those n hard negatives are appended to the in-batch negatives as in the standard dual encoder training foot_2 . note that the hard negatives for one question are treated as in-batch negatives for the other questions in the batch. therefore, for a batch of size b, each question is compared during training against (n + 1) × b passages instead of just b passages in the standard way to train a dual encoder. ## experimental setup we evaluate our proposed approach on two tasks: firstly, we evaluate on the passage retrieval task for open-domain question answering (qa) with the goal of retrieving passages that contain the correct answer spans given a question. secondly, to understand how our approach performs on large-scale text retrieval datasets, we also evaluate on the ms marco passage ranking task. ## conclusions we presented a multi-stage system for neural passage retrieval based on models that combine the use of synthetic data, negative sampling and fusion. we trained bert-based dual encoder models using a two-stage system and demonstrated the positive impact of negative sampling in both the pre-training stage, that uses synthetic data, and the fine-tuning stage, that uses supervised data. results of our pre-training on synthetic data with hard negatives showed the additive benefits of using both methods in combination. we tested our models on passage retrieval tasks and verified that hard negatives in fine-tuning led to considerable gains over previous dense and sparse retrieval models, including on tasks where fine-tuning alone had not shown much improvement. we achieved even greater gains with early-and late-stage fusion. overall, the combined contributions of synthetic data for pre-training, different negative sampling strategies and late fusion allowed us to achieve state-of-the-art retrieval performance on natural questions and squad and highly competitive results on ms marco. our results encourage us to keep exploring this area and investigate similar mechanisms to improve the reranking stage for neural information retrieval and the reading comprehension stage in end-to-end question answering systems.