End-to-End Training of Neural Retrievers for Open-Domain Question Answering (2021)

Abstract: Recent work on training neural retrievers for open-domain question answering (OpenQA) has employed both supervised and unsupervised approaches. However, it remains unclear how unsupervised and supervised methods can be used most effectively for neural retrievers. In this work, we systematically study retriever pre-training. We first propose an approach of unsupervised pre-training with the Inverse Cloze Task and masked salient spans, followed by supervised finetuning using question-context pairs. This approach leads to absolute gains of 2+ points over the previous best result in the top-20 retrieval accuracy on Natural Questions and TriviaQA datasets. We next explore two approaches for end-to-end training of the reader and retriever components in OpenQA models, which differ in the manner the reader ingests the retrieved documents. Our experiments demonstrate the effectiveness of these approaches as we obtain state-of-the-art results. On the Natural Questions dataset, we obtain a top-20 retrieval accuracy of 84%, an improvement of 5 points over the recent DPR model. We also achieve good results on answer extraction, outperforming recent models like REALM and RAG by 3+ points.

## introduction the task of open-domain question answering (openqa) consists of finding answers to the information-seeking questions using a large knowledge source such as wikipedia. this knowledge source is also referred to as evidence and it typically contains millions of documents. most approaches for openqa consist of a two-stage pipeline @xcite . in the first stage, given document 1: bowling is a target sport and recreational activity in which a player rolls a ball towards pins (in pin bowling) or another target (in target bowling) … document 2: "hall of fame" is a song by irish pop rock band the script. it is the lead single from their studio album #3. the track features american hip-hop artist will.i.am of the black eyed peas. a question, a retriever module identifies the most relevant documents, which is often a very small subset of the evidence known as context documents. traditionally, approaches based on document ranking such as bm25 @xcite have been used for the retriever. in the second stage, these relevant documents are given as input to the reader module, which understands them and extracts the answer for the question (figure 1 ). the main drawback of the bm25 method is that it is not trainable and hence it can't be adapted to tasks involving open-retrieval. recent work has addressed this limitation by building upon advances in self-supervised learning, such as bert @xcite . these approaches model both the retriever and reader using neural networks, allowing the retriever to be trained using task-specific datasets @xcite . typically, the retriever model consists of a dual-encoder architecture @xcite , where one encoder processes the question and the other encoder processes the context document. prior work has investigated both unsupervised and supervised approaches to train the retriever. unsupervised approaches include separately training the retriever with inverse cloze task (ict) @xcite or training the retriever and reader jointly by pre-dicting masked salient spans (realm) @xcite , while supervised approaches such as dense passage retrieval (dpr) @xcite train the retriever using human-annotated sets of question and context pairs. however, there is no study that investigates the comparative advantages of using these two styles of training when the retrieval task is challenging, i.e., when the evidence contains millions of documents. it is unclear if the unsupervised approaches can further help to improve the performance of strong supervised approaches, and, if so, under what conditions. a core focus of this work is systematically studying these aspects of retriever training. we propose a unified approach to train the retriever: unsupervised pre-training followed by supervised finetuning. we also investigate key design choices-such as relevance score scaling and longer training-and showcase their effectiveness. our results demonstrate that the proposed approach obtains substantial accuracy gains when evaluated on benchmark openqa datasets. extensive experiments also highlight the relative importance of different pre-training strategies, revealing important trade-offs when varying the amount of supervised data available to train the retriever. furthermore, motivated by recent work @xcite , we also explore two approaches for end-to-end supervised training of the reader and retriever components. in the first approach, the reader considers each retrieved document separately while in the second approach, the reader takes as input all the retrieved documents together. we compare the effectiveness of these approaches on both retrieval accuracy and answer extraction. we show that the first approach leads to an improved retrieval performance, while the second approach results in an improved answer extraction. with end-to-end training, we outperform previous best models to obtain new state-of-the-art results on retrieval accuracy and answer extraction. we also perform experiments by scaling the model size to a large configuration for both retriever and reader and observe consistent improvements, compared with smaller models. in summary, the contributions of this work are: the rest of the paper is organized as follows. sec. 2 and 3 explain the retriever model and endto-end training, respectively. sec. 4-6 describe the experimental details with the results. sec. 7 reviews the related work followed by conclusion in sec. 8. ## neural retriever in this section, we first describe the retriever architecture and then discuss different approaches to train it, including our proposed approach. ## end-to-end retriever and reader training in this section, we explore two supervised training approaches to end-to-end train the reader and retriever components from the task-specific data. in the first approach, the reader considers each retrieved document separately (sec. 3.1) while in the second approach, the reader takes as input all retrieved documents together (sec. 3.2). these approaches are designed such that when predicting the answer conditioned on the question, the learning process improves both the reader and retriever. background and notation: in end-to-end training, the trainable components consists of the retriever (φ) and reader (θ) parameters. for retriever, we use the dual-encoder architecture and train it as discussed previously in sec. 2.3. our reader is a generative model designed according to the sequence-to-sequence modeling paradigm @xcite . specifically, we use pre-trained t5 as the reader. the inputs to the training process are questions (q) and its answers (a), both in string form. given a question, first the retriever obtains the k relevant context documents (k) from the evidence (z) as the reader then takes the question and one or more context documents (z i ) as input to predict the answer, the likelihood of which is defined as ## experimental setup in this section, we describe the datasets and model settings. for reproducibility, we provide training details and list the hyperparameters in appendix a. ## results: retriever training in this section, we compare different approaches to train the retriever. retrieval accuracy is evaluated using the top-k metric (@xmath0, 5, 20, 100}). ## results: answer extraction we next present the results of end-to-end training on answer extraction. to train the model, retriever weights are initialized with ict pre-training and supervised finetuning while the reader is initialized with pre-trained t5 weights. the number of retrieved evidence documents for the reader is tuned on the dev set. results are reported using the conventional exact match (em) metric. ## conclusion we propose approaches to improve the retrieval accuracy of the dual-encoder model for the openqa task. we first perform a systematic investigation of the importance of pre-training with ict and masked salient spans tasks for supervised training of the retriever. we then present two approaches for end-to-end training of the reader and retriever components in openqa. in one approach, the reader considers each retrieved document individually while in the other approach where the reader con-siders all the retrieved documents jointly. overall, these methods help achieve state-of-the-art results on both retrieval and answer extraction.