Retrieve What You Need: A Mutual Learning Framework for Open-domain Question Answering (2024)

Abstract: An open-domain question answering (QA) system usually follows a retrieve-then-read paradigm, in which a retriever is used to retrieve relevant passages from a large corpus, and then a reader generates answers based on the retrieved passages and the original question. In this paper, we propose a simple and novel mutual learning framework to improve the performance of retrieve-then-read-style models via an intermediate module named the knowledge selector, which we train with reinforcement learning. The key benefits of our proposed intermediate module are: 1) no requirement for additional annotated question-passage pairs; 2) improvements in both retrieval and QA performance, as well as computational efficiency, compared to prior competitive retrieve-then-read models; 3) with no finetuning, improvement in the zero-shot performance of large-scale pre-trained language models, e.g., ChatGPT, by encapsulating the input with relevant knowledge without violating the input length constraint.

## introduction recently, there has been a revival of interest in tasks requiring large amounts of knowledge of the world. in such real-world scenarios, an efficient information retrieval system, capable of finding a small subset of relevant and non-redundant information, is needed for applications such as opendomain question answering, in which external knowledge (e.g., wikidata and conceptnet @xcite ) must be integrated in order to generate correct answers. even in the era of large language models like chatgpt and gpt-4, which are capable of encoding extensive knowledge into their parameters, there are still scenarios where retrieval is indispensable, such as when answer ing questions about the most current news events. however, hand-labeling data for training such a retriever is expensive and time consuming, and many datasets and applications lack such annotations. hence, an efficient framework should be capable of learning a retriever, without supervision from annotated query-passage pairs. in this paper, we focus on improving both the inference performance and efficiency of retrievethen-read frameworks. retrieve-then-read frameworks have dominated over current open-domain question answering systems @xcite @xcite as well as other knowledge-intensive tasks such as fact checking @xcite and dialogue systems @xcite . for example, core @xcite , a state-of-the-art open-domain questionanswering system, starts by using a dense retriever @xcite to retrieve a subset of support passages and tables from a large knowledge source, such as wikipedia. then, a generative encoder-decoder (reader) model produces an answer, conditioned on the question and the retrieved knowledge. previous studies @xcite have shown that using a large number of support passages will lead to a significant increase in memory requirement and training time cost. according to @xcite , fid @xcite ) requires approximately 7 × 10 12 floating-point operations (flops) for inference on 100 passages. this high inference cost limits the widespread adoption of such systems in real-world applications, which must trade-off performance for decreased latency. in addition to this, empirical results from previous work @xcite @xcite have suggested that, beyond a threshold number of passages, supplying the reader with additional passages yields only a minimal improvement or even decline in the overall accuracy of the end-to-end qa systems. these two points motivate us to explore whether it is possible to reduce the number of required support passages without compromising the model's performance. to this end, we conducted two preliminary experiments: preliminary experiment 1: given a sample from the tqa @xcite dataset in which each question is accompanied with 100 passages retrieved by dpr @xcite , we achieved an exact match (em) score of 65.0% using a fusion-in-decoder model (t5-base). we then calculated the average em scores when using 10 passages under a range of selection strategies. firstly, by randomly sampling 10 out of the 100 passages retrieved by dpr, the em scored decreases from 65.0% to 53.3%. selecting the top 10 passages ranked by dpr outperformed this random sampling, however the em score still degraded to 59.6%. finally, to further substantiate our hypothesis that retrieving more relevant passages can significantly enhance the reader's generation performance, we employed contriever. 1 contriever is an advanced unsupervised dense information retrieval system trained on extensive corpora. utilizing contriever to select a set of 10 passages, we observed an em score of 65.4%, foot_1 a slight improvement against the original 100 passages. to a certain extent, it shows that the importance of retrieved results lies more in their quality rather than their quantity. ## our method to improve both the inference efficiency and prediction accuracy we propose a simple and novel mutual learning framework for training an open-domain question answering system. our framework inserts a knowledge selector module between the retriever and the reader. crucially, this module requires no additional annotated data and is compatible with any retrieve-then-read models. specifically, given a question q i , the retriever first selects a fixed number of passages d i from a large knowledge source. then, the knowledge selector prunes d i to obtain a smaller subset of passages p i , where p i ⊆ d i . finally, p i is processed by the reader, along with the question, to generate an answer. for the retriever, we use dpr @xcite , which has been demonstrated to perform better than sparserepresentation-based methods, such as bm25 @xcite , in many prior works @xcite . for the reader module, we use the fusion-in-decoder (fid) model @xcite , a sequence-tosequence architecture which we initialize from a pre-trained model such as t5 @xcite or bart @xcite . information retrieval has been studied for many years and there exists an abundance of offthe-shelf retrieval models. after reviewing previous work in open-domain question answering, we find three main classes of retriever: 1) sparse retrievers (e.g., bm25), where both passages and queries are represented as sparse vectors, with each dimension corresponding to a different term; 2) unsupervised dense retrievers (e.g., contriever @xcite ), which are trained without using annotated query-passage pairs; and 3) supervised dense retrievers (e.g., dpr), which represent a cluster of supervised dense retrieval model directly trained on annotated datasets. since it is not the main focus of our work, we directly adopt dpr as our retriever, a state-of-the-art retrieval model. in the following two sections, we outline the training details of the two remaining modules: knowledge selector ( §2.1) and reader ( §2.2). ## zero-shot transfer previous experimental results showed that our mutual learning framework could improve the model performance in the supervised fine-tuning setting. here, we evaluate whether the trained knowledge selector module can also contribute to improving the generation performance of large-scale language models (llms) (e.g., gpt-3 and chatgpt) in a zero-shot setting. in particular, we explore three different settings: 1) without methods nq tqa webqs the prompt we used is from p3 @xcite of the form refer to the passage below and answer the following question. passage: {passages} question: {question}, where {question} and {pas-sages} are replaced by the corresponding question and the retrieved passages. from original question fid-with-dpr's prediction our prediction q: who got the first nobel prize in physics? albert a. michelson ✗ wilhelm conrad röntgen ✓ top-3 passages ranked by dpr: 1. albert a. michelson was an american physicist known for his work on measuring the speed of light . . . in 1907 he received the nobel prize in physics, becoming the first american to win the nobel prize . . . 2. nobel prize in physics the nobel prize in physics is a yearly award given by the royal swedish academy . . . 3. the discovery of x-rays by physicist wilhelm conrad röntgen, first winner of the nobel prize for physics . . . top-3 passages ranked by our method: 1. wilhelm conrad röntgen was a german mechanical engineer and physicist, who, on 8 november 1895 . . . range known as x-rays rays . . . an achievement that earned him the first nobel prize in physics in 1901 . . . 2. the discovery of x-rays by physicist wilhelm conrad röntgen, first winner of the nobel prize for physics . . . 3. . . . when german physics professor wilhelm conrad röntgen discovered the x-ray and noted that, while it could pass through human tissue . . . he received the first nobel prize in physics for his discovery . . . q: who is the president of usa right now? george w. bush ✗ barack obama ✗ top-3 passages ranked by dpr: 1. . . . on january 20, 2009, when barack obama was inaugurated as the 44th president of the united states . . . 2. . . . donald trump was formally elected by the electoral college on december 19, 2016 . . . 3. . . . january 20, 2001, when george w. bush was inaugurated as the 43rd president of the united states . . . top-3 passages ranked by our method: 1. . . . on january 20, 2009, when barack obama was inaugurated as the 44th president of the united states . . . 2. barack obama, a democrat and former u.s. senator from illinois, was first elected president . . . 3. barack obama is an american attorney and politician who served as the 44th president of u.s. . . . ## case study to better understand why our proposed framework can help improve the predictive performance, we manually pick two representative examples as case studies. examples where predicted results of our framework and a strong baseline (fid-with-dpr) together with part of their used passages are in in the first case, we can observe that among the three top passages ranked by dpr, only one is relevant to the question and can provide evidence to generate the correct answer while the other two passages are either off-topic or even providing some incorrect information. for example, the top-1 retrieved passage conveys a seemingly relevant information about the first american winner of the nobel prize for physics, which is considered as a negative factor of leading the reader to generate an incorrect prediction with respect the given question without emphasizing the winner's nationality. in contrast, in terms of the relevance to the given question, we can notice that all the three passages from our method are talking about wilhelm conrad röntgen, based on which the reader correctly gives the answer as we expect. we conjecture that the reader might be negatively distracted by irrelevant knowledge, thus making an incorrect predictions with respect to the given question. in the second case, while the comparison between the two predictions with the ground truth answer (donald trump) is incorrect, the prediction itself should be considered as a correct answer for the question due to the time-dependent property of the question. according to @xcite , the natural questions dataset contains a query: who is the girl in green day 21 guns? ground truth answer: lisa stelly chatgpt [no passage]: lauren german ✗ with top-1 passage by dpr: 21 guns is a song by american punk rock band green day. it was released as the second single from their eighth album . . . ## related work open-domain question answering (odqa) is an important task, aiming at providing precise answers in response to the user's questions in natural language. there are two common types of knowledge sources: one is unstructured textual documents available on the internet, and another is a predefined structured data such as knowledge graphs which are often manually constructed. in this paper, we focus on the former, which is considered to be a more general and challenging task since available unstructured text to obtain answers are fairly common and easily accessible, such as wikipedia, news articles and science books, etc. next, we review two categories of approaches widely explored in current textual based odqa literature. we refer the reader to @xcite for a more exhaustive introduction to this topic. ## conclusion in this work, we explore how to improve the prediction performance and inference cost of reader models in current open-domain question-answer architectures. to this end, we introduce a finegrained knowledge selector into the retrieve-thenread paradigm, whose goal is to construct a small subset of passages which retain question-relevant information. the knowledge selector is trained as a component of our novel mutual learning framework, which iteratively trains the knowledge selector and the reader. we adopt a simple and novel approach employing policy gradients to optimize the knowledge selector, using feedback from the reader to train it to select a small and informative set of passages. this approach avoids brute-force search or manually designed heuristics, without requiring any annotated query-document pairs for supervision. we show that iteratively training the reader and the knowledge selector leads to better predictive performance on some public open-domain question answering benchmarks. finally, our approach matches the accuracy of the top-performing fusion-in-decoder reader, whilst utilizing just 18.32% of its reader inference cost (flops).