Augmenting Pre-trained Language Models with QA -Memory for Open-Domain Question Answering (2023)

Abstract: Existing state-of-the-art methods for open-domain question-answering (ODQA) use an open book approach in which information is first retrieved from a large text corpus or knowledge base (KB) and then reasoned over to produce an answer. A recent alternative is to retrieve from a collection of previously-generated question-answer pairs; this has several practical advantages including being more memory and compute-efficient. Question-answer pairs are also appealing in that they can be viewed as an intermediate between text and KB triples: like KB triples, they often concisely express a single relationship, but like text, have much higher coverage than traditional KBs. In this work, we describe a new QA system that augments a text-to-text model with a large memory of question-answer pairs, and a new pre-training task for the latent step of question retrieval. The pre-training task substantially simplifies training and greatly improves performance on smaller QA benchmarks. Unlike prior systems of this sort, our QA system can also answer multi-hop questions that do not explicitly appear in the collection of stored question-answer pairs.

## introduction open-domain question answering (odqa) is a well-studied knowledge-intensive task. state-ofthe-art methods require retrieving relevant knowledge from a large corpus or datastore before reasoning over this retrieved evidence. most existing methods retrieve documents @xcite @xcite or structured kb triples @xcite . recently, a few works have proposed retrieving from a collection of question-answer (qa) pairs-an approach made feasible by advances in scalable automatic question generation. in this setting, a new question is answered by retrieving paraphrases from a question index, and returning the associated answer @xcite . notably, the repaq system from @xcite won the 2020 efficientqa competition @xcite , outperforming closed-book qa (cbqa) models by a significant margin and matching the prior sota performance on nq @xcite . a collection of qa pairs is appealing for several reasons. as opposed to text passages and much like a kb triple, qa pairs are often concise, tending to express a single relationship. however, unlike kb triples, qa collections have good coverage of actually asked questions like those in standard open qa datasets. repaq demonstrated several advantageous properties such as memory and computational efficiency, strong selective qa performance (i.e. selectively abstaining from answering), and effective ensembling with text-retrieval qa systems. however, question-retrieval qa systems have several limitations as well. first, there is no largescale supervised data for question-question retrieval. this contrasts with the step of retrieving text given a question, where supervised data is used to build retrievers like dpr @xcite . to address this, repaq uses a latentretrieval training process (similar to realm @xcite ), in which the retriever is trained using the downstream end loss from the qa task. this requires asynchronously updating the index as training proceeds, a process that is complex and computationally expensive. this is also a problem for domains with limited qa data: as we will show, repaq's performance is disappointing on smaller datasets like webquestions @xcite , containing only 3k training instances. to address this problem, we introduce a novel pre-training task for question retrieval, which can be applied to any text-qa dataset, which great improves performance on smaller datasets. a second problem is that repaq is limited to answering questions explicitly stored in the index, or paraphrases of such questions. this contrasts with qa systems that retrieve from kbs, which can typically generate complex queries that combine the atomic triples in the kb.to address this, we present an extended model that answers multi-hop questions by iteratively retrieving from a questionanswer corpus, the first question-retrieval-based qa system that addresses this task. in more detail, we propose a new qa-memory-augmented transformer (qamat) with better compositionality paired with a lower complexity training strategy. qamat is based on a t5 encoder-decoder @xcite paired with an integrated key-value memory @xcite populated with question-answer pairs (see figure 1 ). given an input, the encoder generates a query representation scored against the qa memory and retrieves the top-k relevant qa pairs. the encoder then reprocesses the input along with the retrievals forming a qa-injected representation which is passed to the decoder to attend to and generate. to reduce the training (fine-tuning) sample complexity, we propose to first pre-train qamat on a large-scale corpus to teach the model to retrieve and interpret qa pairs. we construct the pre-training corpus by leveraging existing methods for question generation, producing a very large set of potentially interesting questions from text passages @xcite @xcite . for each qa pair and the passage it was generated from, we mask the answer and train the model to fill the mask by retrieving and using an appropriate qa pair. we show that pre-training greatly boosts the model's performance and helps the model generalize to different domains. for example, the pre-trained model can achieve a zeroshot performance of 40% em on nq and triviaqa without any fine-tuning. the effectiveness of this pre-training task means that we can avoid the expensive latent training procedure used by repaq, and instead use an efficient two-stage training pipeline. in the first stage, we use a small local in-batch memory of qa pairs to optimize the qa pair encoder. we then freeze the encoder and construct the index for the global memory. in the second stage, we retrieve from this fixed global memory and continue to optimize the remaining parameters-including the parameters used to construct queries to the global memoryfor better performance. lastly, we extend qamat to build qamat+, which iteratively retrieves from the memory to generate outputs. we demonstrate that qamat+ effectively chains multiple qa-pairs together to answer multi-hop questions in hotpotqa @xcite and musique @xcite . such compositional reasoning capability is nonexistent in repaq @xcite . in summary, we develop a new qa augmented architecture which extends the lines of research considering qa pairs as a representation of knowledge as well as those on memory-augmented language models. when paired with our proposed pretraining strategy (section 4), we address many of the shortcomings of previous qa-indexing-based approaches leading to lower sample complexity training and the ability to perform compositional reasoning (subsection 3.5). ## conclusion in this paper, we propose a more accurate and efficient architecture to utilize qa-pairs as representation units of knowledge. our proposed model qamat outperforms repaq significantly, while leveraging our less expensive training procedure. furthermore, we show how a qa-backed model can perform compositional reasoning and address more complex queries. in the future, we hope to further close the gap with state-of-the-art documentbased retrieve-and-read models and extend this approach to a broader set of tasks.