1- PAGER : One Pass Answer Generation and Evidence Retrieval (2023)

Abstract: We present 1-Pager the first system that answers a question and retrieves evidence using a single Transformer-based model and decoding process. 1-Pager incrementally partitions the retrieval corpus using constrained decoding to select a document and answer string, and we show that this is competitive with comparable retrieve-and-read alternatives according to both retrieval and answer accuracy metrics. 1-Pager also outperforms the equivalent ‘closed-book’ question answering model, by grounding predictions in an evidence corpus. While 1-Pager is not yet on-par with more expensive systems that read many more documents before generating an answer, we argue that it provides an important step toward attributed generation by folding retrieval into the sequence-to-sequence paradigm that is currently dominant in NLP. We also show that the search paths used to partition the corpus are easy to read and understand, paving a way forward for interpretable neural retrieval.

## introduction in recent times, there has been a push to reformulate a wide variety of tasks from nlp and other domains into the sequence-to-sequence paradigm, to make use of large pre-trained transformer networks @xcite . however, despite evidence that large language models can often answer questions @xcite , predict identifiers of documents that support those answers @xcite , or generate text that contains and explains an answer @xcite the dominant paradigm in question answering is still the retrieveand-read approach that pipelines separate retrieval and answer generation modules. this approach has the benefit that it can provide direct and targeted paragraph-level attribution for the generated answers @xcite . however, it also relies on a heterogenous mix of models that are hard to train in concert @xcite . motivated by the observation that language model decoders already perform search over possible sequences @xcite , and that evidence documents themselves are simply sequences of tokens, we present an alternative approach that relies on a single transformer model. in this approach, which we name 1-pager (one pass answer generation and evidence retrieval) or simply 1p, the decoder iteratively partitions a corpus of evidence documents by generating a search path consisting of a set of keywords that identify relevant documents and an answer string that is contained in at least one of these documents. with 1p, we aim to explore the spectrum between cbqa, where the answer is generated without reference to an evidence corpus, and pipelined approaches that feed retrieved documents into the transformer. figure 1 illustrates an example in which the corpus is iteratively partitioned into documents that contain the string 'economy of india', then those that also contain the string 'agriculture', and finally those that also contain the answer string '23%'. while 1p does not yet perform as well as the very best retrieval or open-domain question answering systems in terms of accuracy, the fact that it is competitive with pipelined systems that are trained with the same data and which use similar amounts of inference-time compute suggests a promising path ahead. unlike those systems, 1p can be trained end-to-end along with any other task that fits into the sequence-to-sequence paradigm. additionally, 1p search paths are inherently interpretable, unlike embeddings used in dense retrieval. a particular variant of interest is the iterative retrieval process where the query is reformulated incrementally @xcite leading to an interactive search process @xcite . this query augmentation scheme has similarities with our use of search paths. however, we use the paths to iteratively partition the corpus while prior works have used it for refining the query. to perform well, retrieve-and-read systems will typically retrieve 10s to 100s of passages that must be processed by a language model. in constrast, 1p retrieves and extracts an answer in a single pass of language model generation. closed book question answering with data and parameter scale, llms in a closed-book setting (cbqa) have shown competitive performance @xcite @xcite to retrieval pipelines (odqa), however without producing any attributed passages @xcite . an extension of cbqa is post-hoc retrieval where a large language model llm) is first used to generate an answer and then evidence for the question-answer pair is fetched by a retriever @xcite . while post-hoc retrieval serves the same goal as 1p, it still uses a pipeline of llm and retriever to do so. generative retrieval recently, generative retrieval has emerged as an alternative to the conventional "retrieve-and-read" pipeline @xcite . genre @xcite performed generative entity linking by constraining model's decoding to a set of entities. dsi @xcite showed one of the first proof of llm's ability to memorize docids in the corpus. however, atomic ids or hierarchical clusters, as used in dsi, are opaque identifiers and capture limited information. works such as seal @xcite and ultron @xcite use a semantically richer representation: keywords in the document. in particular, seal constrains the generation to only keywords in the corpus using the fm-index @xcite , a key data structure we borrow in this work. 1p represents docids as keyword paths, which are arguably more interpretable, and learns a soft partition over the corpus instead of the hard partition imposed by dsi's clustering. another crucial distinction is 1p's ability to both retrieve and generate an answer while prior works rely on a external re-ranker/reader for the same. a high-level view of various question-answering systems is presented in figure 2 . attributed question answering standard metrics for open-domain question answering, such as exact match or token-based f1, have received criticism for being imprecise and/or insufficient. several efforts have proposed augmenting answers with textual evidence, via retrieval or citations @xcite @xcite . while this work does not directly evaluate the quality of retrieved answer evidence, our proposed model inherently produces a passage to support the final answer, along with a search path of keywords, which could be used to provide users with answer evidence. ## iterative corpus partitioning and answer prediction we focus on the problem of learning a mapping f (q, d) → (a, d a ) from a question q and corpus of documents d to an answer and supporting document (a, d a ). the predicted document d a is retrieved from d and the answer a is a sub-string of d a . the document d a should be relevant to the question and provide evidence for answer. the goal of this paper is to model the function f using a single sequence-to-sequence model, rather than a pipeline which first retrieves d a and then feeds it into an answer generation module. to achieve our goal, we recast retrieval as an iterative corpus partitioning process illustrated in figure 3 . iterative corpus partitioning adopts the lm decoder's autoregressive search process to partition d by predicting n-gram keywords. an n-gram of tokens k is said to be contained in a document d, denoted by k ≺ d, when k is a sub-sequence of d. we define a keyword corpus partitioning function answer prediction is treated in exactly the same way as keyword selection and in 1p the last keyword from p is taken as the answer. ## constrained decoding and fm-index to avoid generating empty partitions, we constrain 1-pager to only decode search paths that match at least one document. we modify the decoder's beam-search strategy to only allow keyword continuations that are contained in the current partition. given a document subset d p i , which could be the full corpus d at the start of decoding (@xmath0, and a keyword prefix k, which could be empty, the set of all valid continuation tokens is defined as, to efficiently implement these constraints, we need a data-structure that can quickly determine both c(k, d p ), the continuation tokens given a document set and p(d p , k), the subset of documents that contain a given path. for this, we extend the usage of an fmindex @xcite as described in @xcite . the fm-index is a compressed token-based index over a corpus d 0 with a few important properties for our usage: (1) it can efficiently list possible token continuations for a sequence prefix that occur in d 0 i.e., c(k, d 0 ), (2) it can list the set of documents in the corpus that match an n-gram i.e., f(d 0 , k), and (3) it supports search over arbitrary n-grams that occur within documents. note that the fm-index operations are optimized for d 0 , the original corpus it is built over. we extend these to an arbitrary d p ⊂ d 0 at additional cost described in appendix a.1. ## training data generation for training 1p, we produce a dataset with examples of queries and search paths as described above. at a high-level, we generate search paths by iteratively selecting n-grams from an answer passage, and simulating, using the fm-index of the retrieval corpus, the partitioning of the corpus after selecting each keyword, until only a few documents remain. finally, the answer span a is appended to the search path. each example produced can be serialized as sequence-to-sequence pair of inputs and targets as: inputs: generate keywords for: <@xmath1: k_sep k0 k_sep k1 ... k_sep a_sep a eos ## discussion and ablations generating answers while 1p is capable of generating answers, comparison to seal while 1p takes inspiration from seal, in practice, there are a few key differences between the two systems aside from 1p's answer generation. seal generates a large set of keywords (