Relevance-guided Supervision for O pen QA with C ol BERT (2021)

Abstract: Systems for Open-Domain Question Answering (OpenQA) generally depend on a retriever for finding candidate passages in a large corpus and a reader for extracting answers from those passages. In much recent work, the retriever is a learned component that uses coarse-grained vector representations of questions and passages. We argue that this modeling choice is insufficiently expressive for dealing with the complexity of natural language questions. To address this, we define ColBERT-QA, which adapts the scalable neural retrieval model ColBERT to OpenQA. ColBERT creates fine-grained interactions between questions and passages. We propose an efficient weak supervision strategy that iteratively uses ColBERT to create its own training data. This greatly improves OpenQA retrieval on Natural Questions, SQuAD, and TriviaQA, and the resulting system attains state-of-the-art extractive OpenQA performance on all three datasets.

## introduction the goal of @xcite is to find answers to factoid questions in potentially massive unstructured text corpora. systems for openqa typically depend on two major components: a retrieval model to find passages that are relevant to the user's question and a machine reading model to try to find an answer to the question in the retrieved passages. at its best, this should combine the power and scalability of current information retrieval (ir) models with recent advances in machine reading comprehension (mrc). however, if the notions of relevance embedded in the ir model fail to align with the requirements of question answering, the mrc model will not reliably see the best passages and the system will perform poorly. many prior approaches to openqa rely on classical ir models (e.g., bm25; @xcite whose notions of relevance are not tailored to questions. in effect, this reduces the openqa problem to few-passage mrc, imposing a hard limit on the quality of the passages seen by the mrc model. recent work has sought to address this problem by learning to retrieve passages. for instance, @xcite and @xcite jointly train vector representations of both passages and questions to support similarity-based retrieval. this has led to state-of-the-art performance on multiple openqa datasets. however, existing openqa retrievers exhibit two major limitations. first, the representations learned by these models are relatively coarse: they encode each passage into a single high-dimensional vector and estimate relevance via one dot-product. we argue this is not expressive enough for reliably matching complex natural-language questions to their answers. second, existing systems present substantial tradeoffs when it comes to supervision: they expect hand-labeled ''positive'' passages, which may not always exist; or they use simple models like bm25 to sample ''positives'' and ''negatives'' for training, which may provide weak positives and unchallenging negatives; or they conduct retrieval within the training loop, which requires frequently re-indexing a large corpus (e.g., tens or hundreds of times in realm; @xcite or a frozen document encoder that cannot adapt to the task (e.g., as in rag; @xcite . we argue that supervision methods in this space need to be more flexible and more scalable. we tackle both limitations with colbert-qa. 1 to address the first problem, we leverage the recent neural retrieval model colbert @xcite to create an end-to-end system for openqa. like other recent neural ir models, colbert encodes both the question and the document using bert @xcite . however, a defining characteristic of colbert is that it explicitly models fine-grained interactions @xcite . colbert encodes questions and passages into multiple embeddings and allows those to interact via a scalable maximumsimilarity (maxsim) mechanism. sub-figure (b) illustrates our proposed colbert-qa training strategy: we use an existing retrieval model to collect the top-k passages for every training question and, with a simple heuristic, sort these passages into positive (+ve) and negative (-ve) examples, using those to train another, more effective retriever. this process is applied thrice, and the resulting colbert-qa is used in the openqa pipeline. to address the second problem, we propose relevance-guided supervision (rgs), an efficient weak-supervision strategy that allows the retriever to guide its own training without frequent re-indexing or freezing the document encoder. instead of expensive pretraining, rgs starts from an existing weak retrieval model (e.g., bm25) to collect the top-k passages for every training question and uses a provided weak heuristic to sort these passages into positive and negative examples, relying on the ordering imposed by the retriever. these examples are then used to train a more effective retriever, and this process is applied 2-3 times, with the resulting retriever deployed in the openqa pipeline. crucially for scaling to large corpora, rgs only requires re-indexing the corpus once or twice during training and correspondingly only retrieves positives and negatives in 2-3 large batches. in doing so, rgs permits fine-tuning of the document encoder during all of training, freeing it to co-adapt with the query encoder to the task's complexities. to assess colbert-qa, we report on experiments with natural questions @xcite , squad @xcite , and triviaqa @xcite . we adopt the openqa formulation in which the passage is not given directly as gold evidence, but rather must be retrieved, including during training. further, we focus on the standard extractive openqa setup, where the reader model extracts the answer string from one of the retrieved passages. on all three datasets, colbert-qa achieves state-of-the-art retrieval and extractive openqa results. in summary, our contributions are: ## evaluating colbert-qa's retrieval colbert-qa presents many options for training the retriever so it extracts the best passages for the reader. this section explores a range of these options to address the following key questions. our first question concerns colbert's retrieval modeling capacity. can colbert's finegrained interactions improve the accuracy of openqa retrieval when tuned for this task? we consider this under a weak supervision paradigm: training based on bm25 ranking. in §4.2, we find that colbert is highly effective in this setting, considerably outperforming classical and single-vector retrieval. our second question concerns an out-of-domain version of our model. colbert is standardly trained to perform ir tasks. is this form of transfer learning from ir sufficient for effective openqa? if so, then this might be an appealingly modular option for many applications, allowing system designers to focus on the reader. in §4.3, we show that colbert succeeds in this setting. our third set of questions concerns how best to supervise colbert-qa for openqa. in particular, can relevance-guided supervision improve on standard bm25-based supervision by capitalizing on the structure and effectiveness of colbert? we find that the answer is ''yes''; colbert-qa with rgs consistently proves to be the best method ( §4.4), while only requiring 1-2 additional iterations of indexing and training. chen et al. (2017), guu et al. (2020), and karpukhin et al. (2020 ), we study the retrieval quality in openqa by reporting success@k (s@k for short), namely, the percentage of questions for which a retrieved passage (up to depth k) contains the gold answer string. this metric reflects an upper bound on the performance of an extractive reader that reads k passages, assuming it always locates the answer if present. however, readers in practice are affected by the quality and number of passages that contain the answer string and by passages that do not contain the answer. to evaluate this more directly, we suggest employing a bert-large model (with ''whole-word-masking'' pretraining) as a reader, trained for each retriever as described in §3.3. we report the exact-match (em) score corresponding to each retriever-reader combination on the validation set. hyperparmeter tuning is described in appendix b. ## conclusion we proposed colbert-qa, an end-to-end system for openqa that employs the recent colbert retrieval model to improve both retriever modeling and supervision in openqa. to this end, we developed a simple yet effective training strategy that progressively improves the quality of the openqa training data, even without hand-labeled evidence passages. our results show that colbert is highly effective for openqa retrieval; with the proposed training paradigm, colbert-qa can improve retrieval quality over existing methods by over five success@20 points and the resulting openqa pipeline attains state-of-the-art extractive openqa results across three popular datasets. @xcite . we note that the test sets are the same for all methods across both datasets.