Phrase Retrieval Learns Passage Retrieval, Too (2021)

Abstract: Dense retrieval methods have shown great promise over sparse retrieval methods in a range of NLP problems. Among them, dense phrase retrieval—the most fine-grained retrieval unit—is appealing because phrases can be directly used as the output for question answering and slot filling tasks. In this work, we follow the intuition that retrieving phrases naturally entails retrieving larger text blocks and study whether phrase retrieval can serve as the basis for coarse-level retrieval including passages and documents. We first observe that a dense phrase-retrieval system, without any retraining, already achieves better passage retrieval accuracy (+3-5% in top-5 accuracy) compared to passage retrievers, which also helps achieve superior end-to-end QA performance with fewer passages. Then, we provide an interpretation for why phrase-level supervision helps learn better fine-grained entailment compared to passage-level supervision, and also show that phrase retrieval can be improved to achieve competitive performance in document-retrieval tasks such as entity linking and knowledge-grounded dialogue. Finally, we demonstrate how phrase filtering and vector quantization can reduce the size of our index by 4-10x, making dense phrase retrieval a practical and versatile solution in multi-granularity retrieval.

## introduction dense retrieval aims to retrieve relevant contexts from a large corpus, by learning dense representations of queries and text segments. recently, dense retrieval of passages @xcite @xcite has been shown figure 1 : comparison of passage representations from dpr @xcite and densephrases @xcite . unlike using a single vector for each passage, densephrases represents each passage with multiple phrase vectors and the score of a passage can be computed by the maximum score of phrases within it. to outperform traditional sparse retrieval methods such as tf-idf and bm25 in a range of knowledgeintensive nlp tasks @xcite , including open-domain question answering (qa) @xcite , entity linking @xcite , and knowledge-grounded dialogue @xcite . one natural design choice of these dense retrieval methods is the retrieval unit. for instance, the dense passage retriever (dpr) @xcite encodes a fixed-size text block of 100 words as the basic retrieval unit. on the other extreme, recent work @xcite demonstrates that phrases can be used as a retrieval unit. in particular, @xcite show that learning dense representations of phrases alone can achieve competitive performance in a number of open-domain qa and slot filling tasks. this is particularly appealing since the phrases can directly serve as the output, without relying on an additional reader model to process text passages. in this work, we draw on an intuitive motivation that every single phrase is embedded within a larger text context and ask the following question: if a retriever is able to locate phrases, can we directly make use of it for passage and even document retrieval as well? we formulate phrasebased passage retrieval, in which the score of a passage is determined by the maximum score of phrases within it (see figure 1 for an illustration). by evaluating densephrases @xcite on popular qa datasets, we observe that it achieves competitive or even better passage retrieval accuracy compared to dpr, without any re-training or modification to the original model ( to better understand the nature of dense retrieval methods, we carefully analyze the training objectives of phrase and passage retrieval methods. while the in-batch negative losses in both models encourage them to retrieve topically relevant passages, we find that phrase-level supervision in densephrases provides a stronger training signal than using hard negatives from bm25, and helps densephrases retrieve correct phrases, and hence passages. following this positive finding, we further explore whether phrase retrieval can be extended to retrieval of coarser granularities, or other nlp tasks. through fine-tuning of the query encoder with document-level supervision, we are able to obtain competitive performance on entity linking @xcite and knowledge-grounded dialogue retrieval @xcite in the kilt benchmark @xcite . finally, we draw connections to multi-vector passage encoding models @xcite , where phrase retrieval models can be viewed as learning a dynamic set of vectors for each passage. we show that a simple phrase filtering strategy learned from qa datasets gives us a control over the trade-off between the number of vectors per passage and the retrieval accuracy. since phrase retrievers encode a larger number of vectors, we also propose a quantization-aware finetuning method based on optimized product quantization @xcite , reducing the size of the phrase index from 307gb to 69gb (or under 30gb with more aggressive phrase filtering) for full english wikipedia, without any performance degradation. this matches the index size of passage retrievers and makes dense phrase retrieval a practical and versatile solution for multi-granularity retrieval. while traditional passage retrieval models rely on sparse representations such as bm25 @xcite , recent methods show promising results with dense representations of passages and questions, and enable retrieving passages that may have low lexical overlap with questions. specifically, @xcite introduce dpr that has a passage encoder e p (•) and a question encoder e q (•) trained on qa datasets and retrieves passages by using the inner product as a similarity function between a passage and a question: for open-domain qa where a system is required to provide an exact answer string a, the retrieved top k passages p k are subsequently fed into a reading comprehension model such as a bert model @xcite , and this is called the retrieverreader approach @xcite . phrase retrieval while passage retrievers require another reader model to find an answer, @xcite introduce the phrase retrieval approach that encodes phrases in each document and performs similarity search over all phrase vectors to directly locate the answer. following previous work @xcite , we use the term 'phrase' to denote any contiguous text segment up to l words (including single words), which is not necessarily a linguistic phrase and we take phrases up to length @xmath0. given a phrase s (p) from a passage p, their similarity function f is computed as: (3 ). we report top-k passage retrieval accuracy (top-k), mean reciprocal rank at k (mrr@k), and precision at k (p@k). ## phrase retrieval for passage retrieval phrases naturally have their source texts from which they are extracted. based on this fact, we define a simple phrase-based passage retrieval strategy, where we retrieve passages based on the phrase-retrieval score: ## a unified view of dense retrieval as shown in the previous section, phrase-based passage retrieval is able to achieve competitive passage retrieval accuracy, despite that the models were not explicitly trained for that. in this section, we compare the training objectives of dpr and densephrases in detail and explain how densephrases learns passage retrieval. ## improving coarse-grained retrieval while we showed that densephrases implicitly learns passage retrieval, figure 3 indicates that densephrases might not be very good for retrieval tasks where topic matters more than fine-grained entailment, for instance, the retrieval of a single evidence document for entity linking. in this section, we propose a simple method that can adapt densephrases to larger retrieval units, especially when the topical relevance is more important. method we modify the query-side fine-tuning proposed by @xcite , which drastically improves the performance of densephrases by reducing the discrepancy between training and inference time. since it is prohibitive to update the large number of phrase representations after indexing, only the query encoder is fine-tuned over the entire set of phrases in wikipedia. given a question q and an annotated document set d * , we minimize: ## densephrases as a multi-vector passage encoder in this section, we demonstrate that densephrases can be interpreted as a multi-vector passage encoder, which has recently been shown to be very effective for passage retrieval @xcite . since this type of multi-vector encoding models requires a large disk footprint, we show that we can control the number of vectors per passage (and hence the index size) through filtering. we also introduce quantization techniques to build more efficient phrase retrieval models without a significant performance drop. ## related work text retrieval has a long history in information retrieval, either for serving relevant information to users directly or for feeding them to computationally expensive downstream systems. while traditional research has focused on designing heuristics, such as sparse vector models like tf-idf and bm25, it has recently become an active area of interest for machine learning researchers. this was precipitated by the emergence of open-domain qa as a standard problem setting @xcite and the spread of the retriever-reader paradigm @xcite . the interest has spread to include a more diverse set of downstream tasks, such as fact checking @xcite , entity-linking @xcite or dialogue generation @xcite , where the problems require access to large corpora or knowledge sources. recently, realm @xcite and rag (retrieval-augmented generation) (lewis et al., 2020) have been proposed as generalpurpose pre-trained models with explicit access to world knowledge through the retriever. there has also been a line of work to integrate text retrieval with structured knowledge graphs @xcite @xcite . we refer to @xcite for a comprehensive overview of neural text retrieval methods. ## conclusion in this paper, we show that phrase retrieval models also learn passage retrieval without any modification. by drawing connections between the objectives of dpr and densephrases, we provide a better understanding of how phrase retrieval learns passage retrieval, which is also supported by several empirical evaluations on multiple benchmarks. specifically, phrase-based passage retrieval has better retrieval quality on top k passages when k is small, and this translates to an efficient use of passages for open-domain qa. we also show that densephrases can be fine-tuned for more coarsegrained retrieval units, serving as a basis for any retrieval unit. we plan to further evaluate phrasebased passage retrieval on standard information retrieval tasks such as ms marco.