Dense X Retrieval: What Retrieval Granularity Should We Use? (2024)

Abstract: Dense retrieval has become a prominent method to obtain relevant context or world knowledge in open-domain NLP tasks. When we use a learned dense retriever on a retrieval corpus at inference time, an often-overlooked design choice is the retrieval unit in which the corpus is indexed, e.g. document, passage, or sentence. We discover that the retrieval unit choice significantly impacts the performance of both retrieval and downstream tasks. Distinct from the typical approach of using passages or sentences, we introduce a novel retrieval unit, proposition, for dense retrieval. Propositions are defined as atomic expressions within text, each encapsulating a distinct factoid and presented in a concise, self-contained natural language format. We conduct an empirical comparison of different retrieval granularity. Our experiments reveal that indexing a corpus by fine-grained units such as propositions significantly outperforms passage-level units in retrieval tasks. Moreover, constructing prompts with fine-grained retrieved units for retrieval-augmented language models improves the performance of downstream QA tasks given a specific computation budget.

## introduction dense retrievers are a popular class of techniques for accessing external information sources for opendomain nlp tasks @xcite . before we use a learned dense retriever to retrieve from a corpus, an imperative design decision we have to make is the retrieval unit -i.e. the granularity at which we segment and index the retrieval corpus for inference. in practice, the choice of retrieval units, e.g. documents, fixed-length passage chunks or sentences, etc, is usually pre-determined based on how the dense retrieval model is instantiated question: what is the angle of the tower of pisa? ## proposition as a retrieval unit the goal of our study is to understand how the granularity of a retrieval corpus influences the dense retrieval models' performance empirically. aside from commonly-used retrieval units such as 100word passages @xcite or sentences, we propose using proposition as an alternative retrieval unit choice. here, propositions represent atomic expressions of meanings in text @xcite with three defining principles below. the use of proposition as a retrieval unit is inspired by a recent line of work @xcite @xcite , which finds success in representing and evaluating text semantics at the level of propositions. we demonstrate the concept of proposition and how a passage can be split into a set of propositions by an example on the left side of figure 2 . the passage contains three propositions, each of which corresponds to a distinct factoid about the leaning tower of pisa: the angle before the restoration, the current angle, and the horizontal displacement. within each proposition, necessary context from the passage is incorporated so that the meaning of the proposition can be interpreted independently of the original text, e.g. the reference of the tower is resolved into its full mention, the leaning tower of pisa, in the first proposition. we expect each proposition to describe exactly one atomic fact, and so our intuition is that propositions would suitably work as a retrieval unit for information-seeking questions. ## factoidwiki: proposition-level index and retrieval for wikipedia we empirically compare passages, sentences, and propositions as retrieval units on wikipedia, a commonly-used retrieval source for knowledgeintensive nlp tasks @xcite . to allow a fair comparison across granularities, we process an english wikipedia dump from 2021-10-13, as used by @xcite . we segment each document text into three different granularities: passages, sentences, and propositions. we include the details on passage-and sentence-level segmentation of the corpus in appendix a. ## experimental settings to evaluate the impact of the three retrieval unit choices, we conduct experiments on five different open-domain qa datasets with factoidwiki. with each dataset, we evaluate both passage retrieval and downstream qa performance when dense retrievers work with wikipedia indexed in different granularities. ## how does granularity influence passage retrieval? in this section, we report and discuss how indexing the corpus at various granularity influences the passage retrieval performance. surprisingly, despite all of the dense retrieval models being trained on only passage-level documents, all the models demonstrate on-par or superior performance when the corpus is indexed at the proposition level. our results suggest that indexing the corpus at the finergrained units improves the cross-task generalization on passage retrieval. ## how does granularity influence retrieval-augmented lms? in this section, we study how the choice of different granularity used in the prompts affects the retrievalaugmented generation across open-domain qa tasks. to fairly compare different granularity with the same computation budget, we limit the number of retrieved tokens for input to the language model at @xmath0. our results suggest that retrieval by finer-grained units enables a higher density of question-related information in the prompts, leading to better performance. ## related work recent works on dense retrievers typically adopt a dual-encoder architecture @xcite @xcite . with dual-encoders, each query and document is encoded into a lowdimensional feature vector respectively, and their relevance is measured by a non-parametric similarity function between the embedding vectors @xcite . due to the limited expressivity from the similarity function, dual encoder models often generalize poorly to new tasks with scarce training data @xcite . previous studies use techniques such as data augmentation @xcite @xcite @xcite , continual pre-training @xcite @xcite , taskaware training @xcite , hybrid sparse-dense retrieval @xcite , or mixed strategy retrieval @xcite and so on to improve cross-task generalization performance of dense retrievers. the motivation of our work echoes in part with multi-vector retrieval, e.g. colbert @xcite , densephrase @xcite , me-bert @xcite , and mvr (zhang in line with generating retrieval units from the original corpus, @xcite propose using generative summaries as additional retrieval units alongside the original text, enhancing queries with document-level understanding. in contrast, our work generates propositions to improve queries related to long-tailed entities. these approaches are complementary, as they address different aspects of retrieval enhancement. the use of propositions as a unit of text representation dates back to the pyramid method in summarization evaluation @xcite ), where a model-generated summary is evaluated by each proposition. proposition extraction from text has been a long-standing task, with earlier formulations focusing on a structured representation of propositions @xcite . more recent studies have found success in extracting free-text propositions via few-shot prompting with llms @xcite , or fine-tuning compact-sized models @xcite . retrieve-then-read, or more broadly retrieval augmented generation, has recently emerged as a popular paradigm for open-domain question answering @xcite @xcite . while earlier works provide up to the top 100 retrieved passages for the downstream reader @xcite , the amount of context allowed is significantly reduced when using recent large language models @xcite , due to the limited context window length and inability to reason over long context @xcite . recent efforts try to improve the quality of the reader context by filtering or compressing the retrieved documents @xcite . our work offers a new perspective by changing the retrieval granularity, in order to achiev greater information density with a fixed context length. ## conclusion this paper studies how the choice of granularity for indexing a corpus, as well as the granularity used in the prompts, influences retrieval and downstream qa performance. our results show that retrieval by propositions outperforms passage-level and sentence-level retrieval on passage retrieval and downstream qa across five open-domain qa datasets. our analysis shows that indexing a corpus with finer-grained units enhances the cross-task generalization of dense retrievers and increases the density of question-related information in the prompts. we hope that factoidwiki and our findings will facilitate future research on information retrieval and retrieval-augmented generation.