Generative Dense Retrieval: Memory Can Be a Burden (2024)

Abstract: Generative Retrieval (GR), autoregressively decoding relevant document identifiers given a query, has been shown to perform well under the setting of small-scale corpora. By memorizing the document corpus with model parameters, GR implicitly achieves deep interaction between query and document. However, such a memorizing mechanism faces three drawbacks: (1) Poor memory accuracy for fine-grained features of documents; (2) Memory confusion gets worse as the corpus size increases; (3) Huge memory update costs for new documents. To alleviate these problems, we propose the Generative Dense Retrieval (GDR) paradigm. Specifically, GDR first uses the limited memory volume to achieve inter-cluster matching from query to relevant document clusters. Memorizing-free matching mechanism from Dense Retrieval (DR) is then introduced to conduct fine-grained intra-cluster matching from clusters to relevant documents. The coarse-to-fine process maximizes the advantages of GR’s deep interaction and DR’s scalability. Besides, we design a cluster identifier constructing strategy to facilitate corpus memory and a cluster-adaptive negative sampling strategy to enhance the intra-cluster mapping ability. Empirical results show that GDR obtains an average of 3.0 R@100 improvement on NQ dataset under multiple settings and has better scalability.

## introduction text retrieval @xcite is an essential stage for search engines @xcite , question-answering systems @xcite and dialog systems @xcite . traditional retrieval methods include sparse retrieval (sr) and dense retrieval (dr). sr @xcite relies on the assumption that queries and relevant documents have a high degree of word overlap. however, such methods suffer from the zero-recall phenomenon when there is a lexical mismatch between queries and documents. dr @xcite alleviates this issue by training dual-encoders for semantic matching instead of lexical matching, which brings a high hit rate. nevertheless, most queries are semantically related to multiple documents that may not be close to each other in semantic space. thus it is challenging to use a single query representation to recall all the relevant documents with matching mechanism @xcite . recently, generative retrieval (gr) @xcite , which utilizes a language model to memorize document features and autoregressively decodes the identifiers of relevant documents given a query, is considered a promising paradigm. the model is served as a memory bank for candidate documents, and the memorizing process implicitly implements the deep interaction between queries and documents by attention mechanism, which has been proven to be effective in the small-scale corpus settings @xcite . also, beam search, a diversitypromoting decoding strategy, is beneficial for the model to find relevant documents from multiple directions and thus can recall more relevant documents than dr @xcite . however, after empirically comparing the performance of typical gr model nci @xcite and dr model ar2 @xcite , we found that the memorizing mechanism brings three problems: (1) poor memory accuracy for fine-grained features of documents. we calculated the error rate of each position when decoding document identifiers (see as shown in the results indicate that the burden of memorizing more documents causes 5.7 r@100 drop. (3) huge memory update costs for new documents. when new documents come, the document cluster tree needs to be updated, and the model needs to be re-trained to re-memorize all the documents. otherwise, the outdated mapping relationship, i.e., query to document identifiers and document identifiers to documents, will significantly degrade the retrieval performance (see based on the above analysis, a natural idea is to employ memorizing-free matching mechanism from dr to alleviate the burden faced by the memorizing mechanism. however, it is challenging to realize complementary advantages of both mechanisms while ensuring retrieval efficiency. to this end, we propose a coarse-to-fine retrieval paradigm generative dense retrieval (gdr). concretely, memorizing mechanism and matching mechanism are successively applied to achieve coarse-grained inter-cluster (@xmath0. a shared query encoder is used to generate query representations that apply both mechanisms, thereby improving retrieval efficiency. we also explore the strategy of constructing a memory-friendly document cluster tree, including distinguishable document clusters and controllable cluster amounts, so as to further alleviate memory burden. moreover, a cluster-adaptive negative sampling strategy is proposed to enhance the intracluster matching ability of gdr. our contributions are summarized as follows: ## related work given queries, text retrieval task aims to find relevant documents from a large corpus. in this section, we introduce typical paradigms dr and gr that are most related to our work. ## methodology our task is to retrieve a candidate document set d c from a large corpus d l (|d l | >> |d c |) for a given query q, with the objective of including as many documents d from d q as possible, where d q is the set of documents relevant to q. in this section, we introduce the proposed generative dense retrieval (gdr) paradigm (see figure 1 ). to realize complementary advantages of memorizing mechanism and matching mechanism, we need to consider the following issues: ## experiments we empirically demonstrate the performance of gdr and effectiveness of various proposed strategies on text retrieval task in this section. 4 in the following, we will discuss the detailed experimental setups in 4.1, present empirical results in 4.2, verify the effectiveness of proposed modules in 4.3, and conduct specific analysis in 4.4, respectively. ## conclusions in this paper, we empirically demonstrate that the memorizing mechanism of generative retrieval (gr) brings deep interaction characteristics but also causes serious problems. to this end, we propose the generative dense retrieval (gdr) paradigm, which subdivides the text retrieval task into inter-cluster and intra-cluster matching and achieves them by autoregressively generating cluster identifiers and calculating dense representation similarities respectively. gdr focuses the limited memory volume on the deep interaction between query and document cluster and conducts multidirections decoding, thus maintaining the superiority of memorizing mechanism. memorizingfree matching mechanism is further introduced to achieve intra-cluster mapping by fully leveraging fine-grained features of documents. such a coarseto-fine process can also bring better scalability, i.e., stable corpus expansion and low-cost document updates. we further propose a cluster identifier constructing strategy to release the memory burden and a cluster-adaptive negative sampling strategy to provide discriminative signals. comprehensive experiments on the nq dataset demonstrate the state-of-the-art r@k performance and better scala-