AGR a ME : Any-Granularity Ranking with Multi-Vector Embeddings (2024)

Abstract: Ranking is a fundamental problem in search, however, existing ranking algorithms usually restrict the granularity of ranking to full passages or require a specific dense index for each desired level of granularity. Such lack of flexibility in granularity negatively affects many applications that can benefit from more granular ranking, such as sentence-level ranking for open-domain QA, or proposition-level ranking for attribution. In this work, we introduce the idea of any-granularity ranking which leverages multi-vector embeddings to rank at varying levels of granularity while maintaining encoding at a single (coarser) level of granularity. We propose a multi-granular contrastive loss for training multi-vector approaches and validate its utility with both sentences and propositions as ranking units. Finally, we demonstrate the application of proposition-level ranking to post-hoc citation addition in retrieval-augmented generation, surpassing the performance of prompt-driven citation generation.

## introduction dense retrieval methods employ dual-encoder models to obtain vector representations for queries and passages. usually, single-vector methods @xcite produce one embedding per query and passage, utilizing dot product to determine relevance scores. conversely, multi-vector methods @xcite capture more fine-grained interactions when computing query-passage relevance score, resulting in better ranking accuracy. an advantage of multi-vector approaches is the use of token-level embeddings paired with a maxsim operation @xcite which allows for a granular scoring mechanism through dot @xmath0, with entire query encoded, and y indicates the granularity of the retrieval unit being ranked, with entire retrieval unit encoded. in addition to the typical ranking setting (a), our proposed approach enables ranking finer retrieval units (b and d) or using finer query units for ranking @xcite . we make an important observation that tokenlevel embeddings in multi-vector approaches enable discriminative scoring of sub-components within a retrieval unit. we argue that finergranularity scoring used by multi-vector approaches cannot be extended to single-vector approaches since the whole passage is represented by a single embedding, thereby not allowing for subunit scoring. such granular scoring enables multigranularity ranking, which is beneficial for many applications. in open-domain question answering @xcite , ranking sentences within passages can more precisely locate answers. similarly, in attribution @xcite , atomic facts within sentences can be used as queries to retrieve evidence supporting factual claims. to achieve this, we introduce agrame (any-granularity ranking with multi-vector embeddings), a method that permits ranking at different levels of granularity while maintaining encoding at a single, coarser level. our approach enables i) ranking at a finer level than the encoding (or retrieval) unit, and ii) ranking using fragments of the query, as demonstrated in figure 1 . we hypothesize that encoding at a coarser level-such as the entire retrieval unit or query-can provide additional context for the sub-retrieval units being ranked or sub-parts of the query used for ranking. in contrast, achieving such granularity with singlevector approaches requires the use of specialized encoders, such as a sub-sentence encoder @xcite , or necessitates a separate encoding at the desired ranking granularity @xcite . firstly, how well do multi-vector approaches perform when used for ranking at a finer granularity? we investigate this by conducting an experiment ( §2) using colbertv2 @xcite . we observe that the performance of sentence ranking is notably inferior when the encoding is at the passage-level. to improve the model's ability to rank at finer granularity, we propose a multigranular contrastive loss during training (outlined in §3.3). our experiments in §4.1 confirm a significant boost in sentence-level ranking while maintaining passage-level performance. while agrame is generally applicable to arbitrary granularity, we explore the effectiveness for proposition-level ranking, crucial for applications requiring fine-grained attribution @xcite . our results in §4.2 indicate that incorporating a sentence-level contrastive loss further improves proposition-level ranking. additionally, we propose propcite, which utilizes propositions from generated text as queries to rank input context passages and select relevant citations. in §4.3, propcite shows superior performance over traditional methods that prompt models to include citations in rag. the main contributions are as follows: (1) we introduce agrame, that leverages multi-vector embeddings for ranking at various granularities while using the same encoding-level. (2) we introduce a multi-granular contrastive loss for training multi-vector approaches, which we show improves sentence-level ranking even when encoding at passage-level. (3) we demonstrate superior proposition-level ranking using agrame, surpassing existing state-of-the-art methods. (4) we leverage proposition-level ranking to formulate a post-hoc citation addition approach for retrieval- model encoding level ranking level sentence passage p@1 r@5 p@1 r@5 contriever (single vec.) sentence 19.3 45.6 32.4 62.8 passage --37.8 65.1 colbertv2 (multi vec.) sentence 31.6 56.3 40.2 66.8 passage 27.4 48.8 43.4 69.1 ## motivating experiment here, we investigate the effectiveness of col-bertv2 @xcite ), a multi-vector approach, in ranking at a finer granularity than the encoding level. specifically, when encoding is at the passage-level, we measure the sentence-level and passage-level ranking performance.a maxsim operation is applied between query token vectors and token vectors corresponding to the sentence to get a sentence-level score, which is then added to the passage-level score to get the final querysentence relevance score for ranking. when encoding is at the sentence-level, the usual maxsim score gives query-sentence relevance. on the other hand, the query-passage relevance score for ranking using sentence-level encoding is obtained as the maximum of the corresponding passage's querysentence relevance scores. we also include contriever @xcite ), a single-vector approach, for comparison. when encoding is at the passage-level, contriever does not support sentence-level ranking, which is an inherent limitation of single-vector approaches. our evaluation uses the natural questions dev set @xcite and a 22m passage corpus @xcite ) from wikipedia 2018 for retrieval. to keep the retrieval index size manageable, contriever is used to index and retrieve 100 passages, which are then ranked by col-bertv2. when ranking at sentence-level, only the sentences in these top 100 passages are considered. eval metrics are precision@1 and recall@5, based on string exact match @xcite . ## experiments agrame can rank at different granularities, as shown in figure 1 , which involves ranking subparts of the retrieval unit or ranking using sub-parts of the query. in our experiments, we aim to investigate two research questions: rq1: can the training approach proposed in §3.3 improve ranking at a finer granularity than the level of encoding, i.e. @xmath1.1, we show the improvements at sentence-level ranking from our proposed multi-granular contrastive loss, while maintaining performance at passage-level, i.e. @xmath2; rq2: can multi-vector embeddings be used to rank with sub-parts of the query? in §4.2, we demonstrate the application of multi-vector approaches in sub-@xmath3. here, a given proposition within a sentence is used as the query to rank and identify relevant propositions in a corpus of sentences. further, in §4.3, we introduce propcite, a post-hoc citation addition approach based on sub-@xmath4. propcite scores input context passages based on propositions in the generated text to add citations in retrieval-augmented generation. in §2, we saw that with a multi-vector approach, sentence ranking performance drops when changing the encoding from sentence-level to passagelevel. we addressed this in two ways: a) agrame introduces a new query marker (in §3.2) for sentence-level scoring; (b) our multi-granular contrastive loss (in §3.3) providing sentence-level relevance supervision during training. here, we empirically demonstrate the benefits of our proposed approach for sentence-level (sub-retrieval unit) ranking performance when encoding is at passage-level. ## related work the phrase 'multi-granularity' can have different meanings depending on the domain of usage. in the field of image retrieval, it corresponds to representing different regions of the image separately @xcite . for representation learning @xcite , it refers to encoding information at different output embedding dimensions, to adapt to the computational constraints of downstream tasks. our definition of granularity in text ranking corresponds to the level of the ranked sub-units with a given retrieval unit. multi-vector approaches @xcite @xcite have primarily been used for ranking at the same granularity as the encoding level, which is typically at passage-level. single vector approaches, on the other hand, inherently do not support ranking at a finer granularity than the encoding level, thereby needing a separate dense index for each granularity @xcite . hence, specialized models for single-vector embeddings have been introduced for embedding phrases @xcite , propositions @xcite , sentences @xcite or passages @xcite . our approach overcomes this limitation by leveraging multi-vector approaches for ranking at different granularities, while still encoding at a single coarser granularity. prior approaches that rank at different granularities have used custom scoring functions or incorporate separate embeddings. @xcite proposes a multi-granularity matching model that uses a convolutional filter for scoring, instead of cosine similarity, meaning it cannot be scaled to a retrieval-scale corpus due to the matching function. hierarchical ranking approaches @xcite @xcite consider multi-granular ranking but require using a separate embeddings for each ranking granularity. further, our approach uses multi-vector embeddings with a dot product for scoring at all levels of granularity, meaning the same pre-computed dense index can be used for ranking at any granularity. ## conclusion in this work, we introduce agrame, which leverages multi-vector embeddings to rank at finer granularities, while maintaining encoding at a single, coarser level. our proposed multi-granular contrastive loss for training multi-vector approaches improves sentence ranking performance even with encoding at passage-level. we demonstrate that agrame can rank at any granularity, even by using sub-parts of the query for ranking. leveraging multi-vector approaches' superior performance at proposition-level ranking, our post-hoc attribution approach, propcite, outperforms the conventional approach of prompt-driven citation in retrieval-augmented generation.