Query-focused Referentiability Learning for Zero-shot Retrieval (2025)

Abstract: Dense passage retrieval enhances Information Retrieval (IR) by encoding queries and passages into representation space. However, passage representations often fail to be referenced by their gold queries under domain shifts, revealing a weakness in representation space. One desirable concept for representations is ”argmaxable”. Being argmaxable ensures that no representations are theoretically excluded from selection due to geometric constraints. To be argmaxable, a notable approach is to increase isotropy, where representations are evenly spread out in all directions. These findings, while desirable also for IR, focus on passage representation and not on query, making it challenging to directly apply their findings to IR. In contrast, we introduce a novel query-focused concept of ”referentiable” tailored for IR tasks, which ensures that passage representations are referenced by their gold queries. Building on this, we propose Learning Referentiable Representation (LRR), and two strategic metrics, Self-P and Self-Q, quantifying how the representations are referentiable. Our experiments compare three dense model versions: Naive, Isotropic, and Referentiable, demonstrating that LRR leads to enhanced zero-shot performance, surpassing existing naive and isotropic versions.

## introduction there have been notable advancements in the field of information retrieval (ir) through the use of dense representations. however, well-designed dense models such as dpr @xcite , condenser @xcite , and cocondenser @xcite have shown limitations in zero-shot scenarios on beir benchmark @xcite . this limitation suggests that passage representations often fail to align with their corresponding gold queries, highlight- * corresponding author ing a weakness in representation space. this paper explores strategies to overcome these limitations. to ensure the representation of language models meets the minimum criteria, one important condition is "argmaxable". it indicates that the representations should lie in a space where they can be selected through an argmax operation @xcite . for example, in a two-dimensional representation set @xmath0, 1), (1, -1), (-1, 0), (0.5, 0)}, the fourth representation (0.5, 0) is an interior point relative to the first three. as a result, any point satisfying {(x, y)|@xmath1, 2y < x + 1, 2y > -x -1}, including the fourth representation, always yields lower values when multiplied by any vector, as it is overshadowed by the first three representations. while the concept of argmaxable focuses on the theoretical selection of representation by any vector, our interest lies in whether passage representation can practically be selected by query representation in ir. with this argument, we introduce an extended query-focused concept of "referentiable," which determines whether passage representation can practically be referenced as top-1 by its gold query representation. to ensure representations are argmaxable, previous works @xcite @xcite @xcite increased isotropy. this approach mitigates unargmaxable representations by evenly spreading them in all directions, preventing them from being within the interior points of the convex hull. this indiscriminate spreading of representations, without considering queries, is effective for word or sentence level tasks such as semantic textual similarity. however, it is challenging to apply this approach in ir, since thoughtless dispersion may inadvertently render other passages non-referentiable. if all gold queries for each passage were known in advance, it would be straightforward to make all passages referentiable by appropriately spreading passage representations towards their corresponding gold queries. however, this is impractical in real-world scenarios. to address this limitation, we propose learning referentiable representation (lrr) leveraging two strategic metrics, self-p and self-q. these metrics address two types of gold queries: (1) narrow intent queries, finding specific information @xcite , and (2) broad intent queries focused on diverse information @xcite . first, self-p: narrow intent queries contain substantial information overlapping with their relevant passages. thus, self-p uses the passage itself as a proxy for narrow intent query, and quantifies whether each passage is referenced by its own representation, namely passage locality. optimizing self-p encourages passage locality, ensuring the passage can be referenced as top-1 when using itself as gold query. second, self-q: to address broad intent queries, we predict diverse intents from the given passage using query generator. self-q uses their distribution as broad intent gold queries, and quantifies whether the passage can be referenced as top-1 by predicted intent query. optimizing self-q expands passage representations specifically toward the predicted intent query distribution, in contrast to isotropy, which uniformly expands representations in all directions. therefore, lrr optimizes both self-p and self-q together, aiming to 1) ensure referentiable passages through passage locality and 2) achieve referentiable passages through space-effective expansion. to demonstrate the effectiveness of our approach for dense passage retrieval in zero-shot setting, we compare three versions of dense models: naive, isotropic, and referentiable. we then show that our approach generalizes from well-established models like dpr, condenser, and cocondenser to more recent models such as e5 @xcite and bge @xcite . in ## preliminary in this section, we review existing approaches in ir, and the concepts of argmaxable and isotropy. ## proposed methods in section 3.1, we define referentiability (r), and reinterpret the concept of argmaxable and increasing isotropy. we then propose two strategic metrics, self-p and self-q, to approximate referentiability in section 3.2. with these metrics, we introduce learning referentiable representation (lrr) in section 3.3. ## conclusion we tackle the challenge of improving dense representations in zero-shot ir setting. while prior studies have focused solely on increasing isotropy to ensure being argmaxable, we introduce a novel query-focused concept, referentiable. building on this, we proposed lrr to learn referentiable representations. our extensive analysis demonstrates effectiveness of lrr in improving the representation space for ir tasks.