Title: Hierarchical Retrieval: The Geometry and a Pretrain-Finetune Recipe
Venue: NeurIPS 2025
Accepted: yes
Keywords: Information Retrieval

Hierarchical Retrieval

Dual encoders
Abstract: Dual encoder (DE) models, where a pair of matching query and document are embedded into similar vector representations, are widely used in information retrieval due to their simplicity and scalability. However, the Euclidean geometry of the embedding space limits the expressive power of DEs, which may compromise their quality. This paper investigates such limitations in the context of hierarchical retrieval (HR), 
where the document set has a hierarchical structure and the matching documents for a query are all of its ancestors. We first prove that DEs are feasible for HR as long as the embedding dimension is linear in the depth of the hierarchy and logarithmic in the number of documents. 
Then we study the problem of learning such embeddings in a standard retrieval setup where DEs are trained on samples of matching query and document pairs. Our experiments reveal a lost-in-the-long-distance phenomenon, where retrieval accuracy degrades for documents further away in the hierarchy. To address this, we introduce a pretrain-finetune recipe that significantly improves long-distance retrieval without sacrificing performance on closer documents. We experiment on a realistic hierarchy from WordNet for retrieving documents at various levels of abstraction, and show that pretrain-finetune boosts the recall on long-distance pairs from 19% to 76%. Finally, we demonstrate that our method improves retrieval of relevant products on a shopping queries dataset.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper considers hierarchical retrieval where query and documents are organized in a DAG. The paper proves that hierarchical retrieval can be learned with dual encoder through a constructive algorithm, then identified a lost-in-the-long-distance problem under a standard retrieval learning procedure. After that a pretrain-finetuning method is proposed to fix the problem. Two datasets are considered and goo results are shown. However, there are also some concerns on the proposed approach.\n\nSTRENGTHS AND WEAKNESSES:\nStrong points: \n1. The paper is well organized and informative. While the proposed pretrain-finetuning procedure is simple, the process of developing it is informative and I learned something new from it. \n2. The paper proves that dual encoders are feasible to solve hierarchical retrieval as long as the embedding dimension is large enough. \n3. A simple pretrain-finetuning is proposed to fix the lost-in-the-long-distance problem and good empirical results are shown.\n\nWeak points:\n1. It is claimed that the algorithm does not require DAG as input, however, the main algorithm in Section 5.2 requires the distance on the DAG as input. In real-world applications, without the DAG, how can one get the distances?\n2. It is unclear why pretrain-finetuning would fix the lost-in-the-long-distance problem? Why it is better than mixed training? Is there any intuition on this? \n3. There are prerequisites for pretrain-finetuning to work better than mixed training, but it is not discussed at all in the paper. For example, one simple case where there are only a few data point for exact match, so that the model is under-trained when pre-training only on the exact match data, I would imaging pretrain-finetuning to be worse than mixed training. It would be better to discuss when pretrain-finetuning works better.\n\nQUESTIONS:\n1. How to apply the method without distances on the DAG as input?\n2. Any intuition why the proposed pretraining-finetuning works better?\n3. discuss the requirements for pretraining-finetuning\n\nLIMITATIONS:\nlimitation discussion is not complete, for example, one apparent limitation is that the proposed pretraining-finetuning method may not work well if the number of exact match data point is limited.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies hierarchical retrieval (HR), where documents are organized in a hierarchical structure and the goal is to retrieve all ancestors of a query in the hierarchy. The authors first prove that dual encoders are theoretically feasible for HR when the embedding dimension is linear in hierarchy depth and logarithmic in document count. They then identify a \"lost-in-the-long-distance\" phenomenon where retrieval accuracy degrades for documents further away in the hierarchy. To address this, they propose a pretrain-finetune recipe that first pretrains on regular data then finetunes on long-distance pairs. Experiments on demonstrate that this approach significantly improves long-distance retrieval performance.\n\nSTRENGTHS AND WEAKNESSES:\nQuality:\n\nStrengths: Rigorous theoretical analysis with constructive proof establishing feasibility bounds; systematic experimental validation from synthetic to real data; comprehensive evaluation across multiple metrics and datasets\n\nWeaknesses: No statistical significance/cost analysis\n\nClarity:\n\nStrengths: Well-motivated problem with clear real-world applications; excellent progression from theory to synthetic experiments to real applications; clear mathematical exposition with intuitive explanations\n\nSignificance:\n\nStrengths: Highlights a practical failure mode for DEs and offers a simple fix requiring no model changes.\n\nWeaknesses: Limited scope of evaluation (only hierarchical retrieval tasks); unclear generalization to other asymmetric retrieval problems;  unclear impact for very large corpora or LLM-based encoders\n\nOriginality:\n\nStrengths: First formal margin bound linking hierarchy depth to DE dimension\n\nWeaknesses: Pretrain/finetune idea is incremental relative to curriculum or hard-negative mining\n\nQUESTIONS:\n1. Generalization: How does the pretrain-finetune recipe perform on hierarchies with different structural properties (e.g., varying branching factors, irregular depth)? What are the key hierarchy characteristics that determine success?\n\n2. Computational analysis: What is the computational overhead of the two-stage training compared to standard approaches? How does the method scale with hierarchy size and depth?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper studies a new model of \"hierarchical retrieval\" and dual encoder design and training for it. In normal retrieval, the dual encoder is trained with positive example {(q,d+)} to predict relevant docs for a query. This paper extends this to the case where queries form a somewhat hierarchical structure, or rather a DAG. If there is an edge from q1->q2 in the DAG, docs relevant for q1 must also be relevant to q2. The motivation seems reasonable in that we often want to select, and refine relevance matches.\n\nThe paper first studies an entirely toy dataset and simple tree-structured DAG, and studies simple random guassian and learn embeddings. The paper projects and matches the dimensionality needed to get 95% recall. The paper also studies the depth and imbalance of DAG in synthetic settings, and offers ideas on how to adjust the training. The paper then studies wordnet dataset and relations between word categories and trains a model that achieves reasonable accuracy for it.\n\nWhat is not clear is how this applies to the e-commerce scenarios that motivate the problem in intro. If I had a shopping catalog with more complex documents with their own multi-modal embeddings, and weak signal, with multiple docs matched per query category, how does this pipeline extend and scale?\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- Interesting new variant of standard relevance problem \n- good set of toy experiments that set up for the real problem\n- A good answer for small real world datasets\n\nWeakness\n- Unclear whether this formulation can take on and scale to real-world problems\n- Are there alternate ways of training for this problem and how does this approach compare?\n\nQUESTIONS:\n-  Can this formulation extend to product catalogs and multi-modal docs and nebulous query intent. What if the DAG is over precisely stated query categories, but the queries are slightly variants of categories.\nupdate: Authors state this is possible. Concrete proof might be beyond the scope of this paper.\n\n- Are there alternate ways of training for this problem and how does this approach compare? For example, would treating DAG as flat structure and using extreme multi-label classification work?\nupdate: Authors state that XMLC are impractical, however XMLC methods have been proved to be practical and are deployed at millisecond-latency at scale.\n\nLIMITATIONS:\nna", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-04"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies dual encoder (DE) models for information retrieval in the specific context of Hierarchical Retrieval (HR), where documents are organized in a hidden hierarchy, and a query should retrieve all of its ancestor nodes. The authors first provide theoretical guarantees that DEs are capable of solving HR if the embedding dimension scales appropriately with the depth and size of the hierarchy. They then identify a practical challenge -- retrieving distant ancestors becomes harder (termed the \"lost-in-the-long-distance\" problem). To address this, they propose a pretrain-finetune strategy: first training on standard data and then fine-tuning on long-distance query-document pairs. Experiments on synthetic trees, WordNet, and ESCI demonstrate improved performance, particularly for long-distance retrieval. The work bridges theoretical insight with practical system challenges and offers an algorithmic tweak to improve DEs on hierarchically structured data.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- Provides a constructive theoretical proof showing DE feasibility for HR under certain embedding dimension assumptions.\n- Empirical findings consistently validate the theoretical results across synthetic and real datasets.\n- Identifies and articulates a realistic failure case (lost-in-the-long-distance) that affects DEs in HR tasks.\n- Introduces a practical and simple pretrain-finetune training recipe to address the failure case.\n- Experimental results show strong improvements in recall for distant matches, especially in the WordNet dataset.\n\nWeaknesses:\n- Lack of baselines: The paper omits comparison with hyperbolic or other hierarchical embedding models (e.g., Poincare embeddings), which are directly motivated by similar geometric considerations.\n- The pretrain-finetune strategy is intuitive and potentially ad hoc. There is limited analysis on why it helps, beyond empirical results.\n- The novelty of the proposed algorithmic contribution (pretraining then fine-tuning) is limited -- it is a common ML technique and might be seen as an incremental tweak.\n- The theoretical analysis is insightful but assumes knowledge of the matching sets {S(q)}, which are not available in practical settings; this limits the practical relevance of Theorem 3.1.\n- Some claims, like DEs outperforming handcrafted embeddings in low dimensions (Fig. 2), are underexplored and not connected back to theory.\n- The ESCI experiment is a stretch from the core problem setting (HR), and the analogy to \"long-distance\" is loosely defined and not justified.\n\nQUESTIONS:\n- Why is there no comparison with hyperbolic embedding models (e.g., Poincare and hyperbolic embeddings), which also aim to solve hierarchical representation issues? These seem like direct competitors.\n- Can the authors elaborate on why the pretrain-finetune strategy works better than rebalancing? Is there an intuitive or theoretical justification?\n- How sensitive is the performance to the specifics of the fine-tuning stage (e.g., temperature, learning rate, early stopping)? Could you add ablations?\n- The theoretical results (Theorem 3.1) rely on access to S(q) sets. Can you discuss or experiment on how noisy estimation of these sets might affect the embedding quality?\n- Could you provide retrieval time and latency comparisons with other methods, particularly for large datasets? This would help validate the practical relevance.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-13"}