HASH-RAG: Bridging Deep Hashing with Retriever for Efficient, Fine Retrieval and Augmented Generation
Authors: Jinyu Guo, Xunlei Chen, Qiyang Xia, Zhaokun Wang, Jie Ou, Libo Qin, Shunyu Yao, Wenhong Tian
Accepted at: acl25

HASH-RAG: Bridging Deep Hashing with Retriever for Efficient, Fine Retrieval and Augmented Generation

Jinyu Guo1*, Xunlei Chen1*, Qiyang Xia1; Zhaokun Wang1, Jie Ou1, Libo Qin2, Shunyu Yao3 Wenhong Tian1†
1University of Electronic Science and Technology of China 2 Central South University 3 Big data and artificial intelligent institute, China Telecom Research Institute guojinyu@uestc.edu.cn

Abstract
Retrieval-Augmented Generation (RAG) encounters efficiency challenges when scaling to massive knowledge bases while preserving contextual relevance. We propose Hash-RAG, a framework that integrates deep hashing techniques with systematic optimizations to address these limitations. Our queries directly learn binary hash codes from knowledgebase code, eliminating intermediate feature extraction steps, and significantly reducing storage and computational overhead. Building upon this hash-based efficient retrieval framework, we establish the foundation for fine-grained chunking. Consequently, we design a Prompt-Guided Chunk-to-Context (PGCC) module that leverages retrieved hash-indexed propositions and their original document segments through prompt engineering to enhance the LLM’s contextual awareness. Experimental evaluations on NQ, TriviaQA, and HotpotQA datasets demonstrate that our approach achieves a 90% reduction in retrieval time compared to conventional methods while maintaining considerate recall performance. Additionally, the proposed system outperforms retrieval/non-retrieval baselines by 1.4-4.3% in EM scores. Code available at https://github.com/ratSquealer/HASH-RAG.

1 Introduction
In the era of rapidly expanding data, an increasing number of downstream tasks rely on large language models (LLMs). Within these tasks, Retrieval-Augmented Generation (RAG) is a popular technical framework that incorporates external knowledge sources to tackle knowledge-intensive problems (Lewis et al., 2020). By combining a non-parametric retrieval module with the main model, RAG effectively alleviates hallucination issues in large models (Yao et al., 2022; Bang et al., 2023). Moreover, this retrieval-and-generation mechanism expands the capabilities of LLMs in few- or zero-shot settings (Brown et al., 2020; Chowdhery et al., 2023), which is now widely considered a standard solution for addressing factual shortcomings in traditional LLMs (Ma et al., 2023).

Behind the effectiveness of RAG, the huge scale of the knowledge bases is to ensure the quality and professionalism of the output (Izacard et al., 2023). The remarkable effectiveness of RAG systems is fundamentally rooted in the substantial scale of the knowledge bases, which critically ensures output quality and domain expertise. Concurrently, the rapid expansion of both model architectures and knowledge data has established the continuous scaling of RAG knowledge bases (Chen et al., 2021). Consequently, achieving optimal efficiency-performance coordination in knowledge base retrieval has emerged as a paramount concern in modern RAG system design.

Current research has dedicated significant efforts to optimize the retrieval process in RAG systems. Within them, Several approaches focus on multi-round retrieval iterations to obtain more comprehensive and higher-quality content (Yao et al.,

![Framework of Deep Supervised Hashing with Pairwise Similarity. The framework computes similarity-preserving loss by aligning pairwise relationships between hash codes and their corresponding ground truth, which provide a novel perspective for optimizing RAG efficiency.](page_1012_370_388_246.png)

Figure 1: Framework of Deep Supervised Hashing with Pairwise Similarity. The framework computes similarity-preserving loss by aligning pairwise relationships between hash codes and their corresponding ground truth, which provide a novel perspective for optimizing RAG efficiency.
2023). Others employ task-specific or domain-adapted model fine-tuning to enhance performance in targeted scenarios. Alternative strategies involve implementing diversified retrieval techniques or aggregating information from heterogeneous sources (Huang et al., 2023). Chunk optimization strategies demonstrate effectiveness in improving retrieval results by adjusting the chunk size (Sarthi et al., 2024) while mitigating risks of model performance degradation (Zhao et al., 2024). Despite these advancements in performance enhancement, RAG systems consistently encounter significant efficiency challenges as knowledge bases expand.

In the domain of large-scale data retrieval, Approximate Nearest Neighbor (ANN) search attracts significant attention due to its capability to substantially reduce search complexity under most scenarios (Do et al., 2016; Wang et al., 2017). Among ANN techniques (Luo et al., 2021), hashing methods emerged as one of the most widely adopted approaches, offering exceptional storage efficiency and rapid retrieval capabilities (Cantini et al., 2021). In particular, deep hashing methods (Xia et al., 2014) employ deep neural networks to learn discriminative feature representations and convert them into compact hash codes, substantially enhancing retrieval efficiency while reducing storage requirements and computational costs (Lai et al., 2015). Figure 1 illustrates a typical deep supervised hashing framework. This approach marks an unprecedented breakthrough in large-scale image retrieval and significantly outperforms conventional method (Chen et al., 2021), which also provide a novel perspective for optimizing RAG efficiency.

In the era of RAG, the knowledge bases significantly surpass traditional image retrieval datasets in scale and growth velocity. In light of this, ANN techniques exemplified by hashing demonstrate significant potential for RAG applications with their capacity to rapidly target results and reduce computational complexity in large-scale data processing.

In this paper, we introduce ANN-based techniques into RAG frameworks and propose Hash-RAG through systematic integration of deep hashing methods. Specifically, our architecture converts query embeddings into binary hash codes via sign function operations. For knowledge bases, we adopt an asymmetric processing strategy to optimize training efficiency by directly learning binary hash codes without feature learning. Based on this, we achieve fine-grained retrieval through corpus chunking, which filters redundant content while preserving precision. Nevertheless, we notice that existing chunking approaches result in retrieved segments lacking essential contextual information, which substantially degrades the quality of generated outputs. To address this, we also propose a Prompt-Guided Chunk-to-Context (PGCC) module, which splits documents into factual fragments (i.e., propositions) as retrieval units. These propositions are structured in a concise, self-contained natural language format and indexed to their original documents. During generation, LLM processes hash-based retrieved propositions and their contexts through specifically designed prompts to generate, achieving optimal coordination between accuracy and efficiency.

We conducted experiments on open-domain question-answering datasets, including Natural Questions (NQ) (Kwiatkowski et al., 2019), TRIVIAQA (Joshi et al., 2017), and the more complex multi-hop HOTPOTQA (Yang et al., 2018). Experimental results show that our model significantly reduces retrieval time, which requires only 10% of the time needed for conventional retrieval methods while maintaining advanced recall performance. In combination with the PGCC module, we have achieved a performance increase in the generation task while retaining efficiency.

Main contributions of this paper are as follows:

1. We propose HASH-RAG, a framework that systematically integrates deep hashing into RAG with stage-wise optimizations. This approach significantly enhances computational efficiency in large-scale knowledge retrieval, thereby accelerating end-to-end inference throughout the overall RAG.

2. Building upon our hash-based efficient retrieval framework, we propose the PGCC module that enables fine-grained retrieval while enhancing contextual information through prompt-based optimization.

3. Experimental results on multiple datasets demonstrate that HASH-RAG significantly improves the efficiency of the retrieval. With PGCC module, our method surpasses RAG baseline models in overall performance, achieving optimal coordination between efficiency and performance.
2 Related Work

2.1 Retrieval-Augmented Generation
RAG mitigates LLM hallucinations through non-parametric memory integration and compensates for factual deficiencies via external knowledge retrieval (Gao et al., 2023). Early implementations relied on statistical similarity metrics (TF-IDF (Robertson and Walker, 1997), BM25 (Robertson et al., 2009)) before transitioning to vectorized representations (Karpukhin et al., 2020), enabling end-to-end tunable systems with context-aware retrieval. Recent efforts focus on two phases: pre-retrieval query-data matching to enhance precision (Ma et al., 2023) and post-retrieval content re-ranking or transformation to optimize generator input (Glass et al., 2022). A persistent challenge lies in chunking strategy design, balancing granularity trade-offs: coarse chunks risk redundancy despite contextual richness (Shi et al., 2023), while fine-grained units sacrifice semantic completeness for precision (Raina and Gales, 2024). Scalability-induced efficiency bottlenecks in retrieval optimization now critically constrain RAG advancement.

2.2 Deep Hashing Methods
Approximate Nearest Neighbor (ANN) algorithms address large-scale search inefficiency by trading exact precision for efficiency. Unlike tree-based (Annoy (Bernhardsson, 2015)), quantization-based (PQ (Jegou et al., 2010)), or graph-based (HNSW (Malkov and Yashunin, 2018)) methods, hashing reduces memory requirements and search latency while preserving vector locality, which makes it a mainstream ANN solution. Early techniques like Locality-Sensitive Hashing (LSH) (Charikar, 2002; Indyk and Motwani, 1998) relied on predefined mappings to hash buckets, requiring multiple tables for satisfactory recall. Learning-to-hash methods (e.g., Spectral Hashing (Weiss et al., 2008), Semantic Hashing (Salakhutdinov and Hinton, 2009)) later optimized hash functions to improve retrieval efficiency and accuracy. Current research focuses on deep-supervised hashing methods (e.g., Convolutional neural network hashing (CNNH) (Xia et al., 2014): two-phase of binary codes generation and convolutional neural networks (CNNs) training, Deep supervised hashing (DSH) (Liu et al., 2016): pairwise similarity loss with regularization for end-to-end training, Maximum-margin hamming hashing (MMHH) (Kang et al., 2019): discriminative enhancement through t-distribution and semi-batch optimization). These methods demonstrate adaptability to large-scale retrieval, establishing technical foundations for optimizing RAG’s chunking strategies and retrieval efficiency.

3 Method
In this study, we propose a Hash Retrieval-Augmented Generation (Hash-RAG) framework, a pipeline that enhances RAG from the perspectives of deep hashing and contextual enhancement. Figure 2 illustrates the comprehensive architecture of Hash-RAG. Sections 3.1 and 3.2 respectively introduce the Hash-Based Retriever (HbR) and the module Prompt-Guided Chunk-to-Context (PGCC).

3.1 Hash-Based Retriever
Hash-based Encoder (HbE) HbE module comprises two components: a query encoder \( E_q \) and a proposition encoder \( E_p \), which generate binary hash codes for queries \( q \) and propositions \( p \) (derived from knowledge base document segmentation), respectively. Inspired by ADSH (Jiang and Li, 2018), we design asymmetric encoding strategies for queries and propositions. The query encoder \( E_q \) employs BERT-base-uncased (Kenton and Toutanova, 2019) as its embedding model to map queries into a \( d \)-dimensional vector space (\( d = 768 \)). The query embedding vector \( v_q \in \mathbb{R}^d \):

\[
v_q = BERT(q, \theta)
\]

where \( \theta \) denotes BERT’s parameter. Subsequently, the binary hash code of query \( q \) is computed through the sign function in the hashing layer: \( h_q = sign(v_q) \). To address the vanishing gradient problem of the sign function, we approximate it using a scaled \( tanh \) function (Cao et al., 2017).

\[
\tilde{h}_q = tanh(\beta v_q) \in \{-1, 1\}^l
\]

where \( l \) is the fixed hash code length (768 bits) and \( \beta \) is the scaling hyperparameter controlling approximation smoothness (\( \beta \to \infty \) converges to the sign function). We set \( \sigma = 0.1 \) and \( \beta = \sqrt{\sigma \cdot step + 1} \), where \( step \) counts completed training steps.

The proposition encoder \( E_p \) directly learns binary hash codes without embedding model training through specialized loss functions and an alternating optimization strategy to reduce training overhead. For training data \( \Delta = \{ \langle q_i, p_i^+, \{p_{i,j}^- \}_{j=1}^n \rangle \}_{i=1}^m \) containing \( m \) instances, the supervision matrix \( S \in \{-1, 1\}^{m \times (n+1)} \) labels
Figure 2: Framework Overview. (a) Training. The hash-based encoder generates compact query hash codes, while the knowledge base creates binarized propositional codes from factually chunked corpora. Both components are jointly optimized through an asymmetric pairwise loss with similarity constraints. (b) Inference. The hash-based retriever efficiently fetches relevant propositions, augmented by indexed document references for contextual grounding. The generator synthesizes evidence from these elements using optimized prompts to produce responses.

positive (\( S_{i,j} = 1 \)) and negative (\( S_{i,j} = -1 \)) samples. We minimize \( L_2 \) loss between binary code inner products and supervision information:

\[
\mathcal{L}_{HbE} = \sum_{i=1}^m \sum_{j=1}^n \left[ \tilde{h}_{q_i}^T h_{p_j} - lS_{ij} \right]^2
\]

where \( h_p \) denotes the proposition hash code. For proposition-only training data \( \Delta_p = \{ q_j \}_{j=1}^n \), we construct simulated queries by randomly sampling \( m \) propositions from all proposition indices set \( \Gamma = \{ 1, 2, \ldots, n \} \), forming index subset \( \Omega = \{ i_1, i_2, \ldots, i_m \} \subseteq \Gamma \). The extended loss function is as follows:

\[
\mathcal{L}_{HbE} = \sum_{i \in \Omega} \sum_{j \in \Gamma} \left[ \tanh(\beta v_{p_i})^T h_{p_j} - lS_{ij} \right]^2 \\
+ \gamma \sum_{i \in \Omega} \left[ h_{p_j} - \tanh(\beta v_{p_i}) \right]^2
\]

where \( \gamma \) constrains \( h_{p_i} \) and \( \tilde{h}_{p_i} = \tanh(\beta v_{p_i}) \) to be as close as possible, which enables effective optimization of proposition hash codes through iterative parameter updates.

We implement an alternating optimization strategy, alternately fixing and updating the neural network parameters \( \theta \) and the proposition sentence hash code matrix \( H \).

**Update \( \theta \) with \( H \) fixed** With \( H \) fixed, we compute the gradient of \( \mathcal{L}_{HbE} \) w.r.t. \( v_{p_i} \):

\[
\frac{\partial \mathcal{L}_{HbE}}{\partial v_{p_i}} = \left\{ 2 \sum_{j \in \Gamma} \left[ (\tilde{h}_{p_i}^T h_{p_j} - lS_{ij}) h_{p_j} \right] \right. \\
\left. + 2\gamma \left( \tilde{h}_{p_i} - h_{p_i} \right) \right\} \odot \left( 1 - \tilde{h}_{p_i}^2 \right)
\]

The chain rule propagates this gradient through BERT’s parameters \( \theta \), which are then updated via backpropagation.

**Update \( H \) with \( \theta \) fixed** With \( \theta \) fixed, we rewrite Equation 4 in matrix form:

\[
\begin{align*}
\mathcal{L}_{HbE} &= \| \tilde{V} H^T - lS \|_F^2 + \gamma \| H^\Omega - \tilde{V} \|_F^2 \\
&= \| \tilde{V} H^T \|_F^2 - 2\mathrm{tr}(H^T S^T \tilde{V}) \\
&\quad - 2\gamma \mathrm{tr}(H^\Omega \tilde{V}^T) + \mathrm{const}
\end{align*}
\]

where \( \tilde{V} = [\tilde{v}_{p_1}, \tilde{v}_{p_2}, \ldots, \tilde{v}_{p_m}]^T \in [-1, +1]^{m \times l} \), and \( H^\Omega = [h_{p_1}, h_{p_2}, \ldots, h_{p_m}]^T \) denotes the sampled proposition hash codes.

To update \( H \), we adopt a column-wise updating strategy. For the \( k \)-th column \( H_{*k} \) with residual matrices \( \tilde{H}_k \) (excluding column \( k \)), and the \( k \)-th column \( \tilde{V}_{*k} \) with residual matrices \( \tilde{V}_k \) (excluding column \( k \)), the optimisation objective function is:

\[
\mathcal{L}(H_{*k}) = \mathrm{tr} \left( H_{*k} \left[ 2 \tilde{V}_{*k}^T \tilde{V}_k \tilde{H}_k^T + Q_{*k}^T \right] \right) + \mathrm{const}
\]
where \( Q = -2lS^T \widetilde{V} - 2\gamma \nabla \), with the k-th column \( Q_{*k} \). The optimal solution is:

\[
H_{*k} = -\mathrm{sign}\left(2\widehat{H}_k \widetilde{V}_k^T \widetilde{V}_{*k} + Q_{*k}\right)
\] (8)

The alternating optimization between \( \theta \) and \( H \) drives gradual convergence through multiple iterations, ultimately producing an effective query hash function and robust proposition hash code.

3.2 Prompt-Guided Chunk-to-Context

Retrieval Unit Granularity We employ the information bottleneck theory (Tishby et al., 2000) to optimize retrieval unit selection, where proposition-based chunks preserve maximal generator-relevant information while minimizing noise. Given the joint probability distribution \( p(X, Y) \) between document \( X \) and generator output \( Y \), we quantify the information content about \( Y \) contained within compressed proposition \( \widetilde{X} \) through mutual information:

\[
I(\widetilde{X}; Y) = \int_{\widetilde{X}} \int_Y p(\widetilde{x}, y) \log \frac{p(\widetilde{x}, y)}{p(\widetilde{x})p(y)} d\widetilde{x}dy
\] (9)

The compression objective minimizes \( \mathcal{L}_{IB} = I(\widetilde{X}; X) - \beta I(\widetilde{X}; Y) \), where the Lagrange multiplier \( \beta \) balances information retention and compression. Unlike conventional sentence/paragraph units (Karpukhin et al., 2020), we adopt proposition units (Min et al., 2023) that capture atomic semantic expressions. Proposition extraction occurs during the knowledge base preprocessing phase (i.e., index construction). So for document \( Doc \), we extract \( k \) interrelated propositions \( X = [x_1, \ldots, x_k] \), with relevance scores computed through hybrid scoring:

\[
X_f = \alpha X_{doc} + (1-\alpha) \sum_{k=1}^n w_k x_k
\] (10)

where \( X_{doc} \) and \( x_k \) denote document-level and BERT-based proposition scores respectively, with \( \alpha \) and \( w_k \) optimized via cross-validation.

Hash-based retrieval optimizes proposition selection through Hamming distance relationships:

\[
\mathrm{dist}_H(h_{q_i}, h_{p_j}) = \frac{1}{2}(d - \langle h_{q_i}, h_{p_j} \rangle)
\] (11)

where \( d \) is the binary code dimension. We iteratively expand the Hamming radius until selecting the top \( \alpha \) propositions, \( \alpha \) denotes the approximate candidate set retrieved via Hamming distance calculations on hash codes:

\[
\mathrm{Top}\ P_j = \arg \max_{i \in \{1, \ldots, \alpha\}} \langle v_{q_i}, h_{p_j} \rangle
\] (12)

Deduplication over proposition-document mappings yields the final top \( k \) retrieved documents \( \{Doc_1, \ldots, Doc_k\} = \mathrm{Duplicates}(P_1 \cup \ldots \cup P_j) \). This dual optimization of semantic compression and hash-based retrieval ensures maximal information extraction with minimal noise.

Prompt Optimization We employ LLAMA2 as the generator with optimized prompts. The Hash-Retriever identifies top-j propositions \( P_j = \{P_1, \ldots, P_j\} \) and their corresponding document indices \( Doc_k \), forming the generator’s context through three key components: (1) Additional Prompt instructing semantic integration of propositions and indexed documents for precise responses, (2) Retrieved Segments containing similarity-ranked propositions \( P_j \) with document references \( Doc_j \), and (3) Indexed Documents \( Doc_1, \ldots, Doc_k \) providing contextual grounding.

The prompt template activates the generator’s capability through chunk-to-context: propositions supply direct evidence while documents offer broader context, enabling accurate intent understanding with balanced retrieval-context integration. For details, see appendix A.

4 Experiment

4.1 Experimental Settings

Datasets and Retrieval Corpus We evaluated our model on three QA benchmarks using development sets. These datasets contain Wikipedia and web-sourced questions, representing diverse knowledge-intensive tasks: NQ (Kwiatkowski et al., 2019) and TRIVIAQA (Joshi et al., 2017) assess direct knowledge recall, while HOTPOTQA (Yang et al., 2018) requires multi-hop reasoning across documents. Different retrieval granularities from sentences to full documents refer to Appendix B.

Metrics With more retrieval units, we retrieve additional propositions, map them to source documents, deduplicate, and return the top \( k \) unique documents. We evaluate using document recall@k and retrieval efficiency (index size/query time). For generation, Exact Match (EM) assesses whether the ground truth appears exactly in the output.

Implementation Details In our paper, the encoders (Embedding) utilize BERT base, large, AL-
<table>
  <tr>
    <th rowspan="2">Model</th>
    <th colspan="3">Top 5</th>
    <th colspan="3">Top 20</th>
    <th colspan="3">Top 100</th>
    <th rowspan="2">Index size</th>
    <th rowspan="2">Query time</th>
  </tr>
  <tr>
    <th>NQ</th><th>TQA</th><th>HQA</th>
    <th>NQ</th><th>TQA</th><th>HQA</th>
    <th>NQ</th><th>TQA</th><th>HQA</th>
  </tr>
  <tr>
    <td>BM25</td><td>45.2</td><td>55.7</td><td>-</td><td>59.1</td><td>66.9</td><td>-</td><td>73.7</td><td>76.7</td><td>-</td><td>7.4</td><td>913.8</td>
  </tr>
  <tr>
    <td>SimCSE</td><td>28.8</td><td>44.9</td><td>26.7</td><td>44.3</td><td>59.4</td><td>44.1</td><td>47.0</td><td>62.4</td><td>46.1</td><td>64.6</td><td>548.2</td>
  </tr>
  <tr>
    <td>Contriever</td><td>47.8</td><td>59.4</td><td>42.5</td><td>67.8</td><td>74.2</td><td>67.4</td><td>82.1</td><td>83.2</td><td>76.9</td><td>64.6</td><td>608.0</td>
  </tr>
  <tr>
    <td>PQ</td><td>50.7</td><td>63.0</td><td>43.7</td><td>72.2</td><td>73.2</td><td>64.3</td><td>81.2</td><td>80.4</td><td>62.1</td><td>2.0</td><td>46.2</td>
  </tr>
  <tr>
    <td>MEVI<sup>†</sup></td><td><b>75.5</b></td><td>-</td><td>-</td><td><b>82.8</b></td><td>-</td><td>-</td><td><b>87.3</b></td><td>-</td><td>-</td><td>151.0</td><td>222.5</td>
  </tr>
  <tr>
    <td>DPR</td><td>66.0</td><td>71.6</td><td>54.4</td><td>78.4</td><td>79.4</td><td>73.0</td><td>85.4</td><td>85.0</td><td>80.3</td><td>64.6</td><td>456.9</td>
  </tr>
  <tr>
    <td>LSH<sup>‡</sup></td><td>43.2</td><td>48.0</td><td>38.4</td><td>63.9</td><td>65.2</td><td>60.5</td><td>77.2</td><td>76.9</td><td>71.1</td><td>2.0</td><td><b>28.8</b></td>
  </tr>
  <tr>
    <td>DSH<sup>‡</sup></td><td>57.2</td><td>64.7</td><td>44.2</td><td>77.9</td><td>77.9</td><td>66.2</td><td>85.7</td><td>84.5</td><td>80.4</td><td>2.2</td><td>38.1</td>
  </tr>
  <tr>
    <td>HbR(Ours)</td><td>72.4</td><td>78.3</td><td>57.7</td><td>80.3</td><td><b>87.0</b></td><td><b>80.2</b></td><td><b>87.5</b></td><td><b>88.4</b></td><td><b>81.4</b></td><td>4.6</td><td>42.3</td>
  </tr>
  <tr>
    <td colspan="3">±0.2 ±0.2 ±0.1</td><td colspan="3">±0.2 ±0.1 ±0.1</td><td colspan="3">±0.3 ±0.1 ±0.1</td><td></td><td></td>
  </tr>
</table>

Table 1: Top k recall on test sets with the index size(GB) and query time(ms) of HbR and baselines. † Model selected is MEVI Top-100 & HNSW from the main experiments and HNSW backbone retrieval method results in a multiplicative increase in index size, and the retrieval time of MEVI is not SOTA. ‡ Integration of hash with the encoder, selecting DPR, DSH model selected is hash table lookup with candidate = 1000.

<table>
  <tr>
    <th rowspan="2">Model</th>
    <th colspan="3">LLAMA2-7B</th>
    <th colspan="3">LLAMA2-13B</th>
  </tr>
  <tr>
    <th>NQ</th><th>TQA</th><th>HQA</th>
    <th>NQ</th><th>TQA</th><th>HQA</th>
  </tr>
  <tr>
    <td>ToolFormer<sup>◊</sup></td><td>17.7</td><td>48.8</td><td>14.5</td><td>22.1</td><td>51.7</td><td>19.2</td>
  </tr>
  <tr>
    <td>RRR</td><td>25.2</td><td>54.9</td><td>19.8</td><td>27.1</td><td>59.7</td><td>24.4</td>
  </tr>
  <tr>
    <td>FILCO</td><td>25.8</td><td>55.0</td><td>19.4</td><td>27.3</td><td>60.4</td><td>23.9</td>
  </tr>
  <tr>
    <td>REPLUG<sup>♦</sup></td><td>27.1</td><td>57.1</td><td>20.5</td><td>29.4</td><td>62.7</td><td>26.8</td>
  </tr>
  <tr>
    <td>Hash-RAG</td><td><b>28.5±0.1</b></td><td><b>57.1±0.1</b></td><td><b>22.1±0.2</b></td><td><b>34.9±0.2</b></td><td><b>64.5±0.1</b></td><td><b>31.1±0.3</b></td>
  </tr>
</table>

Table 2: EM of open-domain QA. ◊ Generation models in this experiment involve the GPT series, all of which are modified to the LLAMA2 series and w/o train reader in this experiment. ♦ Contriever and a zero-shot setting are selected.

BERT, and ALBERT, with each model initialized using the official pre-trained weights. The number of top j propositions is fixed at 100. The generator (generator) LLMs include LLAMA2-7B and 13B (Touvron et al., 2023). For the HbR model used in our primary experiments, the training batch size is set to 128, with one additional BM25 negative passage per question. Each encoder is trained for 40 epochs, employing linear scheduling with warm-up and a dropout rate of 0.1.

Baselines We compare HbR with BM25 (Robertson et al., 2009), DPR (Karpukhin et al., 2020), SimCSE (Gao et al., 2021), Contriever (Izacard et al., 2021), Model-enhanced Vector Index (MEVI) (Zhang et al., 2024), LSH (Charikar, 2002), and DSH (Liu et al., 2016).

BM25 (Robertson et al., 2009) employs TF-IDF principles for document relevance ranking, while DPR (Karpukhin et al., 2020) utilizes a dual-encoder architecture; SimCSE (Gao et al., 2021), an unsupervised learning architecture, optimizes semantic representations via positive/negative pair discrimination; Contriever (Izacard et al., 2021) employs Transformer-based encoder and optimize a contrastive loss function; MEVI (Zhang et al., 2024) employs the clusters of Residual Quantization (RQ) to search for documents semantically; LSH’s (Charikar, 2002) hash-bucket mapping reduces the scope of nearest neighbor search; DSH (Liu et al., 2016) integrates deep feature extraction with semantic label optimization in hash space.

4.2 Main Result

Table 1 illustrates HbR’s recall@k (\( k \in 5, 20, 100 \)) and latency on NQ, TQA, and HotpotQA benchmarks. Our framework reduces query latency to one-tenth of conventional retrievers while achieving 0.2–8.6% higher recall@20/100. Notably, HbR achieves optimal performance at \( k = 20 \). While H-RAG underperforms MEVI at \( k = 5 \), such small-k
<table>
  <tr>
    <th>Model</th>
    <th>Top 5</th>
    <th>Top 20</th>
    <th>Top 100</th>
  </tr>
  <tr>
    <td>ALBERT</td>
    <td>63.2</td>
    <td>78.1</td>
    <td>82.4</td>
  </tr>
  <tr>
    <td>Bert-base</td>
    <td>72.4</td>
    <td>80.3</td>
    <td>87.5</td>
  </tr>
  <tr>
    <td>RoBERTa</td>
    <td>72.6</td>
    <td>84.7</td>
    <td>87.6</td>
  </tr>
  <tr>
    <td>Bert-large</td>
    <td><b>73.1</b></td>
    <td><b>85.8</b></td>
    <td><b>87.9</b></td>
  </tr>
</table>
Table 3: Top k recall of HbR using different versions of embedding models on NQ Dataset.

scenarios are secondary since users typically require \( \geq 20 \) passages for answer generation. The hashing mechanism maintains index size advantages despite data volume increases from chunking.

Next, Table 2 compares Hash-RAG with baseline RAG systems using LLAMA2-7B/13B. Our framework outperforms retrieval-optimized methods (FILCO (Wang et al., 2023), RRR (Ma et al., 2023)) and LLAMA-Retrieval hybrids (REPLUG (Shi et al., 2024)), with all retrieval-augmented models surpassing Toolformer’s non-retrieval baseline (Schick et al., 2023). By feeding top-20 results to generators, we optimally balance context volume and generation quality. This design choice leverages the complementary strengths of hashing-based retrieval and modern LLMs, demonstrating significant EM improvements across all benchmarks.

4.3 Ablations

Encoder Version We investigated the compatibility of various encoder versions (ALBERT, Bert-base, Bert-large, RoBERTa) with our model. As shown in Table 3, proposition-level chunking achieves significantly superior retrieval performance compared to sentence-level and paragraph-level strategies in terms of Recall@20 metrics. Although BERT-large achieved the highest Top-k recall rates, to ensure a fair comparison with existing models that employ BERT-base as their encoder architecture, we adopt the identical configuration in our implementation.

Chunk Strategy The performance hierarchy in Table 4 demonstrates proposition-level chunking surpassing sentence- and paragraph-level strategies on Recall@20 metrics respectively. Experimental analysis reveals sentence-level segmentation fractures predicate-argument coherence, while paragraph-level processing incorporates extraneous content. The performance hierarchy reflects proposition-level chunking’s dual advantage: preserving self-contained semantic units and systematically eliminating contextual noise.

<table>
  <tr>
    <th>Chunk Strategy</th>
    <th>Recall@20</th>
  </tr>
  <tr>
    <td>sentence</td>
    <td>62.9</td>
  </tr>
  <tr>
    <td>paragraph</td>
    <td>68.8</td>
  </tr>
  <tr>
    <td>prop.</td>
    <td><b>80.2</b></td>
  </tr>
  <tr>
    <th>Prompt Optimization</th>
    <th></th>
  </tr>
  <tr>
    <td>HbR w/o prop.</td>
    <td>25.3</td>
  </tr>
  <tr>
    <td>HbR w/o doc.</td>
    <td>24.8</td>
  </tr>
  <tr>
    <td>HbR</td>
    <td>29.4</td>
  </tr>
  <tr>
    <td>HbR w/ prompt (Ours)</td>
    <td><b>31.1</b></td>
  </tr>
</table>
Table 4: Metrics (Recall@20 and EM) of different chunk strategies and prompt optimizations on HotpotQA dataset, with prop. denoting the propositions and doc. representing the original source documents associated with the retrieved propositions.

Prompt Optimization Table 4 presents our comparative analysis of prompt optimization strategies, where the PGCC module empirically demonstrates superior EM performance over all baselines. Notably, the w/o doc. configuration outperforms w/o prop., suggesting that in multi-hop datasets under Recall@20 settings, self-contained propositions still require cross-verification with source documents even when guided by optimized prompts. The non-prompted configuration achieves secondary performance due to sufficient contextual data supporting LLM reasoning, while prompt integration enhances the model’s focus on retrieved results through structured attention guidance.

5 Analysis

5.1 Chunk-Unit & Prompt-Guided

Information Bottleneck of Proposition To demonstrate how HASH-RAG’s chunking strategy leverages the information bottleneck to enhance text generation capabilities, we analyze content preserved through document segmentation. Experimental analysis based on Table 5 reveals a potential correlation between compression rates and the conciseness of conditional mutual information \( I(\tilde{X}; X|Y; Q) \), comparing exact and greedy search methods across context lengths for generator, mutual information metrics, and EM scores. We propose that applying information bottleneck principles to factual chunking generates concise intermediate answer representations with supporting evidence, outperforming alternative strategies in multi-hop queries. This indicates replacing propositions with complete documents would increased noise, distracted attention, and reduced storage ef-
<table>
  <tr>
    <th>Dataset</th>
    <th>Filtering Candidates</th>
    <th>Words</th>
    <th>\( I(\hat{X}; X|Y; Q) \)</th>
    <th>EM</th>
  </tr>
  <tr>
    <td rowspan="4">NQ</td>
    <td>Exact<br>Paragraph-Level</td>
    <td>78.1</td>
    <td><b>0.597</b></td>
    <td>21.2</td>
  </tr>
  <tr>
    <td>Exact<br>Sentence-Level</td>
    <td>28.4</td>
    <td>0.561</td>
    <td>23.8</td>
  </tr>
  <tr>
    <td>Greedy<br>Query & Answer</td>
    <td>26.2</td>
    <td>0.562</td>
    <td>19</td>
  </tr>
  <tr>
    <td>Greedy<br>Answer</td>
    <td>18.2</td>
    <td>0.556</td>
    <td>24.3</td>
  </tr>
  <tr>
    <td rowspan="4">HotpotQA</td>
    <td>Exact<br>Proposition-Level</td>
    <td>33.6</td>
    <td><b>0.594</b></td>
    <td><b>27.4</b></td>
  </tr>
  <tr>
    <td>Exact<br>Paragraph-Level</td>
    <td>120.0</td>
    <td><b>0.679</b></td>
    <td>26.3</td>
  </tr>
  <tr>
    <td>Exact<br>Sentence-Level</td>
    <td>41.2</td>
    <td>0.619</td>
    <td>27.8</td>
  </tr>
  <tr>
    <td>Greedy<br>Query & Supporting Facts & Answer</td>
    <td>32.5</td>
    <td>0.614</td>
    <td>25.8</td>
  </tr>
  <tr>
    <td></td>
    <td>Greedy<br>Supporting Facts & Answer</td>
    <td>14.8</td>
    <td>0.604</td>
    <td>26.9</td>
  </tr>
  <tr>
    <td></td>
    <td>Exact<br>Proposition-Level</td>
    <td>42.6</td>
    <td><b>0.679</b></td>
    <td><b>28.7</b></td>
  </tr>
</table>

Table 5: The effectiveness of the information bottleneck theory on the filtering data compression rate and concise mutual information in the test sets of the PGCC module for NQ and HotpotQA. The proposition-level chunking in Hash-RAG, combined with the PGCC module guiding LLMs to focus on the correspondence between propositions and documents, is theoretically optimized and experimentally validated as the optimal trade-off between efficiency and accuracy.

![Prompt Guidance Attention Heat Map](page_120_670_482_246.png)
Figure 3: Prompt Guidance Attention Heat Map

![Training time on NQ dataset.](page_728_670_482_246.png)
Figure 4: Training time on NQ dataset.

ficiency, thereby degrading generation accuracy.

Prompt Guidance on Attention To investigate how prompts influence attention mechanisms during LLM text generation, we employ Recall@1 to identify a document providing optimal factual support, thereby validating the effectiveness of prompt optimization. We generate comparative attention heatmaps (Figure 3) illustrating model behavior with versus without prompts. The prompt-free condition exhibits concentrated self-referential attention along the diagonal axis. In contrast, prompted generation demonstrates vertical attention patterns focusing on proposition tokens \( P_j \), accompanied by significant off-diagonal highlights indicating strengthened long-range dependencies between answer generation positions and critical propositions.

5.2 Training of deep hashing algorithms

Time Complexity We compare our hash method to other deep hashing baselines on NQ dataset, with the training time results illustrated in Figure 4. In our evaluation framework, DSH (Liu et al., 2016) and DHN (Zhu et al., 2016) represent conventional deep hashing baselines trained on 10,000 sampled data points, while DSH-D and DHN-D denote their full-database counterparts. The result reveals that full-database training of baselines needs over 80 minutes for convergence, which inspires the sampled training of the large-scale datasets. Moreover, our method achieves significantly faster convergence than both sampled and full-database baselines while maintaining the highest accuracy.

Sensitivity to Parameters Figure 5 illustrates the hashing hyperparameter \( \gamma \) sensitivity on NQ dataset. Our method shows stability across a broad range (\( 1 < \gamma < 500 \)), with Mean Average Precision (MAP) fluctuating within 0.01. It could potentially be attributed to NQ’s hierarchical semantic structure, which exhibits tolerance to hash-induced local perturbations. This parameter invariance reduces deployment optimization complexity while ensuring multi-scenario reliability.
Figure 5: Hyperparameter \( \gamma \) on NQ dataset.

Conclusion

We bridge deep hashing with retrieval-augmented generation for efficient, fine-grained knowledge retrieval and context-augmented generation, balancing the trade-off between the query processing time and recall. Not only as an evaluation framework for hash retrievers, Our proposed PGCC module further improves the accuracy and relevance of retrieval by optimizing the chunking strategy and addressing contextual information limitations. Experimental results demonstrate that our hash retriever significantly outperforms baseline methods and achieves impressive metrics in the generator. In future work, we plan to explore the application of hash techniques to other tasks and structures, such as the knowledge graph.

Acknowledgments

This research is supported by the Sichuan International Science and Technology Innovation Cooperation Project with ID 2024YFHZ0317,the Chengdu Science and Technology Bureau Project with ID 2024-YF09-00041-SN,and the National Natural Science Foundation of China Project with ID W2433163.

Limitations

The focus of this paper is to deeply integrate deep hashing techniques with the RAG model. The experimental framework assumes that the external knowledge base is static. If incremental updates are required, such as adding new documents or revising content, the hash encoder needs to be retrained to incorporate the new data, which is computationally expensive. In the future, developing an efficient adaptation strategy for dynamic hashing encoding remains an open challenge.

References

Yejin Bang, Samuel Cahyawijaya, Nayeon Lee, Wenliang Dai, Dan Su, Bryan Wilie, Holy Lovenia, Ziwei Ji, Tiezheng Yu, Willy Chung, et al. 2023. A multi-task, multilingual, multimodal evaluation of chatgpt on reasoning, hallucination, and interactivity. arXiv preprint arXiv:2302.04023.

Erik Bernhardsson. 2015. Annoy (approximate nearest neighbors oh yeah). URL https://github.com/spotify/annoy.

Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901.

Riccardo Cantini, Fabrizio Marozzo, Giovanni Bruno, and Paolo Trunfio. 2021. Learning sentence-to-hashtags semantic mapping for hashtag recommendation on microblogs. ACM Transactions on Knowledge Discovery from Data (TKDD), 16(2):1–26.

Zhangjie Cao, Mingsheng Long, Jianmin Wang, and Philip S Yu. 2017. Hashnet: Deep learning to hash by continuation. In Proceedings of the IEEE international conference on computer vision, pages 5608–5617.

Moses S Charikar. 2002. Similarity estimation techniques from rounding algorithms. In Proceedings of the thiry-fourth annual ACM symposium on Theory of computing, pages 380–388.

Qi Chen, Bing Zhao, Haidong Wang, Mingqin Li, Chuanjie Liu, Zengzhong Li, Mao Yang, and Jingdong Wang. 2021. Spann: Highly-efficient billion-scale approximate nearest neighborhood search. Advances in Neural Information Processing Systems, 34:5199–5212.

Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. 2023. Palm: Scaling language modeling with pathways. Journal of Machine Learning Research, 24(240):1–113.

Thanh-Toan Do, Anh-Dzung Doan, and Ngai-Man Cheung. 2016. Learning to hash with binary deep neural network. In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part V 14, pages 219–234. Springer.

T Gao, X Yao, and Danqi Chen. 2021. Simcse: Simple contrastive learning of sentence embeddings. In EMNLP 2021-2021 Conference on Empirical Methods in Natural Language Processing, Proceedings.

Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, and Haofen Wang. 2023. Retrieval-augmented generation for
large language models: A survey. arXiv preprint arXiv:2312.10997.

Michael Glass, Gaetano Rossiello, Md Faisal Mahbub Chowdhury, Ankita Rajaram Naik, Pengshan Cai, and Alfio Gliozzo. 2022. Re2g: Retrieve, rerank, generate. In Annual Conference of the North American Chapter of the Association for Computational Linguistics.

Wenyu Huang, Mirella Lapata, Pavlos Vougiouklis, Nikos Papasartanopoulos, and Jeff Pan. 2023. Retrieval augmented generation with rich answer encoding. In Proceedings of the 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1012–1025.

Piotr Indyk and Rajeev Motwani. 1998. Approximate nearest neighbors: towards removing the curse of dimensionality. In Proceedings of the thirtieth annual ACM symposium on Theory of computing, pages 604–613.

Gautier Izacard, Mathilde Caron, Lucas Hosseini, Sebastian Riedel, Piotr Bojanowski, Armand Joulin, and Edouard Grave. 2021. Unsupervised dense information retrieval with contrastive learning.

Gautier Izacard, Patrick Lewis, Maria Lomeli, Lucas Hosseini, Fabio Petroni, Timo Schick, Jane Dwivedi-Yu, Armand Joulin, Sebastian Riedel, and Edouard Grave. 2023. Atlas: Few-shot learning with retrieval augmented language models. Journal of Machine Learning Research, 24(251):1–43.

Herve Jegou, Matthijs Douze, and Cordelia Schmid. 2010. Product quantization for nearest neighbor search. IEEE transactions on pattern analysis and machine intelligence, 33(1):117–128.

Qing-Yuan Jiang and Wu-Jun Li. 2018. Asymmetric deep supervised hashing. In Proceedings of the AAAI conference on artificial intelligence, volume 32.

Mandar Joshi, Eunsol Choi, Daniel S Weld, and Luke Zettlemoyer. 2017. Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1601–1611.

Rong Kang, Yue Cao, Mingsheng Long, Jianmin Wang, and Philip S Yu. 2019. Maximum-margin hamming hashing. In Proceedings of the IEEE/CVF international conference on computer vision, pages 8252–8261.

Vladimir Karpukhin, Barlas Oğuz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense passage retrieval for open-domain question answering. arXiv preprint arXiv:2004.04906.

Jacob Devlin Ming-Wei Chang Kenton and Lee Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of naacl-HLT, volume 1. Minneapolis, Minnesota.

Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, et al. 2019. Natural questions: a benchmark for question answering research. Transactions of the Association for Computational Linguistics, 7:453–466.

Hanjiang Lai, Yan Pan, Ye Liu, and Shuicheng Yan. 2015. Simultaneous feature learning and hash coding with deep neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3270–3278.

Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in Neural Information Processing Systems, 33:9459–9474.

Haomiao Liu, Ruiping Wang, Shiguang Shan, and Xilin Chen. 2016. Deep supervised hashing for fast image retrieval. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2064–2072.

Xiao Luo, Daqing Wu, Chong Chen, Jinwen Ma, and Minghua Deng. 2021. Deep unsupervised hashing by global and local consistency. In 2021 IEEE International Conference on Multimedia and Expo (ICME), pages 1–6. IEEE.

Xinbei Ma, Yeyun Gong, Pengcheng He, Hai Zhao, and Nan Duan. 2023. Query rewriting in retrieval-augmented large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 5303–5315.

Yu A Malkov and Dmitry A Yashunin. 2018. Efficient and robust approximate nearest neighbor search using hierarchical navigable small world graphs. IEEE transactions on pattern analysis and machine intelligence, 42(4):824–836.

Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Koh, Mohit Iyyer, Luke Zettlemoyer, and Hannaneh Hajishirzi. 2023. Factscore: Fine-grained atomic evaluation of factual precision in long form text generation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 12076–12100.

Vatsal Raina and Mark Gales. 2024. Question-based retrieval using atomic units for enterprise rag. arXiv preprint arXiv:2405.12363.
Stephen Robertson, Hugo Zaragoza, et al. 2009. The probabilistic relevance framework: Bm25 and beyond. Foundations and Trends® in Information Retrieval, 3(4):333–389.

Stephen E Robertson and Steve Walker. 1997. On relevance weights with little relevance information. In Proceedings of the 20th annual international ACM SIGIR conference on Research and development in information retrieval, pages 16–24.

Ruslan Salakhutdinov and Geoffrey Hinton. 2009. Semantic hashing. International Journal of Approximate Reasoning, 50(7):969–978.

Parth Sarthi, Salman Abdullah, Aditi Tuli, Shubh Khanna, Anna Goldie, and Christopher D Manning. 2024. Raptor: Recursive abstractive processing for tree-organized retrieval. arXiv preprint arXiv:2401.18059.

Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. 2023. Toolformer: Language models can teach themselves to use tools. Advances in Neural Information Processing Systems, 36:68539–68551.

Freda Shi, Xinyun Chen, Kanishka Misra, Nathan Seales, David Dohan, Ed H Chi, Nathanael Schärtli, and Denny Zhou. 2023. Large language models can be easily distracted by irrelevant context. In International Conference on Machine Learning, pages 31210–31227. PMLR.

Weijia Shi, Sewon Min, Michihiro Yasunaga, Minjoon Seo, Richard James, Mike Lewis, Luke Zettlemoyer, and Wen-tau Yih. 2024. Replug: Retrieval-augmented black-box language models. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 8364–8377.

Naftali Tishby, Fernando C Pereira, and William Bialek. 2000. The information bottleneck method. arXiv preprint physics/0004057.

Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajwal Bhargava, Shruti Bhosale, et al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288.

Jingdong Wang, Ting Zhang, Nicu Sebe, Heng Tao Shen, et al. 2017. A survey on learning to hash. IEEE transactions on pattern analysis and machine intelligence, 40(4):769–790.

Zhiruo Wang, Jun Araki, Zhengbao Jiang, Md Rizwan Parvez, and Graham Neubig. 2023. Learning to filter context for retrieval-augmented generation. arXiv preprint arXiv:2311.08377.

Yair Weiss, Antonio Torralba, and Rob Fergus. 2008. Spectral hashing. Advances in neural information processing systems, 21.

Rongkai Xia, Yan Pan, Hanjiang Lai, Cong Liu, and Shuicheng Yan. 2014. Supervised hashing for image retrieval via image representation learning. In Proceedings of the AAAI conference on artificial intelligence, volume 28.

Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D Manning. 2018. Hotpotqa: A dataset for diverse, explainable multi-hop question answering. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2369–2380.

Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2022. React: Synergizing reasoning and acting in language models. arXiv preprint arXiv:2210.03629.

Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2023. React: Synergizing reasoning and acting in language models. In International Conference on Learning Representations (ICLR).

Hailin Zhang, Yujing Wang, Qi Chen, Ruiheng Chang, Ting Zhang, Ziming Miao, Yingyan Hou, Yang Ding, Xupeng Miao, Haonan Wang, et al. 2024. Model-enhanced vector index. Advances in Neural Information Processing Systems, 36.

Penghao Zhao, Hailin Zhang, Qinhan Yu, Zhengren Wang, Yunteng Geng, Fangcheng Fu, Ling Yang, Wentao Zhang, and Bin Cui. 2024. Retrieval-augmented generation for ai-generated content: A survey. arXiv preprint arXiv:2402.19473.

Han Zhu, Mingsheng Long, Jianmin Wang, and Yue Cao. 2016. Deep hashing network for efficient similarity retrieval. In Proceedings of the AAAI conference on Artificial Intelligence, volume 30.
A  Prompt Template

<table>
  <tr>
    <th colspan="3">Open-domain QA for LLaMA-2-7B</th>
  </tr>
  <tr>
    <td colspan="3"><b>Please consider all relevant details in the retrieved segments and offer a concise, informative, and contextually appropriate response. If necessary, carefully review the corresponding indexed documents to support your answer with just a few words:</b></td>
  </tr>
  <tr>
    <td colspan="3"><b>IdxID:23 Title:</b> Mount Everest<br><b>Propositions:</b> Mount Everest is Earth’s highest mountain above sea level.</td>
  </tr>
  <tr>
    <td colspan="3"><b>IdxID:23 Title:</b> Mount Everest<br><b>Propositions:</b> Mount Everest is known in Tibetan as Chomolungma.</td>
  </tr>
  <tr>
    <td colspan="3"><b>IdxID:59 Title:</b> Lhasa Tibetan<br><b>Propositions:</b> Verbs in Tibetan always come at the end of the clause.<br>....</td>
  </tr>
  <tr>
    <td colspan="3"><b>ID=23 Title:</b> Mount Everest<br><b>Doc:</b> Mount Everest, known locally as Sagarmatha or Qomolangma,[note 4] is Earth’s highest mountain above sea level, located in the Mahalangur Himal sub-range of the Himalayas. The China–Nepal border ....<br><br><b>Question:</b> What is the highest mountain in the world?<br><b>The answer is:</b> Mount Everest</td>
  </tr>
</table>

B  Dataset

B.1  Dataset statistics

<table>
  <tr>
    <th rowspan="2">Dataset</th>
    <th colspan="3"># Examples</th>
  </tr>
  <tr>
    <th>train</th>
    <th>dev</th>
    <th>test</th>
  </tr>
  <tr>
    <td>NQ</td>
    <td>79.2</td>
    <td>8.7</td>
    <td>3.6</td>
  </tr>
  <tr>
    <td>TQA</td>
    <td>78.8</td>
    <td>8.8</td>
    <td>11.3</td>
  </tr>
  <tr>
    <td>HOTPOTQA</td>
    <td>88.9</td>
    <td>5.6</td>
    <td>5.6</td>
  </tr>
</table>

Table 6: Dataset statistics

We use three datasets built from Wikipedia articles as supporting documents for answer, response, and judgment generation, as listed in Table 6.

B.2  Units of Wikipedia

<table>
  <tr>
    <th># units</th>
    <th>Avg. # words</th>
  </tr>
  <tr>
    <td>Passages</td>
    <td>41,393,528</td>
    <td>58.5</td>
  </tr>
  <tr>
    <td>Sentences</td>
    <td>114,219,127</td>
    <td>21.0</td>
  </tr>
  <tr>
    <td>Propositions</td>
    <td>261,125,423</td>
    <td>23.2</td>
  </tr>
</table>

Table 7: Statistics of text units in the English Wikipedia.

We refer to the processed corpus as Prop-WIKI. The statistics of Prop-WIKI are shown in Table 7. Notably, the values presented here correspond to the average segment length of the processed Wikipedia corpus, while Table 5 specifically reports the average input sequence length fed into the text generator during inference.

Reviewer paper summaries: This paper proposes Hash-RAG, a retrieval-augmented generation framework that integrates deep hashing for efficient retrieval over large-scale knowledge bases. The approach replaces dense embedding retrieval with binary hash codes to reduce storage and computation costs. It also introduces a Prompt-Guided Chunk-to-Context (PGCC) module, which combines fine-grained proposition-level retrieval with prompt-based context construction to improve downstream LLM generation.

The paper introduces a framework (called HASH-RAG) that combines deep hashing techniques with a prompt-guided chunking module, which should dramatically speed up retrieval time from large-scale knowledge bases, and enhance the contextual accuracy of generation in RAG systems. The authors’ experimental results on multiple QA benchmarks demonstrate that HASH-RAG reduces retrieval time to 10% of traditional methods while maintaining recall and generation quality.
Summary of strengths: 1. The paper proposes a method from the perspective of improving retrieval efficiency and granularity in RAG systems.

The authors clearly articulate how integrating deep hashing with RAG leads to significant efficiency gains by converting queries and document segments into compact hash codes. Their research thrust is presented in an accessible manner and their innovative methodology of combining prompt-guided chunking with a deep hashing-based retrieval system is supported by solid experimental results.
Summary of weaknesses: 1. The motivation is unclear, and some description or illustrative example such as Fig. 1 does not align well with the actual problem being addressed.
2. The method lacks some theoretical or empirical analysis to justify some key design choices, making it less convincing.

The framework integrates an LLM (LLAMA2) for prompt-guided context integration, but it does not provide an independent evaluation of the LLM’s reliability in this role (concern M1). Additionally, while the use of LLAMA2 in prompt optimization is mentioned, its parameter size is only discussed in the implementation details later on, and it should be introduced earlier for clarity. Along this concern, the appendix would benefit from more implementation details to enhance reproducibility (concern M2).
Comments, suggestions and typos: 1. Motivation Issues
(a) The abstract and introduction are unclear, and the core motivation is not well articulated.
(b) The first two paragraphs of the introduction lack logical coherence, and many sentences are ambiguous. For example: “With the rapid growth of the model scale and xx, the scale of the RAG knowledgebase that is growing increasingly has become a trend.” The connection between model scale and knowledge base expansion is unclear. Please revise and check terminology carefully throughout.
(c) In paragraph 5, the statement “In the era of RAG, xx surpass traditional image retrieval datasets xx” is vague. What specific problem are you trying to solve? Also, in Figure 1, using an image retrieval analogy to explain textual RAG seems misleading, the two modalities should not be treated interchangeably.
(d) The core problem you are addressing seems to be the storage and computation overhead of dense retrieval in traditional RAG frameworks, but this is not clearly emphasized. Specifically: (1) What real-world scenarios actually face this knowledge base scalability problem? (2) What exactly is the efficiency bottleneck in RAG you’re trying to solve? The introduction does not ground the motivation in the nature of the task.
2. Major Method Issues
(a) The concept of “proposition” seems central to your retrieval unit, but it’s not clearly defined or exemplified. How do they differ from sentences in practice?
(b) The asymmetric setup assumes BERT for queries and learned hash codes for propositions. But without a shared encoder or joint representation space, how is semantic alignment between the two ensured, especially for unseen inputs? The approach may rely more on memorized matching than on generalizable semantic understanding.
(c) The paper applies a tanh function to project 768-dim BERT embeddings into an l-dim binary hash space, but tanh does not change dimensionality. It is unclear whether a linear projection is applied before binarization. (see equation 1 and 2)
(d) Moreover, the paper does not specify the value of l, nor analyze its effect on performance or storage. Given the extreme compression from high-dimensional embeddings to low-bit binary codes, how does the method ensure that semantic distinctions are preserved? Is there any evidence that large semantic differences correspond to large Hamming distances?
(e) If proposition codes are learned without an encoder, how do you handle new or unseen propositions at inference time? Is there a mechanism for generalization?
(f) Binary inner product may be too coarse to capture subtle semantic differences, especially when matching short queries to longer, complex chunks. Have you considered this limitation?
(g) You emphasize proposition-level retrieval to reduce noise, but then include full documents in the prompt as well. Doesn’t that reintroduce redundancy and hurt efficiency, especially under input length limits? Also, “prompt optimization” sounds like a learned process, but appears to be just a fixed template.
(h) You mention avoiding encoders for efficiency, yet use BERT-based scoring (x_k) for propositions. Isn’t this inconsistent with the claimed benefits of the asymmetric design?
3. Experiments Issues
(a) Since this is a RAG task, retrieval metrics like top-1/3/10 accuracy are important but missing. 
(b) Why use BERT-base as the encoder? Stronger models or LLM-derived embeddings (with similar dimensionality) would make
for more convincing baselines. 
(c) What does “±” in the tables 1 and 2 represent? Please clarify.
(d) Ensure formatting consistency throughout the paper,  like table captions should appear above the tables.

There are also many grammar and capitalization errors. The volume of these errors (i.e., a mis-capitalized word in the abstract, among multiple other instances later on) and the use of some grammatically confusing sentences (i.e., lines 047-052) severely affect the readability of the paper. The paper would have benefitted from a close read before submission.
Overall assessment scores: 1.5

3.5