HASH - RAG : Bridging Deep Hashing with Retriever for Efficient, Fine Retrieval and Augmented Generation (2025)

Abstract: Retrieval-Augmented Generation (RAG) encounters efficiency challenges when scaling to massive knowledge bases while preserving contextual relevance. We propose Hash-RAG, a framework that integrates deep hashing techniques with systematic optimizations to address these limitations. Our queries directly learn binary hash codes from knowledgebase code, eliminating intermediate feature extraction steps, and significantly reducing storage and computational overhead. Building upon this hash-based efficient retrieval framework, we establish the foundation for fine-grained chunking. Consequently, we design a Prompt-Guided Chunk-to-Context (PGCC) module that leverages retrieved hash-indexed propositions and their original document segments through prompt engineering to enhance the LLM’s contextual awareness. Experimental evaluations on NQ, TriviaQA, and HotpotQA datasets demonstrate that our approach achieves a 90% reduction in retrieval time compared to conventional methods while maintaining considerate recall performance. Additionally, The proposed system outperforms retrieval/non-retrieval baselines by 1.4-4.3% in EM scores.

## introduction in the era of rapidly expanding data, an increasing number of downstream tasks rely on large language models (llms). within these tasks, retrieval-augmented generation (rag) is a popular technical framework that incorporates external knowledge sources to tackle knowledge-intensive problems @xcite . by combining a nonparametric retrieval module with the main model, rag effectively alleviates hallucination issues in large models @xcite . moreover, this retrieval-and-generation mechanism expands the capabilities of llms in few-or zeroshot settings @xcite , which is now widely considered a standard solution for addressing factual shortcomings in traditional llms @xcite . behind the effectiveness of rag, the huge scale of the knowledge bases is to ensure the quality and professionalism of the output @xcite . the remarkable effectiveness of rag systems is fundamentally rooted in the substantial scale of the knowledge bases, which critically ensures output quality and domain expertise. concurrently, the rapid expansion of both model architectures and knowledge data has established the continuous scaling of rag knowledge bases @xcite . consequently, achieving optimal efficiencyperformance coordination in knowledge base retrieval has emerged as a paramount concern in modern rag system design. current research has dedicated significant efforts to optimize the retrieval process in rag systems. within them, several approaches focus on multi-round retrieval iterations to obtain more comprehensive and higher-quality content @xcite . others employ task-specific or domainadapted model fine-tuning to enhance performance in targeted scenarios. alternative strategies involve implementing diversified retrieval techniques or aggregating information from heterogeneous sources @xcite . chunk optimization strategies demonstrate effectiveness in improving retrieval results by adjusting the chunk size @xcite while mitigating risks of model performance degradation @xcite . despite these advancements in performance enhancement, rag systems consistently encounter significant efficiency challenges as knowledge bases expand. in the domain of large-scale data retrieval, approximate nearest neighbor (ann) search attracts significant attention due to its capability to substantially reduce search complexity under most scenarios @xcite . among ann techniques @xcite , hashing methods emerged as one of the most widely adopted approaches, offering exceptional storage efficiency and rapid retrieval capabilities @xcite . in particular, deep hashing methods @xcite employ deep neural networks to learn discriminative feature representations and convert them into compact hash codes, substantially enhancing retrieval efficiency while reducing storage requirements and computational costs @xcite . figure 1 illustrates a typical deep supervised hashing framework. this approach marks an unprecedented breakthrough in large-scale image retrieval and significantly outperforms conventional method @xcite , which also provide a novel perspective for optimizing rag efficiency. in the era of rag, the knowledge bases significantly surpass traditional image retrieval datasets in scale and growth velocity. in light of this, ann techniques exemplified by hashing demonstrate significant potential for rag applications with their capacity to rapidly target results and reduce computational complexity in large-scale data processing. in this paper, we introduce ann-based techniques into rag frameworks and propose hash-rag through systematic integration of deep hashing methods. specifically, our architecture converts query embeddings into binary hash codes via sign function operations. for knowledge bases, we adopt an asymmetric processing strategy to optimize training efficiency by directly learning binary hash codes without feature learning. based on this, we achieve fine-grained retrieval through corpus chunking, which filters redundant content while preserving precision. nevertheless, we notice that existing chunking approaches result in retrieved segments lacking essential contextual information, which substantially degrades the quality of generated outputs. to address this, we also propose a prompt-guided chunk-to-context (pgcc) module, which splits documents into factual fragments (i.e., propositions) as retrieval units. these propositions are structured in a concise, self-contained natural language format and indexed to their original documents. during generation, llm processes hash-based retrieved propositions and their contexts through specifically designed prompts to generate, achieving optimal coordination between accuracy and efficiency. we conducted experiments on open-domain question-answering datasets, including natural questions (nq) @xcite , triv-iaqa @xcite , and the more complex multi-hop hotpotqa @xcite . experimental results show that our model significantly reduces retrieval time, which requires only 10% of the time needed for conventional retrieval methods while maintaining advanced recall performance. in combination with the pgcc module, we have achieved a performance increase in the generation task while retaining efficiency. main contributions of this paper are as follows: ## method in this study, we propose a hash retrieval-augmented generation (hash-rag) framework, a pipeline that enhances rag from the perspectives of deep hashing and contextual enhancement. figure 2 illustrates the comprehensive architecture of hash-rag. sections 3.1 and 3.2 respectively introduce the hash-based retriever (hbr) and the module prompt-guided chunk-to-context (pgcc).