Context-Efficient Retrieval with Factual Decomposition (2025)

Abstract: There has recently been considerable interest in incorporating information retrieval into large language models (LLMs). Retrieval from a dynamically expanding external corpus of text allows a model to incorporate current events and can be viewed as a form of episodic memory. Here we demonstrate that pre-processing the external corpus into semi-structured “atomic facts” makes retrieval more efficient. More specifically, we demonstrate that our particular form of atomic facts improves performance on various question answering tasks when the amount of retrieved text is limited. Limiting the amount of retrieval reduces the size of the context and improves inference efficiency.

## introduction although large language models (llms) demonstrate remarkable capabilities across various tasks, their inability to continuously adapt to dynamic or domain-specific knowledge without parameter updates remains a substantial limitation. to address this limitation retrieval-augmented generation (rag) supplements models with some external knowledge source during inference @xcite @xcite . typically these models treat the external source as a set of arbitrarily segmented blocks of raw text. however, there has also been interest in using more structured external knowledge sources such as knowledge graphs @xcite , compressed documents @xcite or document trees @xcite . in each case one can identify a "unit of retrieval" where one retrieves some set of such units, such as a set of documents or a set of knowledge graph triples. various candidates for units of retrieval, or "atomic facts", have been formulated @xcite @xcite . ... and artificial intelligence. during world war ii, he played a crucial role in breaking the german enigma code, significantly contributing to the allied victory. turing was also interested in biological processes and... ## methodology as shown in figure 2 , our method consists of three main steps: (1) question speculation, (2) atomic fact extraction, and (3) knowledge base (kb) augmentation. this approach allows to decompose long documents into concise, factual entity-description pairs (edps), building up a semistructured kb for rag, reducing retrieval overhead on the lengths of contexts to improve inference efficiency. ## conclusion we have demonstrated that on various challenging question answering benchmarks edp kb retrieval achieves better accuracy when the amount of retrieval (the number of retrieved tokens) is limited. this improves the "context-efficiency" of rag, which has real cost-effective implications for llm inference. we are also optimistic that our formulation of edp kbs is a significant step toward more structured yet expressive internal representations of knowledge, and we encourage future research to build upon and expand this approach.