R ^3 M em: Bridging Memory Retention and Retrieval via Reversible Compression (2025)

Abstract: Memory plays a key role in enhancing LLMs’ performance when deployed to real-world applications. Existing solutions face trade-offs: explicit memory designs based on external storage require complex management and incur storage overhead, while implicit memory designs that store information via parameters struggle with reliable retrieval. In this paper, we propose R ^3 Mem, a memory network that optimizes both information Retention and Retrieval through Reversible context compression. Specifically, R ^3 Mem employs virtual memory tokens to compress and encode infinitely long histories, further enhanced by a hierarchical compression strategy that refines information from document- to entity-level for improved assimilation across granularities. For retrieval, R ^3 Mem employs a reversible architecture, reconstructing raw data by invoking the model backward with compressed information. Implemented via parameter-efficient fine-tuning, it can integrate seamlessly with any Transformer-based model. Experiments demonstrate that our memory design achieves state-of-the-art performance in long-context language modeling and retrieval-augmented generation tasks. It also significantly outperforms conventional memory modules in long-horizon interaction tasks like conversational agents, showcasing its potential for next-generation retrieval systems.

## introduction large language models (llms) @xcite @xcite have demonstrated remarkable capabilities in natural language understanding and generation @xcite @xcite , achieving human-comparable performance on complex reasoning tasks @xcite . deploying llms as controllers to interact with dynamic environments and solve real-world tasks, i.e. , as autonomous agents @xcite , has shown promising success across diverse applications, including conversational assistants @xcite , workflow automation @xcite @xcite , and embodied navigation @xcite @xcite . however, llms have inherent limitations: their stateless nature @xcite makes them struggle with leveraging past experiences for multiturn interactions and cross-task generalization. furthermore, their reliance on fixed context windows and static parameterized knowledge constrains their ability to handle complex tasks requiring dynamic, up-to-date information @xcite . to address these challenges, existing approaches introduce external storage (i.e. , explicit memory), such as knowledge repositories @xcite and vector databases @xcite , to enhance long-term retention and enable cross-task generalization @xcite ) and crossmodel sharing @xcite . in parallel, implicit memory encodes contextual information directly into model parameters, enabling continuous knowledge updates while providing a more compact representation of information, reducing redundancy compared to external storage. modelediting methods modify neurons to update @xcite or forget knowledge @xcite , while context integration @xcite adjusts internal parameters via model distillation. memory-augmented transformers (e.g. , rmt @xcite , associate memory @xcite @xcite , and titans @xcite ) enhance retention by integrating dedicated memory components. however, as illustrated in figure 1 , both explicit and implicit memory involve trade-offs between storage overhead and recall effectiveness. explicit memory grows indefinitely, requiring complex memory management techniques such as merging @xcite and forgetting @xcite . in contrast, implicit memory suffers from unreliable retrieval due to the black-box nature of llms, leading to confabulation and hallucination issues @xcite . as analyzed by @xcite , injected atomic facts can propagate and influence broader inferences, further complicating retrieval accuracy. more recently, adaptive retrieval @xcite and mem-orag @xcite combine explicit and implicit memory in a hybrid retrieval paradigm but remain dependent on large-scale external storage. in this paper, we propose r 3 mem, a novel memory-augmented model that optimizes both memory retention and retrieval while minimizing external storage dependency. r 3 mem leverages a reversible architecture that integrates context compression and expansion, enabling assimilation and reconstruction of input data. specifically, we design a context compression task that learns to generate compressed representations ('query') from raw input ('context'). r 3 mem utilizes virtual memory tokens to encode and retain text that is indefinitely long. to improve compression quality, we introduce a hierarchical compression strategy, progressively refining information at the document, paragraph, and entity levels. for retrieval, r 3 mem adopts a reversible architecture, reconstructing raw input by inverting the model invocation on compressed representations. this is achieved through adapter tuning, allowing seamless integration with pre-trained transformer model while maintaining parameter efficiency. to optimize both memory retention and retrieval, we employ bidirectional training with cycle consistency. the forward process encodes context into compressed memory representations, while the backward process reconstructs the raw content from memory tokens, enforcing consistency between the original and reconstructed information. we evaluate r 3 mem on memory-intensive tasks, achieving state-of-the-art performance in long-context language modeling and retrievalaugmented generation. we also integrate r 3 mem into a real-world conversational agent that requires long-horizon interactions and the ability to recall distant historical context. r 3 mem consistently outperforms existing memory modules, demonstrating superior scalability, retrieval accuracy, and potential for next-generation retrieval systems. ## methodology in this section, we introduce r 3 mem, a memory network that optimizes both memory retention and retrieval. as illustrated in figure 2 , the core component of r 3 mem is context compression, which encodes raw text into model parameters using virtual memory tokens. these trainable tokens are appended to the raw text, summarizing the current context window and propagating information to subsequent windows. this enables the model to absorb and retain indefinitely long input sequences. furthermore, to facilitate more flexible memory usage, i.e. , enabling retrieval of documents for queries with varying semantic granularities, we employ hierarchical compression. this approach chunks documents into multiple levels of semantic representation, including document-, paragraph-, sentence-, and entity-level abstractions. by structuring information hierarchically, our method optimizes retention and retrieval efficiency across different levels of granularity. lastly, we use a pre-trained transformer backbone with an adapterbased reversible architecture, allowing the memory network to operate bidirectionally. this allows the model to be invoked in reverse, which reconstructs raw information from compressed memory akin to a "zip" and "unzip" process, unifying information retention and retrieval within a duplex network. paragraph (𝒑) document (𝒅) sentence (𝒔) entity (𝒆) 𝑹 𝟑 mem reversible adapter 𝑹 𝟑 mem reversible adapter bernard arnault is a french businessman, investor and art collector […] and ceo of the global luxury goods company lvmh […] he overtook elon musk as the richest person in 2024 due to a 21% decline in musk's wealth […] bernard arnault is a french businessman, investor, and art collector who leads lvmh, a global luxury goods company. he overtook elon musk as the richest person in 2024 after a steep decline in musk's wealth. bernard arnault is a french businessman, investor, and art collector who leads lvmh, a global luxury goods company. bernard arnault 𝑹 𝟑 mem reversible adapter 𝑹 𝟑 mem virtual memory token 𝜽 segmented input 𝑹 𝟑 mem 𝑹 𝟑 mem lower level in hierarchy higher level in hierarchy large language model 𝓜 reversible adapter 𝝓 𝑹 𝟑 mem forward model 𝓜 [𝜽;𝝓] : context compression backward model 𝓜 [𝜽;𝝓] -𝟏 : context expansion ## conclusion we propose r 3 mem, a memory network built on a reversible architecture that optimizes both information retention and retrieval. r 3 mem employs hierarchical compression to adaptively process input and utilizes virtual memory tokens to encode long-context information. empirical results demonstrate state-of-the-art performance in long-context modeling and retrieval, with strong scalability and accuracy in real-world conversational agents.