From RAG to Memory: Non-Parametric Continual Learning for Large Language Models
Authors: Bernal Jiménez Gutiérrez
Yiheng Shu
Weijian Qi
Sizhe Zhou
Yu Su
Venue: ICML 2025 poster (2025)
Keywords: retrieval-augmented generation
large language models
long-term memory
continual learning
Primary area: deep_learning->large_language_models
TL;DR: In this work, we propose HippoRAG 2, a non-parametric continual learning framework which outperforms state-of-the-art retrieval methods on factual, sense-making, and associative memory tasks.

Abstract: Our ability to continuously acquire, organize, and leverage knowledge is a key feature of human intelligence that AI systems must approximate to unlock their full potential. Given the challenges in continual learning with large language models (LLMs), retrieval-augmented generation (RAG) has become the dominant way to introduce new information. However, its reliance on vector retrieval hinders its ability to mimic the dynamic and interconnected nature of human long-term memory. Recent RAG approaches augment vector
embeddings with various structures like knowledge graphs to address some of these gaps, namely sense-making and associativity. However, their performance on more basic factual memory tasks drops considerably below standard RAG. We address this unintended deterioration and propose HippoRAG 2, a framework that outperforms standard RAG comprehensively on factual, sense-making, and associative memory tasks. HippoRAG 2 builds upon the Personalized PageRank algorithm used in HippoRAG and enhances it with deeper passage integration and more effective online use of an LLM. This combination pushes this RAG system closer to the effectiveness of human long-term memory, achieving a 7% improvement in associative memory tasks over the state-of-the-art embedding model while also exhibiting superior factual knowledge and sense-making memory capabilities. This work paves the way for non-parametric continual learning for LLMs. Code and data are available at https://github.com/OSU-NLP-Group/HippoRAG.

From RAG to Memory: Non-Parametric Continual Learning for Large Language Models
Bernal Jiménez Gutiérrez"
1
Yiheng Shu
Weijian Qi!
Sizhe Zhou? Yu Su
Abstract
1. Introduction
Our ability to continuously acquire, organize, and leverage knowledge is a key feature of human intelligence that AI systems must approximate to unlock their full potential. Given the chal- lenges in continual learning with large language models (LLMs), aetrieval-uugmente generation (RAG) has become the dominant way to intro- duce new information. However, its reliance on vector retrieval hinders its ability to mimic the dy- namic and interconnected nature of human long- term memory. Recent RAG approaches augment vector embeddings with various structures like knowledge graphs address some of these gaps, namely mense- and associativity, However, their performance on more basic factual memory tasks drops considerably below standard RAG. We address this unintended deterioration and pro- pose HippoRAC AG 2, framework that outperforms standard RAG comprehensively on factual, sense- making, and associative memory tasks. Hip- PoRAG 2 builds upon the Personalized PageRank algorithm used in HippoRAG and enhances it with deeper passage integration and more effective on- line use of an LLM. This combination pushes this RAG system closer to the effectiveness of human long-term memory, achieving a 7% improvement in associative memory tasks over the state-of-the- art embedding model while also exhibiting supe- rior factual knowledge and sense-making memory capabilities. This work paves the way for non- parametric continual learning for LLMs. Code and data are available at https: //github. com/ OLUU-NLP-GOOU
In an ever-evolving world, the ability continuously ab- sorb, integrate, and leverage knowledge is one of the most important features of human intelligence. From lawyers navigating shifting legal frameworks to researchers tracking multifaceted scientific progress, much of our productivity relies on this incredible capacity for continual learning. is imperative for systems to approximate this capability in order to become truly useful human-level assistants.
In recent years, large language models (LLMs) have made remarkable progress in many aspects of human intelligence. However, efforts to endow these models with our evolv- ing long-term memory capabilities have faced significant challenges in both fully absorbing new knowledge (Zhong et al., 2023; Hoelscher- et al., 2023) and avoiding catastrophic forgetting (Cohen al., 2024; Gu et al., 2024), due to the complex distributional nature of their paramet- ric knowledge. Retrieval- augmeeted generation (RAG) has emerged as a way to circumvent these obstacles and allow LLMs to access new information in a non-parametric fash- ion without altering an LLM's parametric representation. Due their simplicity and robustness (Zhong et al., 2023; Xie et al., 2024), RAG has quickly become the de facto continual learning solution for production L.MM systems. However, their reliance simple vector retrieval results in the inability to capture two vital aspects of our intercon- nected long-term memory system: sense-making (Klein et al. (2006); the ability to interpret larger, more complex, or uncertain contexts) and associativity (Suzuki (2005); the capacity to draw multi-hop connections between disparate pieces of knowledge).
Several RAG frameworks that engage LLM to explicitly structure its retrieval corpus have been recently proposed to address these limitations. To enhance cense-making, such structure seuncture-augmenteed RAG methods allow an LLM to either generate summaries (Edge et al., 2024; Sarthi et al., 2024; Chen et al., 2023) or a knowledge graph (KG) structure (Guo et al., 2024) to link groups of disparate but related passages, thereby improving the RAG system's 's ability to understand longer and more complex discourse such as long stories. To address the associativity gap, the authors of HippoRAG (Gutiérrez et al., 2024) use the Personalized
'Equal contribution 'The Ohio State University, Columbus, OHI, USA University of Illinois Urbana- Chaana-Caampain, IL, USA. Correspondence to: Bernal Jiménez Gutiérrez <jimenezgutier- rez.1 @ossu.douuu Yiheng Shu <shu.251 osu.edu>, Yu <su.809 @ss..cuuuu
Proceedings of the 42nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

From RAG to Memory: Non-Parrmct Continual Learning for Large Language Models
Factual Memory
Sense-making
Associativity
What George Rankin's occupation?
How did Cinderella reach her happy ending?
What county is Erik Hort's birthplace a part of?
Oliver Badman is a politician.
Cinderella attended the royal ball.
Erik Hort's birthplace Montebello.
George Rankin is politician
The prince used the lost glass slipper to scarch the kingdom.
X Marina is born in Minsk.
Associated
Thomas Marwick is politician.
When the slipper perfectly, Cinderella was reunited with the prince.
Montebello is a part of Rockland County.
65
26
60-
58.90
50.75
28-
25.70
25.90
26-
$8.81
23,00
54.88
50-
45
16.30
16
RAPTOR
GraphRAG
HippoRAG
NV-Embed-v2
HippoRAG 2
Figure 1. Evaluation of continual learning capabilities across three key dimensions: factual memory (NaturalQuestions, PopQA), sense- making (NarrativeQA), and associativity (MuSiQue, 2Wiki, HotpotQA, and LV-Eval). HippoRAG 2 surpasses other methods across all benchmark categories, bringing one step closer to true long-term memory system.
PageRank algorithm (Haveliwala, 2002) and an LLM's abil- ity to automatically construct KG and endow the retrieval process with multi-hop reasoning capabilities.
PageRank (PPR) methodologies while addressing its query- based contextualization limitations by integrating passages into the PPR graph search process, involving queries more deeply in the selection of KG triples as well as engaging an LLM in the online retrieval process to recognize when retrieved triples are irrelevant.
Although these methods demonstrate strong performance in both of these more challenging memory tasks, bringing RAG truly closer to human long-term memory requires ro- bustness across simpler memory tasks as well. In order to understand whether these systems could achieve such ro- bustness, we conduct comprehensive experiments that not only simultaneously evaluate their associativity and sense- making capacity through multi-hop QA and large-scale dis- course understanding, but also test their factual memory abilities via simple QA tasks, which standard RAG is al- ready well-equipped to handle.
Through extensive experiments, we find that this design pro- vides HippoRAG with consistent performance improve- ments over the most powerful standard RAG methods across the board. More specifically, our approach achieves av- erage 7 point improvement over standard RAG in associa- tivity tasks while showing no deterioration and even slight improvements in factual memory and sense- mansenmaking tasks. Furthermore, we show that our method is robust to different retrievers as well as to the use of strong open-source and proprietary LLMs, allowing for wide degree of usage flex- ibility. All of these results suggest that HippoRAG 2 is a promising step in the development of more numan--ike non-parametric continual learning system for LLMs.
As shown in Figure 1, our evaluation reveals that all pre- vious structure- -uggmenee methods underperform against the strongest embedding-based RAG methods available all three benchmark types. Perhaps unsurprisingly, find that each method type experiences the largest performance decay in tasks outside its own experimental setup. For ex- ample, HippoRAG's performance drops most on large-scale discourse understanding due to its lack of query-based con- textualization, while RAPTOR's 's performance deteriorates substantially the simple and multi-hop QA tasks due to the noise introduced into the retrieval corpora by its LLM summarization mechanism.
2. Related Work
2.1. Continual Learning for LLMs
As the use of LLMs real-world applications grows, becomes increasingly important for them acquire and integrate new knowledge over time while preserving past information-at evidenced by the many benchmarking ef- forts in this direction (Zhong et al., 2023; Liska et al., 2022; Kim et al., 2024; Roth et al., 2024; Li et al., 2024). Given the high computational cost of full-scale L.M pretraining, various techniques have been leveraged to endow these mod-
In this work, we leverage this experimental setting to help us address the robustness limitations of these innovative ap- proaches while avoiding the pitfalls of focusing too narrowly on just one task. Our proposed method, HippoRAG 2, lever- ages the strength of HippoRAG's OpenIE and Personalized
2

---

From RAG to Memory: Non-Parametric Continual Learning for Large Language Models
els with this continual learning capacity. These approaches generally fall into three categories: continual fine-tuning, model editing, and RAG (Shi et al., 2024).
information retrieval capabilities in both low-level and high- level knowledge, integrating graph structures with vector retrieval.
Continual fine-tuning involves periodically training an LLM on new data. This can be achieved through methods like continual pretraining (Jin et al., 2022), instruction tun- ing (Zhang et al., 2023). and alignment fine-tuning (Zhang et al., 2024). While effective in incorporating new linguistic patterns and reasoning skills, continual finettuning suffers from catastrophic forgetting (Huang et al., 2024), where previously learned knowledge is lost as new data is intro- duced. Moreover, its computational expense makes frequent updates impractical for real-world applications.
Although both GraphRAG and LightRAG use a KG just like our HippoRAG 2 approach, our KG is used to aid in the retrieval process rather than to expand the retrieval corpus itself. This allows HippoRAG 2 to introduce less LLM-generated noise, which deteriorates the performance of these methods in single and multi-hop QA tasks.
Associativity is the capacity to draw multi-hop connections between disparate facts for efficient retrieval. It is an impor- tant part of continual learning, which standard RAG cannot emulate due to its reliance on independent vector retrieval. HippoRAG (Gutiérrez et al., 2024) is the only RAG frame- work that has addressed this property by leveraging the PPR algorithm over an explicitly constructed open KG. Hip- PoRAG 2 is closely inspired by HippoRAG, which allows it to perform very well on multi-hop QA tasks. However, its more comprehensive integration of passages, queries, and triples allows to have a more comprehensive performance across sense-making and factual memory tasks as well.
Model editing techniques (Yao et al., 2023) provide a more lightweight alternative by directly modifying specific param- eters in the model to update its knowledge. However, these updates have been found to be highly localized, having little effect on information associated with the update that should also be changed.
RAG has emerged as a scalable and practical alternative for continual learning. Instead of modifying the LLM itself, RAG retrieves relevant external information inference time, allowing for real-time adaptation to new knowledge. We will discuss several aspects of this non- non-parametric con- tinual learning solution for LLMs in the next section.
3. HippoRAG 2
3.1. Overview
HippoRAG (Gutiérrez et al., 2024) is a neurobiologically inspired long-term memory framework for LLMs in which each component is inspired by its neurobiological analog for human memory. The framework consists of three primary components: 1) an LLM that acts as an artificial neocortex, 2) a KG and the Personalized PageRank algorithm mirror the auto-associative qualities of the hippocampus and 3) retrieval encoder that links these two components, reflect- ing one of the functions of the parahippocampal regions. These components collaborate to replicate the interactions observed in human long-term memory.
2.2. -on-Parametric Continual Learning for LLMs
Encoder model improvements, particularly with LLM backbones, have significantly enhanced RAG systems by generating high-quality embeddings that better capture se- mantic relationships, improving retrieval quality for LLM generation. Recent models (Li et al., 2023; Muennighoff et al., 2025; Lee et al., 2025) leverage LLMs, large corpora, improved architectures. and instruction fine-tuning for no- table retrieval gains. NV-Embed-v2 (Lee et al., 2025) serves as the primary comparison in this paper.
HippoRAG's offline indexing process uses an LLM pro- cess passages into KG triples, which are then incorporated into the KG, our artificial hippocampal index. Meanwhile, the retrieval encoder is responsible for detecting synonymy to interconnect information. In HippoRAG's 's online retrieval process, the LLM neocortex extracts named entities from query while the retrieval encoder finds their most similar counterparts in the KG. Then, the nodes in the KG corre- sponding to these entities, which we refer as seed nodes, are used to the Personalized PageRank (PPR) algorithm. More specifically, these seed nodes are used to assign the reset probabilities within PPR, which alter the original PageRank algorithm to distribute probability towards the seed nodes and their neighborhoods, enabling HippoRAG's context- based retrieval. Although HippoRAG seeks to con- struct memory from nan-parametric RAG, its effectiveness
Sense-making is the ability to understand large-scale or complex events, experiences, or data (Koli et al., 2024). Standard RAG methods are limited in this capacity since they require integrating information from disparate passages, and thus, several RAG frameworks have been proposed to address it. RAPTOR (Sarthi et al., 2024) and GraphRAG (Edge et al., 2024) both generate summaries that integrate their retrieval corpora. However, they follow distinct pro- cesses for detecting what to summarize and at what granu- larity. While RAPTOR uses a Gaussian Mixture Model to detect document clusters summarize, GraphRAG uses graph community detection algorithm that can summarize documents, entity clusters with relations, or combination of these elements. LighRAA (Guo et al., 2024) employs a dual-level retrieval mechanism to enhance comprehensive
3

---

From RAG to Memory: Non-Parametric Continual Learning for Large Language Models
Offline Indexing
Phrnse Nude
Passage Node
Opeal by LLM
Seed Node
Synceym detestion by embedding
Relation Edge
Passages
Dense- sparse integratioon
Synonym Fáge
Triples
Contect Edge
Knowledge Graph
Online Retrieval & QA
A nswer
Query
Ranked Passages
Retriving passages and triples
Recognition memory itriple filterin!!
A Assigning seed node weights
PPR graph search
Ranked Triples
Filtered triples
QA reading with selected parseges
Figure 2. HippoRAG 2 methodology. For offline indexing, we use an LLM to extract open KG triples from passages, with synonym detection applied phrase nodes. Together, these phrases and passages form the open KG. For online retrieval. an embedding model scores both the passages and triples to identify the seed nodes of both types for the Personalized PageRank (PPR) algorithm. Recognition memory filters the top triples using an L.M.. The PPR algorithm then performs context-based retrieval the KG to provide the relevant passages for the final QA task. The different colors shown in the KG nodes above reflect their prohability mass; darker shades indicate higher probabilities induced by the PPR process.
is hindered by critical flaw: entity-centric approach that causes context loss during both indexing and inference, well as difficulties in semantic matching.
Finally, this phrase-based KG is combined with the original passages, allowing the final open KG to incorporate both conceptual and contextual information ($3.2).
Built on the neurobiologically inspired long-term memory framework proposed in HGppoRAG (Gutiérrez et al., 2024), the structure of HippoRAG follows a similar two-stage process: offline indexing and online retrieval, as shown in Figure 2. Additionally, however, HippoRAG 2 intro- duces several key refinements that improve its alignment with human memory mechanisms: 1) It seamlessly inte- grates conceptual and contextual information within the KG, enhancing the comprehensiveness and atomicity of the con- structed index ($3.2). 2) facilitates more context-aware retrieval by leveraging the KG structure beyond isolated KG nodes ($3.3). 3) It incorporates recognition memory to improve seed node selection for graph search ($3.4). In the following sections, we introduce the pipeline in more detail and elaborate on each of these refinements.
The query is linked to relevant triples and passages using the encoder, identifying nodes that could used as seed nodes for graph search ($3.3). 2) During triple linkage, the recognition memory functions as filter, ensuring only relevant triples are retained from the retrieved set as the final seed nodes ($3.4). 3) These final seed nodes are then used to assign reset probabilities within the PPR algorithm, enabling its context-aware retrieval and refining the linking results to retrieve the most relevant passages. 4) Finally, the retrieved passages serve as contextual in- puts for the final QA task. Next, we describe each of the improvements in HippoRAG in more detail.
Online Retrieval. 1)
3.2. Dense-Sparse Integration
The nodes in the HippoRAG KG primarily consist of phrases describing concepts, which we refer as phrase nodes in this paper. This graph structure introduces limitations re- lated to the concept-contex tradeoff. Concepts are concise and easily generalizable but often entail information loss. In contrast, context provide specific circumstances that shape the interpretation and application of these concepts, enrich- ing semanties but increasing complexity. However, hu- man memory, concepts and contexts are intricately intercon- nected. The dense and sparse coding theory offers insights into how the brain represents and processes information at different granularities (Beyeler et al., 2019). Dense coding encodes information through the simultaneous activation of many neurons, resulting in a distributed and redundant
Offline Indexing. 1) HippoRAG 2, just as HippoRAG, leverages an LLM extract triples from each passage us- ing OpenIE, which allows the relations and entities to be generated without any constraints or schema. These triples are then arranged into schema-less KG hippocampal index. We call the subject or object of these triples phrases and the edge connecting them relation edge. 2) Next, the retrieval encoder identifies synonyms by evaluating phrase pairs within the KG, detecting those with vector similarity above a predefined threshold, and adding synonym edge between such pair. This process enables the KG link synonyms across different passages, facilitating the integra- tion of both old and new knowledge during learning. 3)
4

---

From RAG to Memory: Non-Parametric Continual Learning for Large Language Models
representation. Conversely, sparse coding relies on minimal neural activation, engaging only a small subset of neurons to enhance efficiency and storage compactness.
nal cues, while recognition relies on identifying information with the help of external stimuli. Inspired by this, we model the query-to-triple retrieval as a two-step process. 1) Query to Triple: We use the embedding model to retrieve the top-k triples T of the graph as described in $3.3. 2) Triple Filter- ing: We use LLMs to filter retrieved T and generate triples T' C T. The detailed prompts are shown in Appendix A.
Inspired by the dense-sparse integration observed in the human brain, we treat the phrase node as a form of sparse coding for the extracted concepts, while incorporating dense coding into our KG to represent the context from which these concepts originate. First, we adopt an encoding ap- proach similar to how phrases are encoded, using the embed- ding model. These two types of coding are then integrated in a specific manner within the KG. Unlike the document ensemble in HippoRAG, which simply aggregates scores from graph search and embedding matching, we enhance the KG by introducing passage nodes, enabling more seamless integration of contextual information. This approach retains the same offline indexing process HippoRAG while en- riching the graph structure with additional nodes and edges related to passages during construction. Specifically, each passage in the corpus is treated as passage node, with the context edge labeled "contains" connecting the passage to all phrases derived from this passage.
3.5. Online Retrieval
We summarize the online retrieval process in HippoRAG 2 after introducing the above improvements. The task involves selecting seed nodes and assigning reset probabilities for retrieval. HippoRAG 2 identifies phrase nodes from filtered triples generated by query-to-triple and recognition memory. If no triples are available, it directly retrieves top-ranked pas- sages using the embedding model. Otherwise, up to l &hraae nodes are selected based on their average ranking scores across filtered triples they originate. All passage nodes are also taken as seed nodes, as broader activation improves multi-hop reasoning. Reset probabilities are assigned based ranking scores for phrase nodes, while passage nodes receive scores proportional to their embedding similarity, adjusted by weight factor ($6.2) to balance the influence between phrase nodes and passage nodes. The PPR search is then executed, and passages are ranked by their PageRank scores, with the top-ranked passages used for downstream QA. An example of the pipeline is in Appendix B and the PPR initialization is detailed in Appendix G.I.
3.3. Deeper Contextualization
Building upon the discussion of the concept-conted trade- off, we observe that query parsing in Hip which re- lies on Named Entity Recognition (NER), is predominantly concept-centric, often overlooking the contextual alignment within the KG. This entity- focused approach to extraction and indexing introduces a strong bias toward concepts, leav- ing many contextual signals underutilized (Gutiérrez et al., 2024). To address this limitation, we explore and evaluate different methods for linking queries to the KG, aiming to more effectively align query semantics with the starting modes of graph searches. Specifically, we consider three ap- proaches: 1) NER to Node: This is the original method used in HippoRAG, where entities are extracted from the query and subsequently matched with nodes in the KG using text emheddings. 2) Query to Node: Instead of extracting indi- vidual entities, we leverage text embeddings to match the entire query directly to nodes in the KG. 3) Query to Triple: To incorporate richer contextual information from the KG, match the entire query to triples within the graph using text embeddings. Since triples encapsulate fundamental contextual relationships among concepts, this method pro- vides a more comprehensive understanding of the query's 's intent. By default, HippoRAG 2 adopts the query-to-triple approach, and we evaluate all three methods later ($6.1) 1)..
4. Experimental Setup
4.1. Baselines
We select three different types of baselines for compari- son. We include three simple baselines: the classic BM25 (Robertson & Walker, 1994) baseline as well as Contriever (Izacard et al., 2022) and GTR (Ni et al., 2022), two popular dense embedding retrievers.
Our second baseline category includes some of the largest embedding models available (7B) that demonstrate strong performance on the BEIR lcaderboard (Thakur et al., 2021):
(Li et al., 2023), GritLL//rriLLM--77 (Muennighoff et al., 2025), and Ividia/NV-EEmeedx22 (Lee et 2025).
In our final baseline category, we include four structure- augmented RAG methods. RAPTOR (Sarthi et al., 2024) organizes the retrieval corpus into a hierarchical structure based on semantic similarity. GraphRAG (Edge et al., 2024) and LightRAG (Guo et al., 2024) leverage a KG struc- ture like ours to generate high-level summaries of the con- cepts present in the corpus. Finally, HíppoRAG (Gutiérrez et al., 2024) uses KG as well but integrates knowledge using PPR rather than summarization.
3.4. Recognition Memory
Recall and recognition are two complementary processes in human memory retrieval (Uner& & Roediger III, 2022).
Recall involves actively retrieving information without exter-
5

---

From RAG to Memory: Non-Parametric Continual Learning for Large Language Models
Table Dataset statistics
NQ
PopQA
MuSiQue
2Wiki
Ho
LV-Eval
NarrativeQA
Num of queries
1, 000
1, 0.0
1,000
1, 1,000
1,000
124
293 4, 111
Num passages
9. 633
8, 676
11,656
6, 119
9, 9,811
22, 849
4.2. Datasets
corpus collection, we segment long-form contexts of LV- Eval into shorter passages while maintaining the same RAG setup as other multi-hop datasets.
To evaluate how well RAG systems retain factual memory while enhancing associativity and sense-making, we select datasets that correspond to three critical challenge types.
This category consists of only NarrativeQA, QA dataset that contains questions requir- ing cohesive understanding of full-length novel. This dataset's focus on large-scale discourse understanding al- lows us to leverage it in our evaluation of sense-making in our chosen baselines and our own method. We randomly select 10 lengthy documents and their corresponding 293 queries from NarrativeQA and collect a retrieval corpus just as in the above LV-Eval dataset.
Discourse Understanding.
1. Simple QA primarily evaluates the ability to recall and retrieve factual knowledge accurately.
2. Multi-hop QA measures associativity by requiring the model connect multiple pieces of information to derive an answer.
3. Discourse understanding evaluates sense- manse- by testing the capability interpret and reason over lengthy, complex narratives.
4.3. Metries
Following HippoRAG (Gutiérrez et al., 2024), we use pas- sage @ evaluate the retrieval task. For the QA task, follow evaluation metrics from MuSiQue (Trivedi et al., 2022) to calculate token-based Fl scores.
We will now list the datasets chosen for each category and describe them in detail. The statistics for our sampled datasets are summarized in Table 1.
Simple QA. This common type of QA task primarily in- volves questions centered around individual entities, making it particularly well-suited for embedding models retrieve relevant contextual information intuitively. We randomly collect 1, 000 queries from the NaturalQuestions (NQ) dataset (collected by Wang et al. (2024)), which contains real user questions with wide range of topics. Additionally, we select 1, 000 queries from PopQA (Mallen al., 2023), with the corpus derived from the December 2021 Wikipedia dump.' Both datasets offer straightforward QA pairs, en- abling evaluation of single-hop QA capabilities in RAG systems. Notably, PopQA from Wikipedia is especially entity-centric, with entities being less frequent than Natu- ralQuestions, making it an excellent resource for evaluating entity recognition and retrieval in simple QA tasks.
4.4. Implementation Details
For HippoRAG 2, we use the open-source -lama-3.3-70B- Instruct (AI @ Meta, 2024) as both the extraction (NER and OpenlE) and triple filtering model, and we use nvidia/NV. Embed-v2 as the retriever. We also reproduce the compared auruetur-augmented RAG methods using the same extrac- tor and retriever for a fair comparison. we use DSPy (Khattab al., 2024) MIPROv2 optimizer and 1lama-.3--70B-Innss the instructions and demonstrations. The resulting prompt is shown in Appendix A. We use top-5 triples ranked by retriever for filtering. Our QA module uses the top-5 re- trieved passages as context for an LLM (GPT-4o-mini or Jlama-3.3-70B-Insttu to generate the final answer. For hyperparameters, we follow the default settings from Hip- AoRAG. More implementation and hyperparameter details can be found in Appendix G.
the triple filter,
to tune the prompt, including
Multi-hop QA. We randomly collect 1, 000 queries from MuSiQue, 2WikiMultihopQA, and HotpotQA follow- ing HippoRAG (Gutiérrez et al., 2024), all requiring multi-passage reasoning. Additionally, we include all 124 queries from IV-Eval (hotpotwikiqa-mixup 256k) (Yuan et al., 2024), a challenging dataset designed to minimize knowledge leakage and reduce overfitting through keyword and phrase replacements. Thus, unlike Wikipedia-basee datasets, LV-Eval better evaluates the model's ability to syn- thesize knowledge from different sources effectively. For
5. Results
We now present our main QA and retrieval experimental results, where the QA process uses retrieved results as its context. More detailed experimental results are presented in Appendix C. The statisties for all constructed KGs are shown in Appendix A.
hhttps://github.e http://www..ccc.ccccooooeeeaaaaa stlas?tab wiw.oo--ffllloooooooo lotoopp
6

---

From RAG to Memory; Non-Parametric Continual Learning for Large Language Models
Table QA performance (Fl scores) on RAG benchmarks using 3.a.-.3.-7---m-eeee as the QA reader. retrieval means evaluating the parametric knowledge of the readers. All structure-aue augmmeetdd RAG baselines and HippoRAG 2 use 3.te--3.-----mna as the LLM generate their structure second-best results, A bootstrapped statistical test was used assess significance; indicates that HippoRAG 2 significantly outperforms the best NV-Embed-v2 baseline (p 0.05).
NV-Embed-v2 their retriever. This table, along with the following ones, highlight the best and
Discourse Understanding
Simple QA
Multi-Hop QA
Retrieval
NQ
PopQA
MuSiQue
2Wiki
HotpntQA
LV-Eval
NarrativeQA
Avg
Simple Baselines
None
54.9
32.5
26.1
42.8
47.3
6.0
12.9
38.4
Contriever (Izacard et al., 2022)
58.9
53.1
31.3
41.9
62.3
8.1
19.7
46.9
BM25 (Rohertson & Walker, 1994)
59.0
19.9
28.8
51.2
63.4
5.9
18.3
47.7
GTR (TS-base ([S-base) (Ni et al., 2022)
59.9
56.2
34.6
52.8
62.8
7.1
19.9
50.4
Large Embedding Models
TTTF--Qwen227B-Innseee GriLLM-7B (Muennigholl et al., 2025)
ii et al., 2023)
62.0
56.3
40.9
60.0
71.0
7.1
21.3
54.9
61.3
55.8
44.8
60.6
73.3
9.8
23.9
56.1
NV-Embed-v2 (7B)
al., 2025)
55.7
45.7
61.5
75.3
9.8
25.7
57.0
Structure
Augmented RAG
RAPTOR (Sarthi et al., 2024)
50.7
56.2
28.9
52.1
69.5
5.0
21.4
48.8
GraphRAG (Edge et al., 2024)
46.9
48.1
38.5
58.6
68.6
11.2
23.0
49.6
AG (Guo et al., 2024)
16.6
2.4
1.6
11.6
2.4
1.0
3.7
6.6
HippoRAG (Gutiérrez et al., 2024)
55.3
55.9
35.1
71.8
63.5
8.4
16.3
53.1
HippoRAG2
63.31
56.2
48.61
71.07
75.5
12.9
25.9
59.8
Table Retrieval performance (passage 5) on RAG benchmarks. denotes the report from the original paper. The compared structure-augne augmented RAG methods are reproduced with the same LLM and retriever for fair comparison. GraphRAG and LightRAG are not presented because they do not directly produce passage retrieval results.
Simple QA
Multi Hop QA
Retrieval
NQ
PopQA
MuSiQue
2Wiki
HotpotQA
Avg
Simple Baselines
BM25 (Robertson & Walker, 1994)
56.1
35.7
43.5
65.3
74.8
55.1
Contriever (Izacard al., 2022)
54.6
43.2
46.6
57.5
75.3
55.4
GTR (TS-base) (Ni et al., 2022)
63.4
49.4
49.1
67.9
73.9
60.7
Large Embedding Models
GTE- Teee--Wen--B--ntteee Instructt (Li al., 2023)
74.3
50.6
63.6
74.8
89.1
70.5
GritLM-7B (Muennighoff al., 2025)
76.6
50.1
65.9
76.0
92.4
72.2
NV-Embod-v2 (7B) (Lee et al., 2025)
75.4
51.0
69.7
76.5
94.5
73.4
Structure-Angmented RAG
RAPTOR (Sarthi et al., 2024)
68.3
48.7
57.8
66.2
86.9
65.6
HippoRAG* (Gutiérrez al., 2024) HGppoRAG (reproduced)
51.9
89.1
77.7
44.4
53.8
53.2
90.4
77.3
63.8
HippoRAG
78.0
51.7
74.7
90.4
96.3
78.2
QA Performance. Table 2 presents the QA performance of various retrievers across multiple RAG benchmarks using Mamma-333-70B-mme
poRAG, HippoRAG shows even greater improvements. validating its courossychology-ispiign approach. These results highlight HippoRAG as state-of-the-au RAG system that enhances both retrieval and QA performance while being effectively powered by an open-source model. Table in Appendix C presents additional QA results (EM and F1) using Llama or HPTT-o-mini as the QA reader, along with an extractor or triple filter. GPT-4o-mini follows Llama's trend, with NV-Embed-v2 outperforming structure- augmented methods in most cases, except for HippoRAG in multi-hop QA. HippoRAG 2 consistently outperforms all other methods across nearly all settings. An analysis
as the QA reader. HippoRAG 2 achieves the highest average F1 score, demonstrating ro- bustness across different settings. Large embedding models outperform smaller ones, with NV-Embed-v2 (7B) scoring 6.6% higher on average than GTR (T5-base). These mod- els also surpass structure-augee auggeeted RAG methods with lower computational costs but excel mainly in simple QA while struggling in complex cases. Notably, HippoRAG 2 outperforms NV-Embed-v2 by 9.5% Fl on 2Wiki and by 3.1 1% on the challenging LV-Eval dataset. Compared to Hip-

---

From RAG to Memory: Non-Parametric Continual Learning for Large Language Models
Table Ablations. report passage @ on multi-hop QA benchmarks using several alternatives to our final design in graph linking, graph construction and triple filtering.
55
MuSiQue
2Wiki
HotpotQA
Avg
Score
HippoRAG2
74.7
90.4
96.3
87.1
50
w/ NER node
53.8
91.2
78.8
74.6
w/ Query to node
44.9
65.5
68.3
59.6
45-
w/o Passage Node
63.7
90.3
88.9
81.0
w/o Filter
73.0
90.7
95.4
86.4
25
50
75
100
Table Reset probability factor. Passage @ with different weight factors for passage nodes on our MuSiQue dev set and NaturalQuestions (NQ) dev set, where each set has 1, 000 queries.
Total Documents
HippoRAG2 2 (NQ)
HippoRAG 2 (MuSiQue)
NV-Embed-v2 (NQ)
NV-Immeed-v2 (MuSiQue)
Weight
0.01
0.05
0.1
0.3
0.5
Figure 3. Continual learning experiment: We partition the NQ and MuSiQue datasets into segments and report the score on randomly chosen segment the other segments are introduced into the retrieval corpus to simulate a continuously evolving corpus.
MuSiQue
79.9
80.5
79.8
78.4
77.9
NQ
75.6
76.9
76.9
76.7
76.4
of the computational resources (tokens, time and memory) required for each method can be found in Appendix F.
Moreover, query-to-node does not provide an advantage over NER-to-node, as queries and KG nodes operate dif- ferent levels of granularity, whereas both NER results and KG nodes correspond to phrase-level representations.
Retrieval Performance. We report retrieval results for datasets with supporting passage annotations and models that explicitly retrieve passages in Table 3. Large embed- ding models (7B) significantly outperform classic smaller LM-based models like Contriever and GTR, achieving at least a 9.8% higher F1 score. While our reproduction of Hip- PoRAG using Llama- 1..3-70B-Inntrue and NV-Embed-v2 shows slight improvements over the original paper, the gains minimal, with only 1.3% increase in F1. Although HippoRAG excels in entity-centric retrieval, achieving the highest recall @5 on PopQA, it generally lags behind recent dense retrievers and HippoRAG 2. Notably, HippoRAG 2 achieves the highest recall scores across most datasets, with substantial improvements of 5.0% and 13.9% Re- @ on MuSiQue and 2Wiki, respectively, compared to the strongest dense retriever, NV-Embed-v2.
6.2. Controlling Reset Probabilities
When setting the reset probability before starting PPR, we find that it is necessary to balance the reset probabilities between two types of nodes: phrase nodes and passage nodes. Specifically, the reset probability of all passage nodes is multiplied by a weight factor to balance the importance of two types of nodes during PPR. Here, we present the results obtained on the validation set in Table 5, which shows that this factor is crucial for the PPR results. Considering the model performance across different scenarios, we set the factor to be 0.05 by default.
6.3. Robustness to Corpus Expansion
6. Discussions
As RAG systems become more widely adopted in the real- world, they must increasingly adapt to continual learning scenarios in which the retrieval corpora grow continuously. To understand how HippoRAG 2's capacity to handle this setting compared to standard RAG, we design an exper- iment in which we partition NQ and MuSiQue into four equal segments, each containing the gold documents and distractors for approximately 250 questions. We then select one segment for evaluation and incrementally add the re- maining segments, measuring how performance evolves as new knowledge is added, allowing us to simulate a continual learning setting. We show the Fl score for HippoRAG and NV-Embed-v2 our strongest baseline, in Figure 3.
6.1. Ablation Study
We design ablation experiments for the proposed linking method, graph construction method, and triple filtering method, with the results reported in Table 4. Each intro- duced mechanism boosts HippoRAG First, the linking method with deeper contextualization leads to significant performance improvements. Notably, we do not apply filtering process to the NER-to-node query-to-node meth- ods; however, the query-to-triple approach, regardless of whether filtering is applied, consistently outperforms the other two linking strategies. On average, query-to-triple improves @ by 12.5% compared to NER-to-node.
8

---

From RAG to Memory: Non-Parametric Continual Learning for Large Language Models
Table We show exemplary retrieval results (the title of passages) from HippoRAG and NV-Emhed-v2 on different types of questions.
Bolded items denote the titles of supporting passages.
Question
NV-Embed-v2 Results
HippoRAG Filtered Triples
HippoRAG Results
Simple QA
In what city was
1. I. Paul
(I. Paul, from, Thrissur)
1. P. Paul
I.P. Paul born?
2. Yinka Ayefele Early life
(I. P. Paul, was mayor of. Thris-
Thrissur
3. Paul Parker (singer)
sur municipal corporation)
3. Yinka Ayefelc
Multi-Hop QA
What county is Erik Hort's birth- place a part of?
1.
Erik Hort
(Erik Hort, born in, Monte-
1. Erik Hort
2. Horton Park (Saint Paul,
bello)
Horton Park (Saint Paul,
Minnesota)
(Erik Hort, born in, New York)
Minnesota)
3. Hertfordshire
3. Monstebello, New York
"Montebello" during the query-to- step, which contains geographic information that implies the answer to the ques- tion. In the subsequent graph search, this passage is also ranked at the top. Apart from this, the error analysis of HippoRAG 2 is detailed in Appendix E.
Table Robust to different dense retrievers. Passage @ on MuSiQue subset.
Retriever
Dense Retrieval
HippoRAG2
GTE- Qwee---e---nstteeeeeeeee
63.6
68.8
GritLM-7B
66.0
71.6
NV-Embed-v2 (7B)
69.7
74.7
7. Conclusion
We introduced HippoRAG 2, novel framework designed to address the limitations of existing RAG systems in approx- imating the dynamic and interconnected nature of human long-term memory. combining the strengths of the Per- sonalized PageRank algorithm, deeper passage integration, and effective online use of LLMs. HippoRAG opens new avenues for research in continual learning and long-term memory for LLMs by achieving comprehensive improve- ments over standard RAG methods across factual, sense- making, and associative memory tasks, showing capabilities that previous methods have either overlooked or been inca- pable of achieving a thorough evaluation. Future work could consider leveraging graph- retrieval methods to further enhance the episodic memory capabilities of LLMs in long conversations.
As we can see in Figure 3, HippoRAG 2's improvements over NV-Embed-v2 v2 remain remarkably consistent in both simple (NQ) and associative (MuSiQue) continual learning settings. We also note that, while both methods retain strong performance on simple QA (solid lines) as more knowledge is introduced, their performance in the more complex asso- ciative task (dotted lines) degrades at a similar rate as more information introduced. This divergence underscores the importance of incorporating varied task complexities into future continual learning benchmarks.
6.4. Dense Retriever Flexibility
As demonstrated in Table 7, HippoRAG 2 consistently sur- passes direct dense retrieval across various retrievers. No- tably, these performance gains remain robust regardless of the specific dense retriever used.
Impact Statement
This paper presents work on Retrieval- Augmenttd Gener- ation (RAG) to advance the field of long-term memory for large language models. While our work may have various societal implications, we do not identify any concerns that warrant specific emphasis beyond those generally associ- ated with large language models and information retrieval systems.
6.5. Qualitative Analysis
We show examples from PopQA and MuSiQue in Table 6. For the first example, "In what city was I. P. Paul born?". NV-Embed-v2 ranks the entity mentioned in the query "I P. Paul" as the top 1, where the passage is enough to an- swer this question. But HippoRAG 2 does even better. It directly finds the answer "Thrissur" when linking the triples, and during the subsequent graph search, places the pas- sage corresponding that entity in the second position, which is perfect retrieval result. For the second multi-hop question, "Whut county is Erik Hort's birthplace parl of?" NV-Embed-v2 also easily identifies the person men- tioned, " "Erik Hon." However, since this question requires two-step reasoning, it is not sufficient to fully answer the question. In contrast, HippoRAG 2 retrieves passage titled
Acknowledgments
We would also like to extend our appreciation to colleagues from the OSU NLP group for their constructive comments. This work is supported in part by ARL W91 INF2220144 NSF 2112606, and a gift from Cisco. We also thank the Ohio Supercomputer Center for providing computational resources. The views and conclusions contained herein

---

From RAG to Memory: Non-Parametric Continual Learning for Large Language Models
those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the U.S. government. The U.S. government is authorized to reproduce and distribute reprints for government purposes notwithstanding any copyright notice herein.
Haveliwala, T. H. Topic-sensitive pagerank. In Lass- ner, D., Roure, D. D., and Iyengar, A. (eds.), Pro- ceedings of the Eleventh International World Wide Web Conference, WWW 2002, May 7-11, 2002, Honolulu, Hawaii, USA, pp- 517-526. ACM, 2002. doi: 10.1145/ 511446.511513. URL https: /dl. acm. org/doil 10.
References
511513.
Al@Meta. Llama 3 model card. 2024. URL
Hoelscher-Obermaic J., Persson, J., Kran, E., Kon- stas, I., and Barez, F. Detecting edit failures in large language models: An improved specificity bench- mark. In Rogers, A., Boyd-Graber J., and Okazaki, N. (eds.), Findings of the Association for Compu- tational Linguistics: ACL 2023, PP- 11548-11559, Toronto, Canada, July 2023. Association for Computa- tional Linguistics. doi: 10. 10.18653/11/2223fmdmmm 733. URL https: / //a//claantt acl.ntccoooooooo....o.o org/ 2023. f Findings-acll 733//
https://githin com caamm/mmtalll aataaaallamaaaaaaaaaaaaaaa /11mma3//
md.
Beyeler, M., Rounds, E. L.,, Carlson, K. D., Dutt, N., and Krichmar, J. L. Neural correlates of sparse coding and dimensionality reduction. PLoS Comput Biol, 15(6): e1006908, 2019. doi: 10.137//joumal.ppbb 1006988.
Chen, H., Pasunuru, R., Weston, J., and Celikyilmaz, A. Walking down the memory maze: Beyond context limit through interactive reading, 2023. URL https: //arxiv. wetrgiirr/////ee.....oooo
Huang, J., Cui, L.,, Wang, A., Yang, C., Liao, X., Song, L.,, Yao, J., and Su, J. Mitigating catastrophic for- getting in large language models with self-synthesized rehearsal. In Ku, L.-.., Martins, A., and Srikumar, V. (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pp. 1416-1428, Bangkok, Thailand, August 2024. Association for Computational Linguis- tics. doi: 10. 10.18655//1/2024..acl / //aclanthology' org//024.//// acl long. 77/.
05029.
Cohen, R., Biran, E., Yoran, O., Globerson, A., and Geva, M. Evaluating the ripple effects of knowledge editing in language models. Transactions of the Association for Computational Linguistics, 12:283 2024. doi: 10. 1162/tacl 00644. URL https: /aalaaaaaattooy..... org/2020
URL https:
Edge, D., Trinh, H., Cheng, N., Bradley, J., Chao, A., Mody, A., Truitt, S., and Larson, J. From local to global: A graph rag approach to query-focused summarization, 2024. URL htttt:::/// /atpii//aaxiv.a/aa org/ abs 240bs//2444. 16130.
Izacard, G., Caron, M., Hosseini, L., Riedel, S., Bo- janowski, P., Joulin, A., and Grave, E. Unsupervised dense information retrieval with contrastive learning. Trans. Mach. Learn. Res., 2022, 2022. URL https:
/ //openrvieww wwwpeeeeeeuneeeeee (iforrmmiddjjaa
Gu, J.-C., Xu, H.-X., Ma, J.,Y., Lu, P., Ling, Z.HH., Chang, K.-W., and Peng, N. Model editing harms general abilities of large language models: Regularization to the rescue. In Al-Onaizan, Y., Bansal, M., and Chen, Y.-N. (eds.), Pro- ceedings of the 2024 Conference Empirical Methods in Natural Language Processing, pp. 16801-16819, Miami, Florida, USA, November 2024. Association for Compu- tational Linguistics. doi: 10. 10.18663/w1120224.cmmmmmmmmm 934. URL ht https: // emn 1-pmpmaam in. 93
Jin, X., Zhang, D., Zhu, H., Xiao. W., Li, S.-W., Wei, X., Arnold, A., and Ren, X. Lifelong pretraining: Contin- ually adapting language models to emerging corpora. In Carpuat, M., de Marneffe, M.-C., and Meza Ruiz, I. V. (eds.), Proceedings of the 2022 Conference of the North American Chapter of the Association for Compu- tational Linguistics: Human Language Technologies, pp. 4764-4780, Seattle, United States, July 2022. Association for Computational Linguistics. doi: 10. 10.186//11/2222. maacl-main.33 URL https: httppp::///caathhloggooo o org 22tp/222..a2 ca ma i......mmm......mmm........ 3511
2024.
Guo, Z., Xia, L., Yu, Y., Ao, T., and Huang, C. LightRAG: Simple and fast retrieval-ane augmented generation, 2024.
Khattab, O., Singhvi, A., Maheshwari, P., Zhang, Z., San- thanam, K., Vardhamanan, S., Haq, S., Sharma, A., Joshi, T. T., Moazam, H., Miller, H., Zaharia, M., and Potts, C. DSPy: Compiling declarative language model calls into self-improving pipelines. 2024.
URL https: / //ax/aaxvvv///////////a////////a v. org/ abs/ 05779.
Gutiérrez, B. J., Shu, Y., Gu, Y., Yasunaga, M., and Su, Y. Hipporag: Neurobiologically inspired long-term memory for large language models. In The Thihty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL https:/ htp///peeeveieeeeee wwweeeiwwwwwweeeeeeeeeeeeeeeeeee forum? id=hkujvAPVsg.
Kim, Y., Yoon, J., Ye, S., Bae, S., Ho, N., Hwang, S. J., and Yun, S.-Y. Carpe diem: On the evaluation of world knowl- edge lifelong language models. In Duh, K., Gomez,
10

---

From RAG to Memory: Non-Parametric Continual Learning for Large Language Models
H., and Bethard, S. (eds.), Proceedings of the 2024 Con- ference of the North American Chapter of the Associa- tion for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pp. 5401-5415, Mexico City, Mexico, June 2024. Association for Compu- tational Linguisties. doi: 10.18653/v www.6663/w1/00244aaacl-oooo 302. URL https: / / //aaa/aaaaanooooo.ooooao org./004....//000....// baaci-long 302/.
(eds.), Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pp. 13604-13622. PMLR, 17-23 Jul 2022. URL htts:::::: /ttpp//roeeenggss. .mlr. press/ v1 62 / siaalsaaaaa.... html.
Lå, X. H. BM25S: Orders of magnitude faster lexical search via eager sparse scoring, 2024. URL https://aaaae org/ abs 22002440.00000 03618.
Klein, G., Moon, B., and Hoffman, R. R. Making sense of sensemaking 1: Alternative perspectives. IEEE intelligent systems, (3():70--3, 2006.
Mallen, A., Asai, A., Zhong, V., Das, R., Khashabi, D., and Hajishirzi, H. When not to trust language mod- els: Investigating effectiveness of parametric and non- parametric memories. Rogers, A., Boyd-Graber, J., and Okazaki, N. (eds.), Proceedings of the 61st Annual Meet- ing of the Association for Computational Linguistics (Vol- ume I: Long Papers), pp. 9802-9822, Toronto, Canada, July 2023. Association for Computational Linguistics. URL https:
Koli, V., Yuan, J., and Dasgupta, A. Sensemaking crisis information. In Blodgett, S. L.. Cercas Curry. A., Dev, S., Madaio, M., Nenkova, A., Yang, D., and Xiao, 7. (eds.), Proceedings of the Third Workshop on Bridging Human Computrr In- teraction and Natural Language Processing. pp. 74-
of socially-mediatee
doi: 10. 10.:86655/1122023aa--oogog / //aclanthologe wwwp:ooooooooo....oooo/o wrg. acl long. 546/.
81, Mexico City, Mexico, June 2024. Association for Computational Linguistics. doi: 10. 10.18453//1/2024 heinlp- 1.7. URL https:://// /tapps::/cclle http://aaaaatttllo...a..o org/ 2024. www.ciiin www.hhnnnnnn......
Muennighoff, N., SU, H., Wang, L., Yang, N., Wei, F., Yu, T., Singh, A., and Kiela, D. Generative repre- sentational instruction tuning. In The Thirteenth In- ternational Conference on Learning Representations, 2025. URL / http:///ppeeeeeeeeeeppppeeoeeeep net/forutt id=BC41IvfS2v.
Kwon, W., Li, Z., Zhuang, S., Sheng, Y., Zheng, L., Yu, C. H., Gonzalez, F.,, Zhang, H., and Stoica, I. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposhum on Operating Systems Principles, 2023.
Ni, J., Qu, C., Lu, J.,, Dai, Z.,, Hernandez Abrego, G., Ma, J., Zhao, V., Luan, Y., Hall, K., Chang, M.-W., and Yang, Y. Large dual encoders are generalizable retrievers. In Gold- berg, Y., Kozareva, Z., and Zhang, Y. (eds.), Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 9844-9855, Abu Dhabi, United Arab Emirates, December 2022. Association for Compu- tational Linguisties. doi: 10.18653/v1/0222.cmmm main. 669. URL http:p::///aaapapaaeae www.e/oogyyrrgg///1 2022. emn 1 ma.in.iammmmmmmmmmmmmm 669/.
Lee, C., Roy, R., Xu, M., Raiman, J., Shoeybi, M., Catan- zaro, B., and Ping, W. NV-embed: Improved techniques for training LLMs as generalist embedding models. In The Thirteenth International Conference on Learning Representations, 2025, URL https: /ppp//eeeeeeeeee/////ee fost/forum?ii i d= gsyLSsDRe.
Li, J., Armandpour, M., Mirzadeh, S. I., Mehta, S., Shankar, V., Vemulapalli, R., Tuzel, O., Farajtabar, M., Pouransari, H., and Faghri, F. Tic-LM: A multi-year benchmark for continual pretraining of language models. In NeurIPS 2024 Workshop on Scalable Continual Learning for Lifelong Foundation Models, 2024. URL https:// openreview
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., Desmaison, A., Köpf, A., Yang. E. Z., Detioo, Z., Rai- son, M., Tejani, A., Chilamkurthy, S., Steiner, B., Fang, L., Bai, J., and Chintala, S. PyTorch: An imperative style, high-performance deep learning library. In Wallach, H. M., Larochelle, H., Beygelzimer, A., d' Alché MicééBuc,, F., Fox, F. B., and Garnett, R. (eds.), Advances in Neural In- formation Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurlPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pp. 8024 8035, 2019.
Li, Z., Zhang, X., Zhang, Y., Long. D., Xie, P., and Zhang, M. Towards general text embeddings with multi-stage contrastive learning. arXiv preprint arXiv:2308.0328/ 2023.
Liska, A., Kocisky, T., Gribovskaya, E., Terzi, T., Sezener, E., Agrawal, D., De Masson D' Autume, C., Scholtes, T., Zaheer, M., Young. S., Gilsenan-Memahon. E.,, Austin, S., Blunsom, P., and Lazaridou, A. StreamingQA: A benchmark for adaptation to new knowledge over time in question answering models. In Chaudhuri. K., Jegelka, S., Song, I.,, Szepesvari, C., Niu, G., and Sabato, S.
Robertson, S. E. and Walker, S. Some simple effective approximations to the 2-poisson model for probabilis- tic weighted retrieval. In Croft, W. B. and van Rijsber- gen, C. J. (eds.), Proceedings of the 17th Annual Interna-

---

From RAG to Memory: Non-Parametric Continual Learning for Large Language Models
tional ACM-SIGIR Conference on Research and Develop- ment in Information Retrieval. Dublin, Ireland, 3-6 July 1994 (Special Issue of the SIGIR Forum), pp. 232-241. ACM/Springer. 1994. doi: 10.1007/978-1 1187788--44447 1-4199778---471-29999 .24
Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtow- icz, M., and Brew, J. Huggingface's transformers: State-of-the-ar natural language processing. CoRR, abs/1910.03771, 2019. URL Abs/1910.0377
Roth, K., Udandarao, V., Dziadzio, S., Prabhu, A., Cherti, M.. Vinyals, O., Henaff, O. J., Albanic, S., Bethge, M., and Akata, Z. A practitioner's guide continual multi- modal pretraining. In NeurlPS 2024 Workshop on Scal- able Continual Learning for Lifelong Foundation Models, 2024. URL https: http://tppppeeeeeeeeeeeee.eeeeep httpeeereeieee.ee//oortrtt id=gkyosluSbR,
Xie, J., Zhang, K., Chen, J., Lou, R., and Su, Y. Adaptive chameleon or stubborn sloth: Revealing the behavior of large language models in knowledge conflicts. In The Twelfth International Conference on Learning Represen- tations, 2024. URL htttps:://ppeeeeee /pwp///eppppeeeeeeee/eeeee net/ Forum?1d=auuAU
Sarthi, P.. Abdullah, S., Tuli, A., Khanna, S., Goldie, A., and Manning, C. D. RAPTOR: recursive abstractive processing for tree- organized retrieval. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenRe- view.net, 2024. URL httph::/tpppeeeeeeeeeeee /owpr//eeeeeeeeeeeeeeeeo .net/ forum? d=GN921 JHCRW.
Yao, Y., Wang. P., Tian, B., Cheng, S., Li, Z., Deng, S., Chen, H., and Zhang, N. Editing large language models: Problems, methods, and opportunities. In Bouamor, H., Pino, J., and Bali, K. (eds.), Proceed- ings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 10222-10240, Singapore, December 2023. Association for Computa- tional Linguisties. doi: 10.18653/1/22233.coom main. 632. URL https: /tc///clla hww.:lllgg.ooooo.o.oeoee. or0/2023. 2023. emnlp main. 632/.
Shi, H., Xu, Z., Wang, H., Qin, W. Wang, W., Wang, Y., Wang, Z., Ebrahimi, S., and Wang, H. Continual learning of large language models: A comprehensive survey. arXiv preprint arXiv:2404. 16789, 2024.
Yuan, T., Ning, X., Zhou, D., Yang, Z., Li, Zhuang, M., Tan, Z., Yao, Z., Lin, D., Li, B., Dai, G., Yan, S., and Wang. Y. LV-EEal: A balanced long-context benchmark with 5 length levels up to 256k, 2024. URL https: //arxiv. org/ abs / 2402.05136. 05136.
Suzuki, W. A. Associative learning and the hippocampus. Psychological Science Agenda, February 2005.
Thakur, N., Reimers, N., Rücklé A., Srivastava, A., and Gurevych, BEIR: A heterogeneous benchmark for zero-shot evaluation of information retrieval models. In Thirty-fitth Conference on Neural Information Process- ing Systems Datasets and Benchmarks Track (Round 2), 2021. URL https::/ http://ppp..eeeeeeeeeee / forum? id=wCu6T5xF
Zhang, H., Gui, L., Zhai, Y., Wang, H., Lei, Y., and Xu, R. Copr: Continual learning human preference through optimal policy regularization, 2024. URL https:// arxiv. arxiv.org/ wwwg/aas////oooooooooeooooeeeeee 1369..161........ 15694.
Zhang, Z., Fang, M., Chen, L., and Namazi-Rad, M.- R. CITB: A benchmark for continual instruction tuning. In Bouamor, H., Pino, J., and Bali, K. (eds.), Findings of the Association for Computational Linguistics: EMNLP 2023, pp. 9443-9455, Singa- pore, December 2023, Association for Computational Linguistics. doi: 1.116633/1/2223..mdiins----sss 633. URL https: http://llaaathhccc org/ 2023.. eeriggs-eenll.ccm 633/.
Trivedi, H., Balasubramanian, N., Khot, T., and Sabharwal, A. MuSiQue: Multihop questions via single-hop ques- tion composition. Transactions of the Association for Computational Linguistics, 10:539-554, 2022, doi: 10. 0002/aclLa_00475 URL https:://///// /aclaaatnannoo...... org/2022. tac.222..
Uner, O. and Roediger III, H. L. Do recall and recognition lead to different retrieval experiences? The American Journal of Psychology, 135(1):33-43 2022.
Zhong, Z., Wu, Z., Manning, C., Potts, C., and Chen, D. MQUAKE: Assessing knowledge editing in lan- guage models via multi-hop questions. In Bouamor, H., Pino, J., and Bali, K. (eds.), Proceedings of the 2023 Conference on Empirical Methods in Nat- ural Language Processing, pp. 15686-15702, Sin- gapore, December 2023. Association for Computa- tional Linguisties. doi: 10.18653//11/22233emmlg 971. URL emnlp 1allp-main.... main.971/.
Wang, Y., Ren, R., Li, J., Zhao, X., Liu, J., and Wen, REAR: A celevance-aware sarieval-augmente frame- work for open-domain question answering. In Al- Onaizan, Y., Bansal, M., and Chen, Y. (eds.), Proceed- ings of the 2024 Conference on Empirical Methods in Natural Language Processing, EMNLP 2024, Miami, FL, USA, November 12-16, 2024, pp. 3pp.5613-555 5613-5626. Association for Computational Linguistics, 2024. URL https:// agl antho wttg 2224. wwnnmmmmmmmm 1o-maiin in 321.
12

---

From RAG to Memory; Non-Parametric Continual Learning for Large Language Models
Appendices
Within this supplementary material, we elaborate on the following aspects:
Appendix A: LLM Prompts
Appendix B: HippoRAG Pipeline Example
Appendix C: Detailed Experimental Results
Appendix D: Graph Statistics
Appendix E: Error Analysis
Appendix F: Cost and Efficiency
Appendix G: Implementation Details and Hyperparameters
A. LLM Prompts
We show LLM prompts for triple filter in Figure 4, including the instruction, the few-shot demonstrations and the input format.
B. Pipeline Example
We show pipeline example of HippoRAG 2 online retrieval Figure 5, including query-to-triple, triple filtering and using seed nodes for PPR.
C. Detailed Experimental Results
We show QA performance and retrieval performance with the proprictary model GPT-4o-mini as well as more metrics here, as shown in Table 8 and Table 9.
QA Performance shown in Table 8, when using GPT-4o-mini for indexing and QA reading, HippoRAG consistently achieves competitive EM and F1 scores across most datasets. Notably, leads in the MuSiQue and 2Wiki benchmarks. Our method also demonstrates superior performance in the Narrative and LV-Eval tasks. When compared the strong NV-Embed-v2 retriever, HippoRAG 2 exhibits comparable or enhanced Fl scores, particularly excelling in the LV-Eval dataset with reduced knowledge leakage.
Retrieval Performance As shown in Table 9, the improvement trend of HippoRAG 2 in recall @2 is similar to that in recall@5.
D. Graph Statistics
We show the knowledge graph statistics using Llama-3.3-70B Instruct- or GPT-4o-mini for OpenIE in Table 10.
E. Error Analysis
We provide an error analysis of 100 samples generated by HippoRAG 2 with @5call less than 1.0. Among these samples, 26%, 41%, and 33% are classified as 2-hop, 3-hop, and 4-hop questions, respectively. Triple filtering and the graph search algorithm are the two main sources of errors.
Recognition Memory In 7% of the samples, phrase from the supporting documents is matched with the phrases obtained by the query-to-trille stage before triple filtering. In 26% of the samples, no phrase from the supporting documents is matched with the phrases after triple filtering. After the triple filtering step, 8% of the samples show a decrease in the proportion of phrases in the triples that match phrases from the supporting passages. For instance, the first case from Table 11 shows an empty list after triple filtering, which eliminates all relevant phrases. Additionally, 18% of the samples are left with zero triples after filtering. Although not necessarily an error filtering, this indicates that the attempt to link to
13

---

From RAG to Memory; Non-Parametric Continual Learning for Large Language Models
Triple Filter
Instruction:
You critical component high-stakes question-answering system used by top researchers and decision- mersionnmmaeess worldwide, Your task is to filter facts hased on their relevance to given query, ensuring that the most crucial information is presented to these stakeholders. The query requires careful analysis and possibly multi-hop reasoning to connect different pieces information.
must select up relevant facts from the provided candidate that have strong connection the query, aiding in reasoning and providing an accurate answer.
The output should be in JSON format, egg. ('fact': [["s1", "p1", *ol"], ["s2", "p2", '22 Ill, and if no facts are relevant, return an empty list, I'fact": I]].
accuracy your response is paramount,
directly impact the decisions made these high-level
stakeholders, You must only use facts from the candidate list and not generate new facts. The future of critical decision- making relies on your ability to accurately filter and present relevant information.
Demonstration:
Question: Are Imperial River (Florida) and Amaradia (Doli) both located in the same country? Fact Before Filter: f'fact:; [l'mmeeria river", "is located in', 'florida"). l'imperial river", "is river in', "united states") ('imperial river", "may refer to', 'south america") I'amaradia', 'flows through', "ro de amaradia") ('imperial river', "may refer to', 'united bates"]]]*
Fact After Filter. ""Yact' [I'mperrial river", "is located in', "florid""" J.[[ ['imperial river", river in', "united states"|[" ll'amaraiia" flows through',
de amaradia"
Question: When is the director of film The Ancestor 's birthday?
Fact Before Filter: 'I' fact:: [['jean jacques annaud", "born on', '1 october 1943"), ['tsui hark', "born on", "15 february 1950"), """0"")''ppaai" l'pablo trapero', "born on", october 19 I'the ancestor', "directed by", 'guido brignone"), ["benh zeitlin', "born on', 'october 1982']]
Fact After Filter:" "['aatt']["" [["the ancestor", directed by" "guido brignone"]]
Question: what geographic region the country where Teafuone located?
Fact Before Filter: 'I' fact:: [|'teafuaniua' "is the', "east"), ['motuloa", "lies between", "teafuaniua", ['motuloa", 'lies between', "teafuanonu") I ['teafuone", "is", "islet"]. ['teafuone', 'located in', "nukufetau"]]] Fact After Filter. ['fact's[[ t(taffuno "aaffuue""""s""" "is"" "islee""]]]] ["teafuone" 'located in", "nukufetau "I]""
Question: When did the director film S.O.B. (Film) die? Fact Before Filter: f'fact": [['allan dwan',
on", "28 december 1981"), o o
"written and directed by", "blake
edwardss], ('robert aldrich', 'died on", "december 1983"), J, ['robert siodmak", "died on', march 1973"), ['bernardo bertolucci', 'died on", '26 november 2018']]]
Fact After Filter: [['actt:[[''s Is's "written and directed by", "blake edwards'
Question: Do both films: Gloria (1980 Film) and A New Life (Film) have the directors from the same country? Fact Before Filter: 'I' fact": [['sebasti n lelio watt', 'received acclaim for directing", "gloora"], I'gloria", " 'is', "1980 american thriller crime drama film"), I ('a brand new life', 'is directed by", "ounie lecomte"), ""gloria", "written and directed by", "john cassavetes") ]], ['a new life", 'directed by", 'alan alda')] (]]
After Filter: "['actt:]l's [['goorroo ""''''"''''"""""""""""" (198 american thriller crime drama film') [][ I''gloria", written and directed by") /john cassavetes
life' dirreced 'alan alda"]]]'
Question:
What the date death of the director of film The Old Guard (1960 Film)
Fact Before Filter: 'I fact": the old guard", "is", "1960 french comedy film'), l'gilles grangier", "directed", 'the old guard'], J, l'the old guard', 'directed by", 'gilles grangier"). J. ['the old fritz', 'directd by", "gerhard lamprecht"). l'oswald albert mitchell", "directed", 'old mother riley series of films']]) Fact After Filter: "["fact'[["thhe [["'he guard" "is", 1990 french comedy film Io[ gilles grangier' "directed", "ateed"""""tteea old guard') I[ the old guard', "directed by", 'gilles grangier") ']]!"
Question: When is the composer film Aulad (1968 Film) birthday? Fact Before Filter: 'a'fatt: [['aulad", "has music composed by', 'chitragupta shrivastava") ['aadmi sadak ka", "has music by', "ravi"]. ['ravi shankar sharma", "composed music for", 'hindi films'], ['gulzar", "was born on', '18 august 1934"], ['aulad', 'is a", "1968 hindi language drama film') ']]
Fact After Filter. I[ "factt:["aaulae haas music composed by", "chitragupta shrivastava "J !''alla'', d 'is "1'"1968 hindi language drama film") I]]"
Input:
Question: []
Fact Before Filter: ()
Fact After Filter:
Figure 4. L.M prompts for triple filtering (recognition memory).
14

---

From RAG to Memory; Non-Parametric Continual Learning for Large Language Models
Query to Triple
Question Triples
What county Erik Hort's birthplace a part of?
("Erik Hort', "born in', "Montebello"), ('Erik Hort", "born in", "New
York"), ("Erik Hort', as,"", "American"), ("Erik Hort", "born on", "February 16, 1987"), ("Erik Hort", "is "Soccer player")
Filtered Triples York")
("Erik Hort', "born in', "Montebello"), ("Erik Hort", "born in", "New
PPR Seed Nodes
Seed Phrase Nodes Seed Passage Nodes (Title) ("Erik Hort", 0.05), ("Horton Park (Saint Paul, Minnesota)', 0.031), ("Hertfordshire", 0.028),
("Montebello", 1.0), ("Erik Hort", 0.995), ("New York", 0.989)
Returned Top Passages
*Top-ranked nodes from PPR are highlighted.
"Final answer to this question is highlighted.
1. Erik Hort
Erik Hort (born February 16, 1987 in Montebello, New York) is American soccer player who currently Free Agent.
2. Horton Park (Saint Paul, Minnesota)
Horton Park is small arboretum in Saint Paul, Minnesota, United States. Known primarily for its variety of trees, Horton Park has become symbol of the Saint Paul Midway community.
3. Montebello, New York
Montebella (Italian: 'Beautiful mountain") is an incorporated village in the town of Ramapo, Rockland County, New York, United States. It is located north of Suffern, cast of Hillburn, south of Wesley Hills, and west of Airmont. The population was 4,526 at the 201 census.
4. Hertfordshire
Hertfordshire is the county immediately north of London and is part of the East of England region, mainly statistical unit. A significant minority of the population across all districts are City of London commuters. To the east Essex, to the west is Buckinghamshire and to the north Bedfordshire and Cambridgeshire.
5. Hull County, Quebec
Hull County, Quebec is an historic county of Quebec, Canada. It was named after the town of the same name (Hull the north shore of the Ottawa River and is part of the Outaouais, one of roughly 12
soggston-upon-ull]) in East Yorkshire, England. It located on
historical regions of Québec.
Figure example of HippoRAG pipeline.
15

---

From RAG to Memory; Non-Parametric Continual Learning for Large Language Models
Table QA performance (EM Fl scores) on RAG benchmarks. No retrieval means evaluating the parametric knowledge of the readers. HippoRAG (and HippoRAG 2) uses the denoted LLM for OpenIE (triple filtering) and QA reading.
Discourse Understanding NarrativeQA
Simple QA
Multi-Hop QA
Retrieval
PopQA
MuSiQue
2Wiki
HotperQA
LV-Eval
Avg
Llama-3.3 3.3-70B -.-rtut-nntttuctt
None
40.2 54.9
28.2 32.5
17.6/ 26,
36.5 42.88
37.0/47.
4.0
6.0
3.4/ 12.9
29.7 /38.4
Contriever (Izacard et al., 2022)
45.07 58.9
41.67
24.0/31.3
38.1 41.9
51.3/62.3
5.7/
6.5/
37.41 46.9
BM25 (Robertson & Walker, 1994)
11.7/59.0
39.1/
20.3/ 28.8
17.9/51.2
52.0/63.4
4.0 5.9
4.17 18.3
38.0/-17
GTR (T5-base) (Ni et al., 2022)
45.5/59.9
43.2/56.
24.8/34.6
19.2/5288
50.6/628
4.8/
6.8/ 19.9
40.0 50.4
Sww--wee--B--nnseeee
(Li
2023)
16.6/62.0
13.5/
30.6/ 10.9
55.1/60.0
58.6/T1.0
5.71
7.9/213
43.8 5
GriLLM-7B (Muennigholl' et al., 2025)
66.8/613
42.8/35.8
33.6/44.8
65.8/60.6
60.7/ 73.3
7.7/ 9.8
8.2/23.9
44.9 56.1
NV WV-Fimee--v2 (7B) (l.ee al., 2025)
41.3/6199
42.9/55.7
34.7/45.7
57.5/61.5
62.8/ 75.3
7.3/
8.9/
45.9 57.0
RAPTOR (Sarthi et al., 2024)
50.7
43.1/56.2
20.7 28.9
47.3/52.1
66.8/69.5
2.4/ 5.0
5.1/214
38. 48.8
GripRRAG AG (Fdgee et al., 2024)
30.8/46.9
31.4748.1
27.3/38..
58.4
55.2/68.6
4.8 11.2
6.6
36.7
LightRAG (Gvo et al., 2024)
8.6/ 16.6
2.17 2.4
0.5 1.6
9.47 11.6
2.0 2.4
0.8
1.0
1.07 3.7
4.2 6.6
HippeRAG (Gutiárrez
2024)
13.0/
12.7/55.9
26.2/35.11
65 71.8
6.5
14/
42.8 13.11
HippoRAG2
48.6 63.3
42.97562
37.2 48.488
65.0/710
62.7 75.5
9.7 12.9
8.9 25.9
48. 18.01 /
GPT-4o-mini
35.2/
11.2/22.0
30.2/36.3
21.6/41.0
2.7/14.1
22.6/33.1
NV-Embed-v2 (7B) (Lec et al., 2025)
43.5/59.9
41.755.8
32.8/46.0
64.4/60.8
57.3/71.0
7.37 10.0
5.1/24.2
42.9 /
RAPTOR (Sarthi et al., 2024)
37.8/515
11.9/5.11
27.7/
39.7/4484
50.6/64.7
5.67 9.2
11/2288
36.91 19.7
GraphRAGi (Eigge et 2024)
38.0/5355
30.7
27.0/
45.7/61.0
51.4/67.6
4.9/11.0
5.4/20.9
36.0 /6.0/52. 52.6
LightRAG (Guo et al., 2024)
2.8/15.4
1.9/14.8
2.0/ 9.3
2./12.1
9.9/20.2
0.9.
1.07 9.0
3.6/
HippoR AG (Gutiérrez
2024)
37.2/52.2
42.5/56. 56.2
24.0/35.9
50.4/67.3
46.3/60.0
4.8.
2.1/
38.9/51.2
HlppoRAG
43.4/6000
45.7/55.7
35.0/49.3
60.5/69.7
56.17111
10.5/14.0
5.8/
4 /8.1
the triples has failed, where HippoRAG 2 directly uses the results from dense retrieval as substitute. Overall, though recognition memory is an essential component, the precision of the triple filter has room for further improvement.
Graph Construction Graph construction is challenging to evaluate, but we find that only 2% the samples do not contain any phrases from the supporting passages within the one-hop neighbors of the linked nodes. Given our dense-sparse integration, we can assume that the graphs we construct generally include most of the potentially exploitable information.
Personalized PageRank In 50% of the samples, at least half of the linked phrase nodes appear in the supporting documents. However, the final results remain unsatisfactory due to the graph search component. For example, in the second case from Table I, the recognition memory identifies the key phrase "Philippe, Duke of Orléans" from the query, but the graph search fails to return perfect results among the top-5 retrieved passages.
F. Cost and Efficiency
For LLM deployment, tensor parallelism via vLLM (Kwon et al., 2023).
Llama-3.3-70B--nsttee
machine equipped with four NVIDIA H100 GPUs, utilizing
For detailed comparison with our baselines, we track the computational resources (# of tokens, indexing time, time per query and GPU memory) usage when indexing and performing QA on the MuSiQue corpus (11k documents) using the 1omma-3.--7BBInsttie model. We compare HippoRAG 2 against NV-Embed-v2 (Lee et al., 2025), RAPTOR (Sarthi et al., 2024), LightRAG (Guo et al., 2024), HG (Gutiérrez et al., 2024) and GraphRAG (Edge al., 2024) Table 12. For the memory requirements, we ignore all memory for model weights since it shared across all systems.
HippoRAG 2 not only outperforms these RAG methods in QA and retrieval performance but also much fewer tokens compared to LightR AG and GraphRRA. In terms of time, we note that HippoR AG 2 is much more efficient than GraphR and Lig while only being slightly less efficient than both RAPTOR and HippoRAG, HippoRAG 2 's use of fact embeddings does increase its memory requirements compared to our baselines, however, we believe that this an acceptable tradeoff given our method's performance benefits. Additionally, while all approaches lag behind standard RAG in terms of time and memory efficiency, HippoRAG is the only one that outperforms this strong baseline substantially.
16

---

From RAG to Memory: Non-Parametric Continual Learning for Large Language Models
Table Passage @ / @5 RAG henchmarks. * denotes the report from the original paper while we reproduce the HippoRAG
results with aligned LLM and retriever.
Simple
Multi-hop 2Wiki
NQ
PopQA
MuSiQue
HotpotQA
Avg
Simple Baselines
Contriever (Izacard al., 2022)
54.1
27.0/ 43.2
34.8/ 46.6
46.6/ 57.5
58.4/ / 75.3
39.2 / 55.4
BM25 (Robertson & Walker, 1994)
28.2 / 56.1
24.0/ 35.7
32.4/ 43.5
55.3/ 65.3
57.3 74.8
39.4 55.1
GTR (T5-base) (Ni et al., 2022)
35.0/ 63.4
40.1 / 49.4
37.4/ 49.1
60.2 67.9
59.3 73.9
46.4/ 60.7
Large Embedding Models
GTE- TIE--wee2-77B Instruct (Li et al., 2023)
44.7/74.3
47.7 / 50.6
48.1 / 63.6
66.7 / 74.8
75.8 89.1
56.6 / 70.5
GritLMM-B (Muennighofff et al., 2025)
46.2/ 76.6
50.1
49.7 / 65.9
67.3 76.0
79.2
57.3 / 72.2
NV-Embed-v2 (7B) (lee et al., 2025)
4 75.4
45.3/51.0
52.7 69.7
67.1 / 76.5
84.1 94.5
58.9 / 73.4
Structure-augmented
RAG
RAPTOR (GPT-4o-mini)
40.5/ 69.4
$7.2/48.1
19.1/61.0
58.4/ 66.0
13.6/90.2
/ 67.0
RAPTOR (Llama-3.3-708- lnsttrtt))tt
40.3/ 68.3
40.2/ 48.7
47.0/57.8
58.3 66.2 66.2
76.8 86.9
52.5 / 65.6
HippoRAG* (Gutiérrez et al., 2024)
40.9/51.9
7
60.5 77.7
HippoRAG (GPT-4o-mini)
21.6/4..1
36.5/52.2
/
68.4/87.0
60.1/78.5
45.7 63.0
HippoRAG (Llama-3.3-700) 1.lama-3..-70--nnstttee Insttrrttt
21.3/44.4 44.4
10.0/53.8
43.2/53.2
71.9
60.4/77.3
47.0 63.8
HippoRAG 2 (GPT-4o-min))
44.4/76.4
52.5
53.5 74.2
74.6 90.2
80.5/95.7
59.3 77.7
HippoRAG (Llama-3.3-70B 3.3-70B-Instructt
45.6/ 78.0
43.9/51.7
56.1 / 74.7
76.2 / 90.4
83.5 / 96.3
61.1 78.2
G. Implementation Details and Hyperparameters
G.1. HippoRAG 2
We provide detailed explanation of the PPR initialization process used in HippoRAG 2 here. The key goal is to determine the seed nodes for the PPR search and assign appropriate reset probabilities to ensure an effective retrieval process.
Seed Node Selection The seed nodes for the PPR search are categorized into two types: phrase nodes and passage nodes. All the scores given by the embedding model below use normalized embedding to calculate. 1) Phrase Nodes: These seed nodes are selected from the phrase nodes within the filtered triples, which are obtained through the recognition memory component. If recognition memory gives an empty triple list and no phrase node is available, HippoRAG 2 directly returns top passages using the embedding model without any graph search. Otherwise, we keep at most 5 phrase nodes as the seed nodes, and the ranking score of each phrase node is computed as the average score of all filtered triples it appears in. 2) Passage Nodes: Each passage node is initially scored using an embedding baased similarity, and these scores are processed as follows. All passage nodes are taken as seed nodes since we find that activating a broader set of potential passages is more effective for uncovering passages along multi-hop reasoning chains compared to focusing only on the top-ranked passages.
Reset Probability Assignment After determining the seed nodes, we assign reset probabilitics to control how likely the PPR algorithm will return these nodes during the random walk. The rules are: 1) Phrase nodes receive reset probabilities directly as their ranking scores. 2) Passage nodes receive reset probabilities proportional to their embedding similarity scores, i.e., balance the influence of phrase nodes and passage nodes, we apply a weight factor to the passage node scores. Specifically, the passage node scores are multiplied by the weight factor discussed in Section 6.2. This ensures that passage nodes and phrase nodes contribute appropriately to the retrieval process.
PPR Execution and Passage Ranking Once the seed nodes and their reset probabilities are initialized, we run PPR over the constructed graph. The final ranking of passages is determined based on the PageRank scores of the passage nodes. Top-ranked passages are then used as inputs for the downstream QA reading process. We manage our KG and run the PPR algorithm using the python-igraph library.
By incorporating both phrase nodes and passage nodes into the PPR initialization, our approach ensures a more effective retrieval of relevant passages, especially for multi-hop reasoning tasks.
hnttps://pythen. httpaa:///ytheen.gggghooooor www.gig....//e//steeeee
17

---

From RAG to Memory; Non-Parametric Continual Learning for Large Language Models
Table 10. Knowledge graph statistics using different LLMs for OpenIE. The nodes and triples counted based unique values.
NQ
PopQA
MuSiQue
2Wiki
HotpotQA
LV-Eval
NarrativeQA
Llama-3.3-70B Insttuuut
of phrase nodes
68, 375
76, 539
85, 288
44, 004
81, 200
175, 195
9, 224
# of passage nodes
9, 633
8, 676
156
6, 119
9,811
22, 849
4, 1,111
of total nodes
78, 008
85, 215
96,944
50, 123
91,011
198, 044
13,335 335
# of extracted edges
125, 777
579
830
68, 881
130, 058
314,324
26, 208
of synonym edges
899, 031
845, 014
1, 1112,991 951
593, 298
994, 187
2, 674, 833
72,494
# of context edges
126, 757
118, 909
132, 586
64, 132
122, 437
375, 424
33, 395
of total edges
1, 165
1, 088, 502
1, 1,399,377 367
726, 311
1, 1,246, 682
3, 363 581
132, 097
GPT-4o-mini
of phrase nodes
86,90 904
85, 744
101, 641
49, 544
95, 105
217, 085
15, 365
# of passage nodes
9, 633
676
11,656
6, 119
9,811
22,849
4,111
of lotal nodes
96, 537
42,
113,297
55, 663
104,916
239, 934
19,476
# of extracted edges
114,900 900
108, 989
125,903
62,
119,630
303, 491
24,373
of synonym edges
1, 0944 651
001,525
1, 1,304,60
715, 763
1, 1,1,6,501
3, 268, 084
14,075
# of context edges
142, 419
127, 568
146,293 293
68,348
133,220 220
404, 210
38, 632
# of total edges
1,351,970
494, 082
1,576,801
846, 737
1,379,351
3, 3,975,77 785
77,080
Table 11. Two examples from MuSiQue where passage recall is less than 1.0.
Query
Where is the district that the person who wanted to reform and address Bernhard Lichtenberg's religion preached sermon on Marian devotion before his death located? Saxony-Anhalt
Answer
Supporting Passages (Title)
1. Mary, mother of Jesus 2. Reformation 3. Wittenberg (district) 4. Bernhard Lichtenberg
Retrieved Passages (Title)
1. Bernhard Lichtenberg 2. Mary, mother of Jesus 3. Ambroise-Maric Carré Reformation Henry Scou Holland (Recall@5 is 0.75)
("Berhaard Lichtenberg", "was", "Roman Catholic Priest")
("Bemhard Lichtenberg", "beatified by". "Catholic Church")
("Bernhard Lichtenberg". "died on", "5 November 1943")
Query to Triple (Top-5)
("Catholic Church", "beatified", "Bernhard Lichtenberg")
('Bernhard Lichtenberg", "was", "Theologian")
All above subjects and objects appear in supporting passages
Filtered Triple
Empty
Query
Who is the grandmother of Philippe, Duke of Orléans? Marie de' Medici
Answer
Supporting Passages (Title)
1. Phitippe I, Duke of Orléans 2. Leonora Dori
Retrieved Passages (Title)
1. Philippe Duke Orléans 2. Louise Élisaheth d' d'Oréén 3. Philip of Spain Anna Lorraine
5. Louis Philippe (Recall 0.5)
("Bank of America", "purchased", "Fleethoston Financial")) ("Fleetbuston Financial", "was acquired by" "Bank America")
("Bank of America", "acquired", "Flectboston Financial")
Query to Triple (Top-5)
("Bank of America". "announced purchase of". "Fleetboston Financial")
("Bank of America". "merged with". "Fleetboston Financial")
All above subjects and objects appear in supporting passages
('Bank of America". "purchased", "Fleetboston Financial") ("Fleetboston Financial", "was acquired by", "Bank of America")
Filtered Triple
All abuve subjects and objects uppear in supporting passages
18

---

From RAG to Memory; Non-Parametric Continual Learning for Large Language Models
Table 12. We report the computational resource requirements (indexing tokens, indexing time, time per query, GPU memory requirements
during QA) for the RAG baselines the MuSiQue corpus (11,656 passages). For each metric, we include the percentage that this method obtains with respect the HippoRAG metric (100%).
NV-Fmbed-x2
RAPTOR
LightRAG
GraphRAG
HippoRAG
HippoRAG 2
Input Tokens
1.7M (18.5%)
68.5M (744.6%)
115.5M (1255.4%)
9.2M (100.0%)
9.2M (100.0%)
Output Tokens
0.2M (6.7%)
18.3M (610.0%)
36.1M (1203.3%)
3.0M (100.0%)
3.0M (100.0%)
Indexing Time (min)
12.1 (12.3%)
100.5 (101.0%
235.0 (236.2%)
277.0 (278.4%
57.5 (57 (57.7%)
99.5 (100.0%)
QA Time/Query (sec)
0.3 (25.0%)
0.6 (50.0%)
13.3 (1008.3%)
10.7 (891.7%)
(75.0%)
1.2 (100.0%)
QA GPU Memory (GB)
1.7 (17.2%)
1.4 (14.1%)
4.5 (45.5%)
3.7 (37.4%)
6.0 (60.6%)
9.9 (100.0%)
Hyperparameters
We perform hyperparameter tuning on 100 examples from MuSiQue's training data. The hyperparame-
ters are listed Table 13.
Table 13. Hyperparameters set on HippoRAG 2
Hyperparameter
Value
Synonym Threshold
0.8
Damping Factor PPR
0.5
Temperature
0.0
G.2. Comparison Methods
We use PyTorch (Paszke et al., 2019) and HuggingFace (Wolf et al., 2019) for dense retrievers and BM25s (Lü, 2024) for the BM25 implementation. For GraphRAG (Edge et al., 2024) and LightR AG (Guo et al., 2024), we adhere to their default hyperparameters and prompts. To ensure a consistent evaluation, the same QA prompt that HippoRAG 2 adopts from HippoRAG (Gutiérrez et al., 2024) is applied to rephrase the original response of GraphRAG and LightRAG.
We keep the default indexing hyperparameters for GraphRAG and LightRAG. For QA, we perform hyperparameter tuning on the same 100 samples as Appendix G.I.
Hyperparameters
Table Hyperparameters on GraphRAG and LightRAG
Hyperparameters
GraphRAG
LightRAG
Mode
Local
Local
Response Type
Short phrase
Short phrase
Top-k Phrases for QA
60
Chunk Token Size
1, 200
2,2
Chunk Overlap Token Size
100
100
Community Report Max Length
2, 000
Max Input Length
80000
Max Cluster Size
10
Entity Summary Max Tokens
500
19


Reviews:
[{"review_id": "fUgFBQFu1T", "reviewer": "Reviewer_gi53", "summary": "**Main findings**:  \n\nThe ability to continuously acquire, organize, and leverage knowledge is a fundamental aspect of human intelligence. To empower LLMs with this capability, retrieval-augmented generation (RAG) has emerged as a critical approach. Recent methods enhance vector embeddings by integrating structures such as knowledge graphs, aiming to improve sense-making and associativity. However, these advanced approaches often suffer from significant performance degradation on basic factual memory tasks compared to standard RAG.\n\n**Main algorithmic/conceptual ideas**:  \n\nTo address this issue, we propose **ContinualRAG**, which extends HippoRAG by introducing three key enhancements: **Dense-Sparse Integration**, **Deeper Contextualization**, and **Recognition Memory**.\n\n**Main results**:  \n\nExperiments conducted on diverse benchmarks demonstrate that ContinualRAG outperforms previous state-of-the-art methods such as HippoRAG, GraphRAG, and NV-Embed-v2.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate the reviewer for the time and effort dedicated to reviewing our work. We are delighted that they found our work clear, convincing, well-motivated, technically solid and empirically validated by impressive performance improvements. We address their suggestions and comments below. \n\n## Continual Knowledge Injection Experiments\n\n> Given that ContinualRAG aims to empower LLMs to continuously acquire, organize, and leverage knowledge, additional experiments could be explored to continually update and expand the knowledge graph, which will make this work more solid.\n\nWe appreciate the reviewer’s thoughtful suggestion and agree with their assessment. \n\nTo address this, we conduct a new experiment on both NQ and MuSiQue. We partition our full corpus into four equal segments (approximately 250 questions and their distractors). We then select one segment for evaluation and incrementally add the remaining segments, measuring how performance evolves as new knowledge is added—simulating a temporal continual learning setting. We report the performance of ContinualRAG and NV-Embed-V2, our strongest baseline, in the tables below using F1 scores.\n\n**NQ**\n\n| # of Documents | ContinualRAG| NV-Embed-V2 |\n|----------------|---------------|-----------------|\n| 5,171          | 60.83         | 60.26           |\n| 6,624          | 61.14         | 60.87           |\n| 8,098          | 61.35         | 60.66           |\n| 9,633          | 61.67         | 60.66           |\n\n**MuSiQue**\n\n| # of Documents | ContinualRAG | NV-Embed-V2 |\n|----------------|---------------|-----------------|\n| 3,316          | 52.83         | 47.44           |\n| 5,496          | 52.02         | 45.52           |\n| 8,127          | 49.93         | 44.13           |\n| 11,656         | 44.95         | 40.42           |\n\nAs we can see, ContinualRAG’s improvements over NV-Embed-V2 remain remarkably consistent in both simple and associative continual learning settings. We note that, while both methods show steady performance on simple QA as more knowledge is introduced, their performance drops almost equivalently in the more complex task as more information is introduced. This behavior shows the strength of RAG in simple temporal tasks, highlights the value of our current experimental setup and points to the need for more complex temporal continual learning benchmarks for LLMs.\n\nWe will add this experiment to the camera-ready version of our paper.", "claims_and_evidence": "Yes, the claims are clear and convincing.", "methods_and_evaluation_criteria": "Yes, the proposed methods make sense for the target problems.", "theoretical_claims": "Not applicable. No theoretical claims.", "experimental_designs_or_analyses": "Yes, I have checked the experimental designs.", "relation_to_broader_scientific_literature": "Introducing **ContinualRAG** can empower LLMs to continuously acquire, organize, and leverage knowledge in a human-like manner.", "essential_references_not_discussed": "No additional related works should be included.", "other_strengths_and_weaknesses": "**Strengths**:\n\n1. The proposed ContinualRAG is well-motivated and clearly explained.\n2. ContinualRAG introduces solid technical improvements over HippoRAG.\n3. ContinualRAG achieves impressively strong performance on various RAG benchmarks.\n\n**Weakness**:\n\n1. Given that ContinualRAG aims to empower LLMs to continuously acquire, organize, and leverage knowledge, additional experiments could be explored to continually update and expand the knowledge graph, which will make this work more solid.", "other_comments_or_suggestions": "No additional comments.", "questions_for_authors": "1. My suggestion is to consider conducting experiments that continuously introduce additional knowledge into the KG.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7tSQ41ERgR", "reviewer": "Reviewer_K3hz", "summary": "This paper presents a method to enhance traditional RAG models for large language models. The proposed approach is based on HippoRAG and introduces a combination of phrase nodes and passage nodes, inspired by how human memory represents and processes information at different granularities. Additionally, the method incorporates query-to-triple contextualization, tightly associating user queries with knowledge graph nodes. Experimental results demonstrate that the proposed approach achieves a 7% performance gain over state-of-the-art embedding models in associative memory while maintaining superior factual recall and discourse understanding.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate the reviewer’s detailed comments and questions, they will surely enhance the quality of our work. We are happy to know that the reviewer found our human memory inspired methodology well-motivated and our experimental settings meaningful, valid and sound. \n\n## Non-Parametric Continual Learning for LLMs: Ideal Experiments\n\n> If this method is designed for continual learning, the experimental setup should be revised to incorporate a temporal dimension.\n\nWe appreciate the reviewer’s suggestion to add a temporal dimension to our experiments; however, we believe that our current setup appropriately evaluates the continual learning abilities of non-parametric methods given their unique strengths and limitations.\n\nAs explained in our paper, non-parametric methods have become the de facto continual learning solution for LLM due to their simplicity and strong performance. In standard continual learning benchmarks, which measure catastrophic forgetting and simple factual learning, standard RAG outperforms parametric alternatives like model editing and continual pretraining by substantial margins (MQuAKE, EvolvingQA).\n\nIn contrast to their strong performance in these simpler settings, non-parametric methods struggle with richer forms of continual learning. More specifically, given that standard RAG acquires isolated knowledge, it is limited in its capacity to enable complex tasks over new knowledge, such as associativity and sense-making. Our experimental setup, which consists of a set of such tasks, is thus designed to explore this limitation in non-parametric continual learning methods.\n\nThat said, we agree that assessing performance as more knowledge is incrementally introduced would be a valuable addition to our paper. We refer the reviewer to the *Continual Knowledge Injection Experiments* section in our response to **reviewer gi53** above for experimental details, results and discussion.\n\nWe will add this discussion and experiment to the camera-ready version.\n\n## ContinualRAG’s Computational Overhead\n\n> How much additional time and computational resources does this approach require compared to the existing models used for comparison?\n\nWe appreciate the reviewer’s question concerning ContinualRAG’s efficiency compared to our baselines. To address this, we report the time and memory resources required for offline indexing and online retrieval, we will add them to Appendix F alongside the token-level costs reported in Table 12. For indexing, we indexed 11k documents using a Llama-3.3-70B model using vLLM on 4 H100s. For the memory requirements, we ignore all memory for model weights since it is shared across all systems. \n\n| Model        | Indexing Time (min) | QA Time per Query (s) | QA Memory (GB) |\n|---|---|---|---|\n| NV-Embed-V2  | 12.12               | 0.33                  | 1.7            |\n| RAPTOR       | 100.50              | 0.61                  | 1.4            |\n| GraphRAG     | 276.95              | 10.70                 | 3.7            |\n| LightRAG     | 234.95              | 13.31                 | 4.5            |\n| HippoRAG     | 57.50               | 0.90                  | 6.0            |\n| ContinualRAG | 99.50               | 1.15                  | 9.9            |\n\n\nAs we can see, in terms of time, ContinualRAG is much more efficient than GraphRAG and LightRAG and only slightly less efficient than both RAPTOR and HippoRAG. For memory usage, ContinualRAG’s use of fact embeddings does increase its requirements, however, we believe this is acceptable given its performance benefits. Additionally, while all approaches lag behind standard RAG in terms of time and memory efficiency, ContinualRAG is the only one that outperforms this strong baseline substantially.\n\n## Novelty Concerns \n\n> The originality of this work is somewhat limited, as it primarily modifies existing modules in HippoRAG rather than introducing an entirely new framework.\n\nWe appreciate the reviewer’s perspective, however, we would like to clarify our view on our work’s originality. While it is true that we build upon the existing HippoRAG framework, our approach systematically explores and enhances key components of the system in ways that are both principled and non-trivial. The space of possible design choices in structure-augmented RAG systems is vast, and identifying which modules to modify and how to do so effectively is itself a meaningful research contribution.\n\nOur modifications are not superficial; rather, they are informed by clear hypotheses about the limitations of the original modules and are validated through rigorous ablation studies (Table 4), which demonstrate substantial and consistent improvements in performance. We believe that such targeted, evidence-based improvements can be as impactful as proposing entirely new frameworks, especially when they advance the capabilities of an already influential baseline.", "claims_and_evidence": "The dense-sparse integration, inspired by human memory processing at different granularities, is well-motivated. However, it seems questionable to connect all edges to phrases linked to passages. While this approach may work well for small documents, it poses a scalability issue, when dealing with numerous documents, the KG can grow excessively large. Moreover, the method does not account for temporal changes in knowledge, which could limit its practical applicability.", "methods_and_evaluation_criteria": "- The evaluation methods and datasets used for comparison with existing RAG systems are meaningful. However, they do not adequately address experiments related to continual learning.\n\n- If the proposed method is intended for continual learning, it is crucial to evaluate how the knowledge graph updates over time such that new documents are continuously introduced. This includes its impact on offline indexing, search, and question-answering.\n\n- Additionally, the proposed method modifies only certain modules of the existing HippoRAG framework. As a result, it does not qualify as a novel framework, as suggested in the conclusion.", "theoretical_claims": "There are no theoretical claims in this paper.", "experimental_designs_or_analyses": "- If the proposed method is intended for RAG-based question answering systems, the experimental design and ablation studies appear valid and sound.\n\n- However, since this approach stores more information in the KG compared to the existing structure-augmented RAG, an analysis is required to determine how much additional storage and computational resources are needed.", "relation_to_broader_scientific_literature": "- The proposed method improves entity-centric retrieval in HippoRAG and enhances alignment in knowledge graphs. Experimental results indicate that the QA performance improves by approximately 3 percentage points compared to SOTA models.", "essential_references_not_discussed": "- Most of the relevant studies are appropriately cited.", "other_strengths_and_weaknesses": "- Strength: The study effectively integrates cognitive-inspired memory representation into the RAG model, reflecting how the brain processes information at different levels of granularity.\n\n- Weakness: The originality of this work is somewhat limited, as it primarily modifies existing modules in HippoRAG rather than introducing an entirely new framework.", "other_comments_or_suggestions": "- If this method is designed for continual learning, the experimental setup should be revised to incorporate a temporal dimension.\n\n- If the paper focuses on traditional RAG systems, the study should emphasize document storage, retrieval, and utilization rather than continual learning.", "questions_for_authors": "- How much additional time and computational resources does this approach require compared to the existing models used for comparison?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "wpGMsREVax", "reviewer": "Reviewer_jXKH", "summary": "This paper proposes ContinualRAG that improves the performance of RAG on natural question answering and multi-hop reasoning benchmarks.\n\nThe method builds on the prior work, HippoRAG, which performs reasoning on a knowledge graph constructed at an offline phase. In the offline indexing phase, HippoRAG extracts knowledge triples of named-entities and detects synonyms to create additional edges in the knowledge graph. In the online retrieval phase, a query is deconstructed to its named-entities, the knowledge graph is queried for the named entities and their synonyms, then Personalized PageRank is used to retrieve the information from the graph as the response to the query.\n\nThis work argues HippoRAG is limited because it is entity-centric and loses information both during indexing and inference. As such, ContinualRAG makes the following modifications to HippoRAG:\n1) Adds passage nodes to the knowledge graph that connect to named-entity nodes in the passage with context edges. The goal is to have more contextual information in the KG.\n2) Improves the linking of queries to the KG from HippoRAG’s named-entity-recognition to a Query-to-triple approach which matches the entire query to triples in the graph using text embeddings.\n3) Improve the retrieval step to two 1) retrieve top-k triples from the graph using an embedding model and 2) filter triples using an LLM.\n\nGiven these improvements, for QA evaluations the online retrieval of ContinualRAG involves assigning scores to retrieved passage and seed nodes and then executing Personalized PageRank (PPR) to retrieve the top-ranked passages as the answer to the question.\n\nThe evaluations are divided into QA and retrieval evaluations. On QA evaluations, they demonstrate on average 6.7% improvement on HippoRAG. Their improvement brings them above the performance of large embedding models utilizing 7B LLMs by 2.8%. On retrieval evaluations they achieve 15% improvement on HippoRAG and nearly 5% better than large embedding models.\n\nThe ablations in Table 4 demonstrate the importance of all 3 proposed modifications. Particularly, the query-triple approach for linking the queries to the KG accounts for the majority of the improvement on most evaluations (15% on average) except for 2Wiki. The other two modifications of adding passage nodes (6% on average) and LLM filtering in retrieval (0.7% on average) are also effective.\n\n## Update after rebuttal\nI thank authors for their response and recommend incorporating the clarifications into a revision. I maintain my rating of weak accept conditioned on applying needed improvements to the text to improve clarity.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to thank the reviewer for thoroughly reviewing our work and noting that our modifications are well-justified and bring strong improvements over all our baselines.\n\n## ContinualRAG is a Continual Learning Method\n\n> The paper is named “ContinualRAG”, however, the method does not seem to have any “continual” aspect.\n\nAs we argued in our work, RAG has become the de facto solution for continual learning in LLMs due to its simplicity and strong performance when compared to parametric alternatives. Our methodology builds on these already robust continual learning systems, enabling them to handle tasks that require more in-depth learning of new knowledge, such as associativity and sense-making. Given its strong performance on these demanding tasks, we argue that ContinualRAG system not only classifies as a continual learning system but also elevates the standard of what it means to continually learn.\n\nTo further show this, we provide an experiment that directly evaluates our method’s performance as more knowledge is aggregated. We refer the reviewer to the *Continual Knowledge Injection Experiments* section in our response to **reviewer gi53** above for experimental details, results and discussion.\n\nFor a more detailed explanation of motivation being our experiments, we refer the reviewer to the *Non-Parametric Continual Learning for LLMs: Ideal Experiments* section of our response to **reviewer K3hz**. \n\n## Why are 7B embedding models “large”?\n\n> The method utilizes Llama-3.3-70B Instruct for extraction and triple filtering. However, the tables refer to models that utilize 7B parameter LLMs as large embedding models. The relation to such methods and why they are referred to as large embedding models while the method utilizes 70B models albeit at a limited capacity is not clear.\n\nWe refer to 7B embedding models as ''large'' because they are much larger than classic ones like GTR (335M) and Contriever (110M). Although smaller than LLMs, they are the largest and strongest models available on the MTEB benchmark.\n\n## How does LLM size impact ContinualRAG?\n\n> It is also not clear what the impact of the model size is in the proposed method. Table 9 in the appendix provides some ablations with GPT-4o-mini but does not ablate on different model sizes of the Llama-3.3 family. \n\nWe report results using Llama-3.1-8B for ContinualRAG (Llama 3.3 family only has 70B models). It shows the 8B model is not sufficiently capable of supporting our system on both types of tasks (MuSiQue & NQ).\n\n| Model                               | MuSiQue |   NQ   |\n|---|---|---|\n| NV-Embed-v2 (7B)                   |  45.70  | 61.90  |\n| ContinualRAG (Llama-3.1-8B-Instruct) |  37.93  | 55.28  |\n| ContinualRAG (Llama-3.3-70B-Instruct) |  48.60  | 63.30  |\n\n## Clarifications\n\n> The paper is sparse in details about prior works such as HippoRAG. In particular, understanding section 3.1 is crucial for the understanding of the contributions, however, definitions and details on the following terms are missing: OpenIE, Personalized PageRank, PHR, reset probability, etc.\n\n**OpenIE:** OpenIE extracts entity–relation–entity triples from text without predefined relation types, in contrast to standard IE.\n\n**Personalized PageRank (PPR):** PPR is a variation of PageRank that measures the importance of nodes in a graph relative to a source node (or source nodes).\n\n**Reset Probabilities:** This vector quantifies the importance of source nodes for PPR.  \n\n**Parahippocampal regions (PHR):** This terminology is borrowed from the HippoRAG paper, which is an analogy between their retrieval encoder and the parahippocampal regions of the human brain.\n\n> The paper is missing descriptions for baselines in section 4.1. At least a description of groups of methods and a comparison is required.\n\nWe include several RAG baselines like BM25, popular retrievers (Contriever, GTR) and 3 SoTA models from MTEB. RAPTOR, GraphRAG, and LightRAG use summarization to enable sense-making capacity. HippoRAG performs well on associative tasks.\n\n> A clear description of the evaluations and execution difference between QA and retrieval setups is missing.\n\nOur QA module uses the top-5 retrieved passages as context for an LLM (GPT-4o-mini or Llama-3.3-70B-Instruct) to generate the final answer. The QA result is evaluated by token-based EM/F1 scores, aligning with MuSiQue/2Wiki/HotpotQA official metrics.\n\n## Continual Pretraining Related Work\n\n> If the paper is position itself as continual method as the title may suggest, it should consider expanding the related works section to discuss more continual pretraining methods.\n> - Roth, Karsten, et al. \"A Practitioner's Guide to Continual Multimodal Pretraining.\" arXiv preprint arXiv:2408.14471 (2024).\n> - Li, Jeffrey, et al. \"Tic-lm: A multi-year benchmark for continual pretraining of language models.\"\n\nWe thank the reviewer for highlighting these important works, we will incorporate them into Section 2.1 of our camera-ready version.", "claims_and_evidence": "The paper claims their 3 modifications to HippoRAG improve the contextual-awareness of the method. Substantial improvements across a diverse set of evaluations, along with ablation studies, substantiate their claim.", "methods_and_evaluation_criteria": "The proposed method involves three well-justified modifications to HippoRAG. The evaluations are similar to prior work and evaluate the reasoning and question-answering capability of the model.", "theoretical_claims": "N/A. The paper makes no theoretical claims.", "experimental_designs_or_analyses": "Lines 370-373 state that the method utilizes Llama-3.3-70B Instruct for extraction and triple filtering. However, the tables refer to models that utilize 7B parameter LLMs as large embedding models. The relation to such methods and why they are referred to as large embedding models while the method utilizes 70B models albeit at a limited capacity is not clear.\n\nIt is also not clear what the impact of the model size is in the proposed method. Table 9 in the appendix provides some ablations with GPT-4o-mini but does not ablate on different model sizes of the Llama-3.3 family. Prior work HippoRAG provided some ablations between Llama-3.1 8B/70B models but the importance of these models in ContinualRAG might be different.", "relation_to_broader_scientific_literature": "This paper advances the question-answering capabilities that require a knowledge graph to answer factual questions.", "essential_references_not_discussed": "If the paper is position itself as continual method as the title may suggest, it should consider expanding the related works section to discuss more continual pretraining methods.\n- Roth, Karsten, et al. \"A Practitioner's Guide to Continual Multimodal Pretraining.\" arXiv preprint arXiv:2408.14471 (2024).\n- Li, Jeffrey, et al. \"Tic-lm: A multi-year benchmark for continual pretraining of language models.\"", "other_strengths_and_weaknesses": "- The paper is sparse in details about prior works such as HippoRAG. In particular, understanding section 3.1 is crucial for the understanding of the contributions, however, definitions and details on the following terms are missing: OpenIE, Personalized PageRank, PHR, reset probability, etc.\n- The paper is missing descriptions for baselines in section 4.1. At least a description of groups of methods and a comparison between the bottlenecks and capacities of methods to the proposed method is required.\n- A clear description of the evaluations and execution difference between QA and retrieval setups is missing.\n- The paper is named “ContinualRAG”, however, the method does not seem to have any “continual” aspect.", "other_comments_or_suggestions": "- Line 355: N otably ->Notably\n- Line 427: It combining -> It combines", "questions_for_authors": "N/A", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "PuJIrRqzzq", "reviewer": "Reviewer_5bA8", "summary": "The paper introduces *ContinualRAG*, a novel retrieval-augmented generation (RAG) framework designed to enhance large language models (LLMs) with a human-like long-term memory system for non-parametric continual learning. Building on the HippoRAG framework, ContinualRAG aims to address limitations in standard RAG by improving performance across three key memory dimensions: *factual memory* (simple QA tasks), *sense-making* (interpreting complex contexts), and *associativity* (multi-hop reasoning). Its main algorithmic contributions include:  \n1. Dense-Sparse Integration: Incorporating passage nodes into the knowledge graph (KG) alongside phrase nodes to better capture context, inspired by dense and sparse coding in human memory.\n2. Deeper Contextualization: Using a query-to-triple matching approach instead of entity-centric methods to improve query-KG alignment.\n3. Recognition Memory: Adding an LLM-based triple filtering step to refine retrieved triples for graph search.\n\nThe authors evaluate ContinualRAG on benchmarks like NaturalQuestions (NQ), PopQA, MuSiQue, 2WikiMultiHopQA, HotpotQA, LV-Eval, and NarrativeQA.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are very thankful for the reviewer’s kind acknowledgment of our work as convincing, well-structured and broadly applicable across memory types. We will address the reviewer’s careful suggestions in the sections below.\n\n## Statistical Significance Testing\n\n> Why were no statistical tests reported? Adding them could confirm result reliability—would significant p-values change your confidence in ContinualRAG’s superiority?\n\nWe thank the reviewer for pointing out the need for statistical significance testing to ensure the reliability of our results. In order to ascertain the significance of ContinualRAG’s performance over NV-Embed-V2 (the best performing baseline), we run a simple bootstrapping statistical test. More specifically, we created 10,000 different datasets by sampling from each set of answers with replacement and thus obtained a distribution over the differences in their QA performance.\n\nThrough this method, we find that ContinualRAG’s performance is significantly larger than that of NV-Embed-V2 (p-value < 0.05) in 4 out of 7 datasets. Additionally, NV-Embed-V2 does not significantly outperform ContinualRAG in the other 3 datasets, demonstrating that ContinualRAG is robustly superior to our strongest baseline. We will include these significance testing results in the camera-ready version of our paper. \n\n| Dataset           | p-value  |\n|-------------------|----------|\n| 2WikiMultiHopQA  | 0.0000   |\n| MuSiQue          | 0.0013   |\n| NQ          | 0.0209   |\n| LVEval           | 0.0484   |\n\n## Novelty Concerns\n\n> Heavy reliance on HippoRAG reduces novelty; In my opinion the only major difference with the former is the passage integration part.\n\nWe appreciate the reviewer’s perspective, however, we would like to clarify our view on our work’s originality. While it is true that we build upon the existing HippoRAG framework, our approach systematically explores and enhances key components of the system in ways that are both principled and non-trivial. The space of possible design choices in structure-augmented RAG systems is vast, and identifying which modules to modify and how to do so effectively is itself a meaningful research contribution.\n\nOur modifications are not superficial; rather, they are informed by clear hypotheses about the limitations of the original modules and are validated through rigorous ablation studies (Table 4), which demonstrate substantial and consistent improvements in performance. We believe that such targeted, evidence-based improvements can be as impactful as proposing entirely new frameworks, especially when they advance the capabilities of an already influential baseline.\n\nMoreover, our work provides actionable insights into the design of RAG systems that others in the community can build upon—offering a path forward for both incremental and architectural innovations.\n\n## Reproduction Fidelity\n\n> Retrieval Performance (Table 3): Recall@5 is a standard metric, and reproducing HippoRAG with the same setup ensures fairness. However, the original HippoRAG paper reports higher R@5 (89.1 vs. 79.4 on 2Wiki), suggesting a potential issue in reproduction fidelity.\n\nWe would like to note that our reproduced HippoRAG results, reported on Table 3, are very close (or slightly better) than the ones reported in the original paper due to the use of a stronger embedding model and LLM for OpenIE. We changed both models in order to effectively compare with all other baselines. \n\nMore specifically, the R@5 scores reported in the original HippoRAG paper are 51.9, 89.1 and 77.7 respectively for MuSiQue, 2Wiki and HotpotQA. Our reproduced HippoRAG R@5 scores are 53.2, 90.4 and 77.3 for the same three datasets.\n\nWe ask the reviewer to let us know if they have any other questions concerning our reproduced HippoRAG results.\n\n## Filtering & Query-to-Node\n\n> Ablation Study (Table 4): Well-designed to isolate contributions (e.g., query-to-triple vs. NER-to-node), with clear recall@5 improvements. The omission of filtering in NER-to-node baselines is justified but could be explored further.\n\nAlthough the reviewer makes an interesting point there are a few reasons why we omit a filter for NER-to-node:\n- NER-to-node is already using an LLM to extract named entities before retrieving nodes.\n- Though a post-extraction filter could be added to NER-to-node, it would require designing a different filtering methodology specifically for this module.\n- Given that the performance of ContinualRAG w/o filter (avg. 86.4) is already much better than with NER-to-node (avg. 74.6), as shown in Table 4, we believe that designing such a filter was not the most worthwhile direction to explore.", "claims_and_evidence": "The paper’s primary claims are well-supported by empirical evidence, though some areas could benefit from deeper analysis:  \n- **Claim: ContinualRAG outperforms standard RAG across all memory tasks.**  \n  - **Evidence**: Tables 2 and 3 provide F1 scores and recall@5 metrics across seven benchmarks, showing consistent superiority over baselines (e.g., 63.3 vs. 61.9 on NQ, 48.6 vs. 45.7 on MuSiQue). Ablation studies (Table 4) further validate the contributions of each component (e.g., query-to-triple improves recall@5 by 12.5% over NER-to-node).\n  - **Weakness**: The claim of \"comprehensive\" outperformance lacks statistical significance tests to confirm robustness across runs or datasets.\n\nOverall, the evidence is convincing, but transparency in metric aggregation and statistical validation could strengthen the claims.", "methods_and_evaluation_criteria": "Yes", "theoretical_claims": "The paper does not present formal proofs but draws theoretical inspiration from neurobiology (e.g., dense-sparse coding, recognition memory). It remains conceptual rather than rigorously proven.", "experimental_designs_or_analyses": "- **QA Performance (Table 2)**: Sound design comparing ContinualRAG against diverse baselines (BM25, NV-Embed-v2, HippoRAG) using Llama-3.3-70B-Instruct. The use of F1 scores across seven datasets is valid, but the lack of error bars or multiple runs limits reliability assessment.\n- **Retrieval Performance (Table 3)**: Recall@5 is a standard metric, and reproducing HippoRAG with the same setup ensures fairness. However, the original HippoRAG paper reports higher R@5 (89.1 vs. 79.4 on 2Wiki), suggesting a potential issue in reproduction fidelity.\n- **Ablation Study (Table 4)**: Well-designed to isolate contributions (e.g., query-to-triple vs. NER-to-node), with clear recall@5 improvements. The omission of filtering in NER-to-node baselines is justified but could be explored further.\n\n**Issue**: No statistical significance tests are reported, weakening confidence in the results’ robustness.", "relation_to_broader_scientific_literature": "ContinualRAG builds on:  \n- *HippoRAG*: Extends its PPR and OpenIE approach, adding passage integration and recognition memory to address context loss.\n- *RAG Evolution*: Cites standard RAG (Zhong et al., 2023) and structure-augmented methods (RAPTOR, GraphRAG), positioning itself as a comprehensive solution.\n- *Neurobiology*: Links to hippocampal indexing (Klein et al., 2006; Suzuki, 2005) mirror HippoRAG’s inspiration, grounding it in memory theory.", "essential_references_not_discussed": "No", "other_strengths_and_weaknesses": "- **Strengths**:  \n  - Integration of passage nodes and recognition memory builds on HippoRAG.\n  - Well-structured, with figures (e.g., Figure 2) and tables enhancing readability.\n  - Broad applicability across memory types.\n\n- **Weaknesses**:  \n  - Heavy reliance on HippoRAG reduces novelty; In my opinion the only major difference with the former is the passage integration part.", "other_comments_or_suggestions": "No.", "questions_for_authors": "1. Why were no statistical tests reported? Adding them could confirm result reliability—would significant p-values change your confidence in ContinualRAG’s superiority?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper presents a method to enhance traditional RAG with a continual learning framework for large language models. While all the reviewers agree that the problem is well-motivated and the solution is intuitive and effective, there are some aspects that can be improved  and integrated into the next version of this work: i) more careful comparison with HippoRAG; ii) experiments emphasizing document storage; iii) experiments that incorporate a temporal dimension.
Decision: Accept (poster)