AssoMem: Scalable Memory QA with Multi-Signal Associative Retrieval
Authors: Kai Zhang
Xinyuan Zhang
Ejaz Ahmed
Hongda Jiang
Caleb Kumar
Kai Sun
Zhaojiang Lin
Sanat Sharma
Shereen Oraby
AARON COLAK
Ahmed A Aly
Anuj Kumar
Xiaozhong Liu
Xin Luna Dong
Venue: ICLR 2026 Poster (2026)
Keywords: memory-augmented LLM
scalable retrieval
memory question answering
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: We propose AssoMem, a novel framework that solves the challenge of accurate, scalable memory QA by forming an associative memory graph and adaptively fusing multi-dimensional retrieval signals, resulting in state-of-the-art performances.

Abstract: Accurate recall from large-scale memories remains a core challenge for memory-augmented AI assistants performing question answering (QA), especially in similarity-dense scenarios where existing methods mainly rely on semantic distance to the query for retrieval. Inspired by how humans link information associatively, we propose AssoMem, a novel framework constructing an associative memory graph that anchors dialogue utterances to automatically extracted clues. This structure provides a rich organizational view of the conversational context and facilitates importance-aware ranking. Further, AssoMem integrates multi-dimensional retrieval signals—relevance, importance, and temporal alignment—using an adaptive mutual information (MI)-driven fusion strategy. Extensive experiments across three benchmarks and a newly introduced dataset, MeetingQA, demonstrate that AssoMem consistently outperforms state-of-the-art baselines, verifying its superiority in context-aware memory recall.

Published as a conference paper at ICLR 2026
ASSOMEM: SCALABLE MEMORY QA WITH MULTI- SIGNAL ASSOCIATIVE RETRIEVAL
Kai Zhang Xinyuan Zhang* Ejaz Ahmed,, Hongda Jiang', Caleb Kumar? Kai Sun', Zhaojiang Lin?, Sanat Sharma², Shereen Oraby?, Aaron Colak?
Ahmed Aly?, Anuj Kumar", Xiaozhong Liu', Xin Luna Dong
'Woreester Polytechnic Institute. MMea Reality Labs i (kkaizh, dylanz426, lunadong) @ meta.com
ABSTRACT
Accurate recall from large-scale memories remains core challenge for memory- augmented Al assistants performing question answering (QA), especially in similarity-dense scenarios where existing methods mainly rely on semantic dis- tance to the query for retrieval. Inspired by how humans link information asso- ciatively, we propose AssoMem, novel framework constructing an associative memory graph that anchors dialogue utterances to automatically extracted clues. This structure provides rich organizational view of the conversational con- text and facilitates importance-aware ranking. Further, AssoMem integrates multi-dimensional retrieval signals-relevance importance, and temporal align- ment-using an adaptive mutual information (MI)-driven fusion strategy. Ex- tensive experiments across three benchmarks and newly introduced dataset, MEETINGQA, demonstrate that AssoMem consistently outperforms state-of-the- art baselines, verifying its superiority in context-awar aware memory recall.
INTRODUCTION
The rapid advancement of large language mod-
els (LLMs) has opened the door to personal as-
Qt: What do usudly y complain ware, can you give some tips to avoid it?
Memory
sistants that function as "second brain" "aiin""a" -a
Brcence Uber: Wie alinge stat phanting too Mes
we'ld
digital companion capable of capturing, orga- nizing, and retrieving information on behalf of
Q2 Whac show did watch fistt the Drown or the Game of Thropes?
Exidenca User: Every time we plan. thone suerra no
unespacted
Ubea: socked late los righta th wusk,
the user. An essential capability of such sys- tems is the ability, with explicit user consent, to store and recall events and facts from the user's life (Jiang et al., 2025). This capabil- ity enables natural memory-recall interactions such as "Summarize the key points from my meeting with Sarah yesterday". In this paper, we focus on settings where textual memories such as meeting notes, and conversational dia- logues are continuously accumulated over time, and we study the problem of answering mem- ory recall questions over large-scale memory repositories.
Top-1
wording
innegrstion
Liee:: like to work onvall
bror
just trtined wendinn the third seasion The Grown on Notks, Took
with peans.
Top-1 Retrieval for
upiosdes
User: Do you hase sinnar show Reggestions
chanaders thesshow aline.
Crown: "the Sone Thrones/?
baggession Thronee"?
you Drown"" "Jo
insufficient retrieval
Relevance solely
Endence
User: staree watching "the Game of Troones about one monttr ago.
100
Decakgi (4)
sessoon evee
Assetren
MonteegA
Lew www.eeew...r
Langlee mill)
Recent research in this area predominantly follows the Retrieval-Augmented Generation (RAG) paradigm, organizing user memories to enable efficient retrieval and accurate response generation. Drawing inspiration from the effectiveness of human memory systems in recalling in- formation from sequential streams, several approaches partition historical dialogues into long- and short-term segments to enhance memory recall (Zhang et al.) 2024: Li et al. 2025 Zhong et al. 2024] Chhikara et al., (025). Other methods organize memory using hierarchical filters such as topics summaries to narrow down the retrieval search space (Tan et al. 2025 Xu et al. 2025b
Figure 1: An example showing limitations in rele- vance solely retrieval. Our AssoMem consistently outperforms SOTA baselines on three datasets.

---

Published as a conference paper at ICLR 2026
Zhou et al. 2025 Graph-based approaches construct entity-relationship knowledge graph from personal memories and answer questions through graph algorithms (Wang et al. 2024 Chhikara
et al. 2025). Despite these advances in memory structure optimization, a critical challenge remains: as the volume of user memory records increases, retrieval performance deteriorates as in Figure D largely because the memory pool accumulates many highly similar items, like repeated meeting top- ics and overlapping conversation snippets, making it harder to distinguish truly relevant information.
We argue that humans do not perceive their memories as isolated entries or as a simple chronological stream. Instead, they organize them associatively, linking pieces of information through clues such as entities, locations, events, and topics. Likewise, pinpointing relevant memory evidence canmot rely solely on refining relevance comparison, as explored in reranking modules (Tan et al.] (2025) or muuti-ggaauulartty retrieval (Xu et al., 2025aj. Among related items, people tend to remember important clues more clearly and revisit them more often; for example, answering "What do I usually complain at work, can you give some tips?" requires identifying the clues that matter most to the user as depicted in Figure
Motivated by these observations, we propose AssoMem, a memory QA framework that leverages associative structures to guide memory selection. At its core is an asxociative memory graph that links each memory utterance to a set of clues, which are automatically extracted by LLMs to enable fine-grained interpretation of a user's memories and to connect memories sharing similar signals. Unlike existing memory graphs that are built entirely on abstractive concepts rather than raw histor- ical data /Wang et al. 2024 Chhikara al.) 2025 Xu et al., 2025b), this graph supports associa- tive connections from abstractive clues to exact memories, facilitating importance-aware ranking. Based on this graph, AssoMem further integrates multiple retrieval rignals--rlevvnnc,, importance, and temporal alignment- and employs a mutual-information (MI)-driven fusion strategy to dynam- ically balance these dimensions according to query intent, yielding more accurate and context-aware memory retrieval. Moreover, fine-tuning the answer generation model with a multi-tast denoising strategy utilized to maximize the QA performance based on retrieved memory records. To the best of our knowledge, AssoMem is the first memory QA system that mimics the structure of associative memory to enhance QA on large-scale, similarity-dena memory collections. We summarize our main contributions as follows:
Unified framework for memory recall QA: We propose AssoMem, a memory QA system that integrates relevance, importance, and temporal signals through mutual information (MI)-driven weight assignment strategy, enabling adaptive and context-aware memory se- lection to improve answer quality.
Associative memory graph: At the core of AssoMem is an associative memory graph that captures semantic relationships between utterances and the clues associated with them. This graph facilitates efficient retrieval and importance-awaar ranking of memories.
New benchmark and evaluation: To foster research in large-scale memory retrieval, we introduce MeetingQA, a benchmark simulating real-world meeting scenarios where multi- turn dialogues form the memory base, paired with diverse QA examples. Extensive exper- iments and ablations on MeetingQA and other memory benchmarks show that AssoMem outperforms existing retrieval approaches by 24.93% on average.
2 RELATED WORK
Large-scale Memory Management and Retrieval Memory has emerged as a promising solution for enhancing LLMs (Madaan al.) 2022. Wang et al. 2023). However, as the historical mem- ory incrementally accumulates, existing methods fail to process the large-scale memory since the incremental information poses noise for both retrieval and generation Yu et al. 2025 Hu et al., 2025 Maharana et al.) 2024). Recent research in this area has advanced along two key aspects: terms of large-scale memory management, early work introduced structural organization by partitioning conversational history into short- and long- term memory (Zhang et al., 2024 Li et al. 2025 Zhong et al. 2024), enabling models to reflect both recent interactions and persistent user preferences. More recent approaches incorporate hi- erarchical structures -uciurreessshhhhhheeee as topics, summaries, and memory graphs to constrain the retrieval space and improve recall performance (Tan et al. 2025 Xu et al., 2025b; Chhikara et al., 2025 |Rezazadeh et al. (024). In terms of retrieval, focus is on directly identifying relevant memories for downstream tasks. Query-centered methods enhance retrieval through improved query formulation
large-scale memory management and retrieval.
2

---

Published as a conference paper at ICLR 2026
AssoMem Frame
Offline
Clue 1
Associative Memory Graph Construction
Aropusor comphakkng rilk
Memory Index
"
Clne Mergs
M
PPR scCuies
1lool
LLM
Up
Clue Tagger
Importance
Memory Graph
Top-K: (M0
Clue
Semantic scores
always planning It' reectngg
conpiiale wart, glu your Sjn sranddd
Relevance
Temporal
Query
User: mely like the eory the crown, charactere the
Sg
Temporal Takens Extractor
Temporal scores
MI-guided Weight Assignment
User
Temporal Utterance
Temporal
$j ......
Historical Dialogue Sessions
Runtime
RITRanker
Figure 2: Overview of the proposed AssoMem framework. A topic-utterance graph constructed from historical dialogues, enabling the integration of relevance, importance, and temporal signals. These are adaptively fused to guide accurate memory retrieval for question answering.
Jiang al. 2023 Jang al. 2024), while reranking-base approaches refine retrieved candidates using trainable scoring mechanisms (Wu et al. 2024bb Du et al.) 2024, Tan et al. 2025). Comple- mentary works have also explored granularity impacts, demonstrating the effectiveness of hybrid retrieval strategies (Xu et al., 2025a Tan et al. 2025 Sarthi et al. (024). Despite their success, these methodologies
predominantly grounded in relevance, aiming to retrieve the most topically similar memories while overlooking key challenges discussed in Section [ In contrast, AssoMem adopts an associative memory to integrate multi-dimensional signals address these challenges.
PageRank (PR) models a random surfer over a directed graph, where node scores reflect the sta- tionary distribution of the walk, yielding an importance prior that complements term-matching rel- evance in web search (Page et al. 1999 Brin & Page 1998 Langville & Meyer (011)). Alongside HITS (Kleinberg) 1999), underpinned early large-scale retrieval by propagating hyperlink endorse- ments and proved effective under sparse noisy lexical signals. Subsequent works improved com- putational scalability via power-method accelerations and linear-algebraic solvers (Bahmani et al., 2010). Personalized PageRank (PPR) biases teleportation toward user-specific needs, effectively yielding a relevance-condditonee importance signal (Bahmani et al., 2010 Wayama Sugiyama 2025). In our work, AssoMem leverages PPR to enable multi-dimensional signals for later retrieval.
METHODOLOGY
3.1 PROBLEM FORMULATION AND SOLUTION FRAMEWORK
Problem formulation Consider a memory bank M ((So, (S,,, (Sx........ dy)), where each session S; {ug, Mi...., ]n contains n utterances and is associated with timestamp d, A question g considered memory question it specifically refers to the user's past, as illustrated in Figure The memory recall problem takes memory question g and outputs answer based on the memory bank M. Note that a memory question can go beyond specific memory seeking such "can you give some tips..." While method can apply to personalized conversations where the user does not explicitly refer to the past, we primarily focus on memory recall questions this paper.
Solution framework At run time, we answer memory questions in two steps as depicted in Figure memory retrieval and answer generation. Given question q. the Retrieval step retrieves set of memory utterances to ground question answering, denoted by E"; the Answer Generation step then generates the answer g based the retrieved memories: à LLM" (q, E*)
3

---

Published as a conference paper at ICLR 2026
3.2 MEMORY RETRIEVAL
The retrieval step aims to select the best memory evidence to support QA. At the core of our retrieval the Axsociative Memory Graph that anchors each piece of memory with the underlying clues, and connects relevant memories. We next discuss the construction of the graph and how we use it for the downstream QA task.
3.2.1 ASSOCIATIVE MEMORY GRAPH CONSTRUCTION
Memory clues A memory clue captures potential cues that help memorization; it can be an aspect phrase like Evening entertainment, key entity like Lumia project, or am event like launching weekly meetings. For each session S, in the memory bank, we employ an LLM agent to generate repre- sentative clue & for the session. This clue is associated with all utterances in S; resulting in an initial clue set C fco, C1,...; CN), where N is the number of sessions.
To reduce redundancy and enhance topic coherence, we merge clues with high semantic similarity. Specifically, for any pair of clues (c), , if their embedding similarity exceeds a threshold 6, they are merged into the same clue. The associated utterances from the merged clues are grouped under the new clue. This process yields refined clue set C' and updated utterance groupings.
Associative Memory Graph An associative memory graph G = (V, E) associates each memory record with the underlying clues and other relevant memories. There are two types of nodes: each clue node represents a clue in the set C' of merged clues, and each ufterance node represents an utterance in U. Each node (clue or utterance) represented by its text embedding, computed using pre-trained embedding model (e.g.. BGE). There are also two types of edges: each ownership edge connects an utterance u E S; with its associated clue o, and each similarity edge connects clues or utterances that are similar. Specifically, we create an edge for pair clues or a pair of utterances whose embedding similarity exceeds pre-decided threshold yy:
sim By) 7 U/ By E C' or U/ By Eu
The structure can extended with conversational metadata (e.g., people, locations) when exists and enriched via external taxonomies such ConceptNet (Speer et al. 2017).
3.2.2 CANDIDATE RETRIEVAL
Inspired by how associative memory works, we perform candidate retrieval in a two-step hybrid mode. First, given query q, retrieve the relevant clues, and denote the Top-K clues related to by Cgi we consider all utterances associated with C, as candidate memories, denoted by Meand = (u clue(u) C.). Next, we rank all candidate utterances by score predicting their usefulness in answering the question, obtaining the final retrieval results: E = arg max
Evee Score(g, u).
&CU,,,, |E|=K
With good scoring system, denoted by Score(y, u), this hybrid retrieval strategy returns accurate memory evidence and ensures that Ex maximizes the utility of the retrieved memories. Designing the scoring system plays critical role to QA quality, which we will describe in detail next.
3.2.3 RITRANKER: RELEVANCE, IMPORTANCE, AND TEMPORAL DYNAMICS
Unlike existing memory-based methods that rely solely on similarity, our retrieval score for each utterance u integrates three dimensions: relevance, importance, and temporal alignment. This fusion enables retrieval of memories that not only align with the recall question but also reflect central aspects of the user's daily life and adhere temporal constraints.
Relevance Existing memory- based methods have verified that semantic relevance serves as an es- sential criteria and ensures that retrieved utterances are contextually aligned with the query. We compute relevance using cosine similarity between semantic representations of question e, and
each memory utterance c.: su = sim (cy. cu), where C2 and c, are semantic embedding vectors obtained from embedding model for the query and utterance, respectively.
Importance Unlike querying specific details. users frequently ask for recommendations where relevance-only retrieval fails to return appropriate memories, seen in Figure T To capture the
4

---

Published as a conference paper at ICLR 2026
importance of utterances within the large-scale memory records, apply graph mining the sociative memory graph. Drawing inspiration from PageRank in web search (Page et al., 1999), we apply Personalized PageRank (PPR) (Wayama & Sugiyama 2025) to decide the importance of each clue and memory utterance w.r.t a given query.
r [ dMr(()) + (1 d)t
(2)
where M {0, 1)NxN is the adjacency matrix from graph connectivity, t E R is the personal- ized teleportation vector, r is the pagerank score vector and is the damping factor. our setting, the utterance cells in t are set to the similarity between query and utterance, and the clue cells are set to O. We initialize rd The importance score for is 8u (imp) Ta after convergence. Notably, we apply PPR rather than global pagerank (r" = (1/ND) to avoid boosting the importance of memories irrelevant the question.
Temporal match Temporal questions are common in the real world yet relevance cannot effec- tively capture the temporal constraints, as illustrated in the example Section [ Recency decay, commonly used in temporal memory retrieval, does not satisfy explicitly specified temporal con- straints (Li et al. 2023). We thus conduct explicit temporal match in three steps. First, we extract temporal tokens from the question to determine if the temporal reasoning is needed. Second, apply temporal embedding (i.e., TimeLlaMMa (Yuan et al.) 2024)) on the extracted temporal tokens. Third, compute similarity between the temporal embeddings of q and the utterance u. Here, the temporal embeddings of utterances are computed based on the temporal tokens and the timestamp (temp) (tsmp)
(temp) sim associated: Si
Cq
Cú
Mutual Information for Score Fusior To balance the information from each dimension with re- spect to the query type, need a strategy that can perceive the importance of each signal. Mutual Information (MI) known for its ability of representing the informativeness of two variables, pro- viding solution to adaptively assign weights for different dimensions w.r.t query types. In scenario, we use Conditional MI (CMI) to indicate how well a signal from each dimension reflects the likelihood that memory utterance is useful for answering a given question.
Initially, the raw score g(d)u of each memory u for dimension is converted into three bins: low, medium, high. By doing so, collect the score-label pairs (3(d)(t) yrn). where b (low, medium, high) denotes the score bin, À E A = {0, represents the memory usefulness la- bel. The probabilities p(s!.)(6) Jyh), p( .q). p(y.. q) can be calculated based on the collected pairs. For each query type q, we compute the conditional mutual information:
p(s. (d) B
4))
CMIa(g) = (3((0))) ;11
Σ Σ (((((()))
ym) log
(3)
p(
(((()))))))
(d)()
q)p(v> (4)
where is the usefulness label. The weight for each dimension is then: w(() (xp(CML.(g)/T) The final score for each memory item u is: exp(CMMa
Score(g, u) (rel) (q) s(mel) + w (imp)) (q) g(imp) w. (tamp) (q) s
g(temp)
(4)
This adaptive fusion mechanism dynamically adjusts the contribution of each dimension based on its relative impact on the current query. temperature parameter T modulates the sharpness of both the score and weight distributions, enabling smoother more selective fusion as needed.
3.3 MODEL FINE-TUNING
Recent advances have highlighted persistent gap between retrieval-based recall and generative performance in large language models (LLMs) (Yang et al. 2024; Ouyang et al.) 2024). This dis- crepancy is often attributed to the presence of irrelevant noisy content among the top-K retrieved candidates. To address this challenge and fully leverage retrieved contextual evidence, we adopt novel fine-tuning approach using augmented datasets informed by targeted negative and positive sampling: I.M" = Fine mimeTTnne[LLMM D.ALM,eDDo Dom-
provide comparisons for different fusion strategies in appendix B.5
5

---

Published as a conference paper at ICLR 2026
Denoising QA Dataset DoA+Mem denotes QA dataset comprising queries, reference answers, and memory context. Specifically, two sampling strategies are adopted for denoising fine-tuning: (1) mixed positive and negative memory contexts to encourage evidence discrimination, and (2) negative-only contexts to improve robustness by preventing over-reliance on supporting evidence.
Multi-task Fine-tuning. Answering question involves recognizing its type better utilize the memory context and generate answers. We jointly train two tasks question rype prediction and an- swer teneration-to capture this process. The model receives an instruction, question, and sampled memories as input, and outputs both the predicted question type and the generated answer. This setup promotes effective memory utilization conditioned on question intent.
AssoMem stands out by a novel associative memory structure and a multi-dimensional scoring sys- tem, jointly enhancing retrieval quality and generation robustness across diverse scenarios.
4
EXPERIMENTAL SETTINGS
4.1 DATASET
To evaluate AssoMem, the dataset must provide question-answer (QA) pairs and large-scale mem- ory annotated with usefulness labels to assess both retrieval and generation quality. We adopt Long- MemEval (Wu et al., (202aa), which meets these criteria and includes six question types covering common real-world scenarios. We use the original small (s) and medium (m) subsets, and addition- ally construct large (I) variant to assess robustness under increasing memory size.
To further test generalizability and promote research in memory recall, we introduce MEETINGQA, synthetic dataset simulating weal-world meetings with multi-speaker dialogues specific topics. provides QA pairs and historical meeting transcripts as memory, with annotated usefulness labels.
All datasets follow consistent structure: QA pair, base memory consisting of historical conver- sations where usefulness labels are available. More dataset details are in Appendix
4.2 BASELINES & EVALUATION METRICS
To assess the effectiveness of AssoMem on conversational memory recall QA, we compare it against suite of representative baselines covering various retrieval granularities. At the utterance level, we include flat utterance retrieval and Long/Short-Ter Term Memory (LST Memory) (Zhang et al. 2024). At the session level, consider flat session retrieval. Under the multi-granularity paradigm, compare against session retrieval first then utterance retrieval (session-utterance)) session summary and utterance retrieval (session summary mem), topic grouping (TTan et al.. 2025). and MemGAS (Xu et al. 2025a). More details about baselines are in Appendix C
We adopt standard evaluation tasks: Memory Recall and Answer Generation. For memory call, we report Recall@k k and nDCG@k; for retrieval generation, we use LLM-as-a-Judge accuracy
(Acc@6, @cc@ 10), BERTScore, and Faithfulness (Zhang al. 2024 Lattimer al. 2023). All retrieval metrics are computed at the utterance level.
4.3 IMPLEMENTATION
For base models, we select a range of models of different sizes: LlaMA-3.3 3.3------3--B----- IIIntructt LlaMA- -nstrrutttt
3.3-70B- Instruct (Grattafiori al. 2024). Qwen2.5-32B (Qwen al. 2025). gpt-oss- 120B (Agar- wal et al. 2025). Notably, for large size models such as 70B and 1208, we do not fine-tune while for other models we follow section 3.3 to perform fine-tuning via trl (von Werra al.
2020 and trans-
formers (Wolf et al. 2020) libraries to reduce the noise of generation. For retrievers. we select three state-of-the-art embedding model DragonPlus, DragonPlusRobertat... et al. 2023), BGB
et al. 2023 For temporal dynamics, we use TimeLlaMA (Yuan et al.) 20244 as the embedding model. For graph construction and usage, we use networkx (Hagberg et al. 2008).
All the retrieval results using DPR as retriever, retriever comparisons and analysis in Appendix B.4

---

Published as a conference paper at ICLR 2026
Table Retrieval and QA performance on LongMemEval medium (the top table), large (the middle table), MeetingQA (the bottom table).
Method
R@1 R@3
R@6 R@ nDCG@3
nDCG@6 nDCG@10 Acc@6
Utterance-level: retrieve top-k utterances as context
Utterance-flat
16.45
54.90
64.25
70.18
56.24
66.14 14
68.04
48.66
LST Memory
52.91
58.03
65.91
70.69
59.82
67.34
68.68
50.96
Sexsion-level: retrieve top-k sessions as context Session-flat
56.91
64.81
70.86
78.93
07.19
71.68
76.37
51.93
Hybrid: first retrieve sestons/npicsssmmaiiies
retrieve utterances as context
Session summary mem.
50.19
59.93
62.95
72.84
61.68
64.25
68.79
49.35
MemGAS
45.75
51.06
66.93
77.02
53.59
66.97
69.46
51.23
Session-utterane
55.37
64.66
70.17
78.97
66.04
71.31
76.50
55.85
Topic grouping
55.98
65.91
76.47
79.14
66.12
77.03
78.86
59.95
AssoMem
59.73
72.96
80.87
84.96
75.36
81.30
82,93
64.01
Method
R@1
R@3
R@6
R@ 10
nDCG@3 nDCG@6 nDCG@ 10
Acc@6
Utterance-level: retrieve top-k ulterances context
Utterance-flat
37.61
41.66
49.91
54.56
42.83
49.94
53.68
41.36
LST Memory
36.19
41.93 51.49
56.16
43.06
51.66
55.93
42.32
Session-level: retrieve top-k sessions context Session-flat
40.82
50.20
55.99
59.34
51.19
56.93
58.87
40.82
Hybrid: first retrieve session/hppic//smmareee
then retrieve
Session summary mem.
39.92
43.26
51.18
55.29
44.51
51.93
54.81
41.89
MemGAS
37.64
44.92
51.12
57.27
46.82
54.67
56.61
43.33
Aession-utterance
40.93
49.87
55.83
59.62
51.92
56.17
58.93
45.38
Topic grouping
40.13
50.78
58.54
62.29
52.27
59.46
61.92
48.36
AssoMem
43.56
59.60
64.93
69.33
62.61
65.87
66.31
52.59
Method
R@1
R@3
R@6
R@ 10
nDCG@3 nDCG@6 nDCG@ 10
Acc@6
Utterance-level: retrieve top-k utterances as context
Utterance-fla flat
23.89
40. 40.27 48.81
55.29
41.91 43.96
53.83
56.19
45.91
LST Memory
23.93
42.77
53.62 58.83
55.78
59.17
48.36
Session-level: retrieve top-k sessions context Session-flat
28.62
47.36
56.72
60.19
48.91
59.31
62.97
51.79
Hybrid: first retrieve ressiors/opiccammmrress/s/e then retrieve utterances context
Session summary
26.56
45.25
52.66
54.17
47.83
53.19
55.93
47.87
Session-utterance
33.78
52.19
67.63
77.91
55.31
69.48
80.19
49.17
MemGAS
32.94
52.59
69.34
80.67
55.86
71.66
82.93
61.26
Topic grouping
39.66
61.69
78.98
89.15
63.15
74.23
83.78
63.56
AssoMem
41.63
64.72
85.17
92.96
66.06
86.93
94.17
69.41
EXPERIMENTAL RESULTS
In this section, we will present the thorough experimental results and concrete analysis with a focus on answering these research questions, supplementary results can be seen in Appendix B
RQ 1. Does AssoMem exhibit performance advantages state-of the-art art solutions?
RQ 2. How does RITRANKER contribute to performance of AssoMem?
RQ 3. How robust is AssoMem against the memory size and question types?
We will answer RQ in Section 5.1 RQ in Section 5.2 and RQ in Section 5.3 and 5.4
5.1 COMPARATIVE STUDY
Retrieval Results. As presented in Table M Leng dataset, the dession-utterance
retrieval raises R@ 10 to 78.97% and nDCG@* 10 to 76.50% over the utterance-level flat retrieval
We adopt 6 reference point based the retrieval-generation transition analysis in Appendix B.3

---

Published as a conference paper at ICLR 2026
Table 2: Generation results of different LLMs using AssoMem recall @ 10 retrieval context.
Accuracy
BertScore
Faithfulness
Model
Plain
Fine-tuned
Plain
Fine-tuned
Plain
Fine-tuned
LlaMA3.2-3B- Instruuct
26.91
33.43
31.19
36.93
33.27
43.06
Dwwen..5332BInnttteu
64.72
73.88
67.91
77.49
55.16
75.73
1iaMA-3.3-70B-Insttee
65.83
71.86
56.96
Gpt-Oss- 120B
76.49
81.76
68.73
70.18% and 68. 68.04%. Topic grouping further improves R@ 10 to 79.14% and nDCG@ 10 to 78.86% Similar patterns are detected on LongMemEvall and MeetingQA datasets. This sug- gests a general finding that hybrid retrieval clearly outperforms single granularayll retrieval. Our proposed AssoMem, on the other hand, makes improvement of 5.82% over the SOTA, topic grouping. Further, AssoMem also makes improvements of 7.04% and 3.81% 1% against the best base- lines on LongMemEval I and MeetingQA, respectively. This suggests that AssoMen does offer advantages compured with SOTA. Why do prior memory methods fail scale? As memory size grows, more similar memory records accumulate where similarity alone cannot discriminate among many near-duplicate or thematically close candidates, so recall collapses and downstream generation suffers. Why does AssoMem help? By ranking with importance and temporal priors in addition to relevance, AssoMem takes the question types into consideration and performs retrieval from a multi- dimensional anchor. As can be seen in Figure Ba). we clearly witness the improvements made in preference and temporal reasoning type questions which suggests the success of our scoring system.
Generation Results Retrieval quality translates directly generation: Acc@6 climbs from 48.66 for flat retrieval to 55.85% for multi-granularity and to 64.01% for AssoMem, with BERTScore moving from 51.71% % to 60.06% and then to 67.56%. This again validates that AssoMem is making improvements against SOTA. Moreover, as previous studies have verified that there exists gap be- tween the recall and generation performance Yang al. 2024); Ouyang et al. (002)) which means the top-k retrieved content may also deliver noise into the context for downstream question answer- ing (We present analysis in Appendix B.3 support this claim). The results presented in Table]allso validate this where the recall @6 result of AssoMem is 80.87% % while the answering accuracy @6 is 64.01 on m dataset, a similar pattern is also observed on dataset. Thus, the base model's 's ability to fully utilize the retrieved memory can be key to improve the downstream generation performance. Following the fine-tuning strategy in Section 3.3 the generation accuracy @ 10 gains improvements of 6.52% and 9.16% for LlaMA3.2-3B and Qwen2.5-32B models, respectively in Table 2 This verifies that our fine-tuning strategy helps the model better utilize the retrieved memory context, contributing to the AssoMem performance.
5.2 ABLATION STUDY
Ablation retrieval dimensions As can be seen in Figure3 (b), the performance temporal rea- soning type questions drops when excluding temporal dimension information and the performance on single-user-prfeerene type questions drops when excluding importance dimension information. These observations validate the challenge mentioned in Section [T that different types of questions require information from different dimensions for retrieval, illustrating that RITRanker contributes to AssoMem's 's performance by well integrating signals from multiple dimensions.
A Alltion on components In AssoMem, the associative memory graph serves as the base for obtaining multi-dimensional information while the MI-guided weight assignment strategy serves as their con- nections. Thus, we further tested how each component impacts the perfor- mance. Specifically, we conduct exper- re-
Table 3: Ablation on components and dimensions. w/o denotes without the component compared to AssoMem. w/o
R@6
R@ 10
Acc@6
Ace@ 10
Temporal
73.39
78.37
57.88
61.19
Importance
75.81
79.62
59.55
61.97
Clue nodes
79.75
84.80
63.06
72.51
Weight Assignment
76.79
81.80
60.38
58.89
iments under these two settings: move the clue nodes within the graph; remove the weight assignment strat- egy and instead, use fixed weighted sum for comparisons. As can be observed in Table the
AssoMem
80.87
84.96
64.01
69.17
8

---

Published as a conference paper at ICLR 2026
retrieval performance drops 1.12% without clue nodes which attribute to the fact that the clue level retrieval would be suboptimal when the importance information is missing. Moreover, using fixed weight assignment results in 4.08% performance drop compared to full AssoMem which further validates the necessity of each component within AssoMen [
5.3 ROBUSTNESS STUDY
Results different question types. As in Figure 3 (a), can clearly observe that the perfor-
100
rerrenat Jabeeees buej cesinc-srrsaary kopto grepprr
Comparison between baselines and AssoMem
(b) Ablation on retrieval dimensions
Figure 3: The radar figure showing performance on different question types.
mance of AssoMem, beyond end-to-end performance in Table [ consistently outperforms all base- lines across six question types. Notably, AssoMem presents the largest margins on preference and temporal type questions compared with other baselines which we attribute the fact that similarity- only retrieval over-emphasizes surface relevance and tends to return most similar memories, which adequate for pure recall but brittle for preference, temporal, and cross-session questions.
Results on memory size As presented in Table AssoMem makes improvements of 6.39%, 7.04% for the @@ @@ 10 against the best baseline topic grouping, respectively. Following the retrieval performance improvements, the generation accuracy performance is also improved by 4.06%. To- gether with the advantages made on Long m, the results suggest that our AssoMem present strong robustness the memory size increases since from from 500 to 2,500 rounds data point. These results and observations on increased memory size and the diverse question types highlight the strong robustness of AssoMem.
the dialogue sessions increase
5.4 ERROR ANALYSIS
As shown in Table 4 AssoMem achieves the highest retrieval fidelity,
Error bucket
LST Mem.
Topic
AssoMem
with a Correct rate of 64.01 % ex-
ceeding Topic Grouping by 4.06% and LST Memory by 13.05% and
Correct
50.96
59.95
64,01
RE: Incorrect retrieval
34.09
23.43
19.13
the lowest Retrieval Error at 13%, reducing errors by 4.30% and 14.96% respectively. It also yields the low- Wrong-grounding
GE: Wrong grounding
5.13
3.82
2.86
GE: Misuse of positives
2.22
4.37
5.71
GE: LLM-Judge error
7.60
8.43
8.29
(2.86% 3.82% and 5. 13%), indicating more accurate and intent-aligned context. However, better retrieval does not fully translate into generation quality: total generation errors remain compa-
Table 4: Error analysis. RE denotes retrieval error, GE de- notes generation error. Wrong grounding means negatives are used for generation, misuse of positives means the posi- tives are misused by LLM.
rable 16.86% for AssoMem, and 16.62% for Topic Grouping. Overall, AssoMem's advantage lies
*Comparisons different fusion strategies are provided in B.8 and B.5

---

Published as a conference paper at ICLR 2026
in reducing retrieval-side failures and confusing negatives; the remaining gap is generation-side suggesting the need for stronger evidence utilization which necessitates our fine-tuning strategy. These observations again validate the effectiveness and robustness of proposed AssoMem.
5.5 LATENCY COST STUDY
Beyond retrieval and generation quality, practical deployment memory QA sys- tems demands low latency and efficient token usage. We benchmark the end-to- end query latency and token consumption of AssoMem against recent graph-based baselines LongMemEval m (Table
Table 5: Latency and token cost Eong
Tokens = average input tokens consumed for generation per query.
Method
R@3
R@6
Acc@6
Tokens
Lat. (s)
HippoRAG
49.66
64.18 18
48.61
8,430
4.51
AssoMem achieves an average query latency of s.30s, which is 3.5x lower than Hip- PoRAG (4.51 s) and 1.9x lower than A-Mem s). This speedup stems from two factors: (i) AssoMem retrieves at the utterance level without requiring an expensive LLM-based Open Information Extraction step query time, and (ii) the RITRanker scoring pipeline-compristee PPR (0.21 s), weight assignment (0.26 s), and answer generation (0.74 lightweight by the structure desigr
Query-time efficiency.
A-Mem
47.82
62. 17
48.06
8,852
2.45
AssoMem
72.96
80.87
64.01
1,846
1.30
Token efficiency. AssoMem consumes only 1,846 tokens average per query, roughly 4.6x fewer than HippoRAG (8,430) and A-Mem (8,852). Because HippoRAG and A-Mem construct fine- grained entity-level graphs, their retrieval pipelines surface a larger volume of fragmented evidence that must be concatenated into the prompt, inflating token cost without proportional accuracy gains.
Offline construction cost. Full graph construction for 2,500 sessions (24.4k utterances) takes ap- proximately 1,950 which is 23% less than MemGAS (2,398 s). More importantly, incremental updates-the the dominant mode in production- require only 0.01 per new node, making AssoMem practical for continuously growing memory stores.
CONCLUSION & FUTURE WORK
In this work, we addressed the critical challenge of accurate, scalable memory recall in conversa- tional AI tackling the limitations of relevance-only retrieval in large-scale memory scenarios. We proposed AssoMem, a novel framework that enhances retrieval quality and generation robustness across diverse query types. At its core, AssoMem constructs memories associative graph and employs the RITRanker system to align relevance, importance, and temporal retrieval signals. To support future research, introduce MEETINGQA, synthetic multi-speaker dataset simulating real-world meeting scenarios, with annotated QA pairs and memory usefulness labels. Extensive experiments across all datasets demonstrate the effectiveness and robustness of AssoMem.
Building upon the success of AssoMem, future research will focus on the following extensions: Extending the framework to manage memory settings that involve the accumulation of hetero- geneous and evolving histories sourced from multiple modalities: 2. Expanding the associative memory clues by incorporating richer semantic concepts such as events, locations, and external knowledge bases; 3. Developing personalized memory compression techniques to facilitate efficient on-device deployment of the associative memory system.
'We further provide comprehensive latency analysis in Appendix B.7
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Sandhini Agarwal, Lama Ahmad, Jason Ai, Sam Altman, Andy Applebaum, Edwin Arbus, Rahul K Arora, Yu Bai, Bowen Baker, Haiming Bao, et gpt-oss-120b. & gpt-oss-20b model card. arXiv preprint arXiv:2508. 10925, 2025.
Bahman Bahmani, Abdur Chowdhury, and Ashish Goel. Fast incremental and personalized pager- ank. Proceedings of the VLDB Endowment, 4(3), 2010.
Sergey Brin and Lawrence Page. The anatomy of a large-scale hypertextual web search engine. Computer networks and ISDN systems, 30(1-7)1007-11,, 1998.
Jianlv Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. Bge m3-embedding: Multi-lingual, multi-functionnality multi-granularity text embeddings through self-knowledge dis- tillation, 2023.
Prateek Chhikara, Dev Khant, Saket Aryan, Taranjeet Singh, and Deshraj Yadav. Mem0: Building production-ready ai agents with scalable long-term memory. arXiv preprint arXiv:2504. 19413, 2025.
Suchana Datta, Debasis Ganguly, Mandar Mitra, and Derek Greene. A relative information gain- based query performance prediction framework with generated query variants. ACM Transactions on Information Systems, -132):1331, 2022.
Yiming Du, Hongru Wang, Zhengyi Zhao, Bin Liang. Baojun Wang, Wanjun Zhong, Zezhong Wang, and Kam-Fai Wong. Perltqa: A personal long-term memory dataset for memory classification, retrieval, and fusion in question answering. In Proceedings of the 10th SIGHAN Workshop on Chinese Language Processing (SIGHAN-10), pp. 152-164, 2024.
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21788 2024.
Aric Hagberg, Pieter Swart, and Daniel A Schult. Exploring network structure, dynamics, and function using networkx. Technical report, Los Alamos National Laboratory (LANL), Los Alamos, NM (United States), 2008.
Yuanzhe Hu, Yu Wang, and Julian McAuley. Evaluating memory in Ilm agents via incremental multi-turn interactions. arXiv preprint arXiv:2507.05257, 2025.
Yunah Jang, Kang-il Lee, Hyunkyung Bac, Hwanhee Lee, and Kyomin Jung. IterCQR:
Iterative conversational query reformulation with retrieval guidance. In Kevin Duh, Helena Gomez, and Steven Bethard (eds.), Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies ( pers), pp. 8121-8138, Mexico City, Mexico, June 2024. Association for Computational Linguis- tics. doi: 10. 10.118663/1100 1/265/////222..naa// hww.o/.......ooooooooo.oo.o URL hat_attts//eeelannacooooooo neacl-long. 4497
(Volume I: Long Pa-
Hongda Jiang, Xinyuan Zhang, Siddhant Garg, Rishab Arora, Shiun-Zu Kuo, Jiayang Xu, Aaron Colak, and Xin Luna Dong. Memory-qa: Answering recall questions based on multimodal mem- ories. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Pro- cessing, pp. 24255-24277, 2025.
Zhengbao Jiang, Frank F Xu, Luyu Gao, Zhiqing Sun, Qian Liu, Jane Dwivedi- Yu, Yiming Yang, Jamie Callan, and Graham Neubig. Active retrieval augmented generation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 7969-7992, 2023.
Jon M Kleinberg. Authoritative sources in a hyperlinked environment. Journal of the ACM (JACM), 46(5):604-633 1999.
Amy N Langville and Carl D Meyer. Google's pagerank and beyond: The science of search engine rankings. 2011.
11

---

Published as a conference paper at ICLR 2026
Barrett Lattimer, Patrick H. Chen, Xinyuan Zhang, and Yi Yang. Fast and accurate factual incon- sistency detection over long documents. In Proceedings of the 2023 Conference on Empirical Methodd in Natural Language Processing, pp. 1691-1703, 2023.
Hao Li, Chenghao Yang, An Zhang, Yang Deng, Xiang Wang, and Tat-Seng Chua. Hello again! LLM-powered personalized agent for long-term dialogue. In Luis Chiruzzo, Alan Ritter, and Lu Wang (eds.), Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume Long Papers), Pp. 2259-52705 Albuquerque, New Mexico, April 2025. Association for Computational 187-6-18996,, doi: 11186653/1/2225.50 11122..aacl--oo sanelllong.227 URL https:
Linguisties. ISBN org naacl -ong.
//aclanthol 2oc.lanntooloog..org/222255
Yang Li, Yangyang Yu, Haohang Li, Zhi Chen, and Khaldoun Khashanah. Tradinggpt: Multi-agent system with layered memory and distinct characters for enhanced financial trading performance. arXiv preprint XXXv:2309.003366 2023.
Sheng-Chiel Lin, Akari Asai, Minghan Li, Barlas Oguz, Jimmy Lin, Yashar Mehdad, Wen-tau Yih, and Xilun Chen. How to train your dragon: Diverse augmentation towards generalizable dense retrieval. In Findings of the Association for Computational Linguistics: EMNLP 2023, pp. 6385- 6400, 2023.
Aman Madaan, Niket Tandon, Peter Clark, and Yiming Yang. Memory-assisted prompt editing to improve gpt-3 after deployment. arXiv preprint arXiv:2201.06000 2022.
Adyasha Maharana, Dong-Ho Lec, Sergey Tulyakov, Mohit Bansal, Francesco Barbieri, and Yuwei Fang. Evaluating very long-term conversational memory of Ilm agents. arXiv preprint 17Xiv:2402.17753. 2024.
Jie Ouyang, Yucong Luo, Mingyue Cheng, Daoyu Wang, Shuo Yu, Qi Liu, and Enhong Chen. Revis- iting the solution of meta KDD cup 2024: CRAG. In 2024 KDD Cup Workshop for Retrieval Aug- mented Generation, 2024. URL httt:::::// /ttp:///oppeeeieeeeeee net wtt/://oommmmmuuu///1/
Lawrence Page. Sergey Brin, Rajeev Motwani, and Terry Winograd. The pagerank citation ranking: Bringing order to the web. Technical report, Stanford infolab, 1999.
Qwen, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li, Tianyi Tang, Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang. Yu Wan, Yugiong Liu, Zeyu Cui, Zhenru Zhang, and Zihan Qiu. Qwen2.5 technical report, 2025.
URL httph: arxiv. org/ abs 2412 15115
Alireza Rezazadeh, Zichao Li, Wei Wei, and Yujia Bao. From isolated conversations to hierarchical schemas: Dynamic tree memory representation for llms. arXiv preprint arXiv:2410. /4052, 2024.
Parth Sarthi, Salman Abdullah, Aditi Tuli, Shubh Khanna, Anna Goldie, and Christopher D Man- ning. Raptor: Recursive abstractive processing for tree-organized retrieval. In The Twelfth Inter- national Conference Learning Representations, 2024.
Kanish Shah, Henil Patel, Devanshi Sanghvi, and Manan Shah. A comparative analysis of logistic regression, random forest and knn models for the text classification. Augmented Human Research, 5(1):12, 2020.
Robyn Speer, Joshua Chin, and Catherine Havasi. Conceptnet 5.5: An open multilingual graph of general knowledge. In Proceedings of the AAAI conference artificial intelligence, volume 31, 2017.
Zhen Tan, Jun Yan, I Hsu, Rujun Han, Zifeng Wang. Long T Le, Yiwen Song, Yanfei Chen, Hamid Palangi, George Lee, et al. In prospect and retrospect: Reflective memory management for long- term personalized dialogue agents. arXiv preprint 2xXx::2503.00066,, 2025.
12

---

Published as a conference paper at ICLR 2026
Leandro von Werra, Younes Belkada, Lewis Tunstall, Edward Beeching, Tristan Thrush, Nathan Lambert, Shengyi Huang, Kashif Rasul, and Quentin Gallouédec. Trl: Transformer reinforcement learning. https::/ggttieii com hwtp///uggiinggge/eeeeae
2020.
Weizhi Wang. Li Dong, Hao Cheng, Xiaodong Liu, Xifeng Yan, Jianfeng Gao, and Furu Wei. Aug- menting language models with long- memory. Advances in Neural Information Processing System,, 36:74530-74444, 2023.
Zheng Wang. Zhongyang Li, Zeren Jiang, Dandan Tu, and Wei Shi. Crafting personalized agents through eerrenal-augmmented generation on editable memory graphs. arXiv preprint arXiv:2409. 19401, 2024.
Hiroshi Wayama and Kazunari Sugiyama. Generalized personalized pagerank with graph convolu- tional networks in recommender systems, In Proceedings of the 2025 International ACM SIGIR Conference on Innovative Concepts and Theories in Information Retrieval (ICTIR), pp. 380-389, 2025.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gug- ger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pp. 48, Online, October 2020. As- sociation for Computational Linguistics. BtLhhtts:: www..wwww......wm..w........ww aclweb. org/ ant holtty/
2020. menlp--dmoss..
Di Wu, Hongwei Wang, Wenhao Yu, Yuwei Zhang, Kai- Chang, and Dong Yu. Longmemeval: Benchmarking chat assistants long-term interactive memory. arXiv preprint arXi::2410.1081., 2024a.
Wei Wu, Zhuoshi Pan, Chao Wang, Liyi Chen, Yunchu Bai, Tianfu Wang, Kun Fu, Zheng Wang, and Hui Xiong. Tokenselect: Efficient long-context inference and length extrapolation for llms via dynamic token-level kv cache selection. arXiv preprint arXiv:241 1.02886, 2024b.
Derong Xu, Yi Wen, Pengyue Jia, Yingyi Zhang. Yichao Wang, Huifeng Guo, Ruiming Tang, Xi- angyu Zhao, Enhong Chen, Tong Xu, et al. Towards multi-granularity memory association and selection for long-term conversational agents. arXiv preprint arXiv:2505. 19549, 2025a.
Wujiang Xu, Kai Mei, Hang Gao, Juntao Tan, Zujie Liang, and Yongfeng Zhang. A-mem: Agentic memory for Ilm agents. arXiv preprint arXiv:2502. 12/10, 2025b.
Xiao Yang, Kai Sun, Hao Xin, Yushi Sun, Nikita Bhalla, Xiangsen Chen, Sajal Choudhary, Rongze Daniel Gui, Ziran Will Jiang, Ziyu Jiang, Lingkun Kong, Brian Moran, Jiaqi Wang, Yifan Ethan Xu, An Yan, Chenyu Yang. Eting Yuan, Hanwen Zha, Nan Tang, Lei Chen, Nicolas Scheffer, Yue Liu, Nirav Shah, Rakesh Wanga, Anuj Kumar, Wen-tau Yih, and Xin Luna Dong. Crag comprehensive rag benchmark. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomezak, and C. Zhang (eds.), Advances in Neural In- formation Processing Systems, volume 37, pp. 10470-10490. Curran Associates, Inc., 2024. URL https: https://proceedingss neur ips cctpaper_raaaaeeeeee files/ htllees/paper/222//5555 Datasetsaaattaaaa. and Benchmarks
4433220ffaaS33308880000053338883 patasaasssspppp...-aaaaassssssss Track.pdf.
Hongli Yu, Tinghong Chen, Jiangtao Feng, Jiangjic Chen, Weinan Dai, Qiying Yu, Ya-Qin Zhang, Wei-Ying Ma, Jingjing Liu, Mingxuan Wang. et al. Memagent: Reshaping long-contex: 11m with multi-conv rl-based memory agent. arXiv preprint arXiv:2507.02255) 2025.
Chenhan Yuan, Qianqian Xie, Jimin Huang, and Sophia Ananiadou. Back to the future: Towards explainable temporal reasoning with large language models. In Proceedings of the ACM Web Conference 2024, pp. 1963-1974, 2024.
Kai Zhang, Kaisong Song, Yangyang Kang, and Xiaozhong Liu. Content- and topology-awe aware representation learning for scientific multi-literature. Houda Bouamor, Juan Pino, and Ka- lika Bali (eds.), Proceedings of the 2023 Conference Empirical Methods in Natural Language
13

---

Published as a conference paper at ICLR 2026
Processing, pp. 7490-7502, Singapore, December 2023. Association for Computational Linguis- tics. doi: 10. 10.18653/w//0222 2e122222...mmpmmmmem www.l--mainnn0n0 465. URL /ttl://pp////lllloooooooooo.... emnlp-main. 4657
Kai Zhang, Yangyang Kang, Fubang Zhao, and Xiaozhong Liu. LLM-based medical assistant per- sonalization with short- and long-term memory coordination. In Kevin Duh, Helena Gomez, and Steven Bethard (eds.), Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume I: Long Papers), pp. 2386-2398, Mexico City, Mexico, June 2024, Association for Computational org/
Linguistics. doi: 10. 18611866/ hww.:224..aa--oon...o URL 2024. www.naaaclllonngo 132/
Wanjun Zhong, Lianghong Guo, Qiqi Gao, He Ye, and Yanlin Wang. Memorybank: Enhancing large language models with long-term memory. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 19724-19731, 2024.
Zijian Zhou, Ao Qu Zhaoxuan Wu, Sunghwan Kim, Alok Prakash, Danicla Rus, Jinhua Zhao, Bryan Kian Hsiang Low, and Paul Pu Liang. Meml: Learning to synergize memory and reasoning 2025.
for efficient long-horizon agents. arXiv preprint
A APPENDIX. DATASETS DETAILS
Table 6: Corpus statistics for LongMemEval and MeetingQA.
Averages
Haystack Totals
Dataset
#Sessions
Evid./Q
Q tokens
A tokens
Utts./sess.
Session tokens
Mem Tokens (m)
LongMemEval.s 50.2
1.9
17.3
9.9
9.83
2053
0.10
LomgMemEval
501.9
1.9
17.3
9.9
9.76
2016
1.01
LongMemEvalJ
2503.4
1.9
17.3
9.76
2016
5.13
MeetingAA
50
1
10.17
2.18
72.32
1848.62
0.09
"Evid./Q" = average number of answer evidences per question; "Utts//ess: = average utterances per session; m stands for million.
As detailed in Table we evaluate all the baselines and proposed AssoMem across four datasets. Among them, the LongMemEval.s and LongMemEval m are open benchmarks. We enlarge this benchmark into larger scale by incorporating more dialog sessions into the memory and form the LongMemEval. which consists of 2,500 dialog sessions. Notably, the added dialog sessions are identical, ensuring the dataset quality.
Further, we mimic the real-world meeting scenario where different speakers will speak during meeting, yielding meeting session with different turns, and construct new dataset named Meet- ingQA. The purpose of this dataset is for evaluating memory recall question answering in multi-turn dialog scenario. The MeetingQA dataset is a collection of 50 meetings, each containing approxi- mately 70-80 messages, with variety of speaker configurations (ranging from 2 to 6 speakers per memo). The dataset was generated by linguistic engineer using L.LMs and is designed to support benchmarking and development of meeting recall systems. It includes 390 QA pairs, where each answer is mapped to at least one specific message within corresponding meeting. The dataset structure provides detailed fields such message IDs, session IDs, speaker identifiers (typically generic unless manually annotated), and transcript messages.
In tandem. the data used in the experiments follows this structure: {question: ... .... question date: sessions: session id: 1, utterance: ..., date: ...}. session id: 2, utterance: date: ...) ...) We organize the dialog memories and retrieve utterances answer the question.
14

---

Published as a conference paper at ICLR 2026
Table 7: Question types and required retrieval dimensions
Question
R
P
T
Where did I have dinner yesterday?
V
V
What do usually say at work?
Most visited coffee shop last month?
How long have been since start my job?
Table 8: Retrieval and QA performance on Long small s.
Method
R@1
R@3
R@6
@@10 10
nDCG@6 nDCG@10 Ace@6 BERTScore
Utterance-level: retrieve top-k utterances and return
context
Utterance-
52.43
65.55
79.63
93.19
75.37
83.23
59.13
68.61
LST Memory
49.20
63.93
82.98
93.50
79.77
83.66
62.17
74.39
Embedding.cam cat
50.13
60.90
76.25
86.17
71.93
80.21
55.96
58.06
Session-level: retrieve top-k sessions and return sessions context Session-level
57.04
66.19
81.63
94.96
78.15
85.72
53.67
56.17
Hybrid: first retrieve topic/session, then utterances within; return utferances context Auuti-granularity
59.92
76.73
83.89
94.08
82.59
85.61
63.74
73.61
Session summary mem.
58.21
77.06
79.96
85.17
77.18
79.48
60.37
70.73
Topic grouping mem.
61.84
81.73
90.07
96.31
88.63
91.47
68.11
78.91
AssoMem
62.73
81.96
93.87
96.96
90.31
91.93
72.13
80.65
B APPENDIX. SUPPLEMENTARY RESULTS
B. RESULTS ON LONGMEMEVAL.S
Table corroborates the main-text trends under the small-s setting. utterance retrieval remains weakest, while session-only retrieval improves recall but limits QA due to context dilution. Hybrid pipelines are consistently stronger: multi granularity reaches 78.97 R@ 10 and 76.50 nDCG@ 10 with 55.85 Acc@6, and topic grouping further lifts retrieval 79.14 R @ 10 and 78.86 nDCG@ 10 with 59.95 Acc@6. AssoMem attains the best results across all metrics
.6 R@ 10 and 82.93 nDCG@ 10 translating into 64.01 Acc@6 6 and 67.56 BERTScore, which improves
the ut- terance baseline by 14.78 R @ 10, 14.89 nDCG@ 10, 15.35 Acc@6, and 15.85 BERTScore; it also surpasses topic grouping by 5.82 R @ 10 and 4.06 Acc @6. The persistence of these gains at smaller indicates that popularity- and temporal-aware re-ranking continues to suppress stale or idiosyncratic items and surfaces recent, widely supported evidence, yielding both higher retrieval concentration and better downstream generation. The results presented in Bbbleccconsittenn validate our findings as in the main text.
B.2
WHY PPR?
To better understand how importance impact on the performance, further provide the result com- parisons between personalized pagerank and pagerank. As presented in Figure f (a), we witnessed that the ppr consistently outperforms the pr in mongle-ssssion-rrfeeeeeee be attributed to the fact that answering preference question requires both the relevance for locating the event and the importance for knowing the most important memories for the located event. In this sense, ppr provides better importance due to the fact that ppr considers relevance in both the algorithm initialization and teleportation.
type question which can
B.3 PERFORMANCE GAP BETWEEN RETRIEVAL AND GENERATION
Across top-k, retrieval monotonically improves for all methods, with AssoMem leading at every k and saturating near 0.85 by k=10, ahead Topic Grouping and Multi-granularity that level off around 0.80 and 0.79. QA accuracy, however, plateaus much earlier: AssoMem rises quickly to about 0.66 by k= and then gains marginally: Topic Grouping continues to climb and ends highest near 0.80, while Multi-granularity trails and tops out near 0.60. The widening retrieval-accuracy
15

---

Published as a conference paper at ICLR 2026
0.7
5o0ne
0.6
Muti gisrusstty Accumey
MuD-gitzMeitt Rebieval
Topic Grouping Mor Acturacy
Granping Marn Retravd
Accoklem Accuacy
Asnoklem Recieval
2
Sending
Top-k
(a) PPR PR different graph structure
(b) Recall Generation of representative baselines and AssoMem
Figure 4: Supplementary results
Granularity
Retriever
LongMemEval Lomg
LongMemEval
MeetingQA
DP
72.54
58.31
44.62
42.94
Utterance
BGE
78.89
64.18
48.73
48.27
DPR
79.63
64.25
49.91
48.81
DP
77.26
64.81
50.33
50.79
Session
DPR
81.63
70.86
55.99
56.72
BGE
83.79
72.17
56.43
55.92
Table 9: Recall performance of three retrievers: DragonPlus, DragonPlus-Roberta and BGE (Large) across datasets in two granularity levels.
gap beyond when is indicates diminishing returns from adding more context and highlights utilization bottleneck: even with superior retrieval, the LLM cannot fully exploit larger evidence sets due redundancy, distraction, context-window limits. Practically, this suggests sweet spot around k=6 and the need for post-retrieval evidence organization (summarization, attribution, or re- (e-weighting) to convert AssoMem's 's retrieval gains into further accuracy improvements.
B.4 RETRIEVER COMPARISONS
Retrievers play a pivotal role the modern memory systems, providing semantic relevance infor- mation for locating useful memory records. As presented in Table we witness that in general, DragonPlus shows sub-optimal performance compared with other two retrievers across 4 datasets in two granularities. Furthermore, DragonPlusRoberta consistently outperforms other two retrievers by 7.09% 5. 1.94%/0.07%% 5.29% /1. 18% and 5.87%/0..54% across 4 datasets in utterance-level retrieval, respectively. This indicates comparable performance between DPR and BGE (Large). Moreover, in session level retrieval, BGE (Large) on the opposite, consistently outperforms other two retrievers by 6.53% 7.36% 1% and 6. 1%/0.44% across three LongMemEval datasets, respectively. We attribute this to the fact that the larger context window of BGE (large) helps boost retrieval performance in session level since the context is longer. However, session-level retrieval on MeetingQA, the performance of BGE (large) is suboptimal compared to DPR, this is because the token length per session MeetingQA is not comparable with LongMemEval and can be handled by DPR. In tandem, better than BGE (Large) and since our most fine -gainer granularity in this work is utterance-level retrieval opt for DPR the main retrieving embedding model.
conclude that within the capability of DP model, DPR is
16

---

Published as a conference paper at ICLR 2026
Question Type
Ground- ory Evidence
Mem-
Top-2 (Relevance-Only)
Retrieval
Top-2 Retrieval (As- soMem)
Q1: Preference reason- ing
always start planning too late, it's like we 're just reacting.
Error: worked late few nights this week, working the new API integration.
honest, I think the results on project memory is too good to trust.
Correet:
"What do usually com-
plain at work, you give some tips to avoid it?"
Every time we plan, there seems no room for unex- pected stuff-
Error: The search team was complaining our work progress in the sync.
honest, I think search team can help us on the project launching.
Correct:
ffigh--mporrtacc opin- ions repeated across his- tory.)
(Semantically related "work," but not represen- tative of core opinions.)
(lligh-mmportance,
repeated memory re- trieved.)
Q2: Temporal reason- ing
I just finished watching the third season of "The Crown' on Netflix.
Do you have similar show suggestions such as 'the Crown', 'the Game of Throne"?
Error:
started watching 'the Game
Correct:
"Which show did watch first, the Crown or the Game of Throne?"
of Throne' about one
I started watching 'the Game of Throne' about month ago.
month ago.
Error: "Game of Throne" is epic show; I'm pretty sure you 're hooked. If 'r looking for some similar shows "Game of Throne", have some recommmenda- tions.
Correct: just finished watching the third sea- son of 'The Crown' on Netflix.
(Provides necessary tem- poral anchors.)
(Accurately supports temporal comparison.)
(Topically related but laeks any temporal sig- nal.)
Table 10: Case study showing limitations of relevance-only retrieval across question types. While relevance methods retrieve semantically similar content, they often miss task-relevant evidence. As- soMem incorporates importance and temporal signals enable more accurate memory selection.
B.5 AsSOMEM WITH DIFFERENT FUSION STRATEGIES
Further, we compare 6 information strategy from two types: Information theory driven weight as- signment: Information Gain (Datta et al. 2022), Mutual Information (Zhang et al. 2023): Learnable weight assignment (Shah et al.]
2020 Logistic Regression (LR), Random Forest (RF). Support Vec-
Table 11: AssoMem with different fusion strategies.
Fusion strategy
Recall@6
Recall@ 10
Acc@6
tor Machine (SVM), Two-layer linear network (LN).
Learnable weight assignment
Logistic Regression Random Forest
72.92
78.55
58.43
Impact of Different Information Fusion Strategy We further present the results of comparing different weight assignment strategies as in Ta- ble [I] We can observe the perfor- mance of information driven weight assignment strategies in general out- performs the learnable weight assign- ment strategies which we attribute this to the fact in the conversational memory recall question answering scenario, the memory evidence for each question sparse which poses difficulties for simply training learnable model to do weight assignments. On the other hand, information driven strategies use the information purity as the signal which mitigates the memory sparsity influence. Another benefit for using information driven strategy is training-fres learnable strategies to handle weight assignment yet the trade-off might also factor that hinders its application in this scenario.
77.92
80.69
61.71
Linear Network Support Vector Machine
77.
80.63
61.74
78.81
81.66
62.11
Information driven weight assignment Information gain Mutual information
80.94
86.33
63.84
82.64
88,91
64.20
see a great potential of
17

---

Published as a conference paper at ICLR 2026
Table 12: Overview of baseline methods by granularity
Granularity
Method
Description
Long- and short-term mem.
Partitions user memory into long- and short-term compo- nents to enable better coordination during memory recall
Utterance level
Zhang et al. 2024
QA.
Flat Retrieval
Retrieves utterances directly based on relevance scores.
Session level
Flat Retrieval
Retrieves entire sessions based overall relevance the query.
Session-utterance
First retrieves relevant sessions (based full session content), then retrieves relevant utterances within those sessions.
Multi Multi-gramularity
Session summary-utterance
First retrieves sessions based on session summaries, fol- lowed by utterance-level retrieval within those sessions. Groups utterances by topic, retrieves relevant topies, then selects utterances within those retrieved topic groups. Uses four gramularities (utterance, session, keyword, summary) with entropy-guided weighting for retrieval across these levels.
Topic-ure utticcuuterance
grouping
TTan et al. 2025) MemGAS
Xu et al. 2025a
B.6 CASE STUDY
Table presents a case study of retrieval behavior across two question types- preference reasoning and temporal reasoning highlighting the limitations of relevance-only retrieval and the improve- ments achieved by AssoMem. In Q1 (Preference reasoning), the asks: "What do usually say at work?" This question requires retrieval of high-importance utterances that reflect consistent pat- terns in user opinions. However, the relevance-only method retrieves memories that are superficially related to "work" but do not reflect the user's repeated viewpoints. In contrast, AssoMem suc- cessfully identifies two high-importance memory entries expressing critical opinions about project memory- these are not only topically relevant but also semantically central to the user's s past behav- ior. This demonstrates AssoMem's 's ability to incorporate an importance signal, which is essential for preference-centric questions. In Q2 (Temporal reasoning), the user asks: "Which show did I watch first, the Crown or the Game of Throne?" This question necessitates precise temporal comparison, which relevance-only methods fail to address. The retrieved responses thematically related the queried shows but offer no chronological cues. In contrast, AssoMem correctly surfaces two time- anchored memory entries indicating both the start time of The Game of Throne and the completion of The Crown, enabling accurate temporal reasoning. In tandem, the case study confirms that rele- vance alone is insufficient for diverse real-world queries. By incorporating importance and temporal dynamics, AssoMem delivers more contextually accurate retrieval, leading to better generation.
B.7 LATENCY ANALYSIS
We report the average latency (in seconds) of each core component in the AssoMempippline to assess its effi- ciency. The most time-consuming operation is mem- ory construction, taking approximately 1948.99 seconds on average, which occurs as a one-time offline process build the structured memory graph. In contrast, in- cremental operations during inference are significantly faster. Node addition and edge addition require only 0.01 and 13.96 seconds respectively, enabling dynamic updates with low overhead. RIT scoring and weight assignment, which compute information-theoreed vance and balance multi-dimensional signals, incur negligible latency of 0.39 and 0.26 seconds. The final inference stage, which utilizes retrieved and weighted memory context for LLM gen- eration, completes within 0.74 seconds on average. These results demonstrate that although initial memory graph construction is computationally expensive, the online inference and memory aug-
Table 13: AssoMem latency statisties.
Operation
Avg. Latency (s)
Graph Construction
1948.99
Node Addition
0.01
Edge Adding
13.96
RIT Scoring
0.39
Weight Assignment
0.26
Answer Generation
0.74
rele-
18

---

Published as a conference paper at ICLR 2026
mentation steps remain efficient, ensuring the system is practical for real-time applications with dynamic memory updates.
B.8 CMI FUSION VS. NON-ADAPTIVE BASELINES
Table 10 (Appendix B.5) compares CMI against sev- eral learnable and information-theoreti fusion strate- gies. Here we further include non-adaptive fixed-weight baseline to isolate the benefit of adaptive weighting. As shown Table CMI outperforms the fixed assignment by 7.7% in @ecall@6 and 8.1% in Acc@ The non- adaptive baseline applies single of weights across all query types, which is suboptimal because different ques- tion categories (e.g., temporal reasoning vs. preference) rely on different signal distributions. CMI captures these conditional dependencies between each scoring dimen-
Table 14: Fusion strategy comparison Long Non-adaptive assigns fixed weights 0.6 0.60033/0 /0.1 to
R
Fusion Strategy
R@6
R@ 10
Acc@6
Non-adaptive SVM
76.70 76.70
80.80
59.38
78.81
81.66
62.11
Information Gain
80.94
86.33
63.94
CMI (ours)
82.64
88.91
64.20
sion and the usefulness label, yielding consistently higher retrieval and generation quality.
B.9 LONG-CONTEXT VS. RETRIEVAL-AUGMENTTE APPROACHES
alternative to retrieval-augmentted generation is to feed the entire conversational history directly into long-context LLM. We evaluate this setting on Long- with its full
Table 15: Long-context vs. RAG on LemEaam.
MemEval using Bi0ma--..-70B-Insttuu 128k context window. As reported in Table 13 the long- context approach achieves only 21.93% Acc@6, substan- tially below even the weakest RAG baseline (HippoRAG, 48.61 %). This confirms that naively extending the con- text window is insufficient for similarity- dense memory scenarios: the model struggles locate and attend relevant evidence among thousands of mantically similar utterances. AssoMem's 's structured retrieval and multi-signal ranking effectively address this challenge, achieving 64.01 % Ace@6 2.9x improvement over the long-context base- line.
Approach
Acc@
Long-Context (Llama-3.3-70B)
21.93
RAG (HippoRAG) AssoMem (ours)
48.61
64.01
C BASELINES
The detailed descriptions are presented in Table 12
D APPENDIX. PROMPTS
Generation Prompt:
### TASK DESCRIPTION
You are helpful assistant that user's question. In this sense, you will have access to user's memory records which contain user's historical information.
Please note you will need to identify if the memories are useful not for you to respond to the query. If the memories are useful then answer the question based on the memories, otherwise answer the question based on your knowledge or answer "IDK".
## INPUT
User memory: (memory) User query: (question)
### OUTPUT REQUI REMENT
19

---

Published as a conference paper at ICLR 2026
Output the or not, please only output the answer and nothing else.
the questicn only. Not matter you the memory
Topic Generation Prompt:
### TASK DESCRIPTION
You are a helpful assistant that helps users to organize their memory records. Next, you'll help me in organizing user's memory zecerds.
Given user's historical dialogue session, please summarize the session into concise topic summary without key information lest. Output the topic summary sentence.
II INPUT Dialogue session: (session)
### OUTPUT REQUIREMENT
Gencrate a topic summary for the given session. Please only output the topic summary and nothing clse.
LIM-as-a-Judge Prompt:
#+ TASK DESCRIPTION
are helpful judge to evaluate the quality of the response user question.
You will be given a user question and two responses: one is the golden response, one is the generated response.
Please evaluate the quality of the generated response based the following criteria:
a) If the response is relevant to the user question.
b) If the response answers the question or not. c) If the response is consistent and coherent.
you think the generated respense meets these criteria semantically responds to the user as the golden response does, you should output "Win", otherwise "Lose".
## INPUT
User query: (question)
Golden Response: (response_l}
Generated Response: (response_2)
### OUTPUT REQUIREMENT
Output "Win" or "Lose" only. Do not output anything else.
Fine-tuning Sample:
of oattr
Cutting Knowledge Date: December 2023
Today Date: 26 Aug 2025
#*# TASK DESCRIPTION
You are helpful assistant that answers user's questions. In this sense, you will have access to user's memory records which contain user's historical information.
memories are useful or If the memories are useful then answer the question based on the memeries, otherwise answer the question based on your knowledge or "IDK".
Please note you will need identify
not for you to answer the query.
### OUTPUT REQUIREMENT
Output the answer to the questien only, No matter you use the menory not, please only output the answer and nothing else.<|eot, < bx<]staat_header__t buaer((end_hadder_idd)
20

---

Published as a conference paper at ICLR 2026
The user query is:
What is the order of the three events: 'I signed up for the rewards progran. at ShopRite', 'I used a Buy One Get One Free coupon on Lavs diapers at Walnart', and 'I redeemed $12 cashback for $10 Amazon gift card from Ibotta'?
#* The memory is:
I' m planning a trip to Walmart this weekend and I'm looking for some deals en baby essentials. Do you have any info their current sales promotions
diapers? By the way, used Buy One Get One Free coupon on luvs diapers great deal!; I'm planning a shopping trip to Target this weekend and I'm wondering if you have any info on their current sales and promotions. By the way, I just redeemed $12 cashback for a $10 Amazon gift card from Ibotta today, about my savings so faz!;I'm trying to plan my grocery shopping trip for this woek. can you help mo find any good deals or salcs on diapers and formula at ShopRite? By the way, signed up for their rewards program today, I'm hoping maximize my points and <errings.<eeot__.. ></ s|arttt__heddrr bassissassnnsseeese [nd__eeade____e >>
walmart today, which was
I'm feeling pretty good
## The answer is:First, I used Buy One Get One Free coupon Luvs diapers Walmart. Then, redeemed $12 cashback for $10 Amazon gift card from Ibotta. Finally, signed up for the rewards program at ShopRite. <let_idd|>>>>>
E USE OF LLM
In paper writing, we use LLMs solely for checking typos and grammar errors; they are not used for any other purposes beyond this.
21


Reviews:
[{"review_id": "YLn8ax6Lyl", "reviewer": "Reviewer_1BTS", "summary": "This paper proposes a novel memory QA framework, AssoMem, designed to address the challenge of accurate recall for AI assistants operating on large-scale, similarity-dense memory repositories. Existing methods, which rely primarily on semantic relevance between the query and memory, perform poorly in these \"similarity-dense\" scenarios. Inspired by human associative memory, AssoMem constructs an \"associative memory graph\" that anchors dialogue utterances to automatically extracted \"clues.\" This graph structure enables \"importance-aware\" ranking of memories. The core of the framework is a retriever called RITRanker, which innovatively integrates three distinct signals: Relevance, Importance, and Temporal alignment. These signals are weighted using an adaptive fusion strategy driven by CMI to dynamically adjust each signal's contribution based on the query's intent. The paper also introduces a new benchmark, MeetingQA. Experiments demonstrate that AssoMem outperforms existing baselines across several datasets.", "questions": "1. The CMI fusion strategy appears to require (query, utterance, usefulness label) triples for training. How is this model trained? Does this imply AssoMem must be deployed only after a significant data annotation effort for a specific user, or is it trained once on the benchmark datasets and then generalized?\n2. The paper mentions using query-personalized PPR. What is the actual latency of this PPR computation on the large graph (LongMemEval_l), and how does this latency scale as the graph grows from 2.5k to 100k sessions?\n\n*Typo:\nLine 82: Associative memory graph -> Associative memory graph:", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. The paper correctly identifies a core weakness in existing RAG and memory systems: relevance-only retrieval is ineffective in \"similarity-dense\" scenarios.\n2. The introduction of the MeetingQA dataset is a useful contribution to the community, addressing the need for benchmarks that simulate real-world, multi-turn dialogue scenarios.", "weaknesses": "1. The paper claims scalability, but the offline graph construction (which includes LLM-based tagging for every session) takes ~1950 seconds (Table 12). This is for a small dataset of 2,500 sessions. When the memory scales to the 100k or 1M entries expected of a \"second brain,\" this offline cost seems prohibitively expensive, challenging the \"scalable\" claim.\n2. The \"importance\" score relies on running Personalized PageRank at query time (Section 3.2.3, \"the utterance cells in t are set to the similarity between query and utterance\"). This requires a graph computation for every query, which can be slow on large graphs. The paper does not analyze this specific query latency, only the overall \"RIT Scoring\" (0.39s).\n3. The paper bundles AssoMem with a generator LLM fine-tuning strategy (Section 3.3). This conflates the contributions. The main results in Table 1 represent the full system (including the fine-tuned LLM). This comparison is potentially unfair, as the baseline methods (e.g., \"Topic grouping\") do not appear to receive this specialized denoising and multi-task fine-tuning. The performance gain attributed to AssoMem might stem not just from the superior retriever but also from a specially-tuned generator.\n4. A key innovation is the adaptive CMI fusion. However, the ablation study (Table 10) comparing it to other fusion strategies (LR, RF, SVM) is relegated to the appendix. More importantly, it is missing the most crucial baseline: a simple, non-adaptive weighted sum.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "qEeVRK8Uje", "reviewer": "Reviewer_xUZR", "summary": "The paper proposes AssoMem, a memory question-answering framework designed to improve large-scale memory retrieval for LLM-based assistants. The key idea is to model associative human-like recall through a multi-signal retrieval mechanism that integrates relevance, importance, and temporal dimensions. It constructs an associative memory graph linking utterances to automatically extracted ”clues“，employs Personalized PageRank for importance scoring, and fuses multi-dimensional scores using a mutual information–based weighting strategy, with a specific fine-tuning strategy for the LLM. In addition, the paper also introduce new benchmark, MeetingQA, to simulate real-world meeting scenarios where multi-turn dialogues form the memory base paired with diverse QA examples. Experiments on LongMemEval and a new synthetic benchmark, MeetingQA, show consistent improvements over baselines.", "questions": "See weakness", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. The benchmark is valueable.\n2. The proposed method is effective as valied by comprehensive experiments.", "weaknesses": "1. The method is overly complex yet underexplained. There are many components in the proposed method, i.e., associative memory graph,  multi-dimensional scores fusion and specific fine-tuning strategy. Desipte putting them together leads to better performance, it is unclear current design is the best way or it is worthy. For example, what if combine MemGAS with later multi-dimensional scores fusion? What if do not finetune the model?\n2. The associative memory graph construction and page rank part is quite similiar with MemGAS, which weaken the contribution in the method side. \n3. Some details or experiments are not clear. for example, how to make sure the quality and diversity of introduced benchmark? the detailed latency comparison with other baselines.\n4. The paper can benefit with improved writing and presentation. The text is technically overloaded, using jargon and formulaic notations without sufficient narrative clarity. Figures and examples provide limited intuition.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Zx0aArDXvj", "reviewer": "Reviewer_P9dL", "summary": "This paper introduces AssoMem, a memory-augmented question answering framework that constructs an associative memory graph to link dialogue utterances with automatically extracted clues. By combining multi-dimensional retrieval signals—relevance, importance, and temporal alignment—using a mutual information-driven fusion strategy, AssoMem achieves importance-aware and context-sensitive memory recall. Extensive experiments across three benchmarks, including the newly proposed MeetingQA dataset, demonstrate its superiority.", "questions": "See weakness.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. AssoMem integrates relevance, importance, and temporal alignment effectively, enabling context-rich and adaptive memory recall for QA tasks.\n\n2. The use of an associative memory graph is a contribution, facilitating semantic connections and improving memory organization.\n\n3. Introduction of the MeetingQA benchmark and extensive experiments validate AssoMem's superiority, with clear performance gains over prior methods.", "weaknesses": "1. The innovation of the paper appears to be limited, as the idea of capturing relationships between memories using graph representations has already been explored by several existing methods, such as HippoRAG [1] and A-Mem [2].\n\n2. The experimental comparisons in the paper seem insufficient. In addition to the aforementioned methods, the authors have not compared their approach with other well-known memory-based methods, such as mem0 [3].\n\n3. I find the model fine-tuning process described in Section 3.3 confusing. For QA tasks, wouldn't prior fine-tuning risk leaking information about subsequent queries? Furthermore, such a fine-tuning process seems to lack practical application value in real-world scenarios.\n\n4.  The construction details of the MeetingQA dataset are insufficiently described, and the dataset has not been made publicly available. This raises concerns about the fairness and reliability of the dataset.\n\n5. Some formulas in the paper lack clear explanations of their symbols. For instance, the meaning of $y_m^\\lambda$ in Eq. (3) is not adequately clarified.\n\n[1] Jimenez Gutierrez, Bernal, et al. \"Hipporag: Neurobiologically inspired long-term memory for large language models.\" Advances in Neural Information Processing Systems 37 (2024): 59532-59569.\n\n[2] Xu, Wujiang, et al. \"A-mem: Agentic memory for llm agents.\" arXiv preprint arXiv:2502.12110 (2025).\n\n[3] Chhikara, Prateek, et al. \"Mem0: Building production-ready ai agents with scalable long-term memory.\" arXiv preprint arXiv:2504.19413 (2025).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "rVTXA3CtQ8", "reviewer": "Reviewer_xRKF", "summary": "The paper proposes AssoMem, a memory question answering framework for large, similarity-dense conversational histories. It constructs an associative memory graph that links LLM-generated “clues” to raw utterances, retrieves top clues, and then ranks candidate utterances. The ranking component, RITRanker, fuses relevance, importance estimated by query-personalized PageRank, and temporal alignment, with mutual-information guided weights that vary by question type. A multi-task denoising fine-tuning step is used to improve small models’ answer generation ability from noisy retrieved context. Experiments on LongMemEval and the new MeetingQA benchmark show consistent gains over utterance, session, and hybrid baselines.", "questions": "Please refer to the weaknesses.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This paper designs novel memory structure and ranking metrics. The method bridges the relevance-only limitation of existing memory and retrieval methods.  \n1. The empirical retrieval performance outperforms the baselines by a large margin. Multiple models and baselines are studied. \n1. This paper contributes a new MeetingQA dataset that can be used for testing long-term memory systems in different scenarios.", "weaknesses": "1. HippoRAG [1,2] designs a graph-structured memory as well and also uses PPR. I would like to see some comparisons as this paper is also a graph-based method.  \n1. Several designs (clue generation, temporal token) are bound to chat-based tasks. By comparison, general memory architectures can be applied to and are actually tested on long-context and RAG tasks. [2, 3]\n1. More details regarding the fine-tuning needs to be discussed. It seems that the fine-tuning is an optional component and does not fit with the entire memory architecture. However, it indeed affects the memory read performance a lot, according to Table 2. It is rare to specially design a fine-tuning procedure just to improve the narrow task of “reading chat memory retrieved from a graph index”. Does the fine-tuning also improve the model’s general long-context ability? Conversely, are existing long-context training recipes/models (e.g., [4] [5]) applicable to this paper?\n1. I would like to see more cost analysis both in terms of latency and in terms of tokens consumed. \n1. I would like to see more details and further analyses to show the value of the new dataset. \n\n[1] Hipporag: Neurobiologically inspired long-term memory for large language models. Gutiérrez et al., 2024.\n\n[2] From RAG to Memory: Non-Parametric Continual Learning for Large Language Models. Gutiérrez et al., 2025. \n\n[3] M+: Extending memoryllm with scalable long-term memory. Wang et al., 2025.\n\n[4] Make your llm fully utilize the context. An et al., 2024. \n\n[5]  Wildlong: Synthesizing realistic long-context instruction data at scale. Li et al., 2025.\n\n\nI am willing to raise my ratings if these concerns are sufficiently addressed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)