Evoking User Memory: Personalizing LLM via Recollection-Familiarity Adaptive Retrieval
Authors: Yingyi Zhang
Junyi Li
Wenlin Zhang
Pengyue Jia
Xianneng Li
Yichao Wang
Derong Xu
Yi Wen
Huifeng Guo
Yong Liu
Xiangyu Zhao
Venue: ICLR 2026 Poster (2026)
Keywords: Large Language Model
Memory Retrieval
Recollection-Familiarity Dual Process
Personalization
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: We propose RF-Mem, a memory retriever for personalized LLMs. Inspired by the Recollection–Familiarity theory, it adaptively switches between Familiarity one-shot and Recollection strepwise retrieval,  enabling evidence reconstruction in retrieval.

Abstract: Personalized large language models (LLMs) rely on memory retrieval to incorporate user-specific histories, preferences, and contexts. Existing approaches either overload the LLM by feeding all the user's past memory into the prompt, which is costly and unscalable, or simplify retrieval into a one-shot similarity search, which captures only surface matches. Cognitive science, however, shows that human memory operates through a dual process: Familiarity, offering fast but coarse recognition, and Recollection, enabling deliberate, chain-like reconstruction for deeply recovering episodic content. 
Current systems lack both the ability to perform recollection retrieval and mechanisms to adaptively switch between the dual retrieval paths, leading to either insufficient recall or the inclusion of noise.
To address this, we propose RF-Mem (Recollection–Familiarity Memory Retrieval), a familiarity uncertainty-guided dual-path memory retriever. 
RF-Mem measures the familiarity signal through the mean score and entropy. High familiarity leads to the direct top-$K$ Familiarity retrieval path, while low familiarity activates the Recollection path. In the Recollection path, the system clusters candidate memories and applies $\alpha$-mix with the query to iteratively expand evidence in embedding space, simulating deliberate contextual reconstruction.
This design embeds human-like dual-process recognition into the retriever, avoiding full-context overhead and enabling scalable, adaptive personalization. Experiments across three benchmarks and corpus scales demonstrate that RF-Mem consistently outperforms both one-shot retrieval and full-context reasoning under fixed budget and latency constraints. 
Our code can be found in the Reproducibility Statement.

Published as a conference paper at ICLR 2026
EVOKING USER MEMORY: PERSONALIZING LLM VIA RECOLLECTION-FAMILAARYY ADAPTIVE RETRIEVAL
Yingyi Zhang! Junyi Li Wenlin Zhang", Penyue Jia², Xianneng Li Yichao Wang* Derong Xu Yi Wen", Huifeng Guo', Yong Liu', Xiangyu Zhao 'Dalian University of Technology, *City University of Hong Kong. 'Huawei Technologies Ltd., "University of Science and Technology of China xi xiannenge dlut. edu. en, wangyichao 50 @huawei. com, xianzhao@cityu..oooooooo edu edu.hk
ABSTRACT
Personalized large language models (LLMs) rely on memory retrieval to incorpo- user-specific histories, preferences, and contexts. Existing approaches either overload the LLM by feeding all the user's past memory into the prompt, which is costly and unscalable, or simplify retrieval into a one-shot similarity search, which captures only surface matches. Cognitive science, however, shows that human memory operates through a dual process: Familiarity, offering fast but coarse recognition, and Recollection, enabling deliberate, chain-like reconstruc- tion for deeply recovering episodic content. Current systems lack both the ability to perform recollection retrieval and mechanisms to adaptively switch between the dual retrieval paths, leading to either insufficient recall or the inclusion of noise. To address this, propose RF-Mem (Recollection -aaniiiii---mmiiaiittt Memory Retrieval), a familiarity uncertainty-guiiee measures the familiarity signal through the mean score and entropy. High famil- iarity leads to the direct top-K Familiarity retrieval path, while low familiarity activates the Recollection path. In the Recollection path, the system clusters can- didate memories and applies a-mix with the query to iteratively expand evidence in embedding space, simulating deliberate contextual reconstruction. This de- sign embeds human- dual dual-process recognition into the retriever, avoiding full- context overhead and enabling scalable, adaptive personalization. Experiments across three benchmarks and corpus scales demonstrate that RF-Mem consistently outperforms both one-shot retrieval and full-context reasoning under fixed budget and latency constraints. Our code can be found in the Reproducibility Statement.
dual-path memory retriever. RF-Mem
INTRODUCTION
Large Language Models (LLMs) have demonstrated remarkable performance when augmented with retrieval mechanisms. Traditional retrieval -aageeeee generation (RAG) primarily targets open- domain corpora, seeking to retrieve and integrate objective facts (Lewis et al., 2020; Li et al., 2025d; Chen et al., 2024; Han et al., 2024). In contrast, memory retrieval focuses on user-specific histories, preferences, and contextualized interactions, aiming to surface evidence tailored to particular user at specific moment (Zhong et al., 2024; Jiang et al., 2025; Tan et al., 2025a; Wu et al., 2025a; Xu al., 2025c), as illustrated the top of Figure The design of memory fundamentally shapes the boundary of personalized LLMs (Zhang et al., 2025b): it can remain a static external index passively queried, or evolve into a dynamic process of recollection, thereby endowing the system with more human-like flow of memory (Wu et al., 2025b; Hatalis et al., 2023). Just as humans sometimes rec- ognize familiar face instantly (i.e., an intuitive "feeling of knowing" without deliberate reasoning) and at other times reconstruct past experiences through slow chains of recollection, so too should memory retrieval move beyond static lookup. A personalized LLM needs to be flexible and alternate between fast recognition and gradual reconstruction, adapting to the demands of the interaction.
"Equal contribution.
1 "ooresponning author.

---

Published as a conference paper at ICLR 2026
User Memory
information
reluted information
User healihicare
genuiscly offoch mife...
flcus comeesiional musliiio;; tie strcctree mechooloogy and mollooesssosee peactionr nomeaate
...lese moduliiis complemant connentiioal
The journay of dlcooeery Htel--ryingg onl sarviss dndhusiuuee..
Human Recognition
Memory Retrieval
Familiarity Retrieval (One-shot)
-ffiiiii dhaloge feslosonns only recoussllatosss
Dvent
Con you sugpert some new evidence-baae pracoies to explore for healthoars desisko--rrkkigg!
asael hoally roupe
Farmiisiit
Inggort gessinely modaliic comphnenr conventionsl Restessee.
Answer
Sigmal
Activate
Recollectin Path
Recullection Retrieval (Stepwise)
Familiarity
Palh
Recellect
Becollect
suttids Lnggeetion gosslwily oficair and sok.. ..-how audistir compiinni contactionnl mectoeer
strervee werhludllgg and eiioncce fooed praceces resoail
Figure 1:
Comparison between standard familiarity based retrieval and recollection based retrieval user health narratives. And the brain figure motivated by (Rugg & Curran, 2007; Yonelinas, 2024).
Insights from cognitive science highlight the Aipollection-Famillann shown in Figure left, which posits that human recognition and memory are driven by two com- plementary mechanisms (Henson et al., 1999; Yonelinas et al., 2002; Merkow et al., 2015; Bastin et al., 2019; Yonelinas, 2024). Familiarity provides rapid but coarse sense of "knowing", enabling efficient yet shallow judgments. Recollection, in contrast, is triggered when familiarity proves insuf- ficient, initiating slower contextual reconstruction process that retrieves time, place, and source- specific details. Importantly, humans regulate these two processes via the familiarity signal: high confidence sustains reliance on familiarity, whereas decreasing familiarity and rising uncertainty prompt the shift to recollection (Rugg & Curran, 2007; Yonelinas, 2024). Applied to user mem- ory retrieval in personalized LLMs, this theory suggests that retrieval should not be conceived one-shot operation. Instead, it should function as dual-proces process controller that adaptively alternates between fast recognition and slow recollection, guided by the system's sense of familiarity.
Theory, as
Current retrieval systems often reduce memory to a static set of vectors, relying on direct similar- ity search without mechanisms to evoke richer user recollections. Retrieval in gemory-auggennte LLMs can be understood along three dimensions: query reformulation (Li et al., 2025a; Zhao et al., 2025; Chen et al., 2025; Shen et al., 2024; Salama et al., 2025), index construction (Zhong et al., 2024; Pan et al., 2025; Xu et al., 2025c; Tan et al., 2025b; Xu et al., 2025d; Ong et al., 2024; Chhikara et al., 2025), and retrieval strategy (Xu et al., 2021). While existing work has advanced the first two, retrieval strategies remain dominated by embedding based one-shot top-K K search (Karpukhin et al., 2020: Lei et al., 2023; Wang et al., 2020; Song et al., 2020; Luo et al., 2024), corresponding to the Familiarity channel: fast yet shallow recall. Two key limitations remain in current memory retrieval systems: 1) they overlook the Recollection path, failing to retrieve evidence chains for ambiguous queries, long-tail knowledge, personalized reasoning; 2) they lack mechanisms for path switching between l'amiliarity and recollection, leading to either under-retrieval that misses deeper contextual cues over-retrieval that introduces more retrieval latency. As shown in Figure 1, the Familiarity path may retrieve only partial fragments (e.g., "I support are genuinely effective and safe"), missing broader context and even introducing irrelevant noise (e.g.. "new healthy recipe"). By contrast, the Recollection path expands iteratively and can introduce more comprehensive ev- idence (e.g., "focus on conventional medicine; and evidence-based practices resonate"). This gap underscores the need for a recollection retrieval path and adaptive switch mechanism to balance efficiency with reliable coverage.
To address these limitations, propose RF-Mem (Recollection-Familaaae Memory Retrieval), an uncertainty-gun guidedd dual-path retrieval framework. RF-Mem begins with probe retrieval that produces initial retrieval list and estimates its familiarity by computing the mean score and en- tropy in the list. When familiar, the system stays on the Familiarity path, returning the top- K candi- dates in a one-shot manner with minimal overhead. Otherwise, RF-Mem activates the Recollection
2

---

Published as a conference paper at ICLR 2026
path: the probe results clustered by KMeans, each cluster centroid is combined with the original query through an a-mixing strategy, and the resulting recollect-queries are expanded iteratively. At each round, new candidates retrieved, clustered, and mixed to form updated queries, allowing the system to stepwise reconstruct evidence chains. 'The process is explicitly bounded by beam width, fanout, and maximum rounds, ensuring controllable computation. In this way, RF-Mem preserves the efficiency of single-pass retrieval when famillirriy is high, while adaptively engaging structured recollection under unfamiliar conditions, embedding chain-like reasoning directly into the retriever.
Our contributions are fourfold: (1) We ground the design of personalized memory retrieval in the Recollection Famillari dual-process theory, formulating retrieval coordination of Familiar- ity and Recollection paths. (2) We introduce familiarity uncertainty-drive selection for adaptive switching between Familiarity and Recollection. (3) We develop recollection retrieval based on clustering and query-centroid mixing, achieving chain-like evidence reconstruction only in embed- ding space. (4) RF-Mem is lightweight, relying solely on vector search and small-scale clustering, achieving high accuracy and recall at near one-shot retrieval latency. Extensive experiments three personalized memory datasets show that RF-Mem consistently surpasses both one-shot top- K retrieval and the full-context method with low latency. An adaptive study shows that RF-Mem complements and generalizes to index bhildiig methods like Memory Bank (Zhong et al., 2024).
METHOD: RECOLLECTION-FAMILLAA
MEMORY RETRIEVAL
We propose RF-Mem, dual-process memory retrieval framework that adapts the retrieval strategy according to uncertainty. As illustrated in Figure 2, RF-Mem consists of five stages: ① user query input, @ user memory retrieval, i.e. the RF-Mem module, is familiarity mncertainty-guider selec- tion introduced in Section 2.1 that adaptively switches between one-shot Familiarity introduced in Section 2.2 and stepwise Recollection retrieval introduced Section 2.2, 3 extracting the memory text, and 4 answer generation by LLM using the memory text.
Question
Famillarity Retricnal
Question
Recallection- Famillarity
Memory Retrieval (RF- (RF-Mem)
M2
Favilionly
Retrieval Top
MK.
Question
Similaity
memory
#
Recollecnion
Familiarity Signol
Question
Centroid
Recollect Query
Seore 5)
Centroid
exp (a(s mas, s))
Retrieval Top N
Retrivaal Top N (N-K)
Yiu orp Mi 5))
Recolleet Query
Centroid
H(p)
kudpo)
KM
Mow
Retrieval Top KMezas
Retrieved Memory
CommonT
Corrad
Batrioar
LLM Amswer
Rocellect
Recellect
The overall architecture of RF-Mem. A dual-proces memory retrieval system dynami- cally switches between the Familiarity and the Recollection paths.
Figure 2:
2.1 FAMILIARITY UNCERTAINTY- DRIVEN RETRIEVAL SELECTION
Guided by dual-process theory, we argue that memory retrieval should not be reduced to a single static top-K similarity search. Instead, retrieval should dynamically select between fast Familiarity path and a deeper Recollection path depending on the familiarity signal.
We first perform probe retrieval for calculating the familiarity signal between this question and the user's 's memory. Let M {m)..... ,.M) denote the set user memory fragments, where m; is encoded embedding vector z; =mm)). Given query q, its embedding x, = Q(q) is used
3

---

Published as a conference paper at ICLR 2026
to compute cosine similarity with each fragment my 8; (x,, Z.), and the retriever returns the candidate set C Top-K ([(ms ({(m4,):)) s)))
The familiarity signal is governed jointly by similarity and uncertainty, thus integrating efficiency with robustness. Formally, given probe scores {s; ) we normalize them as follows:
Ps exp(\(8; max, 8;))
i= i=....., K,
(1)
j= exp(A(8) maxj 8,))
where À controls sharpness. The uncertainty then measured by the entropy:
K
H(p) = Σ Pi log
(2)
The final selection combines the familiarity of the mean score with the uncertainty of the entropy. the mean similarity score 3 = K Σi= exceeds the upper threshold Ohigh the memory is highly relevant and retrieval proceeds along the Familiarity path. Conversely, when falls below the lower threshold 8u.. the memory is considered weakly relevant and the system switches to the Recollection path. For intermediate cases where lies between 8uo and fnig. entropy (p) serves as the disambiguator: low entropy, indicating concentrated evidence, selects Familiarity, whereas high entropy, reflecting uncertainty, triggers Recollection. Formally, the policy is defined as:
Familiarity,
8 2 nnghh
Recollection,
S frows
Strategy(4)
(3)
Familiarity, H S T Recollection, H(p) > T,
Bso Onigh-
This gating mechanism follows our cognitive motivation: when the familiarity signal is strong, the system relies the fast Familiarity path for confident recognition; when the signal is weak, it engages the Recollection path to deliberately reconstruct evidence; and in the intermediate regime, the entropy serves as an additional cue regulate switching. In this way, RF-Mem mirrors the dual-process nature of human memory, dynamically balancing efficiency with retrieval depth.
2.2 FAMILIARITY RETRIEVAL
When the probe retrieval yields a familiarity signal, that is, a sufficiently high average similarity score low uncertainty (p) < r, the system interprets the evidence confident. In this case, it adopts the Familiarity path. Consistent with dual-process theory. this path corresponds to rapid, low-effort recognition. i.c., the retriever directly selects the top-K memory evidence based on raw similarity, ensuring efficiency without invoking further reasoning expansion.
Scoring and Retrieval. Given query gt, we encode as x, <(q+) and each memory entry as ((m;), where all embeddings are unit-normalized. The similarity score is computed as sx,,, 2,) (Xe, zi). We then obtain top-K candidate memory fragments according to their
similarity scores:
C. = =TTop- [[(m, s(x,, 1))
(4)
This retrieval mode reflects the Familiarity process in dual-process theory: when the question is familiar, it signals that sufficient evidence has been retrieved, so recognition can be completed in single step with minimal latency.
2.3 RECOLLECTION RETRIEVAL
When the probe yields an unfamiliar signal, indicated by insufficient mean scores high entropy H(p) > т. the system interprets the evidence as uncertain. then transitions into the Recollection path, corresponding to the deliberate and effortful retrieval mode in dual-process theory. Instead of stopping at surface matches, this path initiates multi-round evidence expansion, progressively reconstructing context and recovering more diagnostic evidence. A case can be found D.7.
4

---

Published as a conference paper at ICLR 2026
Candidate Memory Retrieval. Let x( x, Φ(q:) denote the query embedding. At each (xl*), (.))) where round r, we obtain the candidate memory set as C(r) Top-N (f(m, (o-NN
N (B r) x F < K is determined by beam width B, fanout size F, and round number (0, ..,....................... R}. We enlarge N with prevent the query in round r, which is formulated from previous rounds, from repeatedly retrieving the same memories. Duplicated memories in C(r) excluded if they appeared in earlier rounds.
Relevant Memory Clustering. Given the top-N memory C we group the candidate memory embeddings (z: E )( into B clusters using KMeans. Each cluster G) corresponds to branch in the retrieval tree, and its centroid vector (*)) serves as the base for further expansion:
86 =
Σ M.GG
Zi.
b b=1,..., B.
G
These centroids serve as branching points from which new retrieval paths unfold, simulating a tree- like process of recollection. By clustering memories into semantically coherent sets, they reduce redundancy while highlighting anchors that capture essential cues. In line with dual-process theory, these anchors initiate progressive recollection, supporting chain-like reconstruction of evidence that expands outward yet remains grounded in the user's memory context.
(r) is blended with the current query to Recollect Queries Generation via c-mix. Each centroid g, form a recollect query, with weights matching the annotations a (current query) and 1 a (centroid), and uses residual to maintain original query information:
(r+1) norm(ax! + a) go + xx),
[0, 1].
(6)
Loop. The recollect query xb (r+1) will be used to perform the next round of retrieval to obtain the candidate set:
c(r+1) = Top-N (1(m)) (x) (r+1) -))))
Afterwards, the memory cluster and recollect query mix are conducted. This retrieve-cluster routine is repeated across rounds, progressively expanding evidence chains. To keep the search tractable, we maintain at most active branches per round and caps the recursion depth at R.
Stop and Generation. The process stops when round limit R is reached target number of
items is gathered. The recollection evidence is truncated union C. = Top- K ( c(r))
C(+))
In analogy to human memory, this recollection triggers deliberate, cue-driven reconstruction, where related fragments are progressively retrieved, clustered, and mixed to surface latent context. Through structured retrieve-cluster-mi. iterations under beam and depth budgets, which trade additional la- tency for more diagnostic evidence, enabling the evocation of question-specific memories. The pseudocode provided in Appendix E. And theoretical analysis can be found at Appendix F.
3
EXPERIMENTS
3.1 EXPERIMENTAL SETUP
Datasets We use PersonaMem (Jiang et al., 2025), which includes multiple simulated user-LLM interaction histories over real-world tasks, with memory lengths of 32K, 128K, and 1M tokens. Each history comprises up to 60 multi-turn sessions with evolving user personas and preferences. We also evaluate on PersonaBench (Tan et al., 2025a), a synthetic benchmark composed private user documents and queries probing personal information (e.g., preferences, background), designed to assess the relevant personal memory retrieval ability before generation. And we include Long- MemEval (Wu et al., 2025a), benchmark targeting long-term personalized retrieval, where factual questions require retrieving rask-relevant information under both small and medium context settings. More details can be found Appendix A. And implementation details can be found in Appendix B
5

---

Published as a conference paper at ICLR 2026
Metrics On PersonaMem, performance is measured by Accuracy of generated responses, i.e., the proportion of responses that correctly align with the user's 's current persona and conversation context. On PersonaBench and LongM,mEval, since the focus is on the retrieval of personal memory pieces, we use Recall@ K to assess how well relevant personal memories are retrieved.
Baselines Unlike prior baselines that often rely LLM-generated queries or external indexing strategies, our comparisons are restricted to retr methods to ensure fairnsss all systems operate on the same memory vectors. Since our focus is on the retrieval component itself, we com- pare RF-Mem against four direct baselines: (1) Zero Memory: Following (Pan et al., 2025; Jiang al., 2025), the model answers without using user memory.
(2) Full Context: Following (Pan et al., 2025; Jiang et al., 2025), the entire user history memory is input to the model without re- trieval. (3) Dense Retrieval: Following (Wu et al., 2025a; Pan et al., 2025; Zhong et al., 2024), a standard retriever that returns the top-K memories based on similarity scores. This corresponds to the Familiarity retrieval in dual-process theory. (4) Recollection (ours): The recollection mode we proposed in this paper. This represents the system that only enters the Recollection path.
3.2 OVERALL PERFORMANCE IN PERSONALIZED GENERATION
Table I: Performance comparison over the PersonaMem across different memory corpus sizes. Columns are grouped by question type, rows by retrieval strategy. "NA" indicates the method does not need retrieval. "OOC" means out-of-contex of the LLM input window. The best results are in bold, and the second- best results are underlined. ** indicates the statistically significant improve- ments (i.e., two-sided t-test with < 0.05) over the best baseline.
Retri Time
Avg. Tokens
Revisit Track
Latest Aligned
New Sce- Shared
New Ideas
Method
Overall
Reasons Evolution Pref's Recs
narios
Facts
32K memory corpus data
Zero Memory
NA
464.6
0.7273
0.6259
0.1765
0.2182
0.2105
0.2326
0.1183
0.3854
Full Context
NA
24657.8
0.9394
0.7194
0.7647
0.7455
0.5614
0.5039
0.1828
0.6129
Dense Retrieval
3. 3.14ms
3515.9
0,9091
0.6475
0,6471
0.6364
0.5614
0,5426
0.2151
0.5908
Recol. (ours)
7.09ms
3711.1
0.9495
0.6547
0.7059
0.7818
0.5965
0.5194
0.2688
0.6214
RF-Mem (ours) 5.09ms
3566.6
0.9495
0.6619
0.7059
0.7818
0.6140
0.5659
0.2688
0.6350
128K memory
corpus data
Zero Memory
NA
416.3
0.6766
0.6422
0.2136
0.2751
0.1925
0.2281
0.1737
0.3124
Full Context
NA
115601.4
0.5613
0.3930
0.2783
0.3868
0.2770
0.3977
0.1795
0.3231
Dense Retrieval
3.24ms
3
0.7881
0.6804
0.5346
0.5330
0.3662
0.6082
0.3069
0.5259
Recol. (ours)
7.86ms
3680.3
0.8141
0.6716
0.5254
0.5301
0.3765
0.6140
0.3263
0,5288
RF-Mem (ours) 4.27ms
3565.5
0.8030
0.6862
0.5427
0.5358
0.4131
0.6257
0.3263
0.5394
1M memory
corpus data
Zero Memory
NA
415.1
0.6000
0.6178
0.1797
0.3179
0.1831
0,2569
0.1816
0.2730
Full Context
NA
912148.5
OOC
OOC
00C
OOC
OOC
OOC
OOC
OOC
Dense Retrieval
4.42ms
3816.1
0.7702
0.6933
0.4544
0.4464
0.3085
0.5903
0.3040
0.4518
Recol. (ours)
8.12ms
3847.4
0.7532
0.6800
0.4440
0.4500
0.3593
0,5833
0.3136
0.4544
RF-Mem (ours)
6.28ms
3827.8
0.7787
0.6889
0.4492
0.4536
0.3390
0.6111
0.3150
0.4589
To verify the effectiveness of RF-Mem, evaluate it on PersonaMem across memory corpora of 32K, 128K, and 1M tokens per query. This setup stresses retrieval under different memory scales and question types, allowing us to examine how retrieval methods adapt as corpora grow larger and tasks become more complex. Table I compares zero-memory baselines, full-context input, and three retrieval strategies (Dense, Recollection, RF-Mem). We also report per-category accuracy under three corpus sizes in Appendix D.1 and different K settings in Appendix D.5.
First, RF-Mem delivers the best overall accuracy at every corpus scale while keeping inputs compact. In Table 1, RF-Mem attains the top overall score at 32K (0.6350), 128K (0.5394), and 1M (0.4589). At 32K it surpasses Full Context by +0.0221 with only 3.6k average tokens versus 24.7k for Full Context, and with modest 5.09ms retrieval time. As the memory grows, Full Context deteriorates sharply, reaching 0.3231 at 128K and becoming out of context at IM, whereas RF-Mem remains stable and leads Dense Retrieval (Familarity) by +0.0135 128K and +0.0071 at 1M. These trends validate that when a question is familiar, RF-Mem saves budget; when unfamiliar, it upgrades to structured recollection without committing to the cost of running it unconditionally.
Second, RF-Mem leads on hybrid and transfer-style tasks with lower overhead. At 32K achieves top scores Aligned Recommendations (0.7818), New Scenarios (0.6140), and Shared

---

Published as a conference paper at ICLR 2026
Facts (0.5659). while remaining close Track Evolution. At 128K it remains ahead Track Evolution, Latest Prefs. Aligned Recs, New Scenarios, and Shared Facts, tying on New Ideas. At 1M, where Full Context is infeasible, RF-Mem is strongest Revisit Reasons, Aligned Recs, Shared Facts, and New Ideas, with only small gaps on Track Evolution and Latest Prefs, indicating that adaptive depth better balances precise anchoring and selective expansion than single-mode retrieval.
Third, RF-Mem achieves a favorable accuracy-efficiency trade-off by regulating retrieval depth via entropy. Compared always-on recollection, RF-Mem improves overall accuracy while reducing latency: 5.09ms vs 7.09ms at 32K, 4.27ms vs 7.86ms at 128K, and 6.28ms vs 7. 7.12ms at 1M, with similar token budgets Dense Retrieval. This efficiency arises from treating familiarity as the default and route to the recollection path only when the question is unfamiliar. The result is scalable retrieval controller that avoids the out-of-context cliff of Full Context, outperforms dense retrieval baselines as memory scales, and preserves recollection's 's advantages precisely.
3.3 OVERALL PERFORMANCE IN PERSONALIZED RETRIEVAL
Table 2: Performance comparison over the PersonaBench dataset across multiple question types. The best results are in bold, and the second-hest results underlined. Metries
@
Recall
Basic
Social
Pref
Pref Hard
Basic
Social
Pref
Pref
Method
Time
Overall
Time
Overall
Info
Info
Easy
Info
Info
Easy
Hard
multi multi-qa- MM-MiniLM L6-cos-vI
Famili.
8.40ms 0,4515 0,4852 0.4904 0.3659 0,4484
13.68ms 0,5879 0.6220 0.6442 0,5561 0.5964
Recol.
9.65ms
0.4379 0.4903 0,5128 0.3854 0.4499
17.29ms
0.5924
0.6859 0.5659 0.6267
0.6062
RF-Mem
9.16ms
0.4788 0.5091 0.4872
0.3854 0.4701
15.22ms
0.5924
0.6799 0.5707
0.6267
0.6071
all all-mpnet buseev22
Famili.
7.64ms 0.4242 0.2730
0.4487 0.4049
0.3887
10.55ms 0.5409
0.4434
0.6795
0.5366
0.5333
Recol.
10.94ms
0.4333 0.2918 0.4583 0.4000
0.3976
13.23ms 0.6000 0.4365 0.6378
0.5220
0.5527
RF-Mem
8.33ms 0.4515 0,2730
0.4487 0.4000
0.4009
10.55ms 0.5955
0.4384
0.6378
0.5463
0.5553
Famili.
8.92ms 0.3970 0.3204 0.4583 0.3268
0.3738
10. 19ms 0.5121 0.4748
0.5673
0.4585
0.5002
Recol.
12.14ms
0.3833 0.3619
0.4327 0.3171
0.3722
20.71ms 0.5212
0.4748 0.5673 0.4585
0.5046
RF-Mem
10.14ms 0.4015 0.3619 0.4487 0.3220 0.3836
18ms 0.5303 0.4748 0.5673 0.4585 0.5089
Table 3: Performance comparison over the LongMemEval under small (S) and medium (M) memory versions. The best results are in bold, and the second- best results are underlined.
LongMemEval-S
LongMemEVal-M
Method
R Recall 10 Recall 50
Time
Recall@S Recall @ 10 Recall
Time
Fami. 0.7136
0.8282
0.9761
2s
0.4177
0.5465
0.7518
27.72ms
Recol.
0.7351
0.8425
1.0000
50.62ms
0.4368
0.5585
0.7590
57.93ms
RF-Mem
0.7375
0.8473
1.0000
39.58ms
0.4391
0.5609
0.7613
41.22ms
all-mpnet-bass-v22
Fami.
0.7303
0.8353
0.9832
27.25ms
0.4176
0.5489
0.7637
33.18ms
Recol.
0.7398
0.8305
0.9952
51.79ms
0.4386
0.5871
0.7422
62.1 Ims
RF-Mem
0.7398
0,8377
0.9952
42.39ms
0.439I
0,5894
0.7684
50.80ms
BAAA//bg-basseee--
Fami. 0.7924
0.8926
1.0000
29.65ms
0,4964
0.6611
0,8305
30.77ms
Recol. 0.8162
0.9165
1.0000
43.65ms
0.5131
0.6635
0.8234
58.05ms
RF-Mem
0.8186
0.9189
1.0000
37.34ms
0.5155
0.6635
0.8329
44.74ms
To verify the retrieval performance of RF-Mem, we evaluate it against one-shot Familiarity and stepwise Recollection across both PersonaBench and LongMemEval. PersonaBench covers multi- domain user interactions, while LongMemEval stresses retrieval over extended memory corpora under small (S) and medium (M) settings. We report Recall Recall @ 10, and Recall@ 50, to- gether with average retrieval lateney, under three retriever backbones (MiniLM, MPNet, BGE). This setup allows us to examine how retrieval strategies behave across tasks with varying difficulty and under different embedding models. We also conduct parameter sensitivity studies at D.2 to D.4.

---

Published as a conference paper at ICLR 2026
First, RF-Mem achieves the most balanced and robust performance across retrievers. As shown in Table 2, RF-Mem either matches or surpasses the best baseline in overall 5 and Recall @ 10, while avoiding the pitfalls of single-mode strategies. For example, under MiniLM it achieves an overall Recall@ 10 of 0.6071, slightly higher than Familiarity (0.5964) and Recollection (0.6062). On LongMemEval, in Table 3, RF-Mem further demonstrates stability. Its familiarity uncertainty-driven selection allows the retriever to exploit confident familiarity matches when pos- sible, and to activate deeper recollection only when necessary, yielding consistently strong results.
Second, the comparison between Familiarity and Recollection reveals complementary strengths. Familiarity excels on fact-centric queries such as Basic Information and Preference Easy, where direct surface similarity suffices. Recollection, however, proves highly effective on context-heavy tasks such as Preference Hard and Social queries. PersonaBench, Recollection reaches Recall @ 10 of 0.6267 on Preference Hard under MiniL.M, outperforming Fa- miliarity 0.5561. Similarly, on Eonl,e it consistently lifts Recall @5 by more than 0.02 across multiple retrievers (e.g.. 0.7351 vs. 0.7136 under MiniLM) in Table 3. Its iterative expan- sion uncovers deeper. temporally dispersed cues, making it powerful strategy despite higher cost. These results highlight that Recollection is not merely slower, but offers indispensable diagnostic evidence, and that neither mode alone can achieve robustness across all task types.
Table 2,
trade-off's On PersonaBench (Ta- ble 2). Familiarity is fastest (8-10ms) but shallow, while Recollection is stronger but nearly twice as slow (15-20ms). RF-Mem closes this gap, sustaining latency near Familiarity (9-15ms) with higher accuracy (e.g., Recall@ 10 of 0.6071 vs. 0.5964/0.6062) On Long (Table 3), Familiarity low-latency (25-31 ms) but loses coverage, while Recollection reaches perfect Recall @ @0 much higher cost (40-62ms). RF-Mem balances both, keeping latency lower (37-50ms) while matching or exceeding accuracy.
Third, RF-Mem delivers superior efficiency-effectivvee
3.4 ADAPTIVE EXPERIMENT
3.4.1
ADAPTIVE TO INDEX BUILDING METHOD
Different Offline Index Building
RF-Mem Online Retrieval
rralized that saking time gather relevant putinnt information
Index
Question
sonds like that experience really opened your eyes ID
Summorize the events and bey information the
RF-Mem
Direct save
The attended a health and wellness expo that showessed innovative ...
Retrieved message of origin index
Save summary
Generate Answer
Origin Tum Index MemoryBank Summary Index/
Figure 3: Illustration of adaptive study setup. Offline indexes (e.g.. Memory Bank or origin memory) provide different storage, while RF-Mem serves an online retrieval layer that adapts to them.
Table 4: Results by using MemoryBank summary index on PersonaMem (32K corpus).
Method Avg. Tokens
Revisit Reasons
Track
Latest
Aligned
New See-
Shared Facts
New Ideas
Overall
Evolution
Prefs
Recs
narios
Memory Bank summary index
Familiarity
1267.6
0.7475
0.6187
0,5882
0,5818
0.3333
0.4341
0.1505
0.4941
Recollection
1441.8
0.8182
0.6259
0.5294
0.6909
0.4737
0.4031
0.1398
0.5212
RF-Mem 1421.8
0.8384
0.6259
0.5294 0.6545
0.4211
0.4419
0.1828
0.5314
To further examine the modularity of RF-Mem, integrate it with external indexing schemes beyond raw turn-level memory. As shown in Figure 3, offline methods such as MemoryBank (Zhong al., 2024) first summarize user dialog into indexes (e.g., turn-level summaries), while RF-Mem
8

---

Published as a conference paper at ICLR 2026
operates online module during retrieval. This separation of offline indexing and online retrieval highlights that RF-Mem does not compete with summarization- or graph- basp- memory banks, but instead complements them by enabling human-like remembering. Table 4 highlights that RF-Mem demonstrates robustness across both settings: it achieves the highest overall accuracy under the turn- level index, and crucially, it narrows the performance drop under the summary index compared to single-path baselines. This adaptivity confirms that RF-Mem is modular and can be layered on top of heterogencous memory indices, providing uncertainty-ave dual-process retrieval mechanism that complements, rather than replaces, external indexing methods.
3.4.2 ADAPTIVE TO QUERY EXPANSION METHOD
Nearline Query Expansion
RF-Mem Online Retrieval
Retrieved message of index
Question
RF-Mem
Query Expansion
Expanded
Index
Generate
Question
Answer
Figure 4: Illustration of the adaptive study setup. Nearline query expansion (e.g., HyDE) enriches the query representation, and RF-Mem operates as the online retrieval layer.
Table 5: Results by using HyDE query expansion method on PersonaBench.
Metries
Recall@5
Recall 10
Basic Info
Social Info
Pref
Pref Hard
Basic Info
Social Info
Pref Easy
Pref
HyDE
Overall
Overall
Easy
Hard
Famili.
0.3106
0.3909
0.4615
0,3122
0.3464
0.5000
0.4991
0.5737
0.5220
0.5120
Recol.
0.3015
0.4135
0.4615
0.3171
0.3482
0.4909
0.5028
0.5929
0.4878
0.5046
RF-Mem
0.3061
0.4135
0.4615
0.3171
0.3504
0.5091
0.5028
0.5929
0.5220
0.5194
To further assess the adaptability of RF-Mem, we combine nearline query expansion with online retrieval and examine their interaction on PersonaBench. As illustrated in Figure the nearline expansion methods we adopt, HyDE (Gao et al., 2023), generate pseudo-relevance feedback that enriches the original query. RF-Mem then operates as an online retrieval layer. Table 5 reports the results when applying HyDE-based expansion. Across all categories, RF-Mem consistently matches or surpasses Familiarity baselines, demonstrating that its dual-process mechanism remains effective even when the upstream query representation shifts. These findings confirm that RF-Mem is modular and can be seamlessly integrated into nearline expansion pipelines.
3.4.3 ADAPTIVE TO ITERATIVE RAG METHOD
Query Developing
Retrieval
RF-Mem
Develop New Question
Retrieved message of index
Generate Answer
Question
Question
Index
Interective
Figure 5: Illustration of adaptive study setup. Iterative RAG (c.g., Search-ol) provides a multi-turn retrieval for answer generation, while RF-Mem serves as the retrieval layer that adapts to it.
To further evaluate the adaptability of RF-Mem under iterative retrieval settings, we pair it with a multi-turn reasoning pipeline based on Search-ol (Li et al., 2025d). As illustrated in Figure 5, Search-ol develops refined follow-up queries through iterative question generation, while RF-Mem functions as the retrieval layer that reacts to these evolving queries in real time. Table summarizes results using the Search-ol interactive retrieval the PersonaMem. RF-Mem still achieves the highest overall score. These findings demonstrate that RF-Mem retains its effectiveness in iterative RAG settings.

---

Published as a conference paper at ICLR 2026
Table 6: Results by using Search-ol iterative retrieval on PersonaMem (32K corpus).
Avg. Tokens
Revisit Reasons
Track
Latest
Aligned Recs
New Sce-
Shared Facts
New Ideas
Search-ol
Overall
Evolution
Prefs
narios
Search-ol
Familiarity
4948.7
0.8687
0.6259
0,5882
0.6545
0,5614
0,5271
0.2581
0,5823
Recollection
5103.9
0.8990
0.6619
0.6471
0.7091
0,5789
0.4961
0.2796
0.6010
RF-MEM
5158.2
0.9293
0.6978
0.6471
0.7455
0.5789
0.5349
0.2043
0.6146
4 RELATED WORKS
Personalized memory retrieval for LLMs has emerged to complement the fixed context window and enable user-specific, context-aware responses. Unlike standard knowledge-based RAG that targets factual data, personal memory retrieval draws on a user's own history and preferences (Pan et al., 2025; Wu et al., 2025a; Xu et al., 2025c) (e.g., retrieving long-term user-Al dialogue context (Jiang et al., 2025; Maharana et al., 2024; Wu et al., 2025a) and user-user dialogue (Tan et al., 2025a)) to fill in missing details and tailor responses. LLM with a non-parametric memory to provide relevant background information. We review related works in three areas: (I) query reformulation, (2) index construction, and (3) retrieval frameworks.
cumory-augmenter personalized LLM systems combine an
Query reformulation. These methods expand or refine queries to improve memory retrieval. LD- Agent extracts keyphrases for retrieval (Li et al., 2025a), MemoCue proposes memory-inspired cue query (Zhao et al., 2025), and LQ-TOD generates task-oriented queries (Chen et al., 2025). Other approaches leverage LLM-based query expansion, such as LameR (Shen et al., 2024) and MemInsight (Salama al., 2025), enrich the query with contextual attributes.
Index construction. Prior work has also emphasized structuring user memory into searchable in- dices. Text- methods summarize cluster memories into compact representations, in Mem- oryBank (Zhong et al., 2024), SeCom (Pan et al., 2025), and MemGas (Xu et al., 2025c), while others rely on reflective hierarchical summaries (Tan et al., 2025b). Graph-based approaches instead capture relational structures, exemplified by A-Mem (Xu et al., 2025d), THEANINE (Ong al., 2024), MemO (Chhikara et al., 2025), and Zep (Rasmussen al., 2025). Although differing in representation, these methods share common assumption: retrieval remains static. Most ulti- mately depend on standard dense retrievers to encode queries and memory keys, applying uniform retrieval process regardless of uncertainty or task complexity.
Retrieval frameworks. From early keyword-based search (Robertson et al., 2009) to dense retriev- ers like DPR (Karpukhin et al., 2020) and Contriever (Lei et al., 2023), retrieval methods aim to rank memory items by semantic similarity. More advanced encoders such as MiniL. (Wang al., 2020), MPNet (Song et al., 2020), and BGE (Luo et al., 2024) improve efficiency and accuracy, and are widely adopted in multi-session dialog retrieval (Xu et al., 2021). However, these frameworks largely adopt a single- process retrieval paradigm, treating all queries as homogeneous regardless of confidence or task complexity.
However, existing methods overlook the dual-process nature of human memory retrieval. Most prior works focus on query reformulation, index optimization, or stronger retrievers, yet they im- plicitly reduce retrieval to one-shot recognition process (Familiarity). This not only neglects the crucial role of deliberate (Recollection), but also ignores the need for adaptive switching between the two paths. Our proposed RF-Mem addresses this gap by introducing a recollection retrieval path and familiarity uncertainty-drivee selection that adaptively alternates between fast Familiarity and deeper Recollection retrieval, thereby enabling more personalized memory retrieval.
CONCLUSION
In this work, we revisited personalized memory retrieval through the lens of dual-process theory. Existing methods are limited to one-shot retrieval based on similarity Familiarity, whereas we in- troduce Recollection RF-Mem, a familiarity uncertainty-driien framework that adaptively switches between them. Ex- periments show that RF-Mem achieves robust gains across both generation and retrieval tasks, and scales reliably to million-entry corpora, demonstrating the importance of integrating deliberate rec- ollection into memory retrieval for personalizing L.LMM
deliberate stepwise retrieval mechanism. Building on this, we proposed
10

---

Published as a conference paper at ICLR 2026
ETHICS STATEMENT
This work focuses on improving personalized memory retrieval for large language models. Our study relies solely simulated and publicly available benchmark datasets (PersonaMem, Person- aBench, LongMemEval), which do not contain personally identifiable information or sensitive data. We do not collect or release any private user data. While the proposed framework is designed en- hance efficiency and robustness in memory retrieval, potential misuse could arise if deployed with- out safeguards sensitive applications such as healthcare or personal decision-making. making. We there- fore emphasize that future deployment should follow strict data governance, privacy preservation, and fairness guidelines, and we encourage the research community to consider ethical implications when extending this work to real-world user data. We acknowledge that real-world deployments of preference-basee systems may inadvertently over-amplify user traits or reinforce behavioral bi- ases. Additionally, improper handling of user histories could expose sensitive information or lead to unintended profiling effects, underscoring the need for careful governance.
REPRODUCIBILITY STATEMENT
We make every effort to ensure reproducibility. All datasets used in this study are publicly avail- able, and we provide detailed references to their sources in the main text. Model architectures, hyperparameter choices (B, F, a, т, and thresholds Onigh. Owo) are explicitly documented in Ap- pendix B. We also describe hyperparameter sensitivity analyses in Appendix D to illustrate robust- ness under different configurations. To further facilitate replication, we release code and scripts for reproducing all reported results in the supplementary materials. Our code repository is available at https: /gg.tb...uuu..ggggggggggg..... ctm/ Appl Med-MMcchi Mae--aachhhh-aanneeee ne- waw.rrriinn-nnnnaneeeeeee tag Lab.cc222222 20
ACKNOWLEDGEMENT
This research was supported by the National Natural Science Foundation of China (NSFC) un- der Grants 7207 1029, 72231010, 62502404, and the Graduate Research Fund of the School of Economics and Management of Dalian University of Technology (No. DUTSEMDRFKOI) This research partially supported by Hong Kong Research Grants Council (Research Im- pact Fund No.R1015-23, Collaborative Research Fund No. C1043-43,,, General Research Fund No. 1218325), Institute of Digital Medicine of City University of Hong Kong (No.9229503), Huawei (Huawei Innovation Research Program).
REFERENCES
Christine Bastin, Gabriel Besson, Jessica Simon, Emma Delhaye, Marie Geurten, Sylvie Willems, and Eric Salmon. An integrative memory model of recollection and familiarity to understand memory deficits. Behavioral and Brain Sciences, 42:e281, 2019.
Jiale Chen, Xuelian Dong, Wenxiu Xie, Ru Peng. Kun Zeng, and Tianyong Hao. Lim-enhanced query generation and retrieval preservation for task- dialogue. In Findings of the Associ- ation for Computational Linguisties: ACI. 2025, pp. 14307-14321, 2025.
Jiawei Chen, Hongyu Lin, Xianpei Han, and Le Sun. Benchmarking large language models in retrieval-augmented generation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 17754-17762, 2024.
Prateek Chhikara, Dev Khant, Saket Aryan, Taranjeet Singh, and Deshraj Yadav. Mem0: Building production-ready ai agents with scalable long-term memory. arXiv preprint 25ii::2054.99411. 2025.
Yimin Deng, Yuqing Fu, Derong Xu, Yejing Wang, Wei Ni, Jingtong Gao, Xiaopeng Li, Chengxu Liu, Xiao Han, Guoshuai Zhao, al. Enhancing conversational agents via task-oriented adver- sarial memory adaptation. arXiv preprint arXiv22601.11777, 2026.
Martin Ester, Peter Kriegel, Jörg Sander, Xiaowei Xu, et al. A density- based algorithm for discovering clusters in large spatial databases with noise. In kdd, volume 96, pp. 226-231, 1996.

---

Published as a conference paper at ICLR 2026
Zichuan Fu, Xiangyang Li, Chuhan Wu, Yichao Wang, Kuicai Dong, Xiangyu Zhao, Mengchen Zhao, Huifeng Guo, and Ruiming Tang. A unified framework for multi-domain ctr prediction via large language models. ACM Transactions on Information Systems, 43(5):1-33, 2025.
Jingtong Gao, Xiangyu Zhao, Muyang Li, Minghao Zhao, Runze Wu, Ruocheng Guo, Yiding Liu, and Dawei Yin. Smip4ree: An efficient all-mip architecture for sequential recommendations. ACM Transactions on Information Systems, 42(3):1-23, 2024.
Luyu Gao. Xueguang Ma, Jimmy Lin, and Jamie Callan. Precise zero-shot dense retrieval without relevance labels. In Proceedings of the 61 Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pp. 1762-1777, 2023.
Feijiang Han, Jiaming Zhang, Chuyi Deng, Jianheng Tang, and Yunhuai Liu. Can Ilms handle webshell detection? overcoming detection challenges with behavioral function-awer frame- work. In Second Conference on Language Modeling (COLM 2025), 2025. URL https: // //openreview. net. / Forum? seeeiidd=iiae
Feijiang Han, Zelong Wang. Bowen Wang. Xinxin Liu, Skyler Cheung, Delip Rao, Chris Callison- Burch, and Lyle Ungar. Latex2layout: High-fidelity, scalable document layout annotation pipeline for layout detection. In Proceedings of the AAAI Conference on Artificial Intelli- gence (AAAI-26), 2026a. URL https: latex2layour pdf.
upenn wpt..eeeeeccccc/eeeeeeeeee wetp:/ccb/publccattetee
Feijiang Han, Xiaodong Yu, Jianheng Tang, Delip Rao, Weihua Du, and Lyle Ungar. Zerotuning: Unlocking the initial token's power enhance large language models without training. In ICLR 2026, 2026b. URL https: // arxiv. org/ abs/ 11739.
Haoyu Han, Yu Wang, Harry Shomer, Kai Guo, Jiayuan Ding, Yongjia Lei, Mahantesh Halap- panavar, Ryan A Rossi, Subhabrata Mukherjee, Xianfeng Tang, et al. Retrieval-augmenned gen- eration with graphs (graphrag). arXiv preprint arXiv:2501.00309, 2024.
Kostas Hatalis, Despina Christou, Joshua Myers, Steven Jones, Keith Lambert, Adam Binks, Zohreh Dannenhauer, and Dustin Dannenhauer Memory matters: The need to improve long-term memory in Ilm-agents. In Proceedings of the AAAI Symposium Series, volume 2, pp. 277-280, 2023.
Richard NA Henson, MD Rugg, Timothy Shallice, O Josephs, and Raymond J Dolan. Recollection and familiarity in recognition memory: an event-related functional magnetic resonance imaging study. Journal of neuroscience, 19(10)339223372, 1999.
Bowen Jiang, Zhuoqun Hao, Young Min Cho, Bryan Li, Yuan Yuan, Sihao Chen, Lyle Ungar, Camillo Jose Taylor, and Dan Roth. Know me, respond me: Benchmarking LLMs for dynamic user profiling and personalized responses at scale. In Second Conference on Language Modeling, 2025. URL https: : / /ppp/epeeeeveeeeeeoeeeeeeee net wwr.mormmddeeo wox.:xix00xx00000000
Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick SH Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. Dense passage retrieval for opmn-domain question answering. In EMNLP (1), pp. 6769-67667 6781, 2020.
Yibin Lei, Liang Ding, Yu Cao, Changtong Zan, Andrew Yates, and Dacheng Tao. Unsupervised dense retrieval with relevance-aware contrastive pre-training. arXiv preprint arXiv:2306.03 2023.
Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. Retrieval-augmennt gener- ation for knowlede nnwwledge-intesivee nip tasks. Advances in neural information processing systems, 33: 9459-9474, 2020.
Hao Li, Chenghao Yang, An Zhang, Yang Deng, Xiang Wang, and Tat-Seng Chua. Hello again! llm-powwrr powered personalized agent for long- term dialogue. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume I: Long Papers), pp. 5259-52766 2025a.
12

---

Published as a conference paper at ICLR 2026
Xiaopeng Li, Pengyue Jia, Derong Xu, Yi Wen, Yingyi Zhang, Wenlin Zhang, Wanyu Wang, Yichao Wang, Zhaocheng Du, Xiangyang Li, et al. A survey of personalization: From rag to agent. arXiv preprint ar vrXiv2250..00444 2025b.
Xiaopeng Li, Lixin Su, Pengyue Jia, Suqi Cheng, Junfeng Wang, Dawei Yin, and Xiangyu Zhao. Agenttranking: Semantic robust ranking via personalized query rewriting using multi-agent Ilms. ACM Transactions on Information Systems, 43(6):1-33, 2025c.
Xiaoxi Li, Guanting Dong. Jiajie Jin, Yuyao Zhang, Yujia Zhou, Yutao Zhu, Peitian Zhang, and Zhicheng Dou. Search-ol: Agentic search-enhanced large reasoning models. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, November 2025d.
Shuchang Liu, Qingpeng Cai, Bowen Sun, Yuhao Wang, Jiang, Dong Zheng, Peng Jiang, Kun Gai, Xiangyu Zhao, and Yongfeng Zhang. Exploration and regularization of the latent action space in recommendation.
Proceedings of the ACM Web Conference 2023, pp. 833-844, 2023.
Kun Luo, Zheng Liu, Shitao Xiao, and Kang Liu. Bge landmark embedding: A chunking- embedding method for retrieval augmented long-conted large language models. arXiv preprint arXiv:2402.11573. 2024.
Adyasha Maharana, Dong-Ho Lee, Sergey Tulyakov, Mohit Bansal, Francesco Barbieri, and Yuwei Fang. Evaluating very long-term conversational memory of Ilm agents. arXiv preprint arXiv:2402.17753, 2024.
Maxwell B Merkow, John Burke, and Michael J Kahana. The human hippocampus contributes to both the recollection and familiarity components of recognition memory. Proceedings of the National Academy of Sciences, 112(4))
2015.
Andrew Ng, Michael Jordan, and Yair Weiss. On spectral clustering: Analysis and algorithm.
Advances in neural information processing systems, 14, 2001.
Kai Tzu-iunn Ong, Namyoung Kim, Minju Gwak, Hyungjoo Chae, Taeyoon Kwon, Yohan Jo, Seung-won Hwang, Dongha Lee, and Jinyoung Yeo. Towards lifelong dialogue agents via timeline-based memory management. arXiv preprint arXiv:2406. 10996, 2024.
Zhuoshi Pan, Qianhui Wu, Huiqiang Jiang, Xufang Luo, Hao Cheng, Dongsheng Li, Yuqing Yang. Chin-Yew Lin, H Vicky Zhao, Lili Qiu, et al. Secom: On memory construction and retrieval for personalized conversational agents. In The Thirteenth International Conference on Learning Representationss 2025.
Preston Rasmussen, Pavlo Paliychuk, Travis Beauvais, Jack Ryan, and Daniel Chalef. Zep: a tem- poral knowledge graph architecture for agent memory. arXiv preprint arXi::2501.13956 2025.
Stephen Robertson, Hugo Zaragoza, et al. The probabilistic relevance framework: Bm25 and be- yond. Foundations and Trends@ in Information Retrieval, 3(4):333-389, 2009.
Michael D) Rugg and Tim Curran. Event-related potentials and recognition memory. Trends in cognitive sciences, 11(6):251-277, 2007.
Rana Salama, Jason Cai, Michelle Yuan, Anna Currey, Monica Sunkara, Yi Zhang, and Yassine
Benajiba. Meminsight: Autonomous memory augmentation for Ilm agents. arXiv preprint arXiv:2503.21760. 2025.
Tao Shen, Guodong Long, Xiubo Geng, Chongyang Tao, Yibin Lei, Tianyi Zhou, Michael Blu- menstein, and Daxin Jiang. Retreeval-augmented retrieval: Large language models are strong zero-shot retriever. In Findings of the Association for Computational Linguistics ACL 2024, pp. 15933-15946, 2024.
Kaitao Song, Xu Tan, Tao Qin. Jianfeng Lu. and Tie- Yan Liu. Mpnet: Masked and permuted pre- training for language understanding. Advances in neural information processing systems, 33: 16857-16867, 2020.
13

---

Published as a conference paper at ICLR 2026
Juntao Tan, Liangwei Yang. Zuxin Liu, Zhiwei Liu, Rithesh R N, Tulika Manoj Awalgaonkar, Jianguo Zhang, Weiran Yao, Ming Zhu, Shirley Kokane, Silvio Savarese, Huan Wang. Caim- ing Xiong, and Shelby Heinecke. PersonaBench: Evaluating Al models on understand- ing personal information through accessing (synthetic) private user data. pp. 878-893, July 2025a. doi: 10. 2a055s/w12222..nnnns--aaa 2025. findingn 491.
URL
ogy org/
Zhen Tan, Jun Yan, I Hsu, Rujun Han, Zifeng Wang. Long T Le, Yiwen Song, Yanfei Chen, Hamid Palangi, George Lee, et al. In prospect and retrospect: Reflective memory management for long- term personalized dialogue agents. arXiv preprint arXiv:2503.08022, 2025b.
Wenhui Wang, Furu Wei, Li Dong, Hangbo Bao, Nan Yang, and Ming Zhou. Minilm: Deep self-
attention distillation for task-agnostic compression of pre-trained transformers. Advances in ral information processing systems, 33:5776-5788, 2020.
Yuhao Wang. Xiangyu Zhao, Bo Chen, Qidong Liu, Huifeng Guo, Huanshuo Liu, Yichao Wang, Rui Zhang, and Ruiming Tang. Plate: A prompt-enhanced paradigm for multi-scenario recom- mendations. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 1498 1507, 2023.
Yi Wen, Derong Xu, Pengyue Jia, Yichao Wang. Yingyi Zhang, Maolin Wang, Junyi Li, Yue Liu, Huifeng Guo, Yong Liu, and Xiangyu Zhao. Memory type matters: Enhancing long-term memory in large language models with hybrid strategies, 2026. URL https: httttps://ppeeeee wpwnreeieeeeeeeeeeeeeeeeeeeeeeee .ett/ forum? i idd wCC-2K22........
Di Wu, Hongwei Wang, Wenhao Yu, Yuwei Zhang, Kai-We Chang, and Dong Yu. Longmemeval: Benchmarking chat assistants on long-term interactive memory. In The Thirteenth Interna- tional Conference on Learning Representations, 2025a. URL http:///ppeeeeeeeeccmommm
Corum?id=p2. www.dddddiiyau......mmmmm LyCaVuti.
Yaxiong Wu, Sheng Liang, Chen Zhang, Yichao Wang, Yongyue Zhang, Huifeng Guo, Ruiming Tang, and Yong Liu. From human memory to ai memory: A survey on memory mechanisms in the era of Ilms. arXiv preprint arXiv:2504.15665 2025b.
Derong Xu, Xinhang Li, Ziheng Zhang, Zhenxi Lin, Zhihong Zhu, Zhi Zheng, Xian Wu, Xiangyu Zhao, Tong Xu, and Enhong Chen. Harnessing large language models for knowledge graph question answering via adaptive multi-aspect retrieval-augmeettee Conference on Artificial Intelligence, volume 39, pp. 15570-25578. 2025a.
In Proceedings of the AAAI
Derong Xu, Yi Wen, Pengyue Jia, Yingyi Zhang, Yichao Wang, Huifeng Guo, Ruiming Tang. Xi- angyu Zhao, Enhong Chen, Tong Xu, et al. From single to multi-granularity: Toward long-term memory association and selection of conversational agents. arXiv preprint (xXiv:2505.19549 2025b.
Derong Xu, Yi Wen, Pengyue Jia, Yingyi Zhang, Yichao Wang, Huifeng Guo, Ruiming Tang, Xi- angyu Zhao, Enhong Chen, Tong Xu, et al. Towards multi-granulann granularttll memory association and selection for long-term conversational agents. arXiv preprint arXiv:2505.19555 2025c.
Jing Xu, Arthur Szlam, and Jason Weston. Beyond goldfish memory: Long-term open-domain conversation. arXiv preprint
2021.
Wujiang Xu, Kai Mei, Hang Gao, Juntao Tan, Zujie Liang, and Yongfeng Zhang. A-mem: Agentic memory for Ilm agents. arXiv preprint arXiv:2502.12/100 2025d.
Andrew P Yonelinas. The role of recollection and familiarity in visual working memory: A mixture of threshold and signal detection processes. Psychological review, 131(2):321, 2024.
Andrew P Yonelinas, Neal EA Kroll, Joel R Quamme, Michele M Lazzara, Mary-Jane Sauvé, Keith F Widaman, and Robert T Knight. Effects of extensive temporal lobe damage or mild hypoxia on recollection and familiarity. Nature neuroscience, 5(11): 2002.
2002.
Qinggang Zhang, Shengyuan Chen, Yuanchen Bei, Zheng Yuan, Huachi Zhou, Zijin Hong, Hao Chen. Yilin Xiao, Chuang Zhou, Junnan Dong, et al. A survey of graph retrieval- -aggeeeed generation for customized large language models. arXiv preprini 1399358. 2025a.
14

---

Published as a conference paper at ICLR 2026
Zeyu Zhang, Quanyu Dai, Xiaohe Bo, Chen Ma, Rui Li, Xu Chen, Jieming Zhu, Zhenhua Dong, and Ji-Rong Wen. A survey on the memory mechanism of large language model-basee agents. ACM Transactions on Information Systems. 43(6):1-47, 2025b.
Zijian Zhang, Shuchang Liu, Ziru Liu, Rui Zhong, Qingpeng Cai, Xiangyu Zhao, Chunxu Zhang, Qidong Liu, and Peng Jiang. Lim-powered user simulator for recommender system. In Proceed- ings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 13339-13347, 2025c.
Qian Zhao, Zhuo Sun, Bin Guo, and Zhiwen Yu. Memocue: Empowering Ilm-based agents for human memory recall via strategy-guides querying. arXiv preprint arXiv:2507.2363: 2025.
Xiangyu Zhao. Adaptive and automated deep recommender systems. ACM SIGWEB Newsletter, 1 2022.
Wanjun Zhong, Lianghong Guo, Qiqi Gao, He Ye, and Yanlin Wang. Memorybank: Enhancing large language models with long-term memory. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 19724-19731, 2024.
15

---

Published as a conference paper at ICLR 2026
Technical Appendix
TABLE OF CONTENTS
A Dataset Details
17
Implementation details
18
C Prompt For PersonaMem
20
D Additional Experiment
22
D.I Result Across Category in PersonaMem
22
D.2 Sensitivity Analysis of a and 7
23
D.3 Sensitivity Analysis of B and F
24
D.4 Impact of a Under Varying Retrieval Size K
25
D.5 Sensitivity Analysis of Retrieval Size K
25
D.6 Learning the Strategy Selection Mechanism
26
D.7 Case Study.
................................
27
D.8 Alternative Study
29
D.9 Full-exploration Study
................................
31
D.10 Routing Robustness Across Embedding Models
32
Pseudocode of RF-Mem
32
F Theoretical Analysis
34
F.I
Risk-Minimizing Selection
34
F2 An Entropy Certificate for Correctness
................................
35
F3 Sub-Gaussian Mean Similarity and Gating :elaailyy.......................
35
F44
-overrae Trade-off
35
LLM Usage Disclosure
36
H Limitation and future work
36
16

---

Published as a conference paper at ICLR 2026
A DATASET DETAILS
PersonaMem. We use PersonaMem' (Jiang et al., 2025), a large-scale personalized memory benchmark introduced COLM 2025, as the core dataset for evaluating personalized retrieval and generation. PersonaMem is designed to evaluate memory-augmmntted LLMs by requiring them to trieve user-specific histories and preferences from long-term dialogue traces. To stress-test retrieval under different scales, we follow the original paper and construct three memory corpora of increas- ing size: 32K, 128K, and IM entries. Table 7 reports dataset statistics, showing that query length remains relatively stable across scales while memory size grows by orders of magnitude, creating challenging retrieval environment. Table 8 further summarizes the task distribution, covering diverse domains such family relations, study consultation, legal and medical consults, and recommenda- tion tasks (movies, food, books, etc.). These heterogeneous task types capture both fact-oriented and reasoning-intensive scenarios, making PersonaMem suitable benchmark for evaluating the balance between Familiarity- based fast recall and Recollection-driven contextual reconstruction.
Table 7: Dataset Statisties across Different Memory Corpora.
Dataset name
32k memory corpus
128k memory corpus
1M memory corpus
# of samples
589
2727
2674
Avg tokens of question Avg tokens of memory
464.6
416.3
415.1
24193.2
15185.1
911733.4
# of Revisit Reasons
99
269
235
# of Track Evolution
139
341
225
# of Latest Prefs
17
866
768
# of Aligned Recs
349
280
# of New Scenarios
57
213
295
# of Shared Facts
129
171
144
# of New Ideas
93
518
727
Table 8: Task Distribution across Different Memory Corpora
Dataset name
32k memory corpus
128k memory corpus
1M memory corpus
of Home Decoration
3
153
164
of Family Relations
32
144
193
# of Therapy
2
249
161
of Travel Plan
71
155
192
Medical Consult
19
195
163
# of Legal Consult
32
283
197
of Study Consult
35
157
191
# of Dating Consult
94
197
208
of Financial Consult
72
198
181
of Food Rec
2
229
191
of Movie Rec of Music Rec
104
158
212
56
44
128
of Book Rec
67
172
139
# of Sports Rec Online Shopping
197
148
196
206
PersonaBench. We further evaluate on the PersonaBench" dataset, which benchmarks personal- ized retrieval in multi-source user environments introduced ACL 2025 (Tan et al., 2025a). As shown in Table 9, PersonaBench aggregates heterogeneous user histories across six users, including conversations with friends, user-Al interactions, and e-commerce purchase recordy. Each user con- tributes average 44 queries grounded in corpus of about 88 items. yielding rich signals of both
Phttps://ght http://ggthhh..cmmooeeeeeeugmm Pew.soaaessnnn Phttps://Oo thub. com les hww.rccew...eee.t.....aoeco
17

---

Published as a conference paper at ICLR 2026
social and transactional behavior. The resulting setting allows to test retrieval robustness across diverse memory types, from casual dialogue to structured purchase history. This design contrasts with PersonaMem, which primarily focuses on long user-Al dialogues, and complements study by introducing multi-modal user traces that better capture the breadth of personalization scenarios.
Table 9: Statistics of the PersonaBench dataset across users.
User Id
# of
of Conversations of User-Al Conversation
# of User e-comerce purchase histories
Queries
Corpus
with friends
48
110
84
23
2
43
90
78
8
3
42
64
51
12
4
46
85
71
14
0
5
44
59
21
6
40
94
79
14
Sum
263
527
422
92
13
Avg
43.83
87.83
70.33
15.33
2.17
LongMemEval. We further evaluate on the LongMemEval? dataset, which benchmarks the ability to retrieve task-relevant information from extended user-specific corpora. introduced in ICLR 2025 (Wu et al., 2025a). Each instance in the dataset consists of factual question paired with a synthetic memory corpus simulating historical user data. The dataset includes two set- tings: LongMemEval-s, where each question associated with approximately 50 memories, and LongMemEval-ng where each corpus contains over 500 memories. total, both settings consist of 500 questions, allowing evaluation of memory retrieval precision under varying context lengths. This benchmark enables controlled analysis of personalized retrieval performance under realistic long-context scenarios.
Table 10: Statistics of the Long.emEEall dataset, with different sizes of associated memory corpora.
Statistic
LongMemEyal-s
LongMemEval-m
Total Questions
500
500
Total Session-level Memory
25,1
250,948
Min Session-level Memory per Question
39
501
Max Session-level Memory per Question
506
Avg Session-level Memory per Question
50.22
501.90
B IMPLEMENTATION DETAILS
All experiments are conducted on single NVIDIA A100 GPU with Ubuntu OS, where the GPU is exclusively allocated the process when measuring runtime.
For PersonaMem, we follow the original setup and build the memory corpus at the dialogue- turn level, where each chunk corresponds GPT-4 SPT-4.1-me 1-mini as the generator and multi i qa-MiniiMe Ma--nii-----------
user query and single LLM response. We use
as the retriever. The hyperparameters are set as À 20, B 3, F 2, with thresholds hhgh 0.6 and 6 = 0.3, ensuring stable regulation across turn-level retrieval. And prompt for generation can be found at Appendix C. And we illustrate the mean score & and entropy H() of the PersonaMem dataset in Figure 6. The cumulative distributions reveal a consistent pattern across corpus sizes: as the scale increases from 32K 1M, the mean score distribution shifts slightly toward lower values, reflecting weaker overall familiarity in larger search spaces, whereas entropy remains concentrated within narrow band (0. 1-0.),,, indicating stable uncertainty resolution. The annotated quartiles further con- firm that the median values of both (>~0.50-0.55) and H(p) 1=00.17-0.18 remain largely invariant, which demonstrates that the familiarity signal preserves a stable operating range across different scales. Such stability provides empirical support for RF-Mem's threshold design, ensuring that
Phttps://gr thub. awwwuul 62. Men.MMeaae
18

---

Published as a conference paper at ICLR 2026
the switching mechanism can generalize without costly re-tuning and maintaining robustness under varying corpus sizes. For support our theorical assumption in Appendix F.3, we show the empirical distribution of mean score in the Figure 7. All three datasets exhibit light-tailed, bounded distri- butions without heavy-tail behavior, and the tail shape remains stable as the corpus size grows. This empirically confirms that the similarity landscape does not display the heavy- structure.
32k
-28k
Score -1M
1.0
1.0
1.0
Cunulative Frequency
Cunuiative Frequencyy
Curnilative Frequeccy
0.8
0.8
0.8
0.6
0.6
0.6
25 424
300
0.4
0.2
0.2
0.2
0.6
0.7
Score
Entropy 32k
Entropy 118kk
Entropy 1M
1.0
1.0
1.0
Cunuisive Fraquenccy
cumuistive Frequency
Cumulative Feequency
0.8
0.8
0.8
0.6
0.6
50% 03 179
2044
2332 10
- 206
100
337
215
0.2
-
75%
0.2
50%
75%
0.2
50%
0.15
Score
Score
Score
Figure 6: Mean score and entropy (p) of the PersonaMem dataset.
Histagram
Hastggrm
Histogram
3.5
3.0
1.5
1.0
0.2
0.2
0.5
0.4
0.5
0.5
9.5
0.6
0.2
0.4 0.5
0.5
Maan Score
Mean
Figure 7: Empirical distributions of mean score of the PersonaMem dataset. For PersonaBench, we adopt the session-level memory construction as in the benchmark paper, which enables fair evaluation of retrieval quality using Recall. We evaluate our method under multiple retrievers, including multi -a---aa qa -aa-aaiiaaa LM- vv14.
and Age-base-en-vl-. 5 to ensure generality. For Recall we use B 3, F 1: for Recall @ 10, use B = 3, F = 2, and X 30 for both. The thresholds are set to Onggh 0.6 and flow 0.0, reflecting the lower similarity scores in session-level indices. And we illustrate the mean score 3 and entropy (p) of the PersonaMem dataset in Figure 8. Also, for support theorical assumption in Appendix F.3, we show the empirical distribution of mean score in the Figure
For LongMemEval, we also adopt the session-level memory construction in the paper, with recall as a metric. We also evaluate RF-Mem under multi multiiqa- al 1ill-Minii 1, and bge-base-en-vi
ensure generality. For LengMemE and LongMemEEal-M, we use B = 4, F = 1. and À = 20. The thresholds are set to hhgh = 0.6 and flo 0.0. And we illustrate the mean score and entropy H (p) of the two datasets Figure 10 and Figure 12. The empirical distribution of mean score 8 shown in Figure 11 and Figure 13 also support our theorical assumption in Appendix F.3.
'nttps: hhittpss//huggingffacee wwtpe//.o.eeeeeeeeeeeeeeeeeeeeee trancceetraassooo http://ttteeetttttttttteeaa//aaa 1in--a1i-------ii-66-----------1 cos-vvl Shttps: htap:p/:////gggggfcc.... co/ sentence- connce-tranee formeraraaa//aa hameerr/aallMiiiii -iini ---
"https://hugginggfaas co/ BAAI http://bbbbbbeeeeeeeeeeeee base-en-v1..5
19

---

Published as a conference paper at ICLR 2026
Mean Score MiniLM
Mean Score MPNet
Mean Score BGE
1.0
Cunuistvv Frequency
Curuulative Frequency
0.8
0.6
50N-
7N-00.88
50N-00360
78N--0233
50N=0332
75N-0.500
0.4
0.4
0.4
0.
0.0
0.0
0.0
0.1
0.2
0.3
0.4
0.5
0.1
0.2
0.3
0.4
0.45
0.50
0.55
0.60
0.65
Score
Score
Score
Entropy MiniLM
Entropy MPNet
Entropy BGE
1.0
1.0
1.0
Curdafive Frequancy
Cumdattvv Frequency
Freuuancy
0.8
0.8
0.8
0.6
0.6
50%-
75%
0.4
50%-6
75%-00218
207
23
0.0
0.00
0.05
0.10
0.15
0.20
0.05
0.10
0.15
0.20
0.10
0.15
0.20
Score
Score
Score
Figure 8: Mean score á and entropy (p) of the PersonaBench dataset.
MiniM Hittgrram
HPNet Hestoramm
BGE Hittogram
0.1
6.2
65
6.05
0.10
5.15
6.55
3.35
5.42
0.50
5.53
Maurt Score
Mean Store
Mlan Scord
Figure Empirical distributions mean score of the Personabench dataset.
C PROMPT FOR PERSONAMEM
For the multiple-choice evaluation PerosnaMem dataset, we adopted strict prompt template to ensure consistency and avoid ambiguous model outputs. The instruction explicitly restricts the model to return exactly one option among (a), (b), (c), or (d), without any reasoning or ad- ditional text. This design prevents uncontrolled generation and makes results directly comparable across models.
Prompt Instruction
You are a multiple-choice answer generator. You MUST respond with exactly one option the form of (a). (b), (c), or (d). Do not include any explanation, reasoning, or extra text. Do not output anything else besides the chosen option. If the correct answer is unknown, make the hest guess and still only respond in that format. If your output does exactly match one of (a), (b), (c), or (d), your answer will be considered incorrect.
During evaluation, the prompt is constructed by concatenating the question, the fixed instruc- tion above, and all candidate options. Retrieved memory context (from RF-Mem baselines) is prepended as dialogue history to provide user-specific background. This structure guarantees deter- ministic outputs while isolating the effect of retrieval answer quality.
20

---

Published as a conference paper at ICLR 2026
Mean Score MiniLM
Mean Score - MPNNt
Mean Score BGE
1.0
Cunuistive Frequency
Cunuistive Frequency
Curnulative Frequenccy
0.8
0.8
0.8
75% 75N-00255
15K-0-30
25% 20
0.4
0.4
0.4
0.0
0.0
0.0
0.1
0.2
0.3
0.4
0.05
0.10
0.15
0.20
0.25
0.30
0.35
0.4
0.5
0.6
0.7
Score
Score
Score
Entropy MiniLM
Entropy MPNet
Entropy BGE
Curubiir Frequines
Curubitrr Frequency
Cumulative Coegaaieeeeeegeeeg Frequency
0.8
0.8
0.8
25% 0.04
73% 0.108
8.05S
50% 108
20204
0.4
0.4
0.4
50%%
23%
50%
0.0
0.0
0.0
0.00
0.05
0.10
0.15
0.0
0.00
0.05
0.10
0.15
0.20
0.00
0.05
0.10
0.15
0.20
Score
Score
Score
Figure 10: Mean score 8 and entropy H(p) in the LongMemEval-S dataset.
MiniLM Histogram
MPNit Histogram
BGE Histegram
Figure 11: Empirical distributions of mean score 3 of the LongMemEval--S dataset.
Example Prompt
Can you suggest some new evidence-based practices explore for healthcare decision-making?
Question:
Instruction: You are choiie answer generator. You MUST respond with exactly one option in the form of (a), (b), (c), or (d). Do not include any explanation, reasoning, or extra text. Do not output anything else besides the chosen option. If the correct answer is unknown, make the best guess and still only respond in that format. If your output does not exactly match one of (a), (b), (c), or (d), your answer will be considered incorrect. Options:
(a) Exploring patient-centered communication strategies might be great new evidence-batd based practice to consider...
One practice that could be impactful is the integration of artificial intelligence in predictive analytics for healthcare decision-making...
(c) You might consider exploring systematic reviews and meta-analyses of clinical tri- als, these often provide robust evidence for healthcare practices...
(d) Implementing shared decision-makinn models in clinical practice is promising evidence-based strategy to explore...
Correct Answer: (c) # This is the correct answer and not in the prompt.
21

---

Published as a conference paper at ICLR 2026
Mean Score MiniLM
Mean Score - MPNet
Mean Score BGE
1.0
1.0
Oumulative Frequency
Curnuletiv Frequency
Oarmsaativ Erequuency
0.8
0.8
0.8
0.6
0.6
343
75%--0 505
5013
0.4
0.4
0.4
500G
0.2
25%
0.0
0.0
0.0
0.1
0.2
0.3
0.44
0.5
0.1
0.2
0.3
0.4
0.55
0.6
0.5
0.6
0.7
Score
Score
Score
Entropy MiniLM
Entropy MPNet
Entropy - BGE
1.0
Cumuistive Frequency
Curullative Erequency
Dumulafiuv Erqquency
0.8
0.8
0.8
0.6
0.6
50% 0.158
25R 00007
0.007
50% 0.162
75% 0.202
10%-0.221 21.19
0.4
0.4
0.4
0.2
75%
0.0
0.0
0.0
0.00
0.05
0.10
0.15
0.20
0.00
0.05
0.10
0.15
0.20
0.05
0.10
0.15
0.20
Score
Score
Score
Figure 12: Mean score & and entropy (p) in the LongMemEval-M dataset.
MinLM -Hsttgram
MPNot- Hittoraam
Histogram
0.1
0.3
0.4
0.5
0.3
0.2
0.3
0.4
0.5
0.65
0,50
0.55
0.66
0.55
0.76
075
Mean Score
Vean Score
Nean Seare
Figure 13: Empirical distributions of mean score 3 of the Long dataset.
D ADDITIONAL EXPERIMENT
D.1 RESULT ACROSS CATEGORY IN PERSONAMEM
To further examine retrieval behaviors across different task types, conduct category-level anal- ysis on PersonaMem (Table 11). This evaluation spans factual queries foog recommendation, medical consult), reasoning-intensee intensive domains famiil relations, therapy), and hybrid scenarios (e.g., aligned recommendations, new scenarios). allowing us to disentangle how each retrieval strat- egy responds varying demands of factual precision, contextual reasoning, and personalization.
First, Familiarity excels in direct recall but struggles as complexity grows. As shown in Ta- ble 11, dense retrieval achieves peak scores on fact-centric categories where surface similarity is sufficient. For instance, it reaches perfect accuracy on food recommendation (1.0000 at 32k corpus) and strong results in movie recommendation (0.5759 at 128k). These cases confirm its cognitive role as a fast, coarse recognition process. However, as tasks demand contextual integration- such as track evolution or aligned recommendations Fecomendaiionn-Faaalaatt shows clear degradation, particularly when scaling IM memories.
Second, Recollection proves valuable for grasoning-intensivy tasks. categories like family relations, therapy, and dating consultation, Recollection consistently surpasses Familiarity by constructing dispersed cues across sessions (e.g.. 0.5938 vs. 0.5625 in family relations at 32k, and 0.5280 0.4534 in therapy at 1M). Yet, factual categories such food recommendation or study consultation, where surface matches are already diagnostic, its advantage diminishes or even reverses. This confirms its role slower but more diagnostic mode.
22

---

Published as a conference paper at ICLR 2026
Third, RF-Mem achieves robust gains through adaptive switching. As shown in Table 11, RF- Mem consistently outperforms single-mode baselines by combining the efficiency Familiarity with the diagnostic depth of Recollection. For instance, it improves legal consultation at 32k (0.94 vs. 0.78/0.88) and sustains advantages in family relations at 1M (0.60 vs. 0.57/0.58). These examples illustrate its ability to maintain high aceuracy across both factual and reasoning categories. Impor- lantly RF-Mem also delivers the best overall results at all corpus scales, confirming that uncertainty- aware routing enables robust retrieval without committing to a single mode.
Table 11: Performance comparison on different memory corpus sizes.
32k memory corpus
128k memory corpus
1M memory corpus
Question Category
Famili.
Recol.
RF-Mem
Famili.
Recol.
RF-Mem|
Famili.
Recol.
RF-Mem
Overall
0.5908
0.6214
0.6350
0.5259
0.5288
0.5394
0,4518
0.4544
0,4589
Home Decoration
0.6667
0.6667
0.6667
0.6275
0.6275
0.6471
0,4207
0.4451
0.4207
Family Relations
0,5625
0.5938
0,5938
0,5486
0,5625
0.5764
0.5734
0,5803
0,6010
Therapy
0,5000
0.5000
0.5000
0.5060
0.5301
0.5341
0.4534
0.5280
0.4845
Travel Plan
0.4789
0,5775
0.5634
0.5806
0.6194
0.6452
0.4948
0.4792
0.4896
Medical Consult
0.8421
0.7895
0.8421
0.4718
0,5128
0.5026
0.4294
0.4110
0.4356
Legal Consult
0.7812
0.8750
0,9375
0.5124
0.4735
0.4841
0.4061
0.3909
0.3909
Study Consult
0.6000
0.6000
0.6000
0.5096
0.4904
0.4773
0.4503
0.4293
0.4817
Dating Consult
0,5851
0.5851
0.6277
0.4772
0,5025
0.4975
0.5192
0,5000
0.5096
Financial Consult
0.5556
0.5972
0.6389
0.4899
0.4697
0.4899
0,4365
0.4309
0.4199
Food Rec
1.0000
1.0000
1.0000
0.5240
0.5721
0.5633
0.3717
0.4241
0.3874
Movic Rec
0.6154
0.6154
0.5865
0.5759
0.5696
0.5633
0.4434
0.4481
0.4575
Music Rec
0.6071
0.6250
0.6250
0.4091
0,4773
0.5096
0.4609
0.4688
0.5000
Book Rec
0.5373
0.5970
0.6418
0.5640
0.5407
0.5814
0.4964
0.4532
0.5036
Sports Rec
0.5228
0.4670
0.4975
0.4189
0.4392
0.4392
Online Shopping
0.5408
0.5459
0.5561
0.3738
0.3932
0.3786
D.2
SENSITIVITY ANALYSIS OF a AND
MiniLM (Recal1@5)
MPNet (Recall@5)
BGE (Recali@5)
0.47
0.400
0.395
0.375
0.15
0.390
0.25
0.365
0.44
0.355
MiniLM (Recall@10)
MPNet (Recali@) 10)
BGE (Recall@10)
0.60
0.55
0.1
0.54
0.495
0.2
-0.90
0.485
0.2
0.4
0.2
04
Figure 14: Hyperparameter sensitivity study PersonaBench. Heatmaps report Recall@5 and @ecall@ across different retrievers, varying a qquery-centroid mixing) and 7 (entropy threshold).
To examine the robustness of RF-Mem, conduct systematic study varying two key hyper- parameters, a and т, across different retrievers (MiniLM, MPNet, BGE) and evaluation metrics (Recall@ 5/10). Figure 14 visualizes the results heatmaps, where warmer colors indicate higher retrieval performance. This setup allows us to directly assess how query mixing (ai) and entropy thresholding (r) interact to balance efficiency and coverage.
First, the query-centroid mixing coefficient a controls how strongly recollection expands be- yond the original probe. In Recall moderate (around 0.3-0.6) yields the best results, such
23

---

Published as a conference paper at ICLR 2026
MiniLM improving overall recall ~0.47, while extreme values (a 0.0 1.0) reduce stability. For @ 10, performance is less sensitive, but mid-range a still avoids degradation observed at the boundaries. This indicates that balanced mixing is crucial: too little restricts recol- lection, too much overwhelms with noisy expansions.
Second, the entropy threshold regulates the switch between Familiarity and Recollection. Low 7 values push the system toward frequent recollection, leading to higher gains in complex categories but also exposing variance. For instance, MiniLM Recall @ 10 achieves its highest values (~0.600 when 7 is set around 0.2-0.25, while overly high thresholds (r 2 0.35) reduce adaptivity, as retrieval defaults prematurely Familiarity. The results highlight that moderate entropy gating achieves the best trade-off between efficiency and contextual depth.
D.3 SENSITIVITY ANALYSIS OF B AND
Figure 15 reports the effect of beam width B and fanout F on retrieval performance in LongMemEval-S and Long.MemEval-M, under the retrievaler multi-qe L- cos-vi of recollection retrieval. We observe three consistent patterns.
First, increasing generally reduces recall@5. As shown the top-left and bottom-left plots, recall@5 steadily drops when F grows, since higher fanout expands the search too widely, diluting precision top-ranked results. For example, LongMemEval-S, recall@5 decreases from 0.72 (F 1) to below 0.60 when F = 4.
Second, recall@ 10 is more stable under moderate F, but declines when B or F are too large. The middle plots show that recall@ @ 10 peaks around F = 1 F = 2 (e.g., 0.83 in LongMemEval- S, 0.56 Long but drops once = or = 4, reflecting over-expansion and redundancy. This suggests that moderate fanout provides useful diversification, while excessive expansion introduces noise.
Third, recall@50 50 is robust and even improves with higher F. As shown in the rightmost plots, recall @ 50 remains very high in LongMemEval-S (above 0.98 across all settings), and increases slightly in LongMemEval-M (up to 0.76 when B 3, F' 2). This indicates that large fanout helps cover more relevant memories at longer retrieval depths, consistent with recollection' 's role of broad exploration.
Overall, these results demonstrate that small beam width and low fanout (B = 2 or 3, F = 1 or 2) offer the best balance between early precision and broad coverage, aligning with the intuition of controlled, stepwise recollection.
LongMemEval-S recall@5
LongMemEval-S recall@10 Long recall@50 0.99
0.84
0.70
0.82
Value
0.80
Value
0.99
0.78
0.76
0.99
0.74
LongMemEVal-M recall@5
LongMemEVaI-M recall@10 LongMemEval-M recall@50
0.40
0.50
0.76
Value
0.35
Value
Value
0.75
0.45
0.30
0.40
0.25
0.74
3
2
2
4
Figure 15: Hyperparameter B and F sensitivity of Recollection retrieval on LongMemEval-S and LongMemEval-MM
24

---

Published as a conference paper at ICLR 2026
D.4 IMPACT OF UNDER VARYING RETRIEVAL SIZE K
Effect of a on Retrieval. To investigate the role of the mixing weight a, we conduct a sensitivity study on the LonMMemEval-M dataset under the Recollection retrieval, measuring Recall Re- @ 10, and Recall 50. As shown in Figure 16, the impact of a exhibits a clear dependence on re- trieval depth. For Recall which emphasizes short-hop retrieval precision, performance decreases steadily as a grows, suggesting that placing excessive weight on the original query suppresses ex- ploratory expansion and thereby reduces the system's 's ability to capture nearby but diverse evidence. In contrast, Recall 55 improves markedly with larger a, indicating that stronger query-centroih mixing facilitates broader exploration and enables the retriever to recover more distant but rele- vant memories over long retrieval chains. Recall @ 10 demonstrates intermediate pattern, peaking around a = 0.3-0.5 before declining, which reflects the delicate balance between preserving the specificity of the original query and promoting contextual expansion through recollection.
These results highlight a fundamental trade-off: smaller values of a favor precision in short- short-path re- trieval, while larger values enhance coverage in long-pate path retrieval. The presence of an intermediate optimum for Recall @ 10 further confirms that no single setting universally dominates across depths, underscoring the need to calibrate according application demands. More broadly, this sensitiv- ity analysis validates the design choice in RF-Mem to treat as a tunable parameter, enabling the framework flexibly adjust the balance between efficiency and breadth when navigating different levels of the memory space.
Sensitivity of Recollection Retrieval in LongMemEval-M
Recall@5
Rec 10
Recall@550
0.75
0.43
0.57
0.74
0.43
0.42
0.56
0.73
0.42
0.42
0.55
0.42
0.4
0.8
0.2
0.6
0.4 0.8
Figure 16: Effect of on short- vs. long-path retrieval performance (Recall @ K).
D.5 SENSITIVITY ANALYSIS OF RETRIEVAL SIZE K
To further examine the sensitivity of RF-Mem to the probe retrieval parameter K, vary K from 5 to 50 and report the resulting accuracy under different corpus sizes (32K, 128K, and IM) in PersonaMem. As shown in Figure 17, the overall performance of RF-Mem remains consistently stable across wide range of K. demonstrating the robustness of our familiarity-basse regulation. In contrast, the baselines exhibit stronger fluctuations: for Familiarity, accuracy quickly rises with K but plateaus once K 2 10, while Recollection benefits from larger K due to finer entropy resolution, yet its gains taper off and even slightly degrade after K 20.
RF-Mem combines the advantages of both pathways, achieving higher accuracy than either baseline across all corpus scales, and crucially avoids the ever-expansion problem of Recollection by adap- tively invoking recollection only when necessary. Notably, the performance curves at 128K and IM confirm that the improvement saturates beyond moderate probe sizes, suggesting that small values of K (e.g.. 10-20) are already sufficient for effective familiarity estimation and adaptive switching. These results highlight that RF-Mem does not depend on finely tuned probe parameters, making both robust and practical for deployment in large-scale personalized memory retrieval scenarios.
25

---

Published as a conference paper at ICLR 2026
32k Fami.
32k Recol.
32k RF-Mem
0.64
Accuracy
0.62
0.60
128k Fami.
128k Recol.
128k RF-Mem
0.56
Accuracy
0.54
0.52
0.50
1M Fami.
1M Recol,
1M RF-Mem
0.46
Accuracy
0.44
30
Figure 17: Effect of probe size K on retrieval performance across different corpus scales.
D.6 LEARNING THE STRATEGY SELECTION MECHANISM
Although the gating thresholds in RF-Mem are manually specified, it is important to assess whether the routing policy can also be learned from data. To this end, construct binary classification task in which each memory instance represented by the two gating signals, namely the mean similarity the probe retrieval and the entropy (p) over the top-K similarity distribution. The classifier predicts whether the familiarity path or the recollection path yields higher retrieval quality. This setting allows us to examine how much supervision is needed approximate the decision boundary and (ii) whether the two handcrafted signals contain sufficient information for reliable automatic routing.
Training data. We utilize two subsets of PersonaMem: the 32k corpus (589 samples) and the 128k corpus (2727 samples). For 32k, the first 400 samples are used for training and the remaining 189 for evaluation; for 128k, the first 2000 samples are allocated for training and the remaining 727 for evaluation. Only cases in which one path clearly outperforms the other retained, forming clean -abere-labeled dataset. Table 12 summarizes the distribution of positive and negative labels.
Training Details. A three-layer MLP trained as a binary classifier, with hidden dimensions of 32 and 16, and final sigmoid output unit. ReLU activations are used in the hidden layers. Training follows an 80/20 trin-validation split.
Table 12: Training data statistics for the learned gating classifier.
Dataset
Fami. win
Recol. win
Tie
Training Samples
PersonaMem-327
34
41
325
75 (34+41)
PersonaMem- 128k
178
185
1637
363 (178+185)
Experimental findings. Tables 13 and 14 report detailed performance across evaluation cate- gories. We find while the hand-tuned thresholds remain slightly superior, the learned gate
26

---

Published as a conference paper at ICLR 2026
shows clear improvements when trained on more data, demonstrating its potential applicabil- ity in real-world scenarios where larger-scale logs are accessible. On the smaller 32k set, the learned gate shows high variance across runs and does not consistently outperform simple heuris- ties, indicating that limited data makes the decision boundary difficult to estimate reliably. When moving the larger 128k set, the learned gate becomes considerably more stable and approaches the performance of the manually tuned mechanism. These results suggest that the two handcrafted signals already capture meaningful structural separation between the two retrieval modes, and that automatic tuning becomes increasingly effective as more user data is available.
Table 13: Results of the learned strategy selector on PersonaMem-32k (189 test samples).
Track Evolu- tion
New Scenar- ios
Num. Num. of Famili. Recol.
Revisit Reasons
Shared Facts
Aligned Recs
Latest Prefs
New Ideas
Method
Overall
of
Dense
0.6296
0,9688
0.7297
0.6327
0.8125
0.2857
0.5385
0.2286
189
0
Recol.
0.6667
0.9688
0.7297
0.7551
0.8750
0.1429
0.7297
0.2286
0
189
RF-Mem
0.6720
1.0000
0.7838
0.6939
0.8750
0.1429
0.6154
0.2571
83
106
RF-Mem
0.6482
0.9688
0.7514
0.6878
0.8750
0.1429
0.5385
0.2286
97.5
91.5
(Learned)
=0.0067
+0.0000
+0.0114 +0.0255
1..0000000.0000
+0.0000
=0.0000
16.70
=6.70
Table 14: Results the learned strategy selector on PersonaMem- (727 test samples).
Track Evolu- tion
New Scenar- ios
Num. Num. of of Famili.Recol.
Revisit Reasons
Shared Facts
Aligned Recs
Latest Prefs
New Ideas
Method
Overall
Dense
0.5131
0.7742
0.6410
0.6162
0,5176
0.5122
0.3529
0.3043
727
0
Recol.
0.5158
0.7634
0.5897
0.6162
0.5647
0.5366
0.3824
0.2609
0
727
RF-Mem
0.5199
0.7527
0.6410
0.6162
0.5059
0.5463
0.3824
0.2971
402
325
RF-Mem
0.5175
0.7591
0.6180
0.6091
0.5553
0.5400
0.3794
0.2718
334.4
392.6
(Learned)
+0.0039
+0.0055
+000255
+0.0048
+0.0182
+0.0152
+0.0070
228.33+-28.3
D.7
CASE STUDY.
D.7.1 RECOLLECTION-PATI WINNING CASE
To illustrate the difference between ohe-shot familiarity retrieval and recollection retrieval, we con- duct a case study the PersonaMem dataset. For clarity, we set B 2 and F 2 in the recollection process. Figure 18 presents the outputs under different retrieval modes for the query "Can you suggest some new sridence-basedd practices to explore for healthcare decision-making"" The Familiarity retriever directly returns the top-10 highest-scoring entries, capturing salient but fragmented memories such mentions of "conventional medicine" "therapeutic modalities." with noisy mentions like "try-
While efficient, this strategy surfaces isolated tragments-someting ing new healthy recipe"-aai fails to integrate them into a coherent chain. As result, crucial contextual cues that span across sessions remain overlooked, highlighting the inherent limitation of one-shot retrieval.
By contrast, the Recollection process unfolds in multiple rounds. At each step, retrieved items are clustered into semantically coherent groups, and their centroids are blended with the query to form new probes. As shown in Figure 18, this branching expansion progressively uncovers comple- mentary evidence: for instance, r=1 surfaces "modern science blending with age-old methods," r=2 brings in references to "gathering patient history" and "reviewing health records," and r=3 integrates higher-level anchors like "structured methodology and evidence-based practices." This stepwise en- richment not only reduces redundancy by grouping similar items but also reconstructs temporally dispersed details into a coherent memory trace. Compared with the static list from Familiarity re- trieval, RF-Mem's 's recollection branch demonstrates chain-like reconstruction process, aligning with the process theory by simulating deliberate, effortful recall. Ultimately, this yields more diagnostic and contextually grounded evidence for answering the user's query.
27

---

Published as a conference paper at ICLR 2026
Familiarit Retrioval
support are genuinely effective
seeking belp..
It just doest 't align with my appeoach and focus conventional medicine...
Direct retrieval K-10
70
llikk about how Base modilities could complement conventional seatmmntts..
deep differmnt therapettcc
reecesing offeeecy.
Can you suggos sers new evidmiee- besed practices to explore for heultheaee dacisonnmaking!
69
Jow moder scionce is beginning to Mead with these age-oli mellods, cresting more comprehensive approach health cam
9
nerd explon differrnt meiheds for tracking health treammnns....
a suppective ervironment
elfective dakoue fundamental not only is consuiations but throughout the comcammn of cane.
teed heality specificcllly prepare for better mutition before appoismmnttt.
... varsous unteriscipliuury approaches being utilized cahance patien: being...
Recollection Retrievval
Each Round Retrieval :N- (B-r)"F, F-2 B-2 indieate each round pick tup branches
you suggesi sone new evidance jeactions
explore for henkthcare decision makingg
Chasee
Claster 2
methods I
offective individuals seeking help...
think about how these modalities could complemert convattionl trestments. deveed deep isto difleson therapostic modalities, assessing their efficncy...
just doeen align my approach
36
conventiical medilte....
1D Claster
Claster 2
ID
Claste
Claste
.how moder science beginning blend with these old methods, creating a more compreheniiv sprenach bealth care....
9
...need to explore differest mecladd for tracking my health jomeney.
Chuser
... therapeuti mocalines,
2=
...gthhr relevans information and restewing history could lead to more meaningfti discussions
axecesing ways they reseratc
myself my parents...
45
learn about new develepnents and evidenco lnsed prict lock, peritas inciglt prlissionels...
ID
Cluster
ID
Chuser2 2
fecus on convertional medicine; the stmetured mothodnlogy and exidence- boood practioes resonate much
3=3
methods plants holiatic
healthcaro...
Stop when number retrieved items reaches K-10
Figure 18: Case study from PersonaMem: comparison between Familiarity and Recollection re- trieval where Recollection wins. Familiarity surfaces salient but fragmented evidence, while Recol- lection progressively reconstructs temporally distributed details via clustering and query refinement.
D.7.2 FAMILIARITY-PAT WINNING CASE
To analysis the failure case, we again compare one-shot Familiarity retrieval with the multi-round Recollection process on the PersonaMem dataset. As shown in Figure 19, we show the retrieval result of dual paths for the query "I'm considering developing a tool to manage finances while traveling. How could I ensure it helps me prioritize experiences like local cuisine or guided tours?". Familiarity retrieval outperforms Recollection in serving the user's intent. The one-shot Familiarity retriever returns broad and query-aligned experiences', 'finance management apps", or 'last trip'. These results directly address the user's need to "prioritize experiences while traveling." illustrating that high-scoring surface cues can sometimes be sufficient for intent coverage.
of memories, many which explicitly mention 'local
In contrast, the multi-round Recollection process drifts along a different semantic trajectory. Al- though its iterative clustering-ame aanerringannd-rffeemmeentt mechanism successfully produces deeper and more structured traces, the retrieved branches gradually converge toward 'long-term financial habits' and 'personal finance management', This indicates that the recollection trajectory overfit domi- nant semantic clusters in the memory corpus, especially when several high-density branches are the- matically coherent but misaligned with the user's immediate intent. As result, the recollection path
28

---

Published as a conference paper at ICLR 2026
becomes increasingly anchored in financill-mangenne management narratives, overlooking the travel-related aspects of the query.
Despite being bad case for retrieval accuracy, this example is instructive. It exposes of deliberative recollection: the mechanism excels at reconstructing temporally dispersed, concep- tually rich memory chains, yet it may over-emphasize internal coherence the cost of task-oriented alignment. This case highlights the importance of balancing depth with intent fidelity, and under- strategy selection in future work.
trade-off
scores the need for more precise cumnliarty-unnceraintt--
Fomilivity Retvieval
ID
Direct retrieval K-10
fascinating growing personal management through apps, they can help many travelen...
68
...I Vve commined a personal finance course focused or invel budgeting strategles.. while inveiing, and commend for
Buigering crucial avoid financial
approech...
more structured appronch trakingg finances...
cursidering developing tool munage finances while irveling. How could ensure it helps me prionitoe experienoos lice local cuisine or guided tours?
19
The mare interacioos you lave, the more you cullivatc a suppotive network that cn provide encouragement and additional tips making your planiing even more fruittul.
Embneing boih the prysicality of printed griles and the flexihilit of digitil took
puside wellmonde upprach
...inovative loaming methods,
joinnng
foruns, putticppaing intecactive
halgeting teels nemp apps designed penural fireno...
Our presioss lnvel experianes, whaller Daey were posiivve or negative, have likely influenced how we sppreach planning for fature adventeres.
explom local experiences, can be onlightzning think about they cnrich and community...
my tnp stubling upon local festval or trying hidden pem of resturam...
spontaneous experiences,
Recoilection Retrieval
Each Retrieval (Btr)*F, F=2 B=2 indicate each round pick top branches
conodering developing a tool to manage finnccss while inveling could Unsuar il helps me promisse experiencos like local cuisine or guidal tous??
Clasta
61
...It's fascinating see the growing trend of personal finance
ID
Cluser 2
1-0
...I% through they help mavelers budgecing strategles....
47
... molived might need mone sirictored sepnach to tnac firnasx.
focused
10
... ...doggttingg is crucial to avoid any financial surpriss while taveling.
and command you for uking this prosstive appeouch...
Cluster 2
iruly anazing desper understanding of our firancial habits can lead to significunt clange in our lives..
Claster
Choster
innovative learning methoals such joining onlire forums, parricipoting intemetive budgeting mell, or using sppe designed for personal finance...
Cluster
Choser
ID
Chuoer
... Commitmess significant
S4
... did atlend anvestment
rewards over time, helping you maintain clarity and control over your financial jommy.....
workssop. bur felt redundant...
Claster
Cluster 2
motivstal me consider lomgterm financial well...
45
changed the way you interet your
groally
health...
Stop when the number of retrieved items reaches K-10
Figure 19: Case study from PersonaMem: comparison between Familiarity and Recollection re- trieval where Familiarity wins. Familiarity produces broader and more relevant evidence aligned with prioritizing travel experiences, while Recollection progressively drills into long-term financial management and drifts away from the user's actual intent.
D.8 ALTERNATIVE STUDY
In this section, we evaluate several alternative recollection mechanisms to test the robustness of RF-Mem beyond the proposed KMeann-bassed and a-mix design. We replace the centroid module
29

---

Published as a conference paper at ICLR 2026
with DBSCAN and Spectral Clustering, replace the a-mix with gated mixing, and replace the entire module with graph-basee expansion, while keeping all other components fixed. The comparison results in Table 15 highlight the generality of RF-Mem across diverse structural assumptions.
Table 15: Retrieval performance comparison under different recollection strategies.
Metries
@
Recal @ 10
Basic Info
Social Info
Pref
Pref Hard
Basic Info
Social Info
Pref Easy
Pref
Methods
Overall
Overall
Easy
Hard
Basic KMeans
Famili.
0.4515
0.4852
0.4904
0.3659
0.4484
0.5879
0.6220
0.6442
0.5561
0.5964
Recol.
0.4379
0,4903
0.5128
0.3854
0,4491
0.5924
0.6859
0.5659
0.6267
0.6062
RF-Mem
0.4788
0.5091
0.4872
0.3854
0.4701
0.5924
0.6799
0.5707
0.6267
0.6071
DBSCAN cluster (KMeans alternative)
Recol.
0.4424
0,4940
0.4872
0.3512
0.4431
0.6045
0.6079
0.6891
0.5366
0.6028
RF-Mem
0.4879
0.5129
0.4744
0.3463
0.4669
0.5659
0.6079
0.6667
0.6015
0.6040
Spectral cluster (KMeans alternative)
Recol.
0,4591
0.4739
0,5000
0.3756
0,4522
0.6045
0,5978
0.6474
0.5366
0.5957
RF-Mem
0.4818
0,4928
0.4872
0.3707
0.4651
0.6015
0.6041
0.6474
0,5610
0.6001
Gate (c-mix alternative)
Recol.
0.4439
0,4739
0.5128
0.3902
0.4491
0.5924
0.6016
0.6795
0.5317
0.5936
RF-Mem
0.4667
0,4928
0.5000
0.3756
0.4602
0.5924
0.6079
0.6667
0.5610
0.5988
Graph + Breadth First Search (whole alternative)
Recol.
0.4167
0.4739
0,4872
0.380S
0.4314
0.5591
0.5887
0.5887
0.5366
0.5722
RF-Mem
0.4258
0,4739
0,4872
0.3756
0.4349
0.5742
0.6220
0.6442
0.5561
0.5899
D.8.1 DBSCAN RECOLLECTION (KMEANS ALTERNATIVE)
Experiment setup. To examine whether the assumptions of KMeans (balanced and approximately spherical clusters) influence recollection, we evaluate DBSCAN (Ester et al., 1996) as a density- based alternative. DBSCAN identifies arbitrarily shaped clusters and automatically detects noise points, providing a contrasting clustering geometry. The retrieval backbone, mixing rule, and eval- uation setup remain unchanged ensure clean comparison between recollection mechanisms.
Findings. Results in Table 15 show that DBSCAN- RF-Mem (DBSCAN) consistently improves over its recollection baseline. This demonstrates that the effectiveness of c--mixig is not tied to KMeann-ppecciicc assumptions and remains stable under density-driven neighborhood structures.
recollection performs competitively, and
D.8.2 SPECTRAL CLUSTERING RECOLLECTION (KMEANS ALTERNATIVE)
Experiment setup. We further evaluate Spectral (Ng et al., 2001) Clustering, which identifies clus- ters via graph Laplacian eigenvectors and effectively models manifold-shaped structures. This setup tests whether recollection remains stable when memory clusters deviate from centroid-based geom- etry. All configurations follow the evaluation results summarized in Table 15.
Findings. As reported in Table 15, RF-Mem (Spectral) achieves consistent gains the Spectral baseline, improving R@5 Overall from 0.4522 to 0.4651. These results indicate that a-mixing generalizes well across diverse cluster geometries and maintains its ceometry-respectinn update behavior even under manifold-structuree partitions.
D.8.3 NONLINEAR GATED MIXING (a-MIX ALTERNATIVE)
Experiment setup. Motivated by nonlinear interpolation strategies, we implement gated mixing variant in which the update coefficient is determined by sigmoid gate applied to the similarity between the current query representation and the cluster centroid. Specifically, given the query vector (r) at iteration r and the corresponding centroid (r), the gated interpolation is defined as:
((x(1) E! (xx()) "g
and the updated query is computed via:
(r+1)
norm (x(1)) + +(1+++1--0 (1- (0-g) gi," + Xx
X
30

---

Published as a conference paper at ICLR 2026
where (-) denotes the sigmoid function and x, is the original query. This formulation introduces a nonlimear, content content-adaptive interpolation mechanism that contrasts with the proposed linear a- mixing. All experiments follow the unified evaluation protocol in Table 15 to ensure comparabilityy
Findings. As shown in Table 15, gated mixing improves over Familiarity retrieval but does not surpass the effectiveness of the proposed a-mixing rule. RF-Mem (Gate) obtains 0.4602 R @ Overall, compared with 0.4701 for our K.Means-based implementation. The performance gap is particularly noticeable in Social Info and Pref-Hard subsets, where recollection plays critical role. These results indicate that although nonlinear interpolation is a reasonable alternative, a-mixing provides a stable and geometry-aligned update, consistent with its interpretation as a manifold-an -ware update rule on the unit hypersphere.
D.8.4 BRAPH-BASER RECOLLECTION (WHOLE ALTERNATIVE)
Experiment setup. Finally, we examine a structure-fiee alternative by constructing a KNN graph (Zhang et al., 2025a) over memory corpus and performing Breadth-Finst Search (BFS) expansion. This mechanism collects local neighborhoods via graph traversal rather than via cluster- ing. The full evaluation follows the protocol summarized in Table 15.
Findings. Table 15 shows that graph-based recollection performs weaker than clustering- recollection. Nevertheless, RF-Mem consistently improves its graph baseline, demonstrating the robustness. The comparison suggests that clustering more effectively aggregates semantically coherent neighborhoods, whereas BFS may over-expand into dense yet irrelevant regions.
D.9 FULL-EXPLORATTION STUDY
To further examine whether the effectiveness of RF-Mem depends on the early-stop rule used in the recollection path, we conduct an additional analysis where the expansion process is allowed to continue until all reachable candidates (within predefined maximum depth) have been explored.
Table 16: Retrieval performance under full exploration.
Metries
Recall@5
Recall 10
Basic Info
Social Info
Pref Easy
Pref Hard
Basic Info
Social Info
Pref Easy
Pref Hard
Methods
Overall
Overall
Basic KMeans (Early-Stop)
Famili.
0.4515
0.4852
0.4904
0.3659
0.4484
0.5879
0.6220
0.6442
0.5561
0.5964
Recol.
0.4379
0.4903
0.5128
0.3854
0.4491
0.5924
0.6859
0.5659
0.6267
0.6062
RF-Mem
0.4788
0.5091
0.4872
0.3854
0.4701
0.5924
0.6799
0.5707
0.6267
0.6071
Basic KMeans (Fully Exploration)
Recol.
0.4530
0.4827
0.5128
0.3805
0.4537
0.6015
0.6204
0.6763
0.5415
0.6036
RF-Mem
0.4848
0,5016
0.5000
0.3756
0.4709
0,6015
0.6267
0.6667
0.5659
0.6083
Graph 0.4872
Breadth-First Search (Farly-Stop)
Recol.
0.4167
0.4739
0.3805
0.4314
0.5591
0.5887
0.5887 0.6442
0.5366
0.5722
RF-Mem
0.4258
0.4739
0.4872
0.3756
0.4349
0.5742
0.6220
0.5561
0.5899
Graph Breadth-First Search (Fully Exploration)
Recol. 0.4530
0.4701
0.4776
0.3902
0.4486
0.5818
0.5642
0.6635
0.5659
0.5841 0.5986
RF-Mem
0.4758
0.4890
0.4776
0.3854
0.4629
0.5924
0.6220
0.6442
0.5561
Experiment setup. Unlike the default setting, in which the recollection process terminates once the top-K quota is satisfied, this full-exploration variant aggregates the complete expanded set and per- forms final dense re-ranking step over all collected items. This design enables us to test whether the proposed recollection mechanism remains robust when given a substantially larger search bud- get. Table 16 summarizes the results for both the KMeans-based and Graph-based recollection mechanisms under the early-stop and full-exploration settings. Across all categories, full explo- ration increases the coverage of semantically relevant nodes, providing a stricter evaluation of the recollection path.
Findings. First, we observe that allowing full exploration followed by global re-ranking step con- sistently improves RF-Mem compared to the early-stopped variant. For instance, RF-Mem (Full) achieves an Overall Recall @5 of 0.4709 under KMeans, surpassing the early-stop score of 0.4701. This demonstrates that RF-Mem continues to benefit from deeper recollection when additional evi- dence is made available, reinforcing the stability and scalability of the proposed a-mixing updates.
31

---

Published as a conference paper at ICLR 2026
Second, under the full-exploration setting, the graph-based variant shows clear improvements over both its early-stopped counterpart and the Familiarity (dense) baseline. RF-Mem (Graph+bfs. Full) obtains Overall Recall @ 10 of 0.5986, outperforming the early-stop result of 0.5899. This in- dicates that graph-guided recollection, despite being weaker than KMMans in the default setting, becomes more comperitive when provided with additional traversal depth. These findings support our broader claim that deliberate, structure-aware exploration of the memory space offers a valuable and robust alternative formulation for retrieval.
D.10 ROUTING ROBUSTNESS ACROSS EMBEDDING MODELS
To assess the sensitivity of RF-Mem to the underlying embedding model, we investigate how em- bedding quality and calibration influence routing decisions. Because both the similarity- un- certainty measure and the recollection path depend on the geometry of the embedding space, under- standing the extent to which different retrievers alter routing behavior or introduce failure risks is crucial for evaluating the robustness of the framework.
Experiment Setup To answer the question of whether routing accuracy varies across differ- retrievers, we measure routing statistics for three embedding models PersonaBench: multi -qalii--a-Minnn n L 11--L--cs--2 a -mpnet -aannet--aseee... and bge bege-base--n-v/ 5. For each model, we record (i) routing frequencies into the familiarity and recollection branches, and (ii) retrieval accuracy. All other components of RF-Mem remain fixed to isolate the effect of the embedding backbone.
Findings. Table 17 summarizes the result. We can find: First, Stronger retrievers route more queries into the recollection path, while weaker retrievers rely more heavily on fast familiarity. MiniLM and MPNet trigger substantially more recollection transitions, indicating that their embedding spaces provide more reliable cluster structures for iterative refinement. contrast, BGE, which exhibits less stable similarity distributions, routes significantly larger number of queries to familiarity. Sec- ond, the routing behavior demonstrates that the mechanism adapts to the embedding geometry in principled way. Stronger retrievers provide clearer cluster boundaries, which encourages recol- lection, whereas noisier or weakly calibrated embeddings shift routing toward familiarity as a more reliable fallback.
Table 17: Routing statistics and retrieval performance across different embedding models.
Retriever
Time
Route-RRecol.
R@5 Overall
multi-qa-Me Muult--a-MiniiLM--a Mi--iLL-LL---os-vl
9.16ms
60
203
0.4701
all all-mpnet-base-y -2
8.33ms
15
248
0.4009
bge-basc-en-vi.55
14.s4ms
140
123
0.3836
E
PSEUDOCODE OF RF-MEM
RF-Mem algorithm. Algorithm begins with short probe to estimate retrieval certainty and then switches between fast Familiarity path and a deliberate Recollection path. Given the query embedding X,, the retriever returns the top-kp kp probe candidates and produces a temperature -ccale score distribution p. we compute the list entropy (p) and the mean score s, then apply thresholds (Origh; 0ro)) together with an entropy gate T as specified in Eqs. (1) the familiarity signal is strong, that is $ 2 Ohigh or (p) < T, RF-Mem executes one-shot Familiarity retrieval that returns the top-K items by similarity. If the signal is weak, that is 3 S fjow or H()) > т, or if the probe yields no hits, RF-Mem invokes Algorithm 3 to perform stepwise Recollection. In each round r of recollection, we retrieve top-N candidates with N = (B- F, cluster them by KMeans into at most B groups, form centroid for each group, and update the query by a-mixing the current query with the centroid while retaining a residual from the original query, then continue with the resulting queries as a beam of size B. Unique hits are accumulated across rounds, scores are aggregated per item, and the process stops early when at least K unique items have been collected or when the round limit R is reached. The probe therefore preserves shot efficiency when certainty is high, while the recollection loop builds chain-like evidence under uncertainty, where B and F regulate breadth and a controls the balance between exploration and query stability. And theoretical analysis can be found at Appendix F.
32

---

Published as a conference paper at ICLR 2026
Algorithm RF-Mem: Entropy Entropy-guided Switching Between Familiarity and Recollection (with in- lined entropy)
Require: Retriever R with indexed memories {m4 M and embeddings 2; o(m:): query with embedding = ó(g); probe size kp: temperature X; entropy threshold т; mean-score thresholds (fhigh: ); final budget K. Recollection params: beam B, fanout F, max rounds
R. mix rate a, per-round candidate size N.
Ensure: Ranked memory set with |S| S K.
1: (s, id) t- R. (R,BEE K)
Top-K probe scores
2: 4- mean(s)
3: (Calculate entropy) Let ls < |s| and for i 1..k set
4: 2 + (s; max, sz): p; + exp(2;)/ Σド= cxp(2))
5: Hl - pi logp;
6: if mnigh or H < then
7: return FAMILIARITYTOPK/ZZZ K, R)
8: else if < Bro or H > then
9: return RECOLLECTION( K, B, F, R, a, R,)
10: end if
Algorithm 2 Familiarity Retrieval
Require: Query Is, budget K, retriever
1: For each memory m, compute 8; t- (xt, 2:)
cosine or inner product after normalization
2: S top-K items by s; (optionally filter by floor)
3: return S
Algorithm 3 Recollection Retrieval: retrieve - cluster - a-mix - iterate
Require: Tt, K, B, F, R, a, N, retriever R
Ensure: Ranked set S with |S| < K
1: (0) &- (noooo(;))
Beam + {x);; Seen + 0; Bag + 0
2: for r 0 to R do
3: Next t- 0: N < (B r) x F
D K
4:
for all ,((r) E Beam do
5: (C) + (o N)
& C6) Top-N((mi; ,x(1)
6:
Cluster (2; m, )(1) into min(B, (C(r)) groups by KMeans
7:
for b to k do
(rr) + index set of cluster 9b (r) + norm লল Evecgr 2i)
8:
9:
+ norm -orm((a + a) gb + xz)
Append (z) (r+1) G(r)) to Next
10:
11:
end for
12:
end for
13:
if Next @ then break
14:
end
Score each (2) (1+1) G(r)) by EvEOge (2b
(r+1)
z)); keep top-B new Beam
for all kept (z) (++1) (())) do
16:
17:
for all i E G. do
if E Seen then insent (i, (ab (r+1) 2z)) into Bag: add i Seen
18:
19:
end if
20:
end for end for
21:
22:
if |agl 2 K then break
23:
end if
24:
end for
25: For each id in Bag take top-K as S
26: return S
33

---

Published as a conference paper at ICLR 2026
F THEORETICAL ANALYSIS
Preliminaries. Let the user memory be = [m=]] with embeddings 2; = ((m,)) and a query
q encoded as X2 <(q), with unit normalization. Define similarity scores 8; (x,, (x) and the probe list C = Top K ([(m;, 8:) s;)) Following the paper, define tempered softmax over probe scores
P exp( M(s: max, 8j))
1, K,
(10)
2y=1 exp (Mss max; st))
with entropy H(p) = E Liml Pi log p. and mean similarity 8
K
R Σ iil
81.
The RF-Mem selection
Familiarity,
2 Onigh
Recollection,
8 S frows
Strategy(g)
(1I)
Familiarity, H(p) S T,
0jow < Onigh
Recollection, H(p) > T
In the Familiarity path, the retriever returns Top-K K by 8;. In the Recollection path, the system
iterates retrieve-cluster-mizs at round r it retrieves Top-N, NN, with N2 = (B + F) K, clusters into B groups with centroids g,",, and forms recollect queries
(++1) norm (ax + (1 =a)) + xx).
b=1, 1....., B,
(12)
for r = o,..., R 1, then returns Top-K K from the union U C(r), This matches the method description and notation in the main content.
F.1 RISK-MINIMIZIIG SELECTION
We formalize the selection in equation 11 as minimizing retrieval risk that trades correctness and cost. Let {mode 9) denote the probability of returning an insufficient set for g under mode mode E {Familiarity, Recollection), and let ((mode) denote the expected computation cost. For penalty 3 0 define
L(mode q) {mode q) + 3 ((mode).
(13)
Lemma 1 (Monotonicity of proxy signals) Assume the similarity distribution admits a monotone likelihood ratio in 8; between relevant and nonrelevant items, and that the probe softmax temper- ature X is fixed. Then E( EFaailliarity q) is nonincreasing in 8 and nondecreasing in H H()), while
C (Familliarity < C(Recollection)
Proof. Let Y; E 1] for the relevance label of item means likelihood ratio for scores so the posterior relevance ((s) Pr(Y = s) is nondecreasing in 8. For Familiarity, the miss probability given scores is Ilex (1 - (s:)). Any coordinatewise increase of (s:)iek decreases each factor, hence decreases the product, so {FFaiiliarity 4) is nonincreasing in For entropy, softmax p is Schur-concavee smaller H(p) implies larger Pmaxx and larger top margins S(1) - S()), which increase π (8(1)) and Σierx (8:),, thus the product and its expectation are nonincreasing; therefore & (Familiarity 4) is nondecreasing in Hp). For costs, C(Familiarity) O K Gaim); while Recollection performs at least one extra strieve-cluster-eee round, so C' (Recolleccton) 2 C' (Familiaarity) + R Celust (B) > C(Familiarity) for R 2 1.
a monotone
Theorem (Threshold optimality within monotone policies) Consider the class TI of policies that are monotone in (s, H) in the sense of Lemma I. Within II, two-threshold policy of the form equation 11 minimizes the pointwise risk equation 13.
Proof. By Lemma 1, (Familaarity
q) is nonincreasing in and nondecreasing in H. Define
t == [RReolllectonn I 4)) f(3, H) == [(Familiarity g) {(Failiiarity 9)+ C(amilliartty) (14)
Then the Familiaiity-reiion is the sublevel set
D ((5, H) f(3, H) < ().
(15)
34

---

Published as a conference paper at ICLR 2026
which is down-set in (3 1, H 1) ("south-ease east orthant"). Hence the risk risk-minimizing monotone
policy is 1p. Introduce axis-aligned thresholds
Bhigh == inf(s: sup f(, H) I {t),
8low == sup 8 inf f(3, H) > t}.
(16)
H
and, (fhow; (nigh).
(s( = inf fs, H) S St),
sup
(.).).
(17)
SE(O
Therefore
Familiarity <
(5 miggh) (flow < fhigh & H TT)
(18)
and Recollection otherwise, which is exactly the two-threshold gate.
F.2 AN ENTROPY CERTIFICATE FOR CORRECTNESS
Let Pmax = max; Pi. For fixed Pmax, the maximal entropy occurs when the residual mass is uniform,
that is
(p) < he (Pmax) + (1 Pmax) log(K 1),
(19)
where (2(2) = x lolo x (1 - 2) log(1 - 2)) is the binary entropy. Define the inverse certificate
(x(T) max E [1/K, 1] h2(x) + (1 - log(K K - 1) TT).
(20)
Lemma (Entropy certificate) If H (p) 2 т, then Pmax 2 (x(T).
Proof. Rearrange equation 19 and apply the definition of (K(T).
Proposition 1 (Bound on familiarity error under low entropy) Suppose that returning the Top-K set suffices whenever the true relevant item has pi 2 for some E (0,1). Under H()) I 7 with (x(7) p. the familiarity mode achieves negligible miss probability with respect to this sufficient condition.
Proposition provides a certificate: a small entropy ensures a large Pmax. which guarantees that the best evidence is included by Top-K K under mild sufficiency condition, hence Familiarity is safe in the certified region.
F.3 SUB-GAUSSIAN MEAN SIMILARITY AND GATING RELIABILITY
Assume probe scores {s K independent sub-Gaussian with proxy mean y and variance proxy o². Let fi = 3 and fix thresholds 0jow < Onigh-
Proposition 2 (Gating error bound via concentration) For any O,
K8²
K8
Pr (ī - 2 0) < exp 2x²
Pr (ā 2 5) S exp
(21)
20
If the true regime satisfies μl 2 Qhigh + 6 (familiar) or μ S 6 8 (unfamiliar), the probability of mis selection due to mean estimation is bounded by exp( 882 (00//(222))
Proposition 2 shows that increasing K tightens the reliability of the mean-based selection of the familiarity and the recollection paths.
F.4 COMPLEXITY -OOEEMIITTCCOOAAGEE TRADE-OFF
Let the cost of similarity evaluation be Gám and of B-way k-means update be Cetus (B) batch of size N,.
Proposition 3 (Complexity bounds) Familiarity has time Trom O(K Czim). Recollection with parameters (B, F, R) has time
R
Te 0
Σ 0=0
NNr Crm + (elus(())) = O (BF) Caim + R Cetuon (B))
(22)
35

---

Published as a conference paper at ICLR 2026
since N, (B & S Therefore TRee O +RF)+RF) Caim R Cetust (B). which RRF is polynomial in (B, F, Rt and strictly lower than full-context processing (M) when BF RF & M.
This formalizes the method's s bounded overhead relative to full-contexi, consistent with empirical latency advantages reported in the paper.
Discussion. Lemmas and 2 justify using (s, H) as reliable control signals. Theorems show that the selection is optimal within broad monotone class. Propositions 2 and bound mis-selection and computation. Together these results explain the empirical accuracy-latency improvements of RF-Mem across corpora and tasks.
G LLM USAGE DISCLOSURE
In accordance with ICLR 2026 policy, we disclose our use of large language models (LLMs) in preparing this manuscript. We employed GPT-5 (OpenAl) solely to aid in polishing the writing, specifically for improving clarity, grammar, and sentence structure across sections. All technical content, algorithmic contributions, experimental results, and scientific conclusions remain entirely the authors' own work, without any L.M involvement.
H LIMITATION AND FUTURE WORK
Limitations. While RF-Mem demonstrates consistent advantages across corpora and tasks, several aspects remain simplified. First, current evaluation is confined dialogue-style personalized memory and does not yet explore other modalities (e.g., cross-modal histories). Second, we adopt list entropy lightweight uncertainty proxy; although effective, it may not fully capture finer- grained task difficulty or user intent. Third, our uncertainty signal based on similarity scores and entropy, which may fully capture semantic ambiguity. Finally, retrieval operates static embedding index, without modeling temporal updates memory or potential conflicts across long-term sessions. Moreover, because RF-Mem improves the surfacing of long-term user history, downstream deployments should include safeguards to prevent unintended resurfacing of sensitive information, direction identify important future work for quantifying and mitigating such risks.
Future Work. Several directions arise naturally from the above limitations. First, extending RF- Mem to multi-modal or cross-domain settings (e.g., multimodal dialogue) would allow testing its generality beyond personalized text-based memory. Second, uncertainty estimation could en- hanced by integrating richer signals- such as calibration measures user feedback-tn to comple- ment list entropy and better capture task difficulty. In addition, incorporating semantic or contextual uncertainty cues beyond similarity distributions may further improve the reliability of the strategy selection mechanism. Third, incorporating temporal dynamics into the memory index (c.g., recency weighting, conflict resolution, session-aware updates) may further improve long-term personal- ization. Finally, exploring tighter integration between retrieval and generation for example, jointly optimizing the entropy threshold with the LLM's decoding behavior-coul yield more unified and adaptive personalized reasoning framework.
36


Reviews:
[{"review_id": "soB5nJ9kyv", "reviewer": "Reviewer_eDaP", "summary": "This paper introduces RF-Mem, a dual-process memory retrieval framework inspired by cognitive science, combining a fast Familiarity path and a deliberate Recollection path for personalized LLM memory retrieval. By using a familiarity-uncertainty gating mechanism and a retrieve–cluster–mix procedure, RF-Mem adaptively retrieves evidence under varied user-memory relevance conditions.", "questions": "1. Could the authors consider ablation comparing linear vs. nonlinear mixing (e.g., gating networks, learned interpolation), or reference theoretical grounding such as manifold-preserving mixing used in retrieval augmentation?\n\n2. Could the authors consider learned gating or meta-calibrated thresholds conditioned on query distribution or similarity statistics?\n\n3. Include comparisons to LLM-based query reformulation modules such as LD-Agent or MemoCue, since they also address deeper reasoning retrieval.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. The paper effectively motivates its design using the Recollection–Familiarity Dual-Process Theory (Lines 61–83) and successfully maps the cognitive analogy into a computational retriever.\n\n2. The familiarity-uncertainty gating (Lines 162–201) provides principled decision logic for choosing between retrieval modes, reducing unnecessary expansion while preserving robustness.\n\n3. The adaptive study (Lines 432–454) shows that RF-Mem complements rather than replaces indexing approaches like MemoryBank, strengthening its practicality.", "weaknesses": "1. The α-mix formula (Lines 246–257) is introduced but lacks theoretical clarity on why linear mixing is optimal for query expansion.\n\n2. $\\tau$ (Lines 174–201) is fixed globally but may vary significantly across users, domains, or embedding models.\n\n3. Although the paper compares fairly against retrieval-only baselines (Lines 289–303), retrieval today heavily relies on query rewriting, which is absent from the comparisons.\n\n4. The ethical considerations are brief and do not address challenges like preference over-amplification, exposure to sensitive data, or user-profiling bias.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "fab5PwWeJf", "reviewer": "Reviewer_pdxs", "summary": "The paper “Evoking User Memory: Personalizing LLM via Recollection–Familiarity Adaptive Retrieval” proposes RF-Mem, a dual-process memory retrieval framework for personalized large language models. Inspired by cognitive science, the model integrates two complementary retrieval modes—familiarity, for fast and confident recognition, and recollection, for slower and more deliberate evidence reconstruction. RF-Mem measures familiarity using the mean similarity score and entropy, dynamically switching between the two paths based on uncertainty. When familiarity is high, it performs efficient one-shot retrieval; when uncertainty increases, it activates a clustering-based recollection process that iteratively refines queries through an α-mixing strategy in embedding space. This design allows the system to balance efficiency with depth, maintaining the low latency of one-shot retrieval for straightforward queries while engaging in more thorough evidence gathering for ambiguous ones. Experiments on several benchmarks demonstrate that RF-Mem outperforms traditional one-shot and full-context methods in both accuracy and efficiency, achieving scalable, adaptive, and human-like personalization for LLMs.", "questions": "1. Could adaptive or learned gating mechanisms outperform the hand-tuned parameters used here?\n2.The recollection process relies on clustering in embedding space. Have the authors tested alternative approaches, such as graph-based retrieval or semantic expansion via LLM-generated paraphrases, to validate the robustness of the proposed method?\n3.As the field increasingly uses LLMs for complex retrieval tasks, can you demonstrate RF-Mem's advantage over a baseline that uses a powerful LLM  for a single step of query expansion or decomposition before a dense retrieval step?\n4.The entire framework, including the uncertainty estimation, relies heavily on the quality of the underlying embedding model.  Did you observe significant performance variance across the three retrievers (MiniLM, MPNet, BGE) in terms of the routing accuracy?  Is there a risk that a poorly calibrated embedding model could lead to a cascading failure in the gating mechanism?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "This paper presents a highly original and conceptually grounded contribution by introducing RF-Mem, a dual-process memory retrieval framework inspired by the Recollection–Familiarity theory in cognitive science. The idea of aligning LLM personalization with human memory processes is both novel and intellectually appealing, extending beyond conventional retrieval-augmented generation paradigms. Methodologically, the paper is well executed: the formulation of familiarity uncertainty through mean similarity and entropy is simple yet effective, and the recollection mechanism—implemented via clustering and α-mixing—offers a practical and computationally efficient way to approximate deliberate evidence reconstruction in embedding space. The experiments are comprehensive, covering multiple personalized benchmarks and retriever architectures, and the empirical results convincingly support the method’s claimed benefits in both accuracy and efficiency. The paper is also clearly written and well-structured, maintaining strong conceptual coherence between cognitive theory and technical implementation.", "weaknesses": "Despite its conceptual elegance, the paper’s empirical validation should be further strengthened. \nFirst, the experimental comparisons are restricted to retrieval-only baselines, omitting stronger LLM-based retrieval or reasoning systems such as query rewriting, iterative retrieval (e.g., Search-Mem), or graph-based methods. Including such more advanced baselines would better position RF-Mem within current state-of-the-art approaches and clarify whether its advantages hold beyond standard dense retrieval. \nSecond, the paper lacks an ablation study of the gating mechanism. It is unclear how much the mean similarity score and entropy respectively contribute to routing decisions, and how often each retrieval path is activated. \nThird, the definition of uncertainty is limited to similarity score distributions, which may fail to capture semantic ambiguity or misleading high-confidence matches; this limitation should be acknowledged and discussed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "R2VlknXmuO", "reviewer": "Reviewer_52gJ", "summary": "The paper proposes RF-Mem, a dual path that gates between fast “Familiarity” retrieval and slower “Recollection” based on mean similarity and an entropy signal. \nWhen the gate flags uncertainty, it runs an in-embedding recollection loop that retrieves, clusters with KMeans, α-mixes cluster centroids with the query, and iterates under tight beam/fanout/round caps. \nExperiments on personalized generation and retrieval tasks show a better accuracy–latency balance than one-shot dense retrieval or full-context prompting.", "questions": "1. Would a density-based or spectral clustering help when memory clusters are non-spherical or imbalanced. \n\n\n\n2. Does the gate remain calibrated when swapping embedding models or scaling memory to much larger corpora. \n\n\n\n3. Can you add early-stop checks that detect “recollection loops” and cut branches before they waste budget.", "limitations": "", "rating": 4, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The cognitive grounding is crisp and the gate is concrete (mean score + entropy with a sharpness λ), so the controller is easy to reason about. \n\n2. Recollection is lightweight and model-agnostic since it lives entirely in vector space using simple clustering and linear mixing. \n\n\n3. Compute stays bounded because the loop exposes explicit knobs for beam width, fanout, and depth instead of open-ended expansion.", "weaknesses": "1. The many thresholds and weights feel hand-tuned, with little guidance for auto-tuning across users and domains. \n\n2. KMeans centroids can be unstable on anisotropic or overlapping memory clusters, so α-mixing may drift the query off-intention. \n\n3. Entropy over the top-K score simplex depends on λ and retriever scale, which could make the gate brittle when swapping encoders or normalizations. \n\n4. The controller assumes unit-normalized cosine and a sub-Gaussian mean-similarity story, which may not hold in heavy-tailed similarity landscapes. \n\n5. The latency write-up emphasizes averages, while real deployments care about tail spikes when recollection fans out.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "GJ8hNTiOmL", "reviewer": "Reviewer_jKrT", "summary": "This paper introduces RF-Mem (Recollection-Familiarity Memory Retrieval), a new framework for personalizing Large Language Models (LLMs) by improving how they retrieve user-specific memory. The authors argue that current methods for memory retrieval are inefficient, either by overloading the LLM with the user's entire history (which is unscalable) or by using a simple, one-shot similarity search that only captures surface-level matches. Inspired by cognitive science, RF-Mem is based on the Recollection-Familiarity Dual-Process Theory of human memory. Experiments on three benchmarks show that RF-Mem consistently outperforms both one-shot retrieval and full-context methods, achieving higher accuracy and recall while maintaining low latency.", "questions": "1. To improve robustness and reduce tuning overhead, have the authors considered a learned switching policy? A small, lightweight classifier trained on the probe retrieval's features (mean score, entropy, etc.) might provide a more generalizable switch than fixed thresholds.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The \"Recollection\" path is a novel iterative mechanism that uses KMeans clustering and $\\alpha$-mixing to reconstruct evidence chains purely in the embedding space, without requiring a costly LLM-in-the-loop for query reformulation.\n2. The paper includes a full suite of supporting analyses in the appendix. This includes detailed sensitivity studies for all key hyperparameters ($\\alpha$, $\\tau$, B, F, K), a compelling qualitative case study that visually demonstrates the method's superiority,  and a formal theoretical analysis that mathematically grounds the risk-minimizing properties of the switching mechanism.", "weaknesses": "1. The primary experimental baseline (\"Dense Retrieval\") is a non-iterative, one-shot method. This is a weak baseline, as many advanced RAG systems already use multi-step or iterative retrieval.\n2. This adaptive switch mechanism is heuristic and lacks robustness. The optimal thresholds are highly dependent on the specific embedding model used and the score distribution of the dataset.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "4IOaDWAcEr", "reviewer": "Reviewer_QRxk", "summary": "The paper proposes RF-Mem, a retrieval framework for LLM personalization. The core idea is to adaptively choose between two memory-retrieval modes, inspired by dual-process human memory:\nFamiliarity: fast dense top-K retrieval when the query matches past user memory with high confidence.\nRecollection: iterative, multi-hop expansion when confidence is low — retrieve candidates, cluster them, mix cluster centroids with the original query embedding, and retrieve again to surface contextual / episodic details.\nA simple gate based on mean similarity and similarity-score entropy decides which path to use. The goal is to surface only the most relevant personal memories for context, instead of injecting the full user history.\nThe method is evaluated on multi-session personalization benchmarks with up to ~1M tokens of per-user memory. RF-Mem outperforms (i) no memory, (ii) giving the whole history, (iii) plain dense retrieval, and (iv) recollection-only, while keeping inference context small and latency close to dense retrieval. The authors also show RF-Mem working on top of summarized memory stores, not only raw logs.", "questions": "1. How are the gating thresholds selected? Could the gate be learned?\n2. Can you show qualitative failure cases where the gate picked the wrong mode?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "### 1. Strong motivations and grounding in cognitive science. \n\nThe familiarity vs recollection split is explicitly motivated by dual-process models of human memory. The authors lay out a concrete computational mapping (confidence gating → familiarity, uncertainty → recollection). This gives the method conceptual clarity and makes the design feel principled.\n\n### 2. Clear formalism / presentation.\n\nThe framework is described cleanly, with the retrieval controller, the gating rule, and the recollection procedure all specified in enough detail to reimplement. \n\n### 3. Scales to large memories. \nRF-Mem remains effective under very large user histories (hundreds of thousands to a million tokens) where “put everything in context” is impossible and vanilla dense retrieval starts missing important long-tail details.\n\n### 4. Method is practical with low overhead. \nThe approach adds only modest latency over standard dense retrieval and explicitly tracks context budget. No LLM fine-tuning is required. It is interesting to see how much latency the dual process formulation saves over pure recollection, without losing performance.\n\n### 5. Modularity. \nRF-Mem can sit on top of summarized memories rather than raw transcripts, suggesting it can plug into existing assistant memory stacks.", "weaknesses": "### 1. Lack of real-user evaluations.\n\nAll experiments use synthetic or benchmarked personalization data. There’s no human evaluation of helpfulness or preference alignment, and no analysis of noisy, contradictory, or sensitive histories.\n\n### 2. Heuristic gating mechanism.\nThe decision to enter recollection mode is based on manually chosen similarity / entropy thresholds. There’s no learned controller and no robustness study across domains. Furthermore, the authors do not detail why certain thresholds were selected, or how one should optimise these.\n\n### 3. Baselines.\nComparisons are mostly against: full context, dense top-K, and recollection-only. Stronger long-term memory agents (e.g. reflective / hierarchical memory systems) aren’t fully represented.\n\n### 4. Safety/privacy.\nSince RF-Mem is explicitly better at surfacing detailed personal history, it can also resurface sensitive information the user didn’t mean to bring into the current turn. The authors acknowledge this risk but do not quantify or mitigate it.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)