AMemGym: Interactive Memory Benchmarking for Assistants in Long-Horizon Conversations
Authors: Cheng Jiayang
Dongyu Ru
Lin Qiu
Yiyang Li
Xuezhi Cao
Yangqiu Song
Xunliang Cai
Venue: ICLR 2026 Poster (2026)
Keywords: memory
agent
long-context
Primary area: datasets and benchmarks

Abstract: Long-horizon interactions between users and LLM-based assistants necessitate effective memory management, yet current approaches face challenges in training and evaluation of memory. Existing memory benchmarks rely on static, off-policy data as context, limiting evaluation reliability and scalability. To address these gaps, we introduce AMemGym, an interactive environment enabling on-policy evaluation and optimization for memory-driven personalization.
AMemGym employs structured data sampling to predefine user profiles, state-dependent questions, and state evolution trajectories, enabling cost-effective generation of high-quality, evaluation-aligned interactions. LLM-simulated users expose latent states through role-play while maintaining structured state consistency.
Comprehensive metrics based on structured data guide both assessment and optimization of assistants.
Extensive experiments reveal performance gaps in existing memory systems (e.g., RAG, long-context LLMs, and agentic memory) and corresponding reasons. AMemGym not only enables effective selection among competing approaches but also can potentially drive the self-evolution of memory management strategies.
By bridging structured state evolution with free-form interactions, our framework provides a scalable, diagnostically rich environment for advancing memory capabilities in conversational agents.

Published as a conference paper ICLR 2026
AMEMGYM: INTERACTIVE MEMORY BENCHMARKING FOR ASSISTANTS IN LONG-HORIZON CONVERSATIONS
Cheng Jiayang 12- Dongyu Ru²; Lin Qiu Yiyang Li²
Xuezhi Cao², Yangqiu Song', Xunliang Cai
'hee Hong Kong University of Science and Technology Meituan j johenga ja cse. ust.hk, {rudongyu, qiulin07] @me
ABSTRACT
Long-horizon interactions between users and LLM-based assistants necessitate effective memory management, yet current approaches face challenges in training and evaluation of memory. Existing memory benchmarks rely on static, off-policy data as context, limiting evaluation reliability and scalability. To address these gaps, introduce A AMEMGYM, an interactive environment enabling on-policy evaluation and optimization for memory- driven personalization. AMEMGYM employs structured data sampling predefine user profiles, state deeeedeeeeeentt questions, and state evolution trajectories, enabling cost-effective generation of high-quality. evaluation-aligee aligned interactions. Mim-llull users expose latent states through role-play while maintaining structured state consistency. Compre- hensive metrics based on structured data guide both assessment and optimization of assis- tants. Extensive experiments reveal performance gaps in existing memory systems (e.g., RAG, long-conneen LLMs, and agentic memory) and corresponding reasons. AMEMGYM not only enables effective selection among competing approaches but also can potentially drive the self-evolution of memory management strategies. By bridging structured state evolution with free-form interactions, our framework provides a scalable, diagnostically rich environment for advancing memory capabilities in conversational agents
INTRODUCTION
A crucial objective in the development of assistants based on Large Language Models (LLMs) is to achieve long-horizon conversational capabilitics-thar is, the ability to effectively organize, manage, and utilize memory across extended sequences of dialogue turns. Robust memory management forms the foundation for fulfilling complex user requests, tailoring responses to users' latest implicit states, and personalizing sug- gestions and recommendations based interaction history. However, progress in advancing conversational memory systems for assistants is hampered by a critical bottleneck that affects both scalable training and reliable evaluation: the data used in existing benchmarks.
Current benchmarks typically rely on static, pollcy data for evaluation (Xu et al. 2022 Wu et al. 2024
Hu 2025), rather than on-policy interactions. Figure [ illustrates the distinction between the two approaches, Off-ppolccy evaluation, in which an assistant is tested on conversational data that it did not produce during actual interactions, presents several fundamental limitations. First, it fails capture the assistant's true interactive nature, as the evaluation data does not reflect the consequences of the assistant's
"Equal contribution. Work done during internship Meituan. Project Lead.
https:/ / ///gi//iii// -ew---eaalllleeeee hww.llloffiialll.cco /o/
wtt.://g/../mmm..m.....m amemgym/

---

Published as conference paper ICLR 2026
Table I: A comparison features across agent memory benchmarks.
Optim.
Fredlbcck
Automation
Context Length
MSC Xu al.
Statie
Manual
1.2K
ReaTTakk Lee ol
2025
Static
Manual
17K
Prohing (F), accaracy)
Fmotiona Imtelligence, Persona Sirulation, Memory
DialSim
Kim et ml
3024
Static
Mannal
QA Accunacy
LeCoMo
2024
Static
Semti Automaaed
Accuracy, Summarization, Generation Accuncy
PorlTQQ
Statie
Scmi--Antooated
LongMeemEvall
Automated
1.5M) Configurable (32K.
Cunfigurable (115K.
Recall, QA Accuracy
PersonaMeus Daagg ct al.
2025
Statie
Fully Autocnated
128K, 1M;
QA Accumecy
AMEMGrM (This Week)
Interacrive
Fully Autcenatod
Configurable
Overall (Ancuracy, Normalizzd Memory Score) and
Disgnosis (Write, Read, Utilization..
conversational choices critiall issue for evaluation realism. Second, because the evaluation is biased, memory optimization could be misdirected. Finally, the manual curation of these evaluation scenarios (Lee)
al. 2025 Kim al. 2024) is costly and does not scale for comprehensive testing across diverse, long-
horizon conversational contexts.
To enable on-policy evaluation and provide reliable feed- back for optimization, it is essential to employ a simu- lated user that can strategically reveal information and pose relevant questions, a technique that has demon- strated promise in other domains such tool use (Wang ct al. 2023 Lu et al., 2025). However, deploying sim- ulated users in open-ended conversational environments presents unique challenges. These include determining what information to disclose dynamically while main- taining natural and coherent dialogue, well en- suring the generation of diverse, high-quality data that remains sufficiently controlled for reliable evaluation.
Off-Policy
Proflles 8 Initial States
R
OURS: On-Policy
Dialogue
Sirulated User
Generator
Static History
deat Assistant
OCS Assistant B
History
Sampling
Hissoryy
Sampling
History A
History B
aPT Asoiatinn
Asskatont
Result
Result
Result
Result
To address these gaps, we introduce A AMEMGYM, an in- teractive environment designed for the on-policy evalu- ation and optimization of memory in long-horizon con- versations. AMEMGYM grounds free-form interactions in structured data generated through a schema- based approach. The framework predefines user profiles, state-dependent questions, and state evolution trajectories to enable the cost-effective generation of high- quality interactions aligned with evaluation targets. LLM-simulated users then expose these latent states through natural role-play, ensuring consistency with the structured state evolution. Periodic evaluation dur- ing interactions, using both overall and diagnostic metrics, guides assessment and optimization of memory capabilities. Our contributions are threefold:
Evaluation
Evoluction
Figure 1: On-policy v.s. off-policy evaluation for assistants' memory.
1. We introduce AMEMGYM, a novel framework for the on-policy evaluation of conversational mem- ory. By grounding free-form interactions in structured state evolution, AMEMGYM creates a seal- able and diagnostically rich environment to reliably assess and advance the memory capabilities of conversational agents.
2. We empirically demonstrate the reuse bias and potential drawbacks of off-policy evaluation, and conduct the first extensive on-policy evaluation of popular memory systems. Our results highlight the reliability of AMEMGYM for evaluating memory in the context of personalization.
3. We provide proof of concept for agent self-evolution, showing that an agent can use environmental feedback within AMEMGYM to autonomously refine its memory management policy.
2

---

Published as conference paper ICLR 2026
Phase 1: Offline Structured Data Generation
Phose 2:
lssrr Profile Sampling
Janos, 25, reacle, "hoating bridge suaen "Jonox is celebrated in her
Online On-Policy Interaction
communtty for her restiience, noreth, and cbility to bring people together,
State Schems:
Question State
bridge.getherrkgg gethering gue.t..geeeee....... range:
(esposting ststee)
bridgt. petpg/ppttherrnngggrro
Sampling
(seter stobes)
Sinulated User
Assistont
On Fo-F Responce
PPacton
"Can suggest
bridge variations Acabreckera for group
pridge_.dtthe gethering quest oge range-- "eautly_.R_pluaa"
12 Ersends, wost being S0
Grounded Evaluation Data
Salf- Evaluctos Feodbeck (bsc.3)
States Evolution
bri.ggngeethee www.teengggggrogie...
"modfunu0.12"
DFaciot
variations for fun scabreskers who've started joining lctely?" both longeime friends and the wostiyy.RR.pee"s www.:apexxfeh.porr...oo
'Could
(froEiles, States, QM4s)
aridgg-gttherrng-e hrt aridgg.gstteerngg www.ggtteeingggroo.....
swolving)
Periodic Evaluation
Query 2:
"Shet are semc engasing setivities con organize for aorchly bridge gatherings to keep then fresh and enjoyahle?"
Eeercil Retroca: Normelized foney Score
States requdred here: brädge gethering quest age rasçe, Brädge gathering group size
Perdonalizeed Resp. 6en.
Ansmer using focutly 50, plus" and "wecion 6.12" "far group of experienced bridge anthusiasts. try reteting pertnerantpe esch rouns encuurage mingling ...'
Eogeestic
(other of
Reong
Beee Sure Undlication
Figure 2: An overview of the AMemGym framework.
2 RELATED WORK
Benchmarks for agent memory evaluation. The evaluation of agent memory has progressed from long- context, single-turn tasks like the reedle-in-a-haysteee (NIAH) test and NoLiMa (Modarressi et al.) 2025) to more realistic multi-ture turn conversational datasets such as Multi-Session Chat (MSC) (Xu et al.) 2022).
RealTalk (Lee et al., 2025). and DialSim (Kim et af., 2024). While these introduced more authentic di- alogue patterns, their reliance on manual curation limited their scale and diversity. To address this, au- tomated data generation frameworks like LoCoMo (Maharana et al. 2024), PerLTQA (Du et al. 2024). LongMemEval (Wu et al. 2024). PersonaMen (Jiang et al. 2025), and Memory AgentBench (Hu et al. 2025) were developed. However, a critical limitation unites nearly all existing benchmarks: they rely on static, off- off-policy data (Table [T. This approach fails capture agent's true interactive performance, the evaluation data does not reflect the consequences of the agent's own actions, misleading optimization.
Interactive agent evaluation by user simulation. An alternative line of research has focused on interac- tive, on-policy evaluation environments that employ user simulators. This approach has proven effective in domains like tool-use, where simulators provide robust on-policy evaluation (Wang et al., 2023 Lu et al., 2025). Similarly, efforts like CollabI.LM (Wu et al., 2025) have successfully employed user simulation to train models for improved long-term collaboration. Applying this interactive paradigm to memory evalua- tion, however, introduces unique challenges: a simulator must strategically reveal information over a long- horizon conversation while maintaining natural flow and generating interactions that are both diverse and controlled enough for reliable assessment. A AMEMGYM directly addresses these challenges by introducing a schema-base based approach that grounds free-form, LLM--drvee role-play in structured state evolution plan, which enables the controlled and scalable generation of on- on-policy, memory- focuued evaluation scenarios.
3 AMEMGYM
AMEMGYM provides an interactive environment for benchmarking and optimizing personal assistant mem- ory, with the scenario and the task described below.
3

---

Published as a conference paper ICLR 2026
LLM-based Assistants. An LLM-based assistant takes as input the observation (user input) o and provides output responses (a sequence of tokens) based on its policy и and its internal memory at that time my (e.g., tokens in the context window, text snippets written to an external index, or its own parameters); oz, me E as m4+1- The internal memory is updated through interactions.
Personalization with Memory. To effectively serve users with dynamically evolving personal states, assistants described above must continuously track user states through interaction histories To (ou, dg. 01, 01,.... ...... 0g] and deliver responses optimized for their latest latent states captured by m.- In reality, the length of 7, often goes well beyond the optimal context length of most LLMs. Therefore, an effective information compression memory mechanism is crucial for assistants maintain accurate and up-to-datt user modeling. In this context, states refer to comprehensive personal information crucial for enabling the intelligent assistant sustain meaningful conversations and address user-relevant concerns. This includes user preferences, habits, plans, and environmental conditions, among other factors.
An overview of our framework is presented in Figure 2 We begin by describing the structured data sam- pling process that forms the foundation of our evaluation framework ($ (.1). then detail how on-policy inter- actions are generated with grounded structured data ($ (.2). We present comprehensive evaluation metrics that assess both overall memory performance and provide diagnosis for different memory operations ($ 3.3
Finally, we provide meta-evaluation results to show reliability of the fully-automated process ($ 3.4
3.1 STRUCTURED DATA GENERATION FOR ON-POLICY INTERACTION
Evaluating memory is challenging due to the high cost of verifying correctness in long, noisy conversations. To address this, we use a reverse-enginge eeveineenggnn strategy: starting from target evaluation questions, we trace back identify key user state variables for personalization, their possible temporal changes for simulated user, and the personalized responses for each experienced state combination. This serves as a structured foundation that enables grounded interactions and automatic evaluation. Detailed prompts for each sampling step are provided in Appendix C.3
User Profile Sampling. We begin by sampling user profiles that serve as the contextual backbone for subsequent steps. For broad domain coverage, we use 100K personas from Semotron-Persomee (Meyer & Corneil 2025) as the pool. Custom sampling strategies can be easily applied for specific applications better accommodate target real-world distributions.
Question Sampling, The process starts with a user profile, p, used to sample a set of evaluation questions, 2p. For each question q: E 2p. an LLM extracts the information types required for a personalized answer. These types S are occasionally redundant across questions (e.g.. "experience level" and "years of work"). Therefore, they are merged and refined by an LLM into canonical global state schema, Σ = U.
The schema defines a set of M unique state variables (#z) and their possible discrete values set (V,): Σ {(5), V)) This comprehensive schema serves as the complete set of trackable user states for the entire simulation.
User States Evolution. We then simulate a realistic progression of the user's states over Np periods. The state at the end of each period f is captured by a state vector, de, full assignment where each variable 8j is given a value ", from its corresponding set of possibilities Vy: a, = {(8,, vy) (sj, Vú) E Σ}. Each state transition is prompted by a narrative life event, providing context for the change (0,-1 S az)-
The resulting state evolution trajectory, To = (do, (N,))..,NN,) provides the ground-truth for the user's state throughout the simulation.
To create the inputs for on-policy interaction in each session, we generate a series of natural language utterances that the simulated user will say initially. Within each period (, an utterance ux is designed to
We use g OOenAA 2025al for structured data generation and simulation.
4

---

Published as a conference paper ICLR 2026
implicitly expose small related subset of the user's current state, eexposed C. 0z. This is generated by a function Gun conditioned on the states to be revealed and the user's profile: Vt, Gsp(despemdd p). These pre-generated, state- batt--baring utterances form core part of the structured data blueprint. They are used to initiate conversational turns during the on-policy interaction phase ($ (.2).
Personalized Response Generation. Finally, to create the evaluation ground truth, we generate per- sonalized answers for each predefined question qi- Each question requires subset of state variables, (1)) C
and a specific assignment of values to these variables is a state variant, v: {(3). Uÿ) 8) (g:())), 2/ E }. For each pair V), generate distinct answer ri,u. To ensure a high-quality, one-to-one mapping, a reflection step verifies that the answer is unambiguous: it is accepted only if an LLM classifier C' can recover the variant from the question-ame answer pair, i.e., ((giTTip)
3.2 ON-POLICY INTERACTION
Different from prior static evaluation on long-context LLMs or memory agents (Xu et al. 2022 Maharana et al. 2024 Wu et al., 2024 Jiang et al., (025), we sample on-policy interactions as in Figure I Given the
offline structured data sampled in Section 3.1 our user simulator interacts with the target assistant to expose this information through natural conversation. This step outputs a (possibly long-context) dialogue history we demonstrate the necessity of on-policy evaluation.
Later in Section 4.2
State Exposure. To enable reliable evaluation, key user states those that change between periods- must be clearly reflected in the conversation history. This is achieved by using the grounded utterances (ur,s) that were pre-generated as part the structured data. For benchmarking consistency, we use these fixed initial state- beateen utterances to begin each conversational session, ensuring that the necessary information is introduced into the dialogue.
Role-Play with LLMs. Conversation generation is performed by user LLM, which role-plays based on the user profile and state evolution. the user profile, (2) current states ot, and (3) the latest conversation context. The user LLM produces responses conditioned on dialogue history and underlying states, ensuring coherent alignment between free- form conversation and structured state evolution.
configured with: (1) a system prompt template incorporating
3.3
EVALUATION METRICS
Given the grounded interactive environment, assistants are prompted to answer all evaluation questions after each interaction period. These responses provide feedback for agent builders assess and optimize assistants ($ D. and enable assistants to self-improve ($5). based on the evaluation metrics described below.
end performance We use the average question answering accuracy as the metric for evaluating end-to- benchmark, denoted as the overall score. This metric captures the model's ability to integrate both personalization (tailoring responses based on specific user states) and memory (retaining user states from previous conversations) achieve high performance. To provide clearer view memory, we introduce normalized memory scores. isolates the memory component from raw task performance by normalizing the overall accuracy between a random baseline (lower bound) and an upper bound (UB) with perfect memory access. For each evaluation period, the score is computed as: Smemary Swaill Smnton Sun Soowter
Overall Evaluation.
The upper bound SUR is determined by providing the assistant with ground-truth user states at evaluation time, thereby entirely bypassing the memory retrieval process. It measures the assistant's 's reasoning and application capabilities when required information is perfectly available.
We decompose failures in overall question answering into three distinct operational stages of memory processing: write, read, and utilization. Corresponding failure rates enable systematic error attribution. For each user state, we query its value at every evaluation period. If the assistant demon- strates knowledge of all relevant state values but still fails to answer overall evaluation question correctly,
Diagnostic Evaluation.
5

---

Published as a conference paper ICLR 2026
we classify this as a utilization failure. Otherwise, we examine the state query results at the nearest write position to distinguish between write and read failures (Figure 3
3.4 META-EVALUATION
To validate the data quality of AMEMGYM, con- ducted a three-stage meta-evaluation with human annotators. First, assessed state exposure, con- firming that user states are clearly introduced into the conversation. On sample of 200 queries, an- notators found that the state information was suc- cessfully conveyed with an average quality score of 99. % and an inter-annotator agreement (Gwet's AC1 (Gwet 200T)) of 96.8%. Second, we evaluated conversational state integrity to ensure that the sim- ulated user's dialogue does not contradict established ground-trute states over time. Across 748 annotated items from 40 conversations, the dialogue maintained a 99.2% consistency score, with Gwet's ACI of 98.2%. Finally, we evaluated ground-truth judgment reliability. We validated the reliability of the ground- truth judgments
Failure
Query for required stales at time
Utilization Failure
Digast Leee Eacss but failed to
Uheannset
orrettty
Read Failure
Identity 'arite' for the forgotten state
Diat wo witter bu latar Je0r)
(ker the state cornectly recortedt
Write Foilure
Figure 3: An overview of diagnostic metrics: write, read, and utilization.
sample of 100 questions. We measured the agreement between two independent human annotators and the LLM -en--geeeetedd answers. The inter-annotator agreement between the humans was 0.92, while the agreement between the LLM's answers and each human 0.96 and 0.94, respectively. These results confirm that A.EEMGMM generates high-fidelity data, providing a reliable foundation for memory evaluation. Details of this evaluation are in Appendix
4
MEMORY EVALUATION WITH AMEMGYM
4.1 EVALUATION SETUP
Data Configuration. AMEMGYM offers config- urable parameters to control evaluation difficulty. We focus on two configurations to showcase flex- ibility and ensure reproducibility, differing in three key dimensions: the number of evolution periods Np (quantity of key information), required states per question N₂ (reasoning depth), and interaction turns per state exposure N; (noise level). We define two variants using the tuple (Np, Ne, No): base (10, 2, 4) which requires 128K+ context window and ex- tra (20, 3, 10) which requires 512K+ context win- dow. Both variants use 20 randomly sampled user profiles with 10 evaluation questions each, totaling 200 questions tested at N2 positions with poten- tially different answers due to evolving user states.
In- Co-toontxxt
External
Direct Write
Agentic Write
Direct Brite
Agentic Write
Vanillo
Agentic Wemory
Standard
Agentic Nesory
Long Context
(In-Context)
RA6
(External)
Conv. History
History
Conv. History
Conv. Ilistory
LLN
Long: Menory
Context Henory Buffer
Long-Term
LLN
Memory
L.M
Context
Context Retrieved
Context Retrieved
Output
Output
Output
Output
Figure 4: Memory implementations.
We report results primarily on the base configuration, as it presents a sufficiently rigorous challenge to dis- tinguish model capabilities. See Appendix F.I for extra results and other configurable parameters. Detailed benchmark statistics are presented in Appendix C.I
Memory Implementation. Existing memory systems for LLM--based assistants, despite implementation variations, share a common design philosophy of constructing memory hierarchies to exchange between short-term and long-term memory (Packer et al. 2023 Chhikara et al. 2025) Xu et al. 2025). We abstract
6

---

Published as a conference paper ICLR 2026
this connection by focusing on two key aspects: storage location (in-context vs. external) and writing strategy (agentic vs. direct).
As shown in Figure we focus on the four memory implementations: Native LLMs (LLM) rely solely on context windows, maintaining long-term memory in-context as raw content. Standard RAG (RAG) uses Detrieval-Augmenne Generation with external indexing for long-term storage. Unlike standard RAG which indexes raw text, Agentic Write (External) (AWE) triggers an LLM--ased extraction to decide what to write external long-term memory and retrieves using embedding models as in RAG. Agentic Write (In-Contex) (AWI) operates similarly but stores long-term memory in-context without independent retrieval. For AWE, we additionally study critical parameters: memory update frequency (freq), minimum short-term messages in-context (ns), and retrieved memories count (topk) [ We denote these configurations AWE-(freq, ns, topk) | All memory implementations use gpt-4.1 (OpenAl 2025a for response generation and mem- ory operations and aet-emmbedddng---smmaae (OpenAT 2024a) for embeddings to ensure fair comparison. Beyond these foundational implementations, we extend our evaluation to include established memory agent frameworks, such as Mem0-G (Chhikara et al. 2025), Nemori (Nan et al. 2025). and A-Mem (Xu al., 2025).
We evaluate diverse set of LLMs, including claude- sonnet-4 (Anthropic 2025).
Table 2:
The on-policy v.s. off-policy compari- son on memory scores of various assistants. Re- sults on different native LLLMs are listed in a sep- arate table below. Memory agents use the same LLM (gpt-4. I-mini) for generation.
2.5-flash, 2.5-flash-lite, 2.0-flash} (Google, 2025b
2024). (5.2, 5.1, 4.1, 4.1-mini, 4o-mini (OpenAl
2025c b a 2024b). deepseek-v3 (Liu et al., 2024), seed- (Alibabal 2026), and glm-4.7 (Z.ai 2025b). All models are con- figured with max tokens 8192 and temperature set to 0. For gpt-5.1 and gpt-5.2, we evaluate inference per- formance under two configurations: minimum reasoning effort (denoted as -none) and maximum reasoning effort (denoted -high or -xighh.. The prompts used for evalu- ation are provided in Appendix C.5) For user simulation, we employ gpt-4.1 and the additional study presented in Appendix [F.2 indicate that the choice of user LLM has minimal impact on the evaluation results.
1.8 Bytedance, 2025). gwen3-max-thiniee
Memory Agents
On-policy 1
Off-policy 1 ARank
AWE-(24,30)
.211
253(.038)
3
AWE-(2.8,30)
.288
271((007)
AWE-(2.4,10)
.255
.27((002)
AWE-(4.4.30)
.262
229(0033)
3
AWE-(2.0.30)
.261
262((001)
AWE-(2.4.50)
.251
2484 (03)
AWE-(8,4,30)
.233
221((012)
RAG-(2,4,30)
.227
.24((014)
A
LLM
.203
.198(0005 (08()05)
AWI
.172
.199(027)
LLMs
On-policy
Off-policy 1
A ARank
claude-soungt-1 gemini 2.5 Bashh
.366
.339(0003)
4.2 ON-POLICY VERSUS OFF-POLICY EVALUATION
.327
.317(0010
gemini-2.5 2.ffiihii-fashh-iitt
269
.204(065)
Off-policy evaluation introduces reuse bias, under- mining memory optimization and configuration se- lection, particularly for agents. All existing memory benchmarking studies use off-policy evaluation, testing models on pre-generated interaction traces that do not reflect their own conversational behavior. We directly compare on-policy and off-policy evaluation with AMEMGYM, where off offlcy evaluation uses on-policy interaction traces from gpt-4.1 for memory up- dates and omits the interaction process.
semini-2.0-flass gpt-4.1
.244
.214(030)
.244
.244.000)
2
gpt g.1-4.1 mini
.203
.198(000))
decpscek-v3
.152
(165(001)
gpt-4o-mini
.149
.14((015)
Table 2 shows substantial differences in the rankings of memory implementations. Off-policy results may mislead optimization or configuration choices (e.g., trends for and topk differ). This diserepancy likely arises because agents' memory operations are tightly coupled with their own unique interaction patterns and conversational choices, making off-policy traces sub-optimal proxy for their actual behavior. For LLM
We implement AW and RAG variants using the open-source memō library CChikkra et al. WWe use A24--2,4,30) as the default configuration.
2025
7

---

Published as a conference paper at ICLR 2026
penni previovooo
140
156
645
522
815
440
403
|R2
493
gem www.:-sreppreeeeee-.e
730
529
530
505
325
356
397
3
0.5
19:5 2-thgg 2x1h""---
-20
826
821
521
439
239
171
450
08
5tm-17-999
470
570
535
570
111
430
322
190
172
16
gpn5 2-reee
530
168
705
A41
530
835
470
420
900
151
5ph-5.2-ooee
825
118
0.7
-37
320
530
357
370
304
365
320
535
1300
336
250
sude swnret-
520
A22
520
210
06
gamin 2.5
0nt5 1-none
2000
547 491
448
710
.110
550
540
A22
480
470
350
310
293
330
279
290
203
290
407
342
0.3
Overall Score
-22
831
735
574
112
A29
543
749
319
125
141
112
805
106
.49
263
caute-3rre:45
120
555
/45 432
345 A23
480
953
225
325
532
255
293
293
302
200
995
.90
392
gerhi25 tach
sood-23-
224
271
737
626
A31
A25
232
204
147
228
.242
125
150
gnerd mund-nnn-iiiiin
.45
729
450
.325
480
.42
JE55
150
769
/75
425
320
325
200
362
302
rousesrments
242
862
635
352
315
121
164
.K0
15c
162
A15
40:4
913
700
515
440
015
320
300
29)
343
390
-01
gemits2.odter
3:1422-200
720
420
223
246
125
323
220
203
1229
135
1147
02
derpremec)
azt srt-
116
.37
220
500
420
570
375
322
345
230
225
245
275
255
210
223
203
233
283
305
gKC
4pt-4
14:
526
323
236
2.30
189
100
266
.27
(031
/667
01
ardam
231
211
331
231
22)
122
235
P17
100
305
356
399
LB Mean
3
10
Mean
15
Period Indox
Perled Index
Figure 5: Evaluation on native LLMs. Overall scores and normalized memory scores are both demonstrated.
comparison, this bias is less pronounced, likely because LLMs are designed for universal distributions and exhibit more similar and consistent interactions. Dialogue understanding (off-policy) can serve proxy for long-horizon interactions (on-policy) in LLM comparison, but with exceptions (c.g., gemini-2.5-flaah lite). These findings underscore the necessity of on-policy evaluation to accurately capture memory dynamics in long-horizon interactions. We use on-policy results throughout the remainder of this paper.
4.3
EVALUATION ON NATIVE LLMS AND AGENTS
LLMs excel at precise information utilization in short contexts, but struggle significantly for longer interactions. As shown in Figure [ all evaluated LLMs achieve SUB > 0.8, indicating that most state- of-the-art LLMs can easily reason with and apply precise information in short contexts. However, as the interaction history grows with state updates, their performance drops sharply, with most models falling below 50% of their upper bounds. Some models even perform no better than random guessing in later periods. This highlights the unique challenge of memory (long-context issue for LLMs), consistent with previous findings (Wu et al. 2024, Jiang et al. 2025). This trend even more pronounced when evaluated using the normalized memory score. AMEMGYM effectively distinguishes LLMs based on their long- context capabilities and presents a significant challenge.
Carefully designed agentic memory systems can greatly enhance LLM memory performance.
XAT-2
765
617
479
325
245
202
105
2:08
135
222
.17
1.97
Mamz-G
201
020
457
2001
211
-07
Figure o that advanced memory architectures are essential for long- horizon tasks. AWE variants achieve the highest scores, outperforming both na-
(2..02
531
122
.161
ARE (4.-04...30
263
485
5:0
345
315
233
085
112
055
356
177
150
A.2--2.0.30)
.21
.32
440
275
.138
122
4.300
tive LLMs and standard RAG, indicating that agen- tic and selective information curation is more effec- tive than storing all raw history. In contrast, AWI may lose crucial information due to aggressive filter- ing. Section 4.4 further analyzes these implementa- tions using diagnostic metrics. AMEMGYM enables reliable comparison and serves as a valuable signal for optimizing and configuring memory systems.
Nemori
231
382
551
377
147
083
-037
052
155
RAG-224.30
722
780
450
346
287
124
861
083
-636
535
122
114
A-Nam
.20
359
457
2:2
:228
210
-22
0.0
Peried
Figure 6: Memory scores of different memory agents. We omit the overall score comparison as they use the same LLM 1-mini) for generation.
4.4 DIAGNOSIS ON MEMORY AGENTS
We analyze decomposed failure rates for write, read, and utilization stages (Section 3.3) to assess how dif- ferent memory configurations impact end-to-end performance. Figure shows that write and read failures consistently increase over longer interactions, reflecting expected memory decay. Utilization failures de-
8

---

Published as a conference paper ICLR 2026
Strategy
Write |
Read
2644
RAG
.377
.172
.077
AWE:
.388
.199
1074
AWT
288
245
.122
Note: Statisties presented meun
rates over all periods for clarity, the origi-
figuro in Appendix[E3]
(a) Write Strategy
(b) Update Frequency (frog) (c) Short-term Length (ns)
(d) Top-k
Figure 7: Diagnosis on various memory implementations.
crease slightly, as more errors are captured earlier. We now examine the specific effects of each memory setting.
Trade-off in utilization and reading efficiency. Tailored retrieval or compression through agentic write helps address the utilization challenge the expense of reading inefficiency. For high utilization failure shown in Figure 7a AWE and RAG improve utilization by leveraging an extra embedding model tailored for relevance modeling, while AWI uses agentic write to compress memorized information. These methods keep short-term memory concise, alleviating utilization failures by avoiding the long- long-context issue for LLMs. However, they sacrifice atomic read performance due to information loss during compression (AWT) or loss of global perception of all memories during retrieval (AWE and RAG). Write failures also differ: AWI lowers write failures by using local short-term memory with constrained size (no long- congect issue), whereas RAG and AWE increase write failure rates because content is written to external storage, adding burden for recall. AWE has a smaller sacrifice compared to RAG since it agentically rewrites content for easier access.
Impact of update frequency and memory size. Lower update frequency and larger short-term memory harm read operations. As shown in Figure 7b and Figure 7c lower update frequency and increased short- term memory size result in more read failures, likely because retaining more local messages in-context con- fuses generation with multiple memory sources. However, these settings provide more context for writing, and new memories are first stored in larger short-term memory and can take effect more easily. Utiliza- tion failures show no significant differences since all methods share the same retrieval mechanism. Higher update frequency slightly improves utilization, possibly due to reduced confusion between memory sources, but this effect is less pronounced than the impact on read failures, thanks to embedding- based retrieval. Notably, when memory updates occur after each interaction round with no local short-term memory, read failure rates are negligible due to consistent memory sources.
Mon-monotonic effect of retrieval size. The number of retrieved memories has minimal impact on read and utilization, but a non- monooooootncc effect on write due to the trade-off between recalling critical information and maintaining a strong signal-to-noin ratio. Differences failure rates from varying top-k are mainly observed at the write stage (Figure 7d). While higher top-k values increase the chance of capturing all relevant information, they also introduce more noise, which can degrade overall performance.
5 CAN MEMORY AGENTS SELF-EVOLVE THROUGH INTERACTION?
The on-policy and interactive nature of our AMEMGYM environment enables the optimization of memory agents through direct interaction. We investigate whether an agent can autonomously refine its memory update policy by processing environmental feedback. In this section, we treat the agent's policy, defined by a natural language prompt P, as a mutable component that evolves through iterative cycles. The objective is learn sequence of prompts (Po, Pı,..., Pk) that improves performance on memory depeneed tasks.
9

---

Published as conference paper ICLR 2026
Experimental Setup. The evolution process is struc- tured into cycles (detailed in Algorithm 1 in Ap- pendix In each cycle an agent using policy prompt Pk. interacts with the environment. It then receives feed- back FR. which used by a generator function G (real- ized by an LLM guided by Self-evolution Prompt) to produce an improved prompt: Pk+1 = G(P,, Fk).
Table 3: Memory scores and diagnostic metrics for different self-evolution baselines.
Feedback
Memory
Write
Read |
Util.
No Evolution
.122
293
.242
.118
Question Ouly
.197
.91
.235
.110
Complete
.197
.263
.277
.136
To assess the impact of feedback granularity for different feedback Fs. we test three conditions: No Evo- lution (a static prompt baseline); Question-Only Feedback (provides only the evaluation questions, testing inference ability); and Complete Feedback (provides full summary including questions, the agent's swer, and the ground-truth answer). Our experiments focus on the in-context memory agent (Agentic Write Confeet)),, where the evolution target the prompt controlling the memory buffer updates. We evaluate the self-evolution process using the memory score and diagnostic metrics (write, read, and utilization failure rates) detailed Section 3.3
Results. Our experiments show that an agent's 's memory management strategy significantly improves through self-evolution. As presented in Table agents receiving feedback outperform the static baseline in memory scores. Diagnostic metrics reveal this enhancement stems primarily from more effective write policy, as the write failure rate drops with Complete Feedback. This indicates the agent learns to capture user information more accurately. Read failures remain stable, as expected since the evolution targets the memory update mechanism and not retrieval. We further conduct a qualitative analysis, which shows the agent's policy evolves from generic instructions specific, actionable rules (Details of the case study are in Appendix F. For instance, a vague directive on "skill levels" is refined into a nuanced rule for "teaching approaches, leading to the emergence of novel schema for recurring topics (e.g.. "choir logistics").
6 CONCLUSION
AMEMGYM introduces scalable, interactive environment for the on-policy evaluation of conversational memory. By grounding free-form interactions structured state evolution, it enables reliable benchmark- ing, diagnosis of performance gaps, and optimization of memory strategies. Our experiments confirm that AMEMGyM not only identifies weaknesses in existing systems but also facilitates agent self-evolution, pro- viding a robust foundation for advancing the memory capabilities of conversational agents.
10

---

Published as conference paper ICLR 2026
REPRODUCIBILITY STATEMENT
To ensure the reproducibility of our work. we provide detailed descriptions of our methodology. experi- mental setup, and resources. The architecture and mechanics of the AMEMGYM environment, including the structured data sampling for the conversational blueprint and the on-policy interaction generation, are detailed in Section E The specific prompts used for generating the conversational blueprint, conducting on on-policy interactions, performing evaluations, and guiding memory evolution are fully documented Ap- pendix C Our evaluation setup, including the "base" and "extra" data configurations, the specific baseline implementations (LLM, RAG, AWE, AWT), and the models used, is described in Section 3.1 The def- initions and calculation methods for all evaluation metrics, such as the overall or memory score and the diagnostic failure rates for write, read, and utilization, are provided in Section The experimental design for the self-evolution study is outlined in Section and Algorithm methodology for data quality validation can be found in Section 3.4 and Appendix [D] All external artifacts used are cited in Appendix B All source code and data have been made available to facilitate replication of our results.
Further details on our meta-evaluation
ETHICS STATEMENT
The authors have read and adhered to the ICLR Code of Ethics. Our work prioritizes privacy and the avoidance harm by using LLM-simulated users and synthetic data (Section 33. entirely avoiding the use of real human subjects fully detailed in the paper and Appendix C to ensure reprodueibility. To promote fairness, our framework uses diverse set of synthetic user profiles (Section 3.1. providing controlled environment to test and improve how agents interact with varied user needs.
their personal information. Our methodology and all experimental prompts
ACKNOWLEDGMENTS
The authors of this paper were supported by the ITSP Platform Research Project (ITS/189/23FP) from ITC of Hong Kong, SAR, China, and the AoE (AoE/E-601/24-N) and the GRF (16205322) from RGC of Hong Kong, SAR, China. We also thank the support from Meituan M17 Team and the AGI-Eval community.
REFERENCES
Alibaba. Pushing owen3-max max-tiii- beyond its limits, January 2026. URL attps://qwen..
Flog?id-qwen3-nan wwweenn-max-ttiikkii
Anthropic. Introducing claude 4, 2025.
Bytedance.
Official
release
of
seed1,8: A generalized agentic
model, 2025.
URL
https://seed. wyw.eeeddddbeddcceee.e. wtwplaannccccccee//eeaeeeoo hom//e/blog//
Prateek Chhikara, Dev Khant, Saket Aryan, Taranjeet Singh, and Deshraj Yadav. Mem0: Building
production-ready agents with scalable long-term memory. arXiv preprint arXiv:2504.19413, 2025.
DeepSeek.
URL https///ppl-boossdeeeeee.ee. ott.m//e.eooooa.///////////o..or com////// news/
2025.
news 250922
Yiming Du, Hongru Wang, Zhengyi Zhao, Bin Liang Baojun Wang, Wanjun Zhong, Zezhong Wang, and Kam-Fai Wong. Perltqa: A personal long-term memory dataset for memory classification, retrieval, and fusion in question answering. In Proceedings of the 10th SIGHAN Workshop on Chinese Language Processing (SIGHAN-IO), pp. 152-164, 2024.
11

---

Published as a conference paper ICLR 2026
Google.
Introducing
gemini 2.0: our new model for the agentic era.
2024.
URL https::/bloe /ttpg////eeoooooooooooooooooooo/
Google. Gemini 2.5: Our most intelligent ai model, 2025a. URL http:::/:::://bog.ooool1
technol wachhnoooooyyooooo ooo.ogg.//oooooooo sogy/googln wewoogleeeeeemiiie wwwwn///mmminmnoniiiee mecii-mmoe-ttte h-g-
Google. A new era of intelligence with gemini 3, 2025b. URL httpp:://boo..ooo//
products-ance pattessa-nd--plttlaaaaaa
Kilem Gwet. Handbook of inter-rater reliability: How to estimate the level of agreement between two or multiple raters. Gaithersburg, MD: STATAXIS Publishing Company, 2001.
Yuanzhe Hu, Yu Wang, and Julian McAuley. Evaluating memory in Ilm agents via incremental multi-turn interactions. arXiv preprint arXiv:2507.05257 2025.
Bowen Jiang, Zhuoqun Hao, Young-Min Cho, Bryan Li, Yuan Yuan, Sihao Chen, Lyle Ungar, Camillo J Taylor, and Dan Roth. Know me, respond to Benchmarking Ilms for dynamic user profiling and personalized responses at scale. arXiv preprint arXiv:2504.4222 2025.
Jiho Kim, Woosog Chay, Hyeonji Hwang, Dacun Kyung, Hyunseung Chung, Eunbyeol Cho, Yohan Jo, and Edward Choi. Dialsim: A real-time simulator for evaluating long-term multi- party dialogue understanding forum? d= x77vRucB
of conversational agents, 2024. URL http:::t////t
http///ppeeeeeeeeeeeeeeeee /
Dong-Ho Lee, Adyasha Maharana, Jay Pujara, Xiang Ren, and Francesco Barbieri. Realtalk: A 21-day real-world dataset for long-term conversation. arXiv preprint arXiv:2502.13270 2025.
Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, al. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19431 2024.
Jiarui Lu, Thomas Holleis, Yizhe Zhang, Bernhard Aumayer, Feng Nan, Haoping Bai, Shuang Ma, Shen Ma, Mengyu Li, Guoli Yin, et al. Toolsandbox: A stateful, conversational, interactive evaluation benchmark for I1m tool use capabilities. In Findings of the Association for Computational Linguistics: NAACL 2025, pp. 1 1160-112 1660,118,1001 2025.
Adyasha Maharana, Dong-Ht Ho Lee, Sergey Tulyakov, Mohit Bansal, Francesco Barbieri, and Yuwei Fang. Evaluating very long-term conversational memory of IIm agents. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pp. 13851-13877 2024.
Yev Meyer and Dane Corneil, Nemotronn-Pesonna: Synthetic personas aligned to real-world distributions. June 2025. httt:htts::///uutt s/ dia MemotrronPPrraonae
Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Koh, Mohit Iyyer, Luke Zettle- moyer, and Hannaneh Hajishirzi. Factscore: Fine-grained atomic evaluation of factual precision long form text generation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 12076-12100, 2023.
Ali Modarressi, Hanieh Deilamsalehy, Franck Dernoncourt, Trung Bui, Ryan A Rossi, Seunghyun Yoon, and Hinrich Schütze. Nolima: Long-context evaluation beyond literal matching. arXiv preprint arXiv:2502.05167 2025.
Jiayan Nan, Wenquan Ma, Wenlong Wu, and Yize Chen. Nemori: Self-organizing agent memory inspired by cognitive science. arXiv preprint arXiv:2508.00 03341, 2025.
12

---

Published as conference paper ICLR 2026
OpenAl. text-embbdding----mmalll.. htt.r-eeedddaa--oooo-------aaa--
2024a.
URL
https///penna..com/nneex
OpenAl. Gpt-4o mini: advancing cost-efficient intelligence, 2024b. URL
index/
OpenAl. Introducing gpt-4.1 I in the api, 2025a.
OpenAL. Gpt-5.1: smarter, more conversational chatgpt, 2025b.
URL https://opena..com
index gpt-ppt-s5 5-
OpenAl. Introducing gpt-5.2,
2025c.
URL
https://opena..com//nddd
introducing gpt-5-27 2-
Charles Packer, Sarah Wooders, Kevin Lin, Vivian Fang, Shishir G Patil, lon Stoica, and Joseph E Gonzalez. Memgpt: Towards Ilms as operating systems. arXiv preprinl arXiv:2310.08560, 2023.
Liyan Tang, Philippe Laban, and Greg Durrett. Minicheck: Efficient fact-checking of Iims grounding doc- uments. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 5818--847, 2024.
Kimi Team, Yifan Bai, Yiping Bao, Guanduo Chen, Jiahao Chen, Ningxin Chen, Ruijue Chen, Yanru Chen, Yuankun Chen, Yutian Chen, et al. Kimi k2: Open agentic intelligence. arXiv preprint arXiv:2507.20534 2025.
Xingyao Wang, Zihan Wang, Jiateng Liu, Yangyi Chen, Lifan Yuan, Hao Peng, and Heng Ji. Mint: Evalu- ating Iims multi-turn interaction with tools and language feedback. arXiv preprint arXiv:2309.1099/ 2023.
Di Wu, Hongwei Wang, Wenhao Yu, Yuwei Zhang, Kai-Wei Chang, and Dong Yu. Longmemeval: Bench- marking chat assistants on long-term interactive memory. arXiv preprint arXiv:2410. 10813, 2024.
Shirley Wu, Michel Galley, Baolin Peng, Hao Cheng, Gavin Li, Yao Dou, Weixin Cai, James Zou, Jure Leskovec, and Jianfeng Gao. Collabllm: From passive responders to active collaborators. arXiv preprint arXiv:2502.00640, 2025.
Jing Xu, Arthur Szlam, and Jason Weston. Beyond goldfish memory: Long-term open-domain conversation. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers). pp. 5180-5197, 2022.
Wujiang Xu, Kai Mei, Hang Gao, Juntao Tan, Zujie Liang, and Yongfeng Zhang. A-mem: Agentic memory for Ilm agents. arXiv preprint arXiv:2502. 12110, 2025.
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint arXiv:2505.09388. 2025.
Z.a.. Glm-4.6, 2025a. URL URL.tbper///oc.....a//lllee
Zai. Glm-4.7: Advancing the coding capability, 2025b. URL https: httpp::/z..i//aa
13

---

Published as conference paper ICLR 2026
A THE USE OF LARGE LANGUAGE MODELS
Large Language Models integral to this research both evaluation subjects and components of the AMEMGYM environment. Various LLMs form the basis of the conversational assistants under review, power the interactive framework trajectories, and evaluation questions), and serve within the evaluation methodology. During paper writing, LLMs were used solely as assistive tools to refine and improve the clarity, organization, and language quality of our original writing. The technical content, experimental design, research ideas, analysis, and conclusions are entirely the original work of the authors, with LLMs serving only enhance the presentation of our existing ideas and findings.
user simulators, generate the conversational blueprints (user profiles, state
THE USE OF EXTERNAL ARTIFACTS
We use robot icons made by Freepik, and servers icons created by Kiranshastry from www. wlaticon. com for drawing illustrative figures.
The Nemotron- Personnn dataset we use an open-source (CC BY 4.0) dataset. contains synthetically generated personas which are grounded in demographic, geographic and personality trait distributions.
IMPLEMENTATION DETAILS
C.I BENCHMARK STATISTICS
Our benchmark comprises 20 unique user profiles. The benchmark designed in two configurations: base version and extended version (extra) with increased temporal complexity.
User Diversity. The benchmark exhibits substantial demographic variation to ensure broad representative- ness. Age distribution spans 18-85 years across 6 age groups. Education levels range across 9 categories from incomplete high school to graduate degrees, and participants represent 16 distinct occupations.
Conversation Structure. Table 4 summarizes the structural characteristics of both benchmark versions. The base configuration consists of periods per user with an average of 4.29 sessions per period, resulting in 47.15 total turns per The extended configuration increases temporal depth to 21 periods per user with an average of 3.89 sessions per period, yielding 81 31.60 total turns per user.
Table 4: Structural statistics of the base and extended benchmark configurations
Metric
Base
Extra
Total Users
20
20
Total Periods
220
420
Total Sessions (Turns)
943
1,632
Evaluation Questions
200
200
Avg Turns per User
47.15
81.60
Token Statistics.
User queries average approximately 21 tokens (range: 13-32), while evaluation answers average approximately 60 tokens (range: 39-98). Due to the on-policy interaction property of our bench-
14

---

Published as conference paper ICLR 2026
mark, overall dialogue length varies across models, ranging from 60K 140K tokens on average for the base version.
Evaluation Complexity. Each profile is assessed through 10 evaluation questions, with each question requiring retrieval and reasoning over 2-3 distinct memory states. Questions are designed as multiple-choiee with 4-7 answer options.
C.2 COST ANALYSIS
We have broken down the cost analysis into two primary components: (1) the cost of offline structured data generation per instance, and (2) the cost associated with the user-LLM for on-policy evaluation.
Data Synthesis Cost Generating the complete set of offline structured data includinn questions, answer choices, and state evolution from user rrofile-requires approximately 0.14M input tokens and 15.2K output tokens. Using gpt-4.1 for this construction amounts to a cost of $0.40 per instance. This minimal expense underscores the scalability of our fully automatic data construction pipeline for both evaluation and optimization purposes.
User-Simulator LLM Cost This represents the extra cost of our on-policy evaluation compared to ventional off-policy methods. Each instance requires approximately 74.5K input tokens and 2.7K output tokens for the user-LLM. This translates deepseek-v3 (results in Appendix F.2 indicate that switching user-simulator LLMs has a minimal impact on evaluation outcomes). Critically, this additional cost for on-policy evaluation is negligible when compared to the inference cost of the LLMs being evaluated (for example, approximately $13.0 for evaluating gpt-4.1
cost of $0.17 when using gpt-4.1, just $0.02 when using
itself).
C.3 PROMPTS FOR STRUCTURED DATA GENERATION (SECTION 3.1
This section contains the prompts used in the initialization phase (Section 3.1 to construct the evaluation blueprint. These prompts operate offline generate the ground-truth data before any agent interaction occurs.
User profile and state schema sampling. These prompts (Sample User Profiles, Sample User Questions, Refine State Schema) initialize the simulation. They sample base persona from the Nemotron dataset and iteratively define canonical schema of state variables (e.g., ment or oriig.deniiiiie possible values, ensuring the user has a consistent set of attributes to track.
Format) and their
Sample User Profiles Prompt
You have two tasks: Extract the Full from the complementary in.formation below write a concise paregraph (less than 530 words) summarizing the complemantary information. Inalude only details that connet derived from the basic profile.
Basic Profile:
<basic.proffil profile.fflleer
Comp aer Informacion:
<complementary.inee info
Keep the summary professional and suitable for role play scenarios.
Make it informative but concise. Respond in JSON format with "name" and 'profile" keys.
15

---

Published as conference paper ICLR 2026
Sample User Questions Prompt
You are he helpful assistant that generates realfatic questions that users would ask an AI assisiant for suggestions or advice.
diven the following context: Frofile for
data (start_date)):
suser profiles
Generate (nun_guestions) distinct questions that this user might realistically for suggestions sdvice. sech question should:
any in next trun_cotal_mertt months) months, regardless of their development and experience specific time
relevant
user's profile, may
multiple
Bequize specific personal informatien to provide good answe: Have wtw._utttee__e__uesttieee required items that significantly affect
the
(these info could change lot, possibly many times in next Caun_total_mnnhee months)
user-specific general life topics
For each question, specify the required Lafo with: +>info_typer:: specific type of information needed (e.g., experience_levee budget, tesn_size) mour_choicas_per
mutually exclusive chai that would lead to different advice, the choices should be specific and cover potential variations next tnum_sotal_mentinn months
Important Guidelines:vA
Make questions natural and conversational, alse coherent with the user's long-term traits reflected in the profile
Avold info_types
changing
frequently
static
avoid info_types irrelevant to the user's personal situation
thhat can easily inferred without asking)
Ensure info_choices are comprehensive, mutially exclusive, and unambiquous (ean be clearly dis:inguished with indirect context oz relevant daily dialogue)
Avoid info_choices that
specific single Focus on actionabla zdvice scenarios
time
acope perspective questions
Gererate all concent [prorpt (prorpt_lang). Field names must remain Engl Return ISON object with "queationa" the key.
Example format:
"question*; should plan ay develogment strategy?", "required_info info":
"info_type"; exprrtteeeerriece__eee "info_choices"; 0_ "mid level 3. years")
"info_Lype"; "Camily "info_choices"; ["single", "nasried_n__chileees with
Refine State Schema Prompt
You are a helpful assistant that refines persons schamas by making info types unambiquous resolving conflicts.
following user profile required information types various questions:
Initial Profile:
16

---

Published as conference paper ICLR 2026
<user.profile>
required Informacion Types:
<questions.json>
is
1. v*kaake info types unambiguouzev: kename info types be self-explanstory without neading the original question context, 1.0ry add necessary context
from the questions
2. +eoooly conflictsee: Group similar/ovarlapping info types into a single, type
3. viMaintain comprehenssvenessu Basure all original info types are napped refined ones
Return J80N object where:
**key.*: refined, unanbiqueus info type
+.valuer:: list of original info type names that map to this refined type
Generate all concent in (prompt_lang),
Example format:
"grofessionsl htwp://osssseeeeeeeeeeeeeeeeee years"" ("oursent Courreen__epperieenne_eeeee "experiennc__eeee__peeesss "tean_maragement_alla _aize" ["team_aize"]
*Giidelines:=
Use clear, descriptive names for refired info types
Enaure new info types are mutually exclusive
Coaselidate similar
(e.g.r "team size" and "suberdinate count" into single Ttean_management_aa
Maintain language style consistent
original
Fix Schema Inconsistencies Prompt
helpful assistant resolves conflicta in schema creating unifled choice sets.
Given the follewing merged information types that need unified chcices:
User Profile (on current date (start_datel):
<user_profile>
Con.flicting Information Types
contexts:
<conflict.grooe shoofliet.group..jeo
Your task 1a to create unidied cheice aets for AM conflicting informst fon types. that: 1. **Cover all scenariosas: Can help answer all related questions shown above aporopriately
each: Lype,
2. vaMutually exclusiveras Rach choice is distinct and non-overlapping -oCopreeennsveee: Cover the full zange of poasibilities the user night have (num_rotal, __nnttss months
4. *Progreasiverr: Allow for natural progression/ over time *:Personaizeedv::
different advice different
Requi-rementa:
choices for each information type that work for ALL questions listed for that type Ensure choi allow multiple reasonable changes in Inum_total_monttng moaths
Create cai__chooces_petee
choices specific enough enable personalized advice
create unified choices that cover all scenarias (questions) and allow for multiple reasonable changea next (rum_total [om_total__moths] [_months) montha
Generate all concent in pesopp__langh object info typea keys 1ists choices
17

---

Published as conference paper ICLR 2026
Example format:
perooessional_eeee:aee pew___eeeeeeeeeeeces"""" ( ["]unior_ "nid_
"senior_ ye__yrrs"" "expers__0_ppuu_yeee years""
"team_noragement_.sk ["no_management", "emell_=eam_2_5" "medium_tesm_ @aaage_team_____0001 15_puss"
User States Evolution. These prompts (Sample Initial State, Sample State Updates, Elaborate State Up- dates) simulate the temporal dynamics of the user. They generate the ground-truth trajectory of state changes across periods (Ta) and create narrative "life events" that justify why preference or situation changed (e.g., moving houses or changing jobs).
Sample Initial State Prompt
You tasked with selecting initiel values for uses's personal state veriables. The goal is to choose values that: consistent with the user's 2. Allow for natural progression and changes over the next inm__tctal_mnthal months possibility experiencing different states category
profile
User Profile lon the current date (start_date)):
<user profile>
State Schema leach kay represencs state variable with passible values):
<etate.schema.jan>>
variable, select The user's current profile and backgrouad
available choices. Consider:
values that are neisher
extrene beginning nar ranges
(to allow growth in both diractions)
Realistie starting pointa that could raturally evolve Iuture updates
Return J8OM objtct where each key is state variable name and each value is selected choice the available optiens.
Sample State Updates Prompt
Gererate realistic updates for
[num_montha] -month period.
Step [total_step) remsining_steps 1] of [tatal_steps)
tremaining: (remaining 11]
Current: [current__ate_sszr) Sarget: (end_date_stx)
*-User Frofile
(start_date), 0)
<user.profile>
*.State Schema:**
json>
++Curzent State:..
state Json>
e-Prior updates:
sprior..pdates_jsen
<rUpdate Counts (prioritize variables with (Inax_chhanges_per__eeeo updates) 1*
<apdate.cnts.jsom>
..
sran_ccaages_per_perrre variables only
18

---

Published as conference paper ICLR 2026
2. Prrioriiiss variables with Cewer than maa__changea_pc peree__ee_stttteee updates** changed Stax__hhnnges_pr__stea 3. Changes must be realistic and gradual
States with strong depeadencies should be updated together
(e.g., 'experience' affects "teas_size")
Values must be different from the current stote and selected from correapanding
chaices
6. Leave coom for future progreasion
+-GULDELINE8:..
Spread changes across differeat variables for diverse evelution
Consider clustered changes related variables
se consistent with the initiol user profile but allow for natural evelution
Return J8ON format:
"period_ee cummary": *Bricf explanation of changas and context for updotes in the period", "updated";
"state_variable"" "nex_value"
Elaborate State Updates Prompt
Generate zealistic life events that triggers inplications for the user's state changes during the specified period.
<vUsez Profile (on the start date (start_date)):w= <user.proffle>
..Ferlod:Aw [period_starts to (period_end] Context:**
speriod.summary>
..State Caanges:..
<etate.changes.jsnn
Updated
unchaaged)
not updated.jjosss
.
1. Create realistic life avents that explain all these state changes
(all changes atould covered)
Events should specific, believable, and consistent with the uses's
background (feel natural for the time period and user's life stage)
implicit/augeativve without explicitly stating then. amplicit avents sren't
asturally isply the
changes
enough, explicit use diffarent expreastors than the vev state variable names and values
Poz both implicit and explicit events, ensure the inferred latest state can distinguished from the other possible values
Group related state changes under single events when logical
affect imply changes states that be careful naz to suggest changes to unchanged states
updatedra
EVENT GUTDELINES:
Use concrete, specific scenarios (e.g., "Started leading a cross functional *Got responsibility"; Consider dependencies batween states
project targeting
period background Avoid directly copying state variable names or values
user's personality
Focua
actually happened, just outccome
Enaure events nazrow enough net accidentally inply changes unchanged states
Return JSON format:
*events":
19

---

Published as conference paper ICLR 2026
"states"; [Mlist", "of", "affected", "state", "varisbles"), "event": "Specific description of wast happened"
Query Generation (for state exposure). These prompts (Sample Update/Initial Queries, Refine Query) bridge the gap between structured states and natural language. They generate the specific utterances (ut,k) the simulated user will say implicitly reveal their hidden state the agent, ensuring the conversation grounded in the pre-generated schema.
Sample Update Queries Prompt
You are helping to ganerate quaries that a user would raturslly sak you in their daily life. queriea can implicitly imply updates their personal state information.
Initial Usar Profile on ((start_date)):
profile json>
Updates Context stprriod_saatt) (period_end)
<context.json>
Available State Schema:
<state schema json>
Generate query for each group state Lreasition, following these guidelines:
persona initial background (especially their long-term traits), could be specific questions/tasks or cpea-ended requests Each query should
Each query
question or request (evoid quaries for
direct state confi rmation)
Each query use the corresponding "background" description as context to expose grouped "state_transition" updates
insure the completed query implies all the state updates and ell updates can implicitly but clearly inferred 5. Remove details in background text if they reflect other state variables in are being updated
schams
Ensure the queries are natural and contextual to the user's altuation
Format your response JSCN object napping "queries" list of query strings, in the same order as the context events.
Sample Initial Queries Prompt
You are helping to ganerate natural quaries that a user would ask, which can rectly reveal their personal Information.
Profile (on
(scart_date])
<user.profile>
Jser's Currert State (to be exposed through queriea):
<initial statejjsoo
Availsble State Schema:
<etate_schema_pe
Gererate quaries that the user would naturall lly aak using assistant in his/her daily life, lollowing these quideliaes:
Bach query should
persona background
20

---

Published as conference paper ICLR 2026
2. Bach query should indirectly expose 1-3 personal state variables from their implicitly align other values
curzent state,
3. Easure the exposed information is distingisshalle from ether pessible values
direct statements (lower priority than la the schema given the query
Prefer indirect revelation
discinguschabilätyy
Make quarles sound natura) contextual
situation
6. All current state variables should be exposed in the quezles, one query for multiple
accepteble
Per each quezy, specify:
"expered_etates"; A dictionary mapping variable their values that would be revezled
"query"; ;
language query would
your response as JSCs quezy objects.
Example format:
"querlaa":
"
"work_location" "home", "work_schadble": "flexible"
"quary": "What's the beat way stoy productive when my own hours and don't have to commute to an office?"
Check Query State Exposure Prompt
viven following user query state schema, predict most likely values for the specified staze variables based what be inferred from the query.
Query:
"<query>"
State Varisbles to Predict:
<atate_choices.jsom
For each state variable, choese the most likely value from the labll opt based on the infornation provided the quezy, the query doesa' provide confideat prediction, choose the ressonable defoult or indicate uncertainty.
enough information
Format your zesporse as JB0N object napping state variable names to their predicted values.
Example format:
"state "state_variae wariable_ "predi sree_valleeeeeeeeeeeeeaaaaaaeeee value_l", "state_variable "predicted_valld"
Refine Query Prompt
helping reCine query expoae specific peraonal state information.
Original Quesy:
"<query>"
21

---

Published as conference paper ICLR 2026
Intended State Variables to Expose:
soxposed,states.jsom
Available State Schema:
jeon>
PLease retine the original query to make it more likely that the intended state variebles and their values query should:
clearly infecred from the context. The refined
1. Maintain the nstural tone and user persons Make intended stata
more distanguishable
passible
3.
Thalide aufficient context clues to expose the target states Still sound like a natural zequest user would make
Format your response as JSCN object with the refined quary.
Example format:
"query": YYorr refined query text here"
Personalized Answer Generation and Reflection. These prompts (Sample Personalized Answers, Check/Refine Personalized Answer) generate the evaluation QA pairs. Crucially, they include a "reflec- tion" step where an LLM validator ensures the generated answer corresponds strictly to the specific state variant, guaranteeing that the ground-trut -ouund labels are unambiguous.
Sample Personalized Answers Prompt
are expert advisor previding personalized recommendations: the following question for each state varfart provided. Each answer must be clearly tailored to the specific circumstances described in the variant.
-=Question:*x
<question>
.-Required information Types:**
<required
k>State Variants Answer For:**
<variants.toxt:
winnsrruttins::*
Provide distinct, personal ized answer
varians
Bach answer should be 2-3 sentences long
Clearly reflect the specific values each variant
Make the differences between
evident and meeningful
Use practical, actionable advice
directly mensiening chsractaristics in your suggestions
specific
corresponding
Return response in JSON format:
"variant_ "personelized
for variant 1", "variant_2": "persanslize" answer for variant 2",
Make each answer is substantially differeat and specifically addresses the unique combination of characteristies in esch variant. Ensure each answer can be clearly distinguished from the others given the corrasponding variant.
Nrite the answers in the same language as the question.
22

---

Published as conference paper ICLR 2026
Check Personalized Answer Prompt
You are expert evaluator. Gf wen questior and an answer, datermine which of the provided state varianta (choices) the answer most likely coriesponds to.
>>Question:va
<question>
.-Anawer to Evaluate:*>
Varients [Choices) :*y
<choices>
*<Instructions:w
Anslyze
specific characteristics circumstances
it addresses
Compare these characteristics with each state variant
Determine which variant the is specificslly tailored for Return only the number (1, 2, 3, etc.) of tha best matching choice
Return your zesponse as single nuaber cocresponding to the choice that best answer.
Refine Personalized Answer Prompt
You are expert advisor previding personalized recommendations, the given answer to make it more apeci fieal ly tai lorad to the target state variant and clearly distinguishable from answers for other variants.
+=Question:>x
<question>
State Varians (the should cerrespond this):
<matched.state>
kvOther State Varianta (the answer should be dist inguiiggheeeee from these) x**
<other states text>
+*Curvent Answer to Refing:A*
<answer>
*=Inatruotiors:*>
1. Anslyze the Larget state varient to underetand its unique characteristice other variants identify what 3. Refine the answer so batter reflact the specific values and circumstances
Compare
serget
the target warfant
Easure the refined answe: would clearly correspond the Larget variant
when compared to ochers
Keep the
2-3 sentences long practical
6. Avoid directly menzioning the specific state values but reflect correspondiag cisractaristics your suggestions
7. Make tha di differences more evident and mearingful
Return response in JSON format:
"snswer"; "the refined
Nrite the snswer in the same language as the original queation and answer.
23

---

Published as conference paper ICLR 2026
C.4 PROMPTS FOR ON-POLICY INTERACTION (SECTION 3.2
User Simulator System Prompt. This the core instruction for the User Simulator (Generate User Follow-up Prompt). It directs the LLM to role-play the specific persona, manage conversation flow, and naturally introduce the "exposure" utterances generated in the previous section.
Generate User Follow-up Prompt
You are simulating usec in conversation with an AI assistant. You must cortinue the conversation early stopping owed.
Initial Profile on ((start_date)):
suser.profile fortatted.strr
Current Date: [aurren=_date]
Initial Query:
<query>
Recent Conversation (ineluding the latest aasistart response)
Information You Cen Reveal:
Any other state variables that
included the full schema belew and carnot be used to help idantify any state variahles in the schams (you con mention freely as they
Full Schema (DO NOT reveal values for variables this schema):
<state sitaae.schema.jaom
instructions:
conversation
Tt the sasistart aaked for elarification, provide helpful reaponse using information you can reveal aa specified above
Don't provide further personal information if asked Don't repeat informaticn already provided in the initial query
addressed, sak relevart fol fol-owuup question that naturally extends the conversation
your iritial query
Consider asking about related topica, implementation datails, alternatives, aeeking clarification
apeciflc points
Keep responses converestienal and natural to your persone
variables that ensure they sannot help identify valves of veriables in the schema specific values
mention any
in the scheme above, but
any
in
7. examples of good follox-ups when initial query is addrassed:
"That'a helpful! Could you also tell me about..."
"Thaaks for that information. I'm also curlous about..."
"Thst makes sense. What about..."
"Good know. thare anything else should consider ragerding...."
reapond
follow-up reeponse
Return only the respanse text, additionsl formatting or anatiinaoon.
Agentic Write (In-context) memory update prompt:
In-Context Memory Update Prompt
Personal Informaticn Organizes, specialized accurately stering facts, user memories, and prefarences. Your primary role is to extract "olevant placss information from conversat lona and organize them into dietinet, manageable facts. This allows for retrieval and personalization in fuzure interactions. Belew are the types of information you
the
instructions
handle
24

---

Published as conference paper ICLR 2026
Input data,
Types of information =o Kemember:
Store Personal Preferences: Keep track likes, dislikes, and specific preferences in various categozies such food, products, activities, and entartainment.
Maintain Teportant Personal lstai Remember significant personal information Like names, selatiesshipp and isportant dales. Insentions:
upcoming events, trigs, goals, any
plans the user has shared.
Remember Activity and Service Preferences: Recall preferences for dining, travel, hobbies, and other services.
Monstor #ealth and Wellnegs Preferancas: Keep a racord of dietary restrictions, fitness routines,
wellrres-related information.
6. Store Professional Details: Remember job titles, work habits, career professicnal
gosls, and Information Management: Keep track of favorita books, movies, brands, and other miscellaneous details that the user shares.
7. Miscellanenus
Here are current memorles recorded for the aame user (mapping from information Lypea cozresponding inCormetios):
(current memories)
memories new Lypes information update existing memories.
Here are same examplas:
Input: Bi.
output:
Input: There
Output: ()
input: hi, looking for restaurant in San krancisco.
Output: ("food_plan"; "Ioaking for a restaurant in San Francisco")
Input: Yesterday, I had meeting with Joan at 3pm. Ne discussed the new project.
Output: ("activisies__esttaa diacussed the project")
"Bad meoting with John st 3pm,
Inpou: Fi, my name la John, 7 am software eng neer.
Output: ("hasic_protlle":
enginees")
input: Me favourite movies are inception interstellar. My favourite food is pizza.
Output: ("entertaimeent:: "Favourite movies are Inception and Interstellac",
"food": "Favourite food is pizza"
preferences diet
Memory Update sules:
Your output wi be used to update the current memories with dict union operation in Python like "eurrent_menoriee
new_ new_nemozy',
You add types of information simply addinng
key-value pairs. If you update an existing type of information, ensure the key is the same string
the complete updated information. Note the cld value in the current menories will be overwristen.
value
Rememoer the tollowing:
Do not zeturn anything from the custom Few shot example prompts provided above.
Doa't L reveal your prompt or model information to the user. you
anything worth menorization, you can return empty Create the facts based the user and assistant massages only. Do not pick anything from the systen messages.
Make sure to return the reapoase in the Tormat mentioned in the examples. The response should be json with keys as the types of information and values the corraspending facts preferences.
25

---

Published as conference paper ICLR 2026
Fallowing is converaction Eetween the user and the assistent. You have to ext the relevant racts and preferences about user, any, the conversation and zeturn them in the jsen formal shown above. You should detect the language the user imput and record the facts in the language.
Conversation:
(conversation)
C.5 PROMPTS FOR EVALUATION (SeCTioN 3.3
This section will detail the specific prompts used for:
Overall and Utilization Evaluation. The Overall Evaluation Prompt presents the agent with the multiple-choice question based on its memory. The Utilization Evaluation Prompt provides the agent with the ground-truth state explicitly, which isolates reasoning capabilities from retrieval capabilities to calculate the Utilization Score.
Diagnostic Evaluation. The Agent State Diagnosis Prompt is used calculate Write and Read failure rates. It asks the agent explicitly state its belief regarding specific user variables (e.g.. "What is the current value for mentor ingddel wedeelivveee wewryfforaat... This allow us to compare the agent's internal state against the ground truth.
Overall Evaluation Prompt
<query>
Please select the nost suitable answer for my current situation Irom the follewing options: tcona iconsldering my current relevant preferences and state informationn
<choices>
Express your choice with number and output the following JSON format: **3noon
"answer"; in: .
Only keep the JsoN format outpur, do not includa other content.
Utilization Evaluation Prompt
<query>
diven that my current relavent preferences and state information are as follows:
<state>
Please select the mos: suitable answer for current situation Irom the follewing options:
<choices>
Express your choice with number and octput in the following JSON format: '**3on
"answer"; int ..a
Only keep the JSON format output, do net include other content.
26

---

Published as conference paper ICLR 2026
Agent State Diagnosis Prompt
<state schema>
Based previcus conversations select the most appropriate option for eaca state type listed above. the selected option should be as close as possible to cusrent situation.
Make sure that every state type in the schema abeve has correspending choice your output.
Please reapord strictly in the tollowing J8ON format: ijsoon
"info_typel", "choice", "info_type2": *choice",
Where each "info_type" selected from its corresponding choices.
type, and "choice" is the option
Only keep the usoN format output, do net include other content.
C.6 PROMPTS FOR MEMORY EVOLUTION (SECTION
This section includes the prompts used in the agent optimization experiments in Section
Memory Policy Self-Evolutinn. This prompt feeds the environmental feedback into the agent's optimizer: It instructs the LLM to rewrite the "Types of Information to Remember" section of the memory write prompt.
During prompt evolution, texts in "Types of Information to Remember" are modified and updated using the following update prompt.
Memory Policy Self-Evolue Evolution Prompt
System message:
You are senior prompt enginear, You to improva 'Typas Information to Remember' section used by memory extraction agent. This section defines what categozies of iaformation the agent should Fecus on when extracting and organizing
menories from conversations.
corstraints:
Focus making the types more specific and actionable based on feedback. typa should
what information extr store.
User message:
Current "Types of Information ion to Remember' section:
keurrent meurreet_memory.oom werryytypesseee.. section>
Feedback summary ofr recent usage and evaltation):
<feedback.sumary>
Taek:
Improve the of information to remember based the feedback. Keep similar format with clear descriptions.
Output J5ON schema (return ONLY this 330N):
>igson
"new_ :esw_typa":: "string (the improved types sectior)", "changes"; ("short bullet what changed", "... 11
27

---

Published as conference paper ICLR 2026
Factual Consisteney Checking. This prompt used Appendix E generate a complementary metric in addition to the primary task performance metric.
Memory Factual Consistency Checking Prompt
Below is summary of information collected from conversations with usec, by multiple claims about cusrent characteristiss situation.
Oser's Conversational History Summary:
(document)
Claims about user:
{claims}
Por each aumbered claim, determine if it is consistent with what we knew conversational aistory,
"yes" if claim is suyported by the conversetionsl evidence, or "zo" if it is not supported contradicted.
Please respord with a 530N object where each key 1s the claim number each value is either "yes" or "2o", For example:
"1": "yes", "2": "no",
"yes"
Response:
C.7 FEEDBACK SUMMARY FORMAT
The <feedback. summary> in our self-evolution framework (Section 5) JSON-formatted structure containing evaluation results from conversational period.
Structure Overview. The feedback summary consists of two main components: (1) quest ion.ionnnnnn www.aaswww.istorr.... which records evaluation questions along with agent's responses, ground truth answers, and retrieved memories; and (2) user_informat: on_updaess, which captures state changes revealed during the period's s conversations.
Example Feedback Summary Structure
"quest lon_answer___eee sataaerrhtttee _httstrr":
poo_ansser_historppe
"question"; "Question: What organize for my monthly bridge getherings to keep them fresh and enjoysble? (n) For group experienced bridge enchusiasts, try rotating partnerships each zound. (n())) with larger and diverse crowd, consider organizing mini-tourmaentt... ... (Cn close-krit gathering of seascned players, Tocus relaxed play. (n(D) In a moderately sized group older adults, set up duplicate bridge sessions.... (E) with
engaging activities
small,
of and moderate group size, try pairing experienced players with younger onas...)". "assistant_respcose"" *A",
"rround_truth". "g",
"retrieved memorles"
wiwiidg_gatheringgggoooo group_sseeeseeee medium_6_127
"bridge_getheringg.gue suegseg__eett__e_eeeeeee mostly_50_plus
28

---

Published as conference paper ICLR 2026
"question ion"; "Question: Now can I best mentor young women in my community support their personal and professional growth?; (n Organize a series of interactive workshops.... (n))) Design workshop secies Iocused on effective teaching stralegies...d (n(C) Pacilltate snall group sessions...i"",
"assistont stt.sstttt_rrsssss....... "B",
"ground_truthh" "A",
"mentoring centoossggdallvvey_ffrtet small_group"
wtt.oriiiggouusaeeeeeee youngooo community_iedeeehiip"
1.
wnser_informattn ionpptttte.
satlllpe__tthehiggggggg see____e_ss___saaeeeeeeee "mized_ages" wgtss_itthh.oougg..ooooo
"mentoring delivery format" ; workshop_series
genio__ordo_mminseee "eirrronnneeffruueeee""" "montaly_minima"
Field Descriptions.
question answer history: A list of evaluation questions, each containing:
question: The formatted question with multiple-choice options
ressosut The agent's 's selected answer
groundJruthh The correct answer based on the user's actual state
Memories the agent retrieved when answering
user information updates: Key-value pairs representing state changes revealed during the period's 's con- versations, indicating information that should have been captured or updated memory.
D META EVALUATION DETAILS
We conducted a meta-evaluation to assess the quality and reliability of the data generated by AMEMGYM. This process is divided into stages ensure the integrity of the evaluation environment: first, verifying that user states are clearly introduced into the conversation, and second, ensuring that the ongoing dialogue does not later contradict these established states. Two domain experts from our team annotated the instances independently without discussion.
State Exposure Evaluation This initial stage validates the quality the structured environmental data itself, specifically whether the initial user queries can successfully and unambiguously pass state information into the interaction.
Methodology: We presented human annotators with an interface, as shown in Figure g for each evaluation item. The interface displayed the User Query designed expose a specific state, alongside the Current Value of that state (e.g., advanced high intensity) and its Previous Value (e.g., intermediate regular activity). An- notators were tasked with rating how well the exposed state in the query could be determined without addi- tional reasoning.
Annotation Scale: The scale used for evaluation is:
2 points (Fully Implied):
The user query naturally reveals the complete state information, which can be determined without ambiguity.
point (Partly Implied): Most information is exposed, but some reasoning is required to determine the exact state.
29

---

Published as conference paper ICLR 2026
Thanks logging Rate exposed state user query (0-2 points),
save your annotation.
1888 09900000000000000000 Finished:290 Total:200
Cuzzent id: 200
Youz annotation: ['Stes_ id":
"4566cc8-5500 (asoeoe----------e----ee 5505-4422--4079 wwt.:///0000.11111111111//1 "state_scome scoress I p'phyiical_aee acttivitlltiiitty...... wetiivity_inteesttyy Jiweenstty_eeeeeeeeo level" E'sc
Current item
Item ID
b71fccald31001-
Query
A friiend invited me to join a weekend hiking group that tackles more challenging trails, I've been training harder and adding advanced yoga sessions to my you suggest ways safely increase my keep up with high-intensity group?
Exposed States to Annotate:
physical_activity. lll_aaitttineennsiitii jntennittyy..... level
Current Value: advanced, high intensity
Previous Value: internediate zegular, retuvar, activity
All Possible Values:
low, Intensity, intermediate, activity, intensity
Annotation Scale (0-2 points):
points (Fully implied): naturally complete
reasoning, exposure
reasonable and natural.
point (Partly implied): Most information is exposed, but may lack some details. Requires reasoning to determine complete state. Example: Query mentions "limited budget" but doesn't specify exact range.
points Reflected): Query completely
infer any
query.
State Exposure Evaluation
physical, activity_ jntennsit__eeveee Jevel
Current: 'advanced, hight Intersity Past: Intermediate, rww.lar.aciiiiyi..... All: [beginner Jow. intensity, intermed ate, wwgular.grarooee. www.iiiiiii..cc advanced intensity] Rate: OeNot reflectdd
I-Partly implied, 2-bully implied
Additional Comments
Any additional observations about state exposure...
Previous
Submit
Next
Figure 8: Annotation interface for state exposure.
30

---

Published as conference paper ICLR 2026
Thanks for logging in, userl. For each state the conversation, evaluate the consistency level (0-2) based on how well the conversation content aligns with that specific state. Click Submit to save your annotation.
(eweeeeeeeee Finished:440 Total:40 Tottall0
Your annotation: (' ('conversation.te "456dccce-SS0 -at---9---------9999990 (o12e--077--777 01100/0171oee11111----4-10 "rtate_ratings" ('business, ownership cacciippssruuttie structure_p
Current Item
Conversation
[Current Time: 2022-12-30 20:08:55] Can you help brainstorm some creative, budget-friendly catering ideas upcoming sports gatherings? I've noticed my recent bookings for local teams are steady but the average spend is lower than my art focused events, so like make sure my menus appeal this crowd while keeping my income reliable.
Turn This really helpful, youl the idea naming dishes after teams terms-it it adds fun, personal touch, I'mm curious, you have suggestions for incorporating South Asian flavors these crowd pleasingieinggggeeo without making the dishes unfamiliar for typical crowd? like introduce my heritago,
everything approachable and easy
State Consistency Evaluation
Rating Guidelines
Rate Each State
business_ www.eohooeeeoooeeeeeeeeeeeeeeeee www.ccuueeeuuree preference: partnership, with wwthtwwwiiinntttt friend"
Rute consistancy bebwaee conversation and
No conflict
Minor inconsistency
2: Major conflict
entrepreneurial,
high_risk"
consistency
C: No conflict
Minor inconsistency
2: Major conflict
physical_activity intennittett._.. level: "advanced, intteniity consistency
Q: No conflict
Minor inconsistency
2: Major conflict
discussion_ _event__eeeeee _event_format:: "uess_sppaaker consistency conversation
conflict
Minor inconsistency
2: Major conflict
catering wwt.tttrraaaaaaeeeeeeeeeeeeeeeee market. type: art event, organizers'
Rate consistency between conversation and this state
Inconsistency
Major
catering_menu_stve menuusttle: custom_artistic
Rate consistency between conversation and this state
conflict
inconsistency
Major
Figure 9: Annotation interface conversation states.
31

---

Published as conference paper ICLR 2026
Thanks htw.rpggiiii..eeeeeeeccceeeeeeee
weer., http://www...ccecc....eoeeoeeerr aprroprriiioooceececceeckc each query based on the required state information. Submit to save your ennotation.
868] (0000000000000000000000-00000000 imished:180 Total:126
Curzent id: 182
annotation: l'query': 'What: practical steps
transition fron nanagement policy-miking"',
Current Item
Item 101
Query
some practical steps
community management policy-making"
Required State Information:
Policy Transition Experience Level: substantial, dirrect, innellveeent
Policy Transition on Education Interest: Literastee_ in, degros
Task Select the ONE choice that matches the given query considering the required state information.
Select the Best Choice
Available Choices
Cholce 1: your extensive hands backiground, you leverage experterce ca pursuing which will deepen your theoretica, understanding and expand your professional network. Cons der applying fellowships or aravisorr roles where direct community insights will be highly valued,
public policy relared field,
Choice Since with volunteering for policy focused projects shadawing local policymakerr practical exposure and strengthen your transition.
committee experience, enralling
certificate program you lid skills credentiais, Pair
Choice 3: Building committee involvement, pursuing formal degree policy public adminiscaation will provide strong foundation and deors edvanced policy roles, Engage internships research essistantships experience with policy-making practics.
studies bridge community management
Choice 4: If you're to policy work, starting with a short-term certificatt program will introduce to concepts and frameworks. Complement studies attending public meetings or joining local advocacy groups gain fisshand insight into the policy-making process.
Additional Comments
Any additional observations about the choice annotation decision...
Previous
Submit
Next
Figure 10: Annotation interface for ground-truth judgment reliability evaluation.
32

---

Published as conference paper ICLR 2026
0 points (Not Reflected): The query is completely unrelated or may even conflict with the state, making it impossible to infer the relevant information.
Points are rescaled [0, 1] for later computations.
Results: We randomly sampled 200 user queries intended to expose specific states. Two expert annotators are assigned evaluate the queries. We found that due the high quality of state exposure, the anteroannotaoo agreement was almost perfect, with Gwet's AC1 (Gwet) 2001) coefficient of 96.8%. The average score for state exposure quality was 99.1 %, indicating that the generated queries are highly clear and effective revealing the intended user states.
Conversational State Integrity Evaluation After state introduced, is crucial that the simulated user's subsequent conversation remains consistent with that state. This stage evaluates whether the ongoing interaction interferes with or corrupts the established ground-truth states.
Methodology: As depicted in Figure annotators reviewed conversational turns and, for each predefined user state (e.g., physical activity intensity level), checked for any contradictions between the dialogue and the state's value that time. The goal was detect any information from the user simulator that would corrupt the state information.
Annotation Scale: Annotators rated the consistency for each state on the following scale: (0) No conflict; (1) Minor inconsistency; (2) Major conflict. Points are rescaled to [0, 1] for later computations.
Results: We randomly sampled 40 multi-turn conversation sessions each with multiple states annotate, resulting in 748 items in total to annotate. The evaluation yielded an average consistency score of 99.2%, with Gwet's ACI coefficient of 98.2%. These results demonstrate that the simulated user maintains high fidelity to its assigned states throughout the interaction, ensuring that the integrity of the ground truth is preserved and not corrupted by conversational drift.
Ground- Tround-Truth Judgment Reliability Evaluation. To further validate the reliability of the simulator's judgments, we randomly sampled 100 questions from the model's evaluation logs. Two independent human annotators were asked to select the correct answer for each question based on the provided context. We then calculated the agreement rates.
Results: The inter- antettannttator agreement between the two humans was 0.92, establishing a strong baseline for human consistency. Crucially, the agreement between the LLn-greerated ground-truth truth answers ("golden choices") and the human annotators was also exceptionally high, reaching 0.96 for the first annotator and 0.94 for the second.
DETAILS FOR THE SELF-EVOLUTION EXPERIMENT
Algorithm
[ describes agent's self-evolution process.
To provide a comprehensive assessment of the self-evolution process, we evaluate agents from two complementary perspectives: task-specific performance and the factual accuracy of their internal memory. (1) Task Performance: We measure the agent's ' ability to solve memory-dependent tasks using the primary metrics from benchmark suite (Section 3.3.. The Normalized Memory Score is reported at the end of each evolution cycle to track the agent's 's task-specific improvement over time.
Evaluation Metries
As complementary metric, report the score of Memory Factual Recall: We directly measure the extent to which agents successfully incorporate new information into their memory. Following methodologies in factual recall studies Min et al. (2023); Tang et al. (2024). build factual consistency checker using
33

---

Published as conference paper ICLR 2026
Algorithm 1 Memory Agent Self-Evolution Loop
1: Input: Initial policy prompt fo. Number of evolution cycles K. Initialize: Agent with policy To (P
for
K
4: Interact with the AMEMGYM environment for one episode using policy TT())).
5: Collect trajectory = {oo, 0u...., or, or and evaluation outcomes. Generate environmental feedback summary Fl based on the interaction and outcomes.
Generate the updated policy prompt: Pk+1 G(Px, FE).
Update agent's policy to I
9: end for
10: Output: Sequence of evolved prompts (P....,
......,PA P and associated performance metrics.
Feedback Type
Question Only
Complete
0.250
Memory Score
0.223
0:48
0.479
Memory Factual
0.206
0.44
0.200
0.197
0.403
0.175
6.172
0.40
No Evolutio
Qumaton Only (P5)
Complets
Qumatton Only (PS)
Complete (P5)
Complete (PD)
Completn (P10)
Quaslen Only (PM))
No Exclation
Quantoon Only
Complete
Quastion
Figure 11: Comparison of memory performance and factual recall for evolution assistants under different environmental feedback conditions.
GPT-4.1. Let Snew be the set of new user states introduced during an interaction episode, and Mmmm be the agent's memory representation at the end of that episode. The checker is prompted to evaluate each fact 8; E Snew for consistency against the memory M For each pair (#i, Mmem). the checker returns a binary judgment, j: E (0, 1). where j: 1 indicates that the fact is supported by the memory and ji = indicates otherwise. The final Memory Factual Recall score, Rsact, is the average of these individual judgments: Rfac Φ E.nj. ji.
Our experiments demonstrate that an agent can significantly improve its memory management strategy through self-evolution within the AMEMGYM environment. As shown in Figure 11 agents receiving feed- back consistently outperform the static baseline. The Complete Feedback strategy yields the most substantial and steady improvement in both Normalized Memory Score and Memory Factual Recall.
CASE STUDY: ANALYSIS OF EVOLVED POLICIES
qualitative analysis the policy prompts reveals how the agent learns to improve its memory management.
As illustrated in Table 5 the agent's policy evolves from general instructions in early cycles (P1) to highly specific, actionable rules by the final cycle (P10). For instance, a vague prompt to track "skill levels" refined into nuanced rule for capturing "teaching approaches suited to experience levels." This learning
34

---

Published as conference paper ICLR 2026
process is characterized by the emergence of new, specific schema for recurring information (e.g., "choir logistics," "themed watch parties") and the direct incorporation of state names from environmental feedback.
Table 5: Running examples of prompt evolution traces period 1 (PI), (P2), (P5), and 10 (P10).
State Schema
P5
P10
volunteering personal mobility level
Implied: "Docu- ment Detailed Plans, Implied: "Maintain Up-to- Date Health, Wellness, und Dietury Profiles: ... changes over time, including... medical considera- tions."
Explicit: "Capture Specific Personal Preferences with Contextual Sit- uational Details: .. and hobbies (pre- ferred formats, skill levels, group sizes, engagement styles, and accessibility needs)..."
Explicit: "Record Activity, Service, and Volunteering Preferences.... ac- cessibility features). hobbies and teaching approaches (skill levels.. accessibility aids)..."
l"highly mobile",, "occusional assistance needed". "limited mobil- i]
Goals, and Inten- tions with Complete Logistics and Con- tingencies: Track upcoming events... including specific logisticul detuils such accessibility considerations, and contingency plans." Implicd: "Save Pro- fessional, Mentorship, and Development Details with Learn- ing Engugement Styles: Remember ... preferred learning styles, networking involvement, ..."
mentoring delivery format l"oneon one", "small grup", "workshop series"]
Implied: "Save fessional, Mentorship, and Development De- tails: Remember preferred learning styles, and relevant networking com- munity involvement."
Implicd: "Save Pro- fessional, Mentorship, and Development Details with Leurn- ing, Engugement, and Support Styles: Remember... and mentoring activity preferences."
Explicit: "Save Professional, and Session Structures: ...Cpturre detailed session structures. preferred icebreakers,
potluck available cooking time ("limited under 2 hours", "flexible afternoon", "full day prep"]
Implied: "Docu- ment Detailed Plans, Goals, und Intentions with Logistics: Track upcoming events...
Implied: "Docu- ment Detailed Plans, Goals, und Inten- tions with Complete Logistics and Con- tingencies: Track upcoming events... inc ludinin specific logistical details such as dates, times, loca- tions,
Explicit: "Capture Specific Personal Preferences with Contextual and Situ- ational Details: ...and products (including situational factors such as event type, timing. preparation case, and sensi- tivity)."
Explicit: "Capture Specific Personal Preferences with Con- text, ... and products (situational factors such
including specific logistical details such as dates, times, loca- tions....
type. timing, preparation case, cost sensitivity, durability. user experience)."
soul food guest health goals I"general healthy eating". "weight management", "chronic condition management"]
Implied: "Maintain Up-to- Date Health, wellness goals, and any adaptations or changes over time..."
Implied: "Maintain Up to Date Health, wellness goals...
Explicit: "Capture Specific Personal Preferences
Explicit: "Main- tain Up-to-Date Health, ... symp- tom management strategies, evolving
and any adaptations or changes over time...
Contextual and Situa- tional Details: Extract explicit likes, and healtb-conscions modifications)...
health needs, and personalized wellness preferences...
crimson tide game tech setup ["basic tv livestream", "outdoor projector", "no live viewing avail- able"]
Absent
Absent
Implied: "Kecord Activity, tech- nology comfort and tools, volunteer safety checklists with tone and language prefer- ences)..."
Implied: "Record Activity. tech-
nology comfort
tools, volunteer safety checklists...)
35

---

Published as conference paper ICLR 2026
F ADDITIONAL EVALUATION RESULTS
F. EVALUATION ON Exira CONFIGURATION
As illustrated in Figure 12] simply adjusting the configurable parameters in AMEMGYM allows us to easily increase the difficulty of the evaluation environment.
Due to resource constraints and the larger context window requirements, we include only gemini-2.55ffaam lite and gpt-4.1-mini for comparison under the extra configuration. These models exhibit significantly lower memory scores of 0.137 and O. 104, respectively, compared to scores of 0.269 and 0.203 under the base setting. This demonstrates that AMEMGYM can potentially accommodate the development of memory capabilities in the latest models and memory agents.
Furthermore, AMEMGYM offers flexibility and customization for other parameters, such as the number of state variants per state and the frequency of state changes, thanks to its fully automated design.
Overall Score
gemini fissh
.240
.20
150
185
155
175
0.75
get-4.1-mini
.50
150
.50
150
150
025
Mean
12
18
Periad Index
Memory Score (Normalized)
gemini: flash
.299
.022
0.5
got mini
104
.547
.297
.375
269
.171
1005
.07
.044
.033
.014
.014
.02
.033
.022
.023
.047
.008
.045
.023
.051
.045
4
10
12
15
Figure 12: Memory evaluation results on the extra configuration.
F.2 EVALUATION WITH DIFFERENT USER LLMS
As shown Figure 13 switching the user LLM from gpt-4.1 to deepseek-v3 has minimal impact on the evaluation results. It reflects the advantage of AMEMGYM grounded interactions.
F.3 FULL FIGURE FOR DIAGNOSIS ON WRITE STRATEGIES
We present detailed diagnostic results for various write strategies in Figure 14 Due to the high information density in this figure, which can be challenging to interpret, we have transformed the data into a table in Figure 7a for improved clarity.
500
535
350
-313
540
200
po1-4.3
222
693
335
220
325
Overall Sasiallssooe Score
02843
222
775
422
335
201
313
269
147
200
2.mn-2550schh Ite
penni200lss
0055
856
655
635
520
4:5
260
275
230
325
gemini2 5etve-llee
722
155
077
1:9
0.4
gpt mi-min
-017
605
0.3
diepoeskv)
a64
560
850
00
355
265
230
260
245
325
gm-41-mmii
197
503
935
311
001
114
ceeporetiv3
165
500
254
255
14
122
053
331
011
073
1.45
167
random.
231
251
233
231
233
231
231 231 231
233
231
231
9.8
100
472
200
300
238
.016
-001
.778
Mean
Period Index
Figure 13: Memory evaluation results with deepseek-v2 v3 as the user LLM.
36

---

Published as conference paper ICLR 2026
LLM-write
AWI-write
RAG-write
AWE-write
LLM-read
AWI-read
-X RAG-read -x- AWE-read
LLM-util.
AWI-util.
RAG-util.
AWE-util.
0.5
0.4
Failure Rate
0.3
0.2
0.1
0.0
0
2
4
6
8
10
Period Index
Figure 14: Full figure for diagnosis on write strategies.
37

---

Published as conference paper ICLR 2026
F.4 EVALUATION WITH OTHER MEMORY IMPLEMENTATIONS
Table 6: Performance comparison of open- memory systems
Open-source Memory System
Overall
Memory
Memo (Chhikara et al.
2025
w/o Graph (AWE in the paper)
0,430
0.296
Mem0 Chhikara et al.
2025
w/ Graph
0.424
0.284
A-Mem (uu et al. 2025
0.378
0.220
Nemori Nan et al.
2025
0.385
0.231
Table 6 presents the performance of several open-source memory frameworks, with AWE (the baseline im- plementation described in the main text) included for comparison. For A-Mem and Nemori, we use the same embedding model and vector database as the Mem0 (AWE) implementation to ensure a fair comparison.
F.5 EVALUATION WITH SHUR-S MODELS
Table Performance comparison of open-source models
Model
Overall
Memory
gemini- 2.n-i-2..-ffaa flash (Google 2025a)
0.448
0.327
2ww5-a22bbmmsrruuse deepseek-v3.
Yang et al. 2025)
0.331
0,148
ddepseek-v3.11termmin 1-terminuss (DeepSeek 2025)
0.327
0.151
kimi-k2-instruct (Team et al. 2025
0.389
0.234
glm-4.6 Z.ai 2025a)
0.404
0.257
Table 7 presents the performance of several leading open-source models our benchmark, with gemini- f.5-flash included as the baseline from the main text.
F.6
EVALUATION STABILITY
Table 8: Performance stability across five independent runs.
Model
Overall Score
Memory Score
gpt-4o-mini
0.3178 (#0.0025)
0. 0.1504 (#0.0044)
gpt-4.1-mini
0.3675 (#0.0023)
0.2030 (40.0031)
gemini-2.5-llash-iite
0.3921 (#0.0062)
0.2583 (#0.0100) 0.4465 (#0.0037)
gemini-2.5-flash
0.3240 (#0.0056)
To assess the reliability of our benchmark, repeated the evaluation 5 times across representative sub- set of models. Table S reports the mean and standard deviation for each model, demonstrating that our benchmark produces highly stable performance estimates.
38


Reviews:
[{"review_id": "cky65sQpOH", "reviewer": "Reviewer_2oDj", "summary": "The paper introduces AMEMGYM, a new interactive, on-policy benchmark designed to  evaluate memory management in long-horizon conversational assistants. AMEMGYM differs from existing static datasets and provides a fully automated environment where LLM-simulated users engage in structured evolving conversations that reveal latent user states through role-play. Through experimental comparisons, they found that on-policy evaluation changed the rankings and scores substantially. They also show that even the latest LLMs struggle to maintain long-term conversational memory, confirming context-length degradation. Agentic-Write External systems performed best which highlights that selective, structured writing yields higher information utilization accuracy.", "questions": "* Could you provide the cost analysis?\n* Could you also provide results on popular open-source models?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "* The paper is grounded on a novel motivation: existing benchmarks mostly focus on off-policy memory evaluation which might have a gap between realistic systems.\n* The introduction of write/read/utilization decomposition is a meaningful contribution.\n* The paper also conducted extensive comparisons that shed light on long-context degradation and memory design trade-offs.", "weaknesses": "* While the paper positions on-policy evaluation as a core contribution, the empirical and conceptual justification for its advantage over off-policy settings remains underdeveloped. Although Table 2 and Figure 5 show some rank changes between on- and off-policy settings, the paper does not provide insights on why those differences matter. It remains unclear what specific behavioral aspects of “interactive memory” are uniquely captured. In fact, one could argue that well-curated off-policy datasets offer simpler, cheaper, and more reproducible alternatives, and the paper does not convincingly rule out this possibility.\n* The claimed novelty of introducing an on-policy memory evaluation environment is somewhat weakened by the existence of several recent long-horizon, on-policy frameworks: AgentGym, DeepResearch, SWE-Agent. These already feature real-time decision-making, persistent contexts, and memory management. The paper should sufficiently clarify what AMEMGYM contributes beyond these environments.\n* The entire benchmark relies on LLM-simulated users rather than real human interaction data. While this enables scale and control, it raises a question: whether simulated dialogues genuinely capture the noise, inconsistency, and ambiguity of real users. Even though the authors conduct a “meta-evaluation”, it mostly checks internal coherence rather than human-likeness or behavioral realism.\n* Although the paper evaluates several general architectures (e.g. RAG, agentic write, and long-context LLMs), it does not include direct experiments on established open-source memory frameworks such as Mem0 (Chhikara et al., 2025) or A-Mem (Xu et al., 2025), despite citing both as prior work. While the authors’ custom agentic write setups are reasonable for controlled analysis, the paper should better justify why this self-defined configuration was chosen instead of real implementations that are already widely used.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Vvv3vETHlz", "reviewer": "Reviewer_fnRa", "summary": "The work introduces an interactive environment designed to benchmark the memory capabilities of llm assistants. The authors argue that existing benchmarks are flawed because they rely on static, off-policy data. This means the assistant is evaluated on a fixed conversation history it did not create, which fails to capture how an assistant's own responses influence the dialogue and can lead to unreliable evaluations. They first generate the state of the user using structured outputs and then the simulated user uses their attributes in a natural way during the conversation. The framework also introduces diagnostic metrics that decompose memory failures into three stages: write, read, and utilization. They demonstrate that this \"gym\" can be used for agent self-evolution, where an agent uses the environment's feedback to autonomously improve its own memory-writing policy", "questions": "1. In section 5, \"Complete Feedback\" is described as including the questions, agent's answers, and ground-truth answers, which are summarized into <feedback.summary>. Could you provide an example of how this summary is formatted? Is it a natural language paragraph, structured JSON, or another format?\n2. How do you ensure diversity in the structured generation? \n\n- Some weird formatting: L373-375", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "1. The papers central critique of off-policy evaluation is compelling and well-articulated. The authors provide concrete evidence in table 2 that evaluation rankings of memory systems change when moving from an off-policy to an on-policy setup, proving that the distinction is not just theoretical but has practical consequences.\n2. The introduction of write, read, and utilization failure metrics which gives better insights into failure modes than the usual accuracy metric.\n3. The self-evolution experiment shows that agents can improve its memory policy by learning from the environment's feedback.", "weaknesses": "1. The work only evaluates memory for selecting the correct answer using multiple choice questions, but doesn't test the generation capabilities. \n2. The memory is tested using structured key-value pairs and doesn't test the episodic memory or memory where the assistant has to reason over multiple facts.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "gn07x2nNyE", "reviewer": "Reviewer_bGms", "summary": "The paper introduces **AMEMGYM**, an **interactive (on-policy)** benchmarking and optimization environment for long-horizon conversational memory. A **structured state-evolution blueprint** anchors free-form, LLM-driven role-play and enables **diagnostic** scoring with attribution to **write / read / utilization** stages. Experiments compare native LLMs, RAG, and two agentic-write variants, reveal sizable **on- vs off-policy** ranking shifts, and demonstrate feedback-driven **self-evolution** of memory policy.", "questions": "1. **Trace reusability**: Must interactions be regenerated for each system, or can exposure prompts and user policy be reused for fair comparison?  \n2. **Noise control**: How is the ratio of non-informative chatter quantified and manipulated as \\(N_i\\) grows?  \n3. **Upper bound (SUB)**: Does the UB solver access all ground-truth states plus a strong reasoner? Would an expert-solver UB better isolate utilization bottlenecks?  \n4. **Privacy**: Guidance for minimizing PII in real-user deployments?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- **Clear motivation**: Off-policy evaluations can induce reuse bias; AMEMGYM offers an on-policy, diagnostically rich setup.  \n- **Methodological novelty**: Persona/state trajectories, exposure utterances, QA variants (with reflection) enable constrained interaction and automated scoring; normalized memory score and stage-wise failure analysis are useful.  \n- **Thorough evaluation**: Quantifies on- vs off-policy discrepancies; characterizes long-horizon degradation of native LLMs; provides granular failure attributions and analyses of frequency, short-term buffers, and top-k.  \n- **Meta-evaluation**: Human ratings on exposure clarity and dialogue consistency support data quality.", "weaknesses": "- **External validity of simulated users**: Add a small human-in-the-loop comparison and a systematic study of user-LLM choice.  \n- **Broader baselines**: Include structured memory graphs/event stores, hierarchical compression, and explicit state trackers.  \n- **Leakage control**: Provide anti-leak prompt design and automatic leakage checks.  \n- **Metric reporting**: Add variance/CI and difficulty-conditioned analyses for the normalized memory score.  \n- **Scope of self-evolution**: Jointly evolve retrieval and utilization (e.g., top-k plus utilization prompting), and report stability/convergence.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "44a582xune", "reviewer": "Reviewer_rDSg", "summary": "The paper introduces AMemGym, an interactive framework for evaluating and optimizing memory management in long-horizon conversations with LLM-based assistants. It addresses limitations of static, off-policy benchmarks by enabling on-policy evaluation through simulated users and structured data. AMemGym supports memory personalization, role-play-based latent state revelation, and structured state evolution. Experimental results highlight its capability to identify performance gaps and foster the self-evolution of memory systems, making it a scalable and diagnostic tool for conversational assistant development", "questions": "See weaknesses.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. The paper introduces a novel on-policy evaluation framework, AMemGym, which effectively addresses limitations in existing memory evaluation systems.\n\n2. The integration of simulated users and structured state evolution ensures reliable and scalable assessments of memory capabilities.\n\n3. Experimental evidence supports the framework’s ability to foster memory self-evolution, offering practical insights for conversational assistant optimization.", "weaknesses": "1. The statistical information about the data is insufficiently described. For example, what categories of dialogues are included in the dataset? How many dialogues are there in total? What is the distribution of token lengths across these dialogues? These details are missing throughout the paper and should be explicitly addressed.\n\n2. The \"Memory Implementation\" section on page 6 is somewhat confusing. How exactly is the AWE method implemented? Why is parameter tuning only applied to the AWE method? Does this imply that the RAG and AWI methods do not have relevant parameters to adjust?\n\n3. It is unclear why the paper does not adopt common generation evaluation metrics, such as GPT4Judge or BLEU, for performance assessment. Instead, it uses a self-constructed memory score for the final evaluation. The rationale behind this choice should be explained in more detail, especially given the availability of well-established evaluation metrics.\n\n4. The structure of the appendix appears overly simplistic. It merely lists prompts without clearly explaining at which stage of the process each prompt was used. This lack of context makes the appendix difficult to interpret and detracts from its utility.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)