MemGen: Weaving Generative Latent Memory for Self-Evolving Agents
Authors: Guibin Zhang
Muxin Fu
Shuicheng YAN
Venue: ICLR 2026 Poster (2026)
Keywords: Agent Memory
Latent Reasoning
LLM Agent
Primary area: foundation or frontier models, including LLMs

Abstract: Agent memory shapes how Large Language Model (LLM)-powered agents, akin to the human brain, progressively refine themselves through environment interactions. Existing paradigms remain constrained: parametric memory forcibly adjusts model parameters, and retrieval-based memory externalizes experience into structured databases, yet neither captures the fluid interweaving of reasoning and memory that underlies human cognition. To address this gap, we propose MemGen, a dynamic generative memory framework that equips agents with a human-esque cognitive faculty. It consists of a \textit{memory trigger}, which monitors the agent’s reasoning state to decide explicit memory invocation, and a \textit{memory weaver}, which takes the agent's current state as stimulus to construct a latent token sequence as machine-native memory to enrich its reasoning. In this way, MemGen enables agents to recall and augment latent memory throughout reasoning, producing a tightly interwoven cycle of memory and cognition. Extensive experiments across eight benchmarks show that MemGen surpasses leading external memory systems such as ExpeL and AWM by up to $38.22\\%$, exceeds GRPO by up to $13.44\\%$, and exhibits strong cross-domain generalization ability. More importantly, we find that without explicit supervision, MemGen spontaneously evolves distinct human-like memory faculties, including planning memory, procedural memory, and working memory, suggesting an emergent trajectory toward more naturalistic forms of machine cognition.

MemGen
MEMGEN: WEAVING GENERATIVE LATENT MEMORY FOR SELF-EVOLVING AGENTS
Guibin Zhang, Muxin Fu, Shuicheng Yan National University of Singapore
ABSTRACT
Agent memory shapes how Large Language Model (LLM)-powered agents, akin to the human brain, progressively refine themselves through environment inter- actions. Existing paradigms remain constrained: parametric memory forcibly adjusts model parameters, and retrieval-basee memory externalizes experience into structured databases, yet neither captures the fluid interweaving of reason- ing and memory that underlies human cognition. To address this gap, we pro- pose MemGen, dynamic generative memory framework that equips agents with a human- cognitive faculty. It consists of a memory trigger, which moni- tors the agent's 's reasoning state decide explicit memory invocation, and a mem- ory weaver, which takes the agent's s current state stimulus to construct la- this
tent token sequence as machine-nativa memory to enrich its reasoning. way, MemGen enables agents to recall and augment latent memory throughout rea- soning, producing tightly interwoven cycle of memory and cognition. Exten- sive experiments across eight benchmarks show that MemGen surpasses leading external memory systems such as ExpeL and AWM by up to 38.22%, exceeds GRPO by up to 13.44%, and exhibits strong cross-domain generalization abil- ity. More importantly, we find that without explicit supervision, MemGen sponta- neously evolves distinct human-like memory faculties, including planning mem- ory, procedural memory, and working memory, suggesting an emergent trajec- tory toward more naturalistic forms of machine cognition. Codes are available https: /// //t////ttttt/t/ withhu........ wwom/binggeek/.. /MemGen.
INTRODUCTION
The ascent of Large Language Model (LL.M) agents marks paradigm shift across diverse domains (Luo et al., 2025b; Yang et al., 2024b; Qian et al., 2025; Singh et al., 2025; Pantiukhin et al., 2025; Ren et al., 2025). Pivotal to this success the concept of agent memory (Zhang et al., 2024b; Wu et al., 2025b), which enables LLM agents learn progressively from environmental interac- tions (Zhang et al., 2025a; Qiu et al., 2025b). Crucially, this conception of agent memory extends beyond that of conversational agents (i.e., personalized memory (Wu et al., 2025b)), whose primary role is to sustain coherence across long-horizon, multi-turn dialogues (Chhikara et al., 2025; Xu al., 2025a; Packer al., 2024; Zhong al., 2023). Rather, the scope of this paper is primarily on enabling agents to internalize experience, simulate human-like cognitive iteration, and progressively enhance problem-solving competence (Gao al., 2025).
The memory serving as this self-evolving engine typically manifests in two dominant paradigms. The isrst (1) parametric memory, which internalizes experiences by directly updating agents' pa- rameters (Yao et al., 2024; Yang et al., 2023; Zeng et al., 2023; Chen et al., 2024b; 2025). While this approach can yield substantial performance gains, its reliance on parameter modification inevitably entails catastrophic forgetting. i.e., the erosion of general knowledge (Dou et al., 2024). Conversely, the second paradigm is (II) retrieval-based memory, which externalizes past experiences into structured database, such as (i) raw trajectories (Luo et al., 2025a; Zhang et al., 2025a; Zhao et al., 2024), (ii) high-level experiences (Zhao et al., 2024; Tang et al., 2025; Fang et al., 2025; Wang et al., 2024c), and (iii) condensed skills like reusable APIs (Zheng et al., 2025) MCP boxes (Qiu et al., 2025b;a). Although this non on-invasive approach circumvents catastrophic forgetting. its efficacy is fundamentally tethered to context engineering, It adheres rigid execution pipeline, providing retrieved context to the agent without achieving the fluid, seamless integration characteristic of truly internalized memory (Su al., 2025).

---

MemGen
Parametric Memory
Latent Memory (for MemGen)
Rollout
Task Query: Which D.6, acate has the Iongest
Agent
Experience
ocavet Enn? Yoy can rea the Interret.
Policy
Reward
Data
will
Lateot
SFT, GRPO, DPO, etc.
garadign...
Voonedt hst Tasks'
Think Paradigm
Experience Retroive
External +
Formatting
Knowledge
Experience
Retrieval-based Memory
Task
Previous
Latent Space
Query
REC
Trajectories
(k++1)
(k+-2)
In-contextt
M-Mem, Memo, AWM, etc.
prompting
Condensed
Agent
Insights & Exp.
Internal Memory
External Memory
Policy
Expel, Mobile-Agent-E,
Seen lasks: PopQA. TriviaQA
RAG system:
Figure 1: The comparison among parametric memory, retrieval-based memory and MemGen. We drew inspiration from the layout presented in Figure of Li et al. (2025a).
Given these deficiencies, latent memory offers compelling alternative, leveraging latent states as machine-native high-densite medium for memory. Existing approaches either use the (i) key-value (KV) cache maintain dynamic memory set (Gim al., 2024; Jin et al., 2024: Hongkang Yang et al., 2024), yet which is primarily confined to addressing long-context issues, or (ii) latent token embeddings to store agent experiences (Wang et al., 2024b; 2025b), which still rely on invasive LLM parameter updates. More fundamentally, these methods diverge from human cognition in two critical dimensions: they lack the seamless interleaving of reasoning and memory, a process where thought and memory dynamically reshape one another, and remain largely retrieval-basee fetching memories by embedding similarity (Wang et al., 2024b) rather than generatively reconstructing them into novel, coherent insights. This leads to the pivotal research question that motivates work:
How can we architect agent memory as dynamic cognitive faculty, capable of fluid, reconstructive pro- cesses that interweave seamlessly with reasoning?
To address this challenge, introduce MemGen, dynamic and generative memory framework de- signed to endow any LLM agent with a more human- esque cognitive faculty. At its core, MemGen continuously monitors agent's s cognitive state, enabling it to dynamically invoke generative pro- cess that synthesizes a bespoke latent memory at any critical juncture during its reasoning process. Practically, MemGen comprises two synergistic components: a reinforcement learning (RL)-trained & memory trigger, which acts as a metacognitive monitor to discern the opportune moments for explicit memory invocation; and a A memory weaver, which takes the agent's current state stimulus to draw upon relevant implicit parametric memory (potentially augmented with externaly retrieved information) and then reconstructs this synthesis into succinct, machine-native latent memory. With the reasoning core fixed, MemGen inherently mitigates catastrophic forgetting when exposed to new data, and equips agents with a generative memory deeply integrated with reasoning.
Experimental Observation. Extensive experiments across nine benchmarks and four baseline categories demonstrate that MemGen delivers O substantial performance gains, with improve- ments of up to 31.7% on ALFWorld (Shridhar et al., 2021) and 27.1% on KodCode (Xu et al., 2025d) with Qwen3-8R, surpassing parametric memory (REINFORCE++, +5.8% and the GRPO method (+5.32%) 2 strong cross-domain generalization, where training in the math domain not only avoids degradation elsewhere but also boosts performance in science reasoning (+6.06%) and code generation (+5.1%); and @ continual learning ability, maintaining stable performance in previously trained domains even after fine-tuning on three additional ones.
Analysis Interpretation. Beyond quantitative evaluation, we also analyze the functional be- havior of the latent memories learned by MemGen. Through post-hoc interventions examining the impact removing specific latent memory on different agent failure modes, we found that differ- ent latent memory tokens play distinct computational roles within the agent's reasoning process, including 1 planning memory, where certain latent tokens specifically support high-level task planning, @ procedural memory, where some latent memory tokens facilitate the agent's 's recall of sask-spe procedural skills, such as tool usage and answer formatting, and @ working mem- ory, where certain tokens help the agent maintain coherence and understanding long contexts
2

---

MemGen
within single task session. These specializations strongly reveal that MemGen endows the agent with precise, functionally distinct memory.
2 RELATED WORK
LLM & Agent Memory. As outlined in $1, existing memory mechanisms designed to evolve the problem-solving capacity of L.M agents can be broadly categorized into three classes: (I) para- metric memory. which either integrates past experiences directly into agent parameters through finetuning, as in FireAct (Chen et al., 2023), AgentLumos (Yin et al., 2024), and others (Zhang al., 2024a; Fu et al.. 2025), or maintains them in external parameter modules (Tack et al., 2024; Wang et al., 2024a); (II) retrieval-baset memory, which abstracts prior experiences into trans- ferable knowledge (Zhang et al., 2025a; Zhao et al., 2024), or distills them into reusable tools and skills (Zheng et al., 2025; Wang et al., 2025c; Qiu et al., 2025b;a); and (III) latent memory, which leverages implicit representations to encode and retrieve experience (Wang et al., 2024b; 2025b; Hu et al., 2025b; Liu et al., 2024; Sun et al., 2025). Our MemGen falls within the latent memory paradigm, yet distinguishes itself from prior approaches through its interweaving of reasoning and memory, as well as generative reconstruction mechanism.
Latent Computation. Our method is also closely related to latent computation, wherein latent states are employed to intervene in or reshape the LLM's reasoning process (Zhu al., 2025). Prominent paradigms include: (I) architecturally enabling native latent reasoning, exemplified by Coconut (Hao et al., 2024), CODI (Shen et al., 2025b), LatentR3 (Zhang et al., 2025b) and CoLaR (Tan et al., 2025), which render the L.M's inference process inherently latent and machine- native; and (II) employing latent computation to steer LLM generation, as in LaRS (Xu et al., 2023), LatentSeek (Li et al., 2025a), SoftCoT (Xu et al., 2025c;b), and Coprocessor (Liu et al., 2024), which leverage latent representations to modulate the quality of generated outputs.
PRELIMINARY
Notation. We formalize the agent's interaction within an environment E. An agent, powered by an LLM parameterized by 6, is denoted as ny. For a given task x, the agent's interaction unfolds as a high-level trajectory, denoted as follows = (so, 4g. $1, AI..... (i,.....T), where s; represents the state of the environment and a, is the high-level action taken by the agent. More internally, each action a is essentially a sequence of tokens, 0 (Z,,1, .e,2..... Lt,,,), generated autoregressively by
the LLM. The generation of the j-th token is conditioned on the current state St and all previously generated tokens within that action:
Zt. To(. Z1,z),
(1)
After entire action sequence a, is generated, it is executed in the environment, which transitions the state from S, to 11+1- The success of the trajectory T is evaluated by reward function R(TT).
Problem Formalization Given history of past experiences H. {(x, 7, T the objective is to leverage this history to maximize the agent's 's performance on new tasks. The policy Ty and a memory system M are thus jointly optimized to maximize the expected reward over task distribution D:
max Ez=D, T [R(7)], 6.M
during which M is produce memory representation, m, which conditions the agent's policy. The action at any timestep t thus sampled as a ~ T &t. m.), where m, is the inserted memory at that step. Crucially, the nature and timing of memory generation, which we denote the function fM. vary across different paradigms. We express the generation of the memory m, as:
m, fM m.).
(3)
which accommodates diverse memory invocation granularities. For task-level memory (e.g., Ex- pel (Zhao et al., 2024) and G-Memory (Zhang et al., 2025a)), J' is invoked only mo for all subsequent steps. For step-level memory (e.g., AgentKB (Tang et al., 2025)), fM is invoked every step to update the memory. In parametric memory, the influence of H compiled into 0, rendering memory generation implicit in the model parameters. Our work, which introduces dynamic latent memory, focuses on designing a more fine- fine-grained fM that decides for itself the optimal moments to regenerate m, at the token level during the agent's 's reasoning process.
0, and
3

---

MemGen
Explicit Reason
Reaonser Tt
Task Query: lo, please flight
Friday
(should be earlier than 12ar)), and book ride from the airport.
okay, to achíeve this, 1'11 start by searching Flight on Google Flights.
cheapest
aannl
watt, 1 can do this faster by..
Obs.
Env.
the API that not usable.
Latent
Context
Trigger T
Context
7(H) = No Memoor
Notations
Menory Insert
Prompt
Completion
tokenss
Fnkoons
T: R {o,
Implicit Memory
memsory tokeens
Token Sequenee
Token Sequence
Hug g
T(B)
Invocation
Memory
Latent
sequence M
Latent
Memory
He:y 2
Memory y Inseert
Invocation
Weaver W
Memory Model R
Externol Mem DB
Memory
W:
Purumetric
Raw Trajectories (o...))....))) High-level abstracts Kate (+ f
W R
Memory
Latest Memory
Memory
M
Rich
M RRD
Figure 2: The overview of our proposed MemGen.
METHODOLOGY
4.1 MemGen: INTERLEAVING MEMORY AND REASONING
Just person is the sum of their past experiences (McAdams, 2001), memory profoundly shapes an agent's behavior (Xiong et al., 2025). Yet existing agent memory systems lack the adaptive fluidity characteristic of human cognition. Human reasoning and recollection operate in a continuous interplay, whereas most agent memory frameworks retrieve information only once at task initiation and append it to the query in a coarse and static manner. MemGen is designed to close this gap by enabling memory to participate dynamically the reasoning process.
As shown in Figure 2, the reasoning process in agent equipped with MemGen unfolds autore- gressively, driven by frozen core LLM, the reasoner Tg. For a given state 8t, my generates action [Z......., L., ). MemGen continuously monitors the token-by-token generation process and performs on-demand memory insertion. At each eoken-generatin step j. a memory trigger Tuigggrr monitors the reasoner's internal cognitive state determine if moment of reflection, i.e., memory invocation, is necessary. Specifically, as the reasoner generates the token sequence 2,,xj- produces corresponding sequence of hidden state vectors, H, (h..,....,, ht.j-),, where each he E I dudke The trigger takes the current hidden states Hr,ej as representation of the reasoner's current metacognitive state and computes an invocation probability:
P; = (Triigger (h.,1...., b.))
(4)
from which binary decision, d, Bernoulli(ps) (INVOKE, SKIP), is sampled. If the decision to [SKIP], To proceeds with its standard autoregressive generation, i.e., 2t,j ~ To 8t, 2,,s)) However, the decision is to INVOKE, the reasoning process is momentarily paused. This summons the second core component of our framework: the memory weaver Wweaver. which takes the same cognitive state Hi,ej as stimulus to perform generative act of recollection. It synthesizes bespoke, machine-natee nativve latent memory, formalized as M, E I
K Xdndti with fixed length K:
M [ 14 _m,k] = Wweaver (Weaver(HH,<.)))
(5)
where the memory generated not merely from the parametric knowledge encoded within Wweaver but may also incorporate cues retrieved from external memory databases (detailed implementation is elaborated in $4.3). Crucially, M, is not verbatim restatement of prior content but selective reconstruction, filtered and integrated through Wweasers akin to the memory consolidation process in human brain (Spens& & Burgess, 2024). Once formed, the latent memory is woven seamlessly into reasoner's ongoing dynamics: its hidden states are prepended H, upon which the reasoner resumes generation conditioned on this enriched context,
Zt.j no( (.1 8t, Zt, M,).
4

---

MemGen
This iterative eycle of generation, monitoring, invocation, weaving, and reintegration elevates rea- soning from a linear unfolding to a recursive dialogue with memory, all without altering the frozen reasoner Ty. and thereby preserving its general capabilities. In the following sections, we detail the implementations of the memory trigger (t $4.2) and the memory weaver (> $4.3).
4.2 MEMORY TRIGGER: LEARNING TO INVOCATE MEMORY
In this section, we describe the concrete instantiation and training recipe of the memory trigger Trigger. Recall from $4.1 that Trigger serves as metacognitive monitor, observing the evolving rea- soning state of the frozen reasoner Th and producing a binary decision d, E {INVOKE, SKIP [( <> SKIP, 1 ++ INVOKE) that determines whether the memory weaver should be invoked at token j.
Instantiation. We instantiate Trigger as a lightweight LoRA adapter attached to the reasoner na- At the decoding step j of the timestep t, it receives the sequence of all hidden states, Hi,< E R(--1)xdaaat conditioned conditiooe on this context, Trigger outputs the action probability P(d, = INVOKE) E [0, We use hidden states rather than textual context input because, following prior work in latent reasoning (Hao et al., 2024; Shen et al., 2025a), latent embeddings retain richer context- sensitive information that would otherwise be lost after softmax decoding. To avoid excessive computational overhead, we adopt a sentence garanlaarrr activation strategy, inspired by recent studies on LLM interpretability (Anthropic, 2025: Chen al., 2024a), which find that interventions between sentences can more effectively guide LLMs' reasoning path. Specifically, we define delimiter token set 'D (e.g., commas, periods) and let the trigger act only when the current token falls in D. The invocation decision is computed as:
00 dj Bernoulli(p)). Pj 1 Taiggerr (H,e))
if $ D, zf E ED,
which ensures that Tuigger is invoked only at semantically significant boundaries, preserving decod- ing efficiency. We validate that MemGen does not incur excessive inference delay in Appendix E.3.3.
Training Recipe. The memory trigger is trained via reinforcement learning, motivated by the need to balance two competing desiderata: ensuring that critical latent memories invoked to improve task performance, while avoiding unnecessary or spurious invocations that could disrupt reasoning incur computational overhead. Given batch of seen tasks H {(x,, Tx) ) the frozen reasoner generates candidate trajectories while the memory weaver Wweaver remains fixed. At each activated step, the trigger selects an action d, E { INVOKK,, SKIP and receives a reward r(T,) reflecting the quality of the resulting trajectory with respect the task objective. To encourage sparse yet strategically critical memory invocation, we introduce a awarddaddapive penalty, which discourages unnecessary activations while preserving essential ones, into the objective:
max E
H(T.)
Σ max(0, de, - -p)
(8)
TLNNT To
where p is computed as the mean activation probability across high-reward trajectories, iee,, those with reward exceeding the batch median:
p
(Higgh
Jes. Hnigh R(T) 2 median, (N(r))))
(9)
FE #iiah
where ensures that Tuigger learns to invoke memory selectively at key decision points, maximizing task reward while maintaining computational efficiency.
4.3 MEMORY WEAVER: SYNTHESIZING AND INSERTING LATENT MEMORY
In this section, we elaborate on the weaver Wwaserr the memory carrier within the MemGen frame- work. When the agent assimilates new experiences, this information is exclusively internalized into the parameters of Wwesvers leaving the core reasoner entirely unmodified. At junctures where the reasoner requires experiential support, context-dependent hook activates the weaver to synthesize and externalize pertinent knowledge as usable memory. To be more specific, recall from Equa- tion (5) that after the Trigger signals the need for memory at step j. Wweaver accepts H,,y (as the hook) and generates latent token sequence M, (as the memory) for ne.
5

---

MemGen
Instantiation. We instantiate Wweaver using anthoer LoRA adapter attached to Te. Formally, given the incoming hook Hr E (RR(1)) Xcmute, the weaver outputs a latent memory matrix: M, Weaver (H,,s)) E R R-xxdusat, where K denotes the fixed length of the latent memory sequence and O' are the trainable LoRA parameters. The synthesized M, is first projected through a linear layer to align it with the reasoner's 's token- ooken-embedding space. and then prepended the current hidden states of ITy to guide subsequent token generation, as described in Equation (6).
W
Training Recipe. The training of Wweaver proceeds over batch of past trajectories H
{( J. Distinct from conventional agent tuning, which directly integrates experiential data into the parameters of тe (Chen et al., 2025; Yin et al., 2024), MemGen internalizes experiential knowledge solely into Wweaver, which ensures that To 's general capabilities remain intact.
Crucially, this separation makes MemGen agnostic to optimization strategies and compatible with diverse LLM backbones. Whether employing supervised fine-tuning (SFT) or RL-based objectives such as GRPO or DAPO, the weaver can be updated under a unified goal: optimizing the generation
process of latent memory so as to maximize downstream reward. Formally, let ITg "T."" (. x>) denote the process of rolling out trajectory for a task by Ty in conjunction with weaver We and trigger T. Given a reward functional Rt the objective updates only 0' by maximizing the expected reward:
max B( E. Okn
"r-nl Forn" (-1x;) [R(x,,
(10)
where the gradients from R are propagated solely to 6', thereby equipping the weaver supply precisely the memories that improve end-to-end performance without altering πy. Equation (10) enables Wwewer to absorb diverse experiential signals and externalize them as dynamic, context- sensitive latent memories, independent of the architectural or training paradigm of the base reasoner.
In practice, we first train the memory weaver using a random inserter as a lightweight surrogate for the trigger, and then freeze the trained weaver and proceed to train the trigger. For a complete description of the training procedure, please refer to Appendix D.I.
Integration with Retrieval-based Memory. Although the memory generation above primarily draws on the weaver's parametric knowledge, it can be combined with external memory sources. When triggered, any retrieval-base system (e.g., MemoryBank ExpeL) can provide textual mem- ory, which is merged with the hook Hı,ej and fed into W to produce latent memory. This allows W to integrate internal knowledge and external information, supplying the reasoner with richer memory support. Implementation details and results are placed in Appendix F.
5 EXPERIMENTS
In this section, we conduct extensive experiments to answer the following research questions:
(RQ1) Can MemGen surpass both parametric and retrieval- based memory?
(RQ2) Is the memory learnt by MemGen generalizable across task domains? And why?
(RQ3) Can MemGen facilitate continual learning and mitigate catastrophic forgetting?
(RQ4) Does MemGen implicitly evolve human-like memory hierachy?
5.1 EXPERIMENTAL SETUP
Evaluation and Benchmarks. Our evaluation covers nine datasets from five domains, including web search: TriviaQA (Joshi et al., 2017) and PopQA (Mallen et al., 2023); @ embodied action: ALFWorld (Shridhar et al., 2021); @ math reasoning: AQuA (Ling et al., 2017), GSMSK (Cobbe et al., 2021), and MATH (Hendrycks et al., 2021); @ scientific reasoning: GPQA (Rein et al., 2023); and ① coding: KodCode (Xu et al., 2025d) and BigCodeBench (Jain et al., 2024).
Baselines. We compare MemGen against twelve baselines, categorized into four groups: (I) Prompt-based methods: Vanilla model, CoT (Wei et al., 2023); (II) Parametric memory, where experiential knowledge directly modifies model parameters via: SFT, GRPO (DeepSeck-Al et al., 2025), REINFORCE (Williams, 1992), REINFORCE++ (Hu al.. 2025a), Agent-FLAN (Chen et al., 2024b); (III) Retrieval-based memory, where processing tasks sequentially and storing the experiences an external database, represented by MemoryBank (Zhong et al., 2023), Expel. (Zhao et al., 2024), Agent Workflow Memory (AWM) (Wang et al., 2024c); and (IV) Latent computation, where leveraging latent tokens as carriers of experiential knowledge, including SofTCoT (Xu et al., 2025c) and Co-processor (Liu et al., 2024).

---

MemGen
Table 1: Results on SmoILM3-3B and Qwen3-8B. All values represent the performance metric for
each task (e.g., accuracy %). We highlight the best and second best results.
Backbone
Method
ALFWorld
TrivialQA
PopQA
KodCoode
BigCodeBlench
GPQA
MATIΙ
Vanilla
18.96
10.47
8.23
37.05
35.96
9.35
47.63
16.22
9.95
38.45
20.70
56.33
SFT
32.36
55.25
37.22
59.25
40.79
19.70
63.48
45.65
GRPO
55.35
65.88
45.16
68.48
72.44
22.73
80.03
61.23
REINFORCE
53.13
63.20
46.81
65,53
14
23.44
82.03
58.75
REINFORCE++
53.95
63.20
44. 10
65,90
68.80
22.73
81.50
59.89
Smo 1M3-33
Agent-FL.An FLAN
34.00
56.70
39.50
56.80
37.20
17,80
59.60
36.84
Expel.
36.18
46.20
28.16
51.14
40.22
15.15
56.23
38.11
MemoryBank Bank
32.80
43,30
25.81
44.50
31.80
10,20
58.30
43.53
AWM
40,50
49,80
29.60
SoftCoT
34.90
59.20
39.10
17:22
44.62
Co: po-processor
38.36
53.28
38,96
56.25
45.40
20.
57.60
38.81
MenGen
50,60
68.13
42.34
62,65
42.99
26.75
70.42
57.44
Manden
63.60
79.30
$8.60
72.85
74.24
25.20
83.47
63.65
49.30
Col
$7.10
$3.80
33.20
51.25
35.59
35.15
87.67
78.24
SETT
83.59
74.55
$1.12
64.75
41.33
40.33
90.76
81.35
GRPO
85.60
76.15
$8.90
73.35
70.24
39.54
92.30
83.54
REINPORCE
82.10
75.22
57.96
72.11
70.20
37.12
91.25
83.27
REINTORCE++
84,80
75.90
$8,30
72,90
71.88
37.68
91,90
85.24
Qwen3
Agent-FL.An FLAN
80.32
70.32
50.08
62.99
43.40
39.50
87.60
80.05
Expel_
78.97
65.54
40.33
$7,20
34.23
35.15
86.20
77.40
McmoryBane Bank
70.41
60.56
41.60
56.39
40.61
35.66
90.35
80.35
AWM
80.33
69.30
43.69
SofCoT
75.60
59.42
39.42
63.28
38.27
39.60
86.30
76.23
Cosprocessor
73,28
61.42
45.55
64.90
42.19
39.15
76.23
79.20
MenGen
85.82
77.22
54.65
66.15
40.35
(43.23
91.25
83.30
ManGen
90.60
80.65
62.30
76.16
75.56
40.24
93.20
88.24
Implementation Details. We select LLM backbones of varying sizes and sources, including 2. (Yang et al., 2024a), HuggingFace's SmolLM3 3B (HuggingFace, 2025), and Qwen3-88 3-BB (Yang al., 2025). The length of each latent memory sequence K is set among {2, 4. 8}. As described in Equation (10), MemGen does not rely on a specific optimization algorithm, so we implement two variants: MemGen SFT and MemGen GRPO, in which the weaver is updated using SFT and GRPO signals. Details on these variants are provided in Appendix C. More training setup and parameter configurations are listed in Appendix D).
5.2 MAIN RESTULS
[For RQIJ MemGen provides high- seeh-perforiing memory across domains. As shown Tables and 4, existing baselines exhibit clear limitations in cross-domain adaptivity. Retrieval-based memo- ries (e.g., (e.g.,ExpeL. MemoryBank, AWM) occasionally surpass parazmetric tuning in embodied action; for instance, AWM reaches 36.18% ALFWorld with Smol LM3 3B, exceeding SFT by 3.15%. Yet their effectiveness deteriorates on bassoning-intensiv tasks: Expel. achieves only 8.12% on GPQA+Qwen2. 5-1. 53, and even underperforms the vanilla model by 6.9% on TriviaQA. under- scoring its heavy reliance on backbone capacity. Parametric finetuning methods display the opposite tendency: they excel in structured domains such as code generation, where REINFORCE+ reaches 63.33% on KodCode with Qwen2. 5-1 5B, but remain weak in knowledge-intensive intensive reasoning, with GPQA below 14%. In contrast, MemGen consistently advances performance across all do- mains. For example, on ALFWorld+ and 63.60%, improving over vanilla by 31.66 6.4% and 44. 44.64%, respectively. Similar gains appear with the larger Qwen3-88 BB, where MemGen GRPO achieves +27.06% on KodCode and +28. 17% on PopQA, surpassing GRPO by up to 3.4%. Overall, the dynamic memory insertion of MemGen delivers substantial improvements across diverse task domains.
LM3 3B, MemGen SFT and MemGen GRPO attain 50.60%
FFor RQ2] MemGen Exhibits Strong Cross-Domain Generalization. To evaluate whether the memory learned by MemGen can transfer across tasks, we train MemGen on one dataset and test it on several others. We include two out-of-domma datasets, ScienceWorld (Wang et al., 2022) and FEVER (Thoree et al., 2018), to further probe this. As shown in Figures 3, 9 and 10, baselines such as SFT and MemoryBank achieve gains within their training domains (e.g., on ALFWorld, SFT + +1.11% and MemoryBank +5.4% compared with vanilla), yet fail to generalize, with performance

---

MemGen
Trained on ALFWorld
Trained on TriviaQA
AL.Woordd
SFT
ALFWorld
SFF
MeneyBeck
MemorBank
TriviqQA
ManGee
TriviaQA
Memaam
World
ScienceWorld
FEVER
FRVER
500
Figure 3: The generalization study of MemGen. We train MemGen SFT on one dataset (ALFWorld or TriviaQA) and evaluate on four datasets (TriviaQA, ALFWorld, Science World, and FEVER).
dropping sharply on FEVER by 16.2%. In contrast, MemGen not only attains substantial in-domain improvements (24.55% + 58. 16% on KodCode, Figure 10), but also exhibits effective transfer:
when trained KodCode, performance on MATH rises from 36.6% + 54.2%. Having empirically validated MemGen's generalizability. a question naturally arises: what underlies this capability?
[For RQ2] The Memory Trigger Intelligently De- termines When to Activate Memory Insertion, Mit- igating Domain Conflict. After training MemGen on GSM8K, evaluate 150 samples each from GSM8K, KodCode, and GPQA, visualizing the frequency with which the memory trigger invoked the memory weaver at each relative position in the model output. We ob- serve that the invocation frequency varies across domains and correlates directly with performance in Figure 9: GSM8K exhibits the largest improvement (+19.64%) and maximal invocations, GPQA achieves moderate gains (+6.06%) with medium invocations, and KodCode shows the smallest improvement (+-3. 1%) with the fewest invo- cations. This indicates that MemGen autonomously assesses, based on task-specific context, when memory insertion will be beneficial, invoking the weaver less frequently in unfamiliar domains.
Invocation Count
GSMRK (hoge $6.07)
GPQA (Avg 46.9t)
KadConke (Ange 21.6)
80
60
(%)
Figure 4: Memory invocation frequency across benchmarks inferenec (trained on MemGen + +GSM8K).
[For RQ3] MemGen Effectively Mitigates Catastrophic Forgetting. train on four datasets and evaluate on all benchmarks after each stage, where MemGen exhibits stronger knowledge retention ability compared to bascline methods. For example, unlike SFT which primarily improves performance on the most recent task (54.10% KodCode but only 2.53% on GPQA), MemGen MemGen demonstrates more balanced cross-task generalizzzinnnn attaining 38, 38%43% on AQuA and 21.72% GPQA after GSM8K training. Finally, it mitigates forgetting serving 40.3 40.34% on AQuA following KodCode training compared to 27. 14.14% for Expel. and 28.61% % for SFT, indicating more stable continual learning ability. More analysis in Appendix E.
Table 5, we sequentially
earlier tasks, pre-
5.3 FRAMEWORK ANALYSIS
Having established the expressive capabilities MemGen, we further investigate its underlying mechanisms: what do the learned latent memories look like? Do they have specialized functions?
[For RQ4J The Latent Memory Is Machine-Native and Human-Unreadable. We first visu- alized the latent memory sequences learned by MemGen across differrnt datasets using t-SNE in Figures 5 and 11. As shown in Figure (Left), sequences from distinct domains form separate distributions, with related domains clustering closely (e.g., KodCode and BigCodeBench, GSM8K and MATH). Examining latent memories within the same dataset, we observed pronounced clus- tering patterns (as shown Figure (Middle and Right). To explore potential commonalities within these clusters, we forcibly decoded the latent tokens. Although the decoded sequences are human readablee they exhibit intriguing regularities: many tokens within a cluster share struc- tural conventions. For example, Cluster 0 in TriviaQA frequently follows the pattern "I "[... SOC", whereas Cluster in GSM8K often adopts the format " [[... pick". A large corpus of latent memory tokens is provided in Appendix G. Despite these sequences being machine-native and human-unreadable,, we further investigate whether their underlying semantics can be interpreted.
[For RQ4] MemGen Implicitly Learns Human-like Memory Hierarchy. To uncover the func- tional roles of different latent memory clusters, we conducted a post-hoc intervention study. Follow- ing the taxonomy from (Song et al., 2025), we study eight distinct types of agent failure, including
8

---

MemGen
Latent Memoory Virmalization
Latent Memorr Viuunlization
Latent Memury Visualization (on GSMMBS)
(acrosa all benchnurrk)
(oe TrivissAA)
GLSMIK MATII CITGA
Clutern
Chuders
Chatrr
Chite
I-SND Comsprornt
MATTH
AlPMield
BCB
201
YeiraQu
p-000 1-000 Chopsooeeee
TrMSoOA
1177
Sindergeta
Kodccale
ALFWorrdd
Clatero
Chsters
pi
L-STE Componiut
1-5NI Composeen11
Figure 5: (Left) t-SNE visualization of latent memories generated by MemGen +Qwen 8B across datascts; (Middle and Right) Latent memory visualization within the TriviaQA and GSM8K datasets, clustered using K K-means. The text at each cluster center represents the common pattern shared by many memory sequences in the cluster, such as Cluster O in GSM8K, where many sequences end with check".
68
KodCode TriviaQA
Vanilla MemGen
w/lo Cluster
w/o Cluster
w/o Cluster
1e/l Chuster
66
Planning Memory
Procedural Memory
Working Memory
Performance
64
Error Count
62
10
60
58
38
Planing Paflere
Componitional Beaaoning
Tool Response Ertor Tool Parsing Heror Answer Formanting Faiinee Demand Kissnddrstanding
Think- Art Inconsintency
Filse Delief (ToM))
Latent Token Length
Figure 6: (Left) Parameter sensitivity analysis on the latent memory length K; (Right) Effects of selectively removing latent memory clusters on different agent failure modes the TriviaQA dataset.
planning errors, tool response/parsing failures, answer formatting mistakes, etc, providing a struc- tured framework to assess how memory influences performance. During evaluation, we selectively removed latent tokens close to a specific cluster while keeping others intact, measuring the resulting changes in these failure modes. Details on (1) the visualization process, (2) failure mode annotation, and (3) token filtration are in Appendix H. As shown in Figure 6 (Right), distinct memory clusters exhibit varying influence on failure modes and can be mapped different memory functions:
Planning Memory supports high-level task planning and strategic reasoning. Removal of Cluster 2 substantially increases planning and compositional reasoning failures, indicating that this cluster is crucial for guiding the L.M agent's decision- maainng and sequencing of reasoning steps.
Procedural Memory captures task-specific operational knowledge, such as tool usage and for- matting ability. Cluster corresponds to this role, as its removal leads to a marked increase in tool response errors, parsing failures, and answer formatting mistakes.
Working Memory manages the retention and effective use of prior context to maintain reason- ing consistency. Clusters 1 and 4 contribute to this function: for instance, removing Cluster I's memory tokens results in more frequent task misunderstandings and think-act inconsistency.
Nevertheless, these memory clusters are not entirely independent: for example, removing Cluster also negatively affects planning ability. indicating that these memory faculties interact and jointly enable the I.IM to leverage past experience effectively. This analysis reveals that MemGen sponta- neously organizes latent memory into a structured, human-like hierarchy.
Ablation Study & Sensitivity Analysis. We conduct a sensitivity analysis on the length of the latent memory sequence K, as shown in Figure (Lefi). can be observed that as the latent token length increases from 2 + 32, MemGen 's performance correspondingly improves, likely reflecting the expanded memory capacity. We then perform an ablation study on the memory trigger module in Table 6, demonstrating the necessity of dedicatedly trained trigger for effective memory invo- cation. Furthermore, we analyze different training paradigms of the memory weaver in Table 7. For additional results and discussions, please refer to Appendix E.3.

---

MemGen
Efficiency Analysis. To confirm that the memory insertion process of MemGen does not introduce significant inference overhead, we show in Appendix E.3.3 that, while achieving up to 57.66% performance improvement below the baseline, ranging from 24% to 94% of the vanilla L.M latency. This clearly demonstrates that MemGen delivers substantial performance gains without compromising efficiency.
vanilla LLMs, the per-query inference delay remains consistently
CONCLUSION
In this work, we introduced MemGen, a dynamic and generative memory framework designed for LLM Agents. By interleaving reasoning with memory synthesis through a reinforcement-learee memory trigger and generative memory weaver, MemGen transcends the limitations of paramet- ric and retrieval-based paradigms. Extensive experiments showcase substantial performance gains, robust cross- rross-domain generalization, strong continual learning ability, and MemGen 's explicitly mod- eled memory hierarchy (i.e., planning, procedural, and working memory). These results suggest promising path toward self-evolving LLM agents capable of fluid and reconstructive intelligence.
ETHICS STATEMENT
This work presents a latent memory architecture designed for LLM agents. All experiments are con- ducted publicly available academic benchmarks across reasoning, scientific problem-solving and embodied tasks, without deployment real-world decision- making or safety-critical applications. Therefore, believe this research does not pose direct ethical risks.
REPRODUCIBILITY STATEMENT
We aim to ensure the reproducibility of our work by releasing anonymous repository linked in the abstract, as well detailing experimental settings (hyperparameters, L.M backbones, etc.) in $5.1 and appendix D.
CONTRIBUTIONS
G. Zhang and M. Fu contributed equally to this work. G. Zhang led the initial framework imple- mentation, manuscript preparation, and experimental analysis. M. Fu made key contributions to improving the efficiency of MemGen and conducted large-scale experiments and visualization. S. Yan is the corresponding author.
ACKNOWLEDGEMENTS
This research was supported in part by the NUS Start-up Grant (A-0010106-00-00) and by the National Natural Science Foundation China under Grant No. 62320106007.
REFERENCES
Anthropic. On the Biology of Large Language html, 2025. [Accessed 24-08-2025].
Tianle Cai, Yuhong Li, Zhengyang Geng, Hongwu Peng, Jason D. Lee, Deming Chen, and Tri Dao. Medusa: Simple Ilm inference acceleration framework with multiple decoding heads, 2024. URL
https: /arxvv org/ a /20.s./411... 10774.
Baian Chen, Chang Shu, Ehsan Shareghi, Nigel Collier, Karthik Narasimhan, and Shunyu Yao. Fireact: Toward language agent fine-tuning, 2023. URL https:://axxi......//// org/ htsgg/abs/33e 05915.
Guoxuan Chen, Han Shi, Jiawei Li, Yihang Gao, Xiaozhe Ren, Yimeng Chen, Xin Jiang, Zhenguo Li, Weiyang Liu, and Chao Huang. Seplim: Accelerate large language models by compressing one segment into separator. arXiv preprint arXiv:2412. /2094, 2024a.
10

---

MemGen
Zehui Chen, Kuikun Liu, Qiuchen Wang. Wenwei Zhang, Jiangning Liu, Dahua Lin, Kai Chen, and Feng Zhao. Agent-flam: Designing data and methods of effective agent tuning for large language models, 2024b. URL hwtps:ttt///tttte
org/ abs http://b//2200/0 22403.403. 12881.
Zhixun Chen, Ming Li, Yuxuan Huang, Yali Du, Meng Fang, and Tianyi Zhou. Atlas: Agent tuning via learning critical steps, 2025. URL, http:::tt///ot //apxis/////vvv./// org/ abs 2503. 020 97.
Prateek Chhikara, Dev Khant, Saket Aryan, Taranjeet Singh, and Deshraj Yadav. Mem0: Building production-ready ai agents with scalable long-term memory, 2025. URL https://arxxe
htt///b//220...9490
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reüichiro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:21
2021.
DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang. Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangbo Hao, Guanting Chen, Guowei Li, H. Zhang. Han Bao, Hanwei Xu, Haocheng Wang. Honghui Ding, Huajian Xin, Huazuo Gao. Hui Qu, Hui Li, Jianzhong Guo, Jiashi Li, Jiawei Wang, Jingchang Chen, Jingyang Yuan, Junjie Qiu, Junlong Li, J. L. Cai, Jiaqi Ni, Jian Liang, Jin Chen, Kai Dong, Kai Hu, Kaige Gao, Kang Guan, Kexin Huang, Kuai Yu, Lean Wang. Lecong Zhang, Liang Zhao, Litong Wang, Liyue Zhang, Lei Xu, Leyi Xia, Mingchuan Zhang, Minghua Zhang, Minghui Tang, Meng Li, Miaojun Wang, Mingming Li, Ning Tian, Panpan Huang. Peng Zhang. Qiancheng Wang, Qinyu Chen, Qiushi Du, Ruiqi Ge, Ruisong Zhang, Ruizhe Pan, Runji Wang, R. Chen, R. L. Jin, Ruyi Chen. Shanghao Lu, Shangyan yn Zhou Shanhuang Chen, Shengfeng Ye, Shiyu Wang. Shuiping Yu, Shunfeng Zhou, Shuting Pan, S. S. Li, Shuang Zhou, Shaoqing Wu, Shengfeng Ye, Tao Yun, Tian Pei, Tianyu Sun, Wang. Wangding Zeng. Wanjia Zhao, Wen Liu, Wenfeng Liang, Wenjun Gao, Wenqin Yu, Wentao Zhang. W. L. Xiao, Wei An, Xiaodong Liu, Xiaohan Wang, Xiaokang Chen, Xiaotao Nie, Xin Cheng, Xin Liu, Xin Xie, Xingchao Liu, Xinyu Yang, Xinyuan Li, Xuecheng Su, Xuheng Lin, X. Q. Li, Xiangyue Jin, Xiaojin Shen, Xi- aosha Chen, Xiaowen Sun, Xiaoxiang Wang, Xinnan Song, Xinyi Zhou, Xianzu Wang, Xinxia Shan, Y. K. Li, Y. Q. Wang. Y. X. Wei, Yang Zhang, Yanhong Xu, Yao Li, Yao Zhao, Yaofeng Sun, Yaohui Wang. Yi Yu, Yichao Zhang, Yifan Shi, Yiliang Xiong, Ying He, Yishi Piao, Yisong Wang, Yixuan Tan, Yiyang Ma, Yiyuan Liu, Yongqiang Guo, Yuan Ou Yuduan Wang, Yue Gong, Yuheng Zou, Yujia He, Yunfan Xiong, Yuxiang Luo, Yuxiang You, Yuxuan Liu, Yuyang Zhou, Y. X. Zhu, Yanhong Xu, Yanping Huang, Yaohui Li, Yi Zheng, Yuchen Zhu, Yunxian Ma, Ying Tang, Yukun Zha, Yuting Yan, Z. Z. Ren, Zehui Ren, Zhangli Sha, Zhe Fu, Zhean Xu, Zhenda Xie, Zhengyan Zhang, Zhewen Hao, Zhicheng Ma, Zhigang Yan, Zhiyu Wu, Zihui Gu, Zijia Zhu, Zijun Liu, Zilin Li. Ziwei Xie, Ziyang Song, Zizheng Pan, Zhen Huang, Zhipeng Xu, Zhongyu Zhang, and Zhen Zhang. Deepseek-rl: Incentivizing reasoning capability in llms via reinforce- ment learning, 2025. URL https: // //tpx//avvvv//////////e/e/.aaavl org/ abs/ 2501 12948.
Shihan Dou, Enyu Zhou, Yan Liu, Songyang Gao, Jun Zhao, Wei Shen, Yuhao Zhou, Zhiheng Xi, Xiao Wang, Xiaoran Fan, Shiliang Pu, Jiang Zhu, Rui Zheng, Tao Gui, Qi Zhang, and Xuanjing Huang. Loramoe: Alleviate world knowledge forgetting in large language models via moe-style plugin, 2024. URL https: //tp///aa/vvvv/////////ea//aaavv org/ alg//bb//// 22122111222 09979.
Runnan Fang, Yuan Liang. Xiaobin Wang, Jialong Wu, Shuofei Qiao, Pengjun Xie, Fei Huang, Huajun Chen, and Ningyu Zhang. Memp: Exploring agent procedural memory. arXiv preprint arXiv:2508. 254.035500.04333. 2025.
Dayuan Fu, Keqing He, Yejie Wang, Wentao Hong, Zhuoma Gongque, Weihao Zeng, Wei Wang, Jingang Wang. Xunliang Cai, and Weiran Xu. Agentrefine: Enhancing agent generalization through refinement tuning. 2025. URL https: h/tp://axvvvvvo/////ee org/ abs////////// /aab////1.... 01702.
Yichao Fu, Peter Bailis, Ion Stoica, and Hao Zhang. Break the sequential dependency of Ilm infer- ence using lookahead decoding, 2024. URL /tt.tts//////iive org/ abs abs//402. 2202 02057.
11

---

MemGen
Huanang Gao, Jiayi Geng, Wenyue Hua, Mengkang Hu, Xinzhe Juan, Hongzhang Liu, Shilong Liu, Jiahao Qiu, Xuan Qi. Yiran Wu, Hongru Wang, Han Xiao, Yuhang Zhou, Shaokun Zhang, Jiayi Zhang, Jinyu Xiang, Yixiong Fang, Qiwen Zhao, Dongrui Liu, Qihan Ren, Cheng Qian, Zhenhailong Wang, Minda Hu, Huazheng Wang, Qingyun Wu, Heng Ji, and Mengdi Wang. A survey of self-evolving agents: On path artificial super intelligence, 2025. URL https: (ttpsvv..rg/abb/2277220000
In Gim, Guojun Chen, Seung-seob Lee, Nikhil Sarda, Anurag Khandelwal, and Lin Zhong. Prompt cache: Modular attention reuse for low-latency inference. Proceedings of Machine Learning and Systems, 6:325-338, 2024.
Raghavv Goel, Sudhanshu Agrawal, Mukul Gagrami, Junyoung Park, Yifan Zao, He Zhang, Tian Liu, Yiping Yang, Xin Yuan, Jiuyan Lu, Chris Lott, and Mingu Lee. Vocabtrim: Vocabulary pruning for efficient speculative decoding in llms, 2025. URL httpst:/teeeeee /ttp:///xxvvvvv/ org/abs/ 2506. 22694.
Sachin Goyal, Ziwei Ji, Ankit Singh Rawat, Aditya Krishna Menon, Sanjiv Kumar, and Vaishnavh Nagarajan. Think before you speak: Training language models with pause tokens, 2024. URL
https: org/ /tg/abs/////aaooo 2210. 02226.
Shibo Hao, Sainbayar Sukhbuatar, DiJia Su, Xian Li, Zhiting Hu, Jason Weston, and Yuandong Tian. Training large language models reason in continuous latent space, 2024. URL https: / //arxiv..rrg org/ abs/ /aabb//2410 06212.66666.
Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardr. Measuring mathematical problem solving with the math dataset, 2021.
URL https: http:///aaxvvv..gaa///////s
03874.
Hongkang Yang Hongkang Yang. Zehao Lin Zehao Lin, Wenjin Wang Wenjin Wang. Hao Wu Hao Wu, Zhiyu Li Zhiyu Li. Bo Tang Bo Tang, Wengiang Wei Wenqiang Wei. Jinbo Wang Jinbo Wang, Zeyun Tang Zeyun Tang. Shichao Song Shichao Song, Chenyang Xi Chenyang Xi, Yu Yu Yu Yu, Kai Chen Kai Chen, Feiyu Xiong Feiyu Xiong, Linpeng Tang Linpeng Tang, and Weinan Weinan E. Memory Language modeling with explicit memory. Journal of Ma- chine Learning, 3(3):300-346, January 2024. ISSN 2790-203X. doi: 10.4208/jm1.240008 URL http:/ //tp://dxx.// doi. org/ 100. 4208/ jml- 240708.
Jian Hu, Jason Klein Liu, Haotian Xu, and Wei Shen. Reinforce++: An efficient rIhf algorithm with robustness both prompt and reward models, 2025a. URL https://arxis: wwt. abs/ 2501. 03262.
Zhiyuan Hu, Yibo Wang, Hanze Dong, Yuhui Xu, Amrita Saha, Caiming Xiong, Bryan Hooi, and Junnan Li. Beyond' aha'': Toward systematic meta-abilities alignment in large reasoning models. arXiv preprini ar0iv:2505. 10554, 2025b.
HuggingFace. SmolLM3: smol, multilingual,, long-context reasoner - huggingface.co. https: / //huggingface. co /blog/ smollmm 1m3, 2025, [Accessed 23-09-2025).
Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar-Lezama, Koushik Sen, and Ion Stoica. Livecodebench: Holistic and contamination free evaluation of large language models for code, 2024. URL https://aaaa /atpsi///arvvv..// org/ / 2403.00 07974.
Chao Jin, Zili Zhang, Xuanlin Jiang, Fangyue Liu, Xin Liu, Xuanzhe Liu, and Xin Jin. Ragcache: Efficient knowledge caching for eetrieval-augmente generation. arXiv preprint arXiv:2404.1244 12457, 2024.
Mandar Joshi, Eunsol Choi, Daniel Weld, and Luke Zettlemoyer. TriviaQA: A large scale distantly supervised challenge dataset for reading comprehension. In Regina Barzilay and Min-Yes Kan (eds.), Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1601-1611, Vancouver, Canada, July 2017. Association for Com- putational Linguistics. doi: 10. 10.18653/v1/P11-111 1147. URL https: : / //a/aaaannt /acaaaannnnoogyyy. .org P17-1147/
12

---

MemGen
Hengli Li, Chenxi Li, Tong Wu, Xuekai Zhu, Yuxuan Wang. Zhaoxin Yu, Eric Hanchen Jiang, Song-Chun Zhu, Zixia Jia, Ying Nian Wu, and Zilong Zheng. Seek in the dark: Reasoning via test-time instance-level policy gradient in latent space, 2025a. URL https: /ttp:///axvvvv...//eeeeeeeawella org/ abs/2505. 13308.
Yuhui Li, Fangyun Wei, Chao Zhang, and Hongyang Zhang. Eagle: Speculative sampling requires rethinking feature uncertainty, 2025b. URL https::s..... / sett://axxii wwwxii.or/aaba........oooaaaa. /4001. 15077.
Wang Ling, Dani Yogatama, Chris Dyer, and Phil Blunsom. Program induction by rationale gener- ation: Learning to solve and explain algebraic word problems. arXiv preprint urXiv:/7055 7775.04446. 2017.
Luyang Liu, Jonas Pfeiffer, Jiaxing Wu, Jun Xie, and Arthur Szlam. Deliberation latent space via differentiable cache augmentation. arXiv preprint 17747
Hanjun Luo, Shenyu Dai, Chiming Ni, Xinfeng Li, Guibin Zhang, Kun Wang, Tongliang Liu, and
Hanan Salam. Agentauditor: Human-level safety and security evaluation for Ilm agents, 2025a.
URL wwwptttpp////...... /ttppp://axivvi /ttpiva.ivv////////e//e..eaawaea org/ abs 2506 6. 00641.
Michael Luo. Naman Jain, Jaskirat Singh, Sijun Tan, Ameen Patel, Qingyang Wu, Alpay Ariyak, Colin Cai, Shang Zhu Tarun Venkat, Ben Athiwaratkum, Manan Roongta, Ce Zhang. Li Erran Li, Raluca Ada Popa, Koushik Sen, and Ion Stoica. Deepswe: Training a state-of-the-art coding agent from scratch by scaling rl. https: lepprtttyraddoo---o DeepSWB- Training Fully Salllyy-penn-- sourced State- of aache-rtt-- Art-Cooing Aagg-AAgentt- by Scaling RL- 2025b. Notion Blog.
notion. site//
Laurens van der Maaten and Geoffrey Hinton. learning research, (NNov) 2605, 2008
Visualizing data using t-sne. Journal of machine
Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, and Hannanch Hajishirzi. When not trust language models: Investigating effectiveness of parametric and non-parametric memories, 2023. URL https: /arxivvv../////////////a//////aa orggabbs 2212 10511.
Dan P McAdams. 2001.
The psychology of life stories. Review of general psychology. 5(2):100-122,
Charles Packer, Sarah Wooders, Kevin Lin, Vivian Fang, Shishir G. Patil, Ion Stoica, and Joseph E. Gonzalez. Memgpt: Towards 1lms as operating systems, 2024. URL https:t//axxii...t///// abs/2310, 08560.
Dmitrii Pantiukhin, Boris Shapkin, Ivan Kuznetsov, Antonia Anna Jost, and Nikolay Koldunov. Ac- celerating earth science discovery via multi-agent Ilm systems, 2025. URL https://aaxxv org/ htgg/bb//ee htt/abbs/55333 2203. 05854.
Cheng Qian, Emre Can Acikgoz, Qi He, Hongru Wang, Xiusi Chen, Dilek Hakkani-Türr, Gokhan Tur, and Heng Ji. Toolrl: Reward is all tool learning needs, 2025. URL https://arxii org/ brrg/bb// 2504. 13958.
Jiahao Qiu, Xinzhe Juan, Yimin Wang, Ling Yang, Xuan Qi, Tongcheng Zhang, Jiacheng Guo, Yifu Lu, Zixin Yao, Hongru Wang, Shilong Liu, Xun Jiang, Liu Leqi, and Mengdi Wang. Agentdistill: Training- free agent distillation with generalizable mep boxes, 2025a. URL https://arxiv org/abs/ 2 14728.
Jiahao Qiu, Xuan Qi. Tongcheng Zhang, Xinzhe Juan, Jiacheng Guo, Yifu Lu, Yimin Wang. Zixin Yao, Qihan Ren, Xun Jiang, Xing Zhou, Dongrui Liu, Ling Yang, Yue Wu, Kaixuan Huang. Shilong Liu, Hongru Wang, and Mengdi Wang. Alita: Generalist agent enabling scal- able agentic reasoning with minimal predefinition and maximal self-evolution,, 2025b. URL https: /ttss///rxivvi org/ /
David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R. Bowman. Gpqa: A graduate-level google-proof q&a benchmark, 2023. URL httptt/////aaa org/ 2211 12022.
13

---

MemGen
Shuo Ren, Pu Jian, Zhenjiang Ren, Chunlin Leng, Can Xie, and Jiajun Zhang. Towards scientific intelligence: A survey of llm-based scientific agents, 2025. URL abs/2503.2400 24047.
Zhenyi Shen, Hanqi Yan, Linhai Zhang, Zhanghao Hu, Yali Du, and Yulan He. Codi: Compressing ofain-of-thought into continuous space via self-distillation. arXiv preprint arXiv:2502.21074,
2025a.
Zhenyi Shen, Hanqi Yan, Linhai Zhang, Zhanghao Hu, Yali Du, and Yulan He. Codi: Compressing chain-of-thought into continuous space via self-distillation, 2025b. URL https: //rxivv.
org/abs. 2502. 21074.
Mohit Shridhar, Xingdi Yuan, Mlec-Alexandre Côté, Yonatan Bisk, Adam Trischler, and Matthew Hausknecht. Alfworld: Aligning text and embodied environments for interactive learning, 2021.
URL htttthtst//aiiiee org/ abs/2010 03768.
Joykirat Singh, Raghav Magazine, Yash Pandya, and Akshay Nambi. Agentic reasoning and tool integration for Ilms via reinforcement learning, 2025. URL
2505. 01441.
Peiyang Song, Pengrui Han, and Noah Goodman. A survey large language model reasoning failures. In 2nd Al for Math Workshop @ ICML 2025, 2025. URL https://...... /openeeiiww. net/Eorum? iininiddiee G
Eleanor Spens and Neil Burgess. A generative model of memory construction and consolidation. Nature human behaviour, 8(3):526-543, 2024.
Weihang Su, Yichen Tang, Qingyao Ai, Junxi Yan, Changyue Wang, Hongning Wang, Ziyi Ye, Yujia Zhou, and Yiqun Liu. Parametric retrieval augmented generation, 2025. URL https: //arxivvorg abs 2200225111111 15915.
Yuchang Sun, Yanxi Chen, Yaliang Li, and Bolin Ding. Enhancing latent computation in transform- ers with latent tokens, 2025. URL https::///// http//rxvvvv///////////a
12629.
Jihoon Tack, Jachyung Kim, Eric Mitchell, Jinwoo Shin, Yee Whye Teh, and Jonathan Richard Schwarz. Online adaptation of language models with a memory of amortized contexts, 2024.
URL https:t//////. arxiv. org/ 2os/2403. 04317.
Wenhui Tan, Jiaze Li, Jianzhong Ju, Zhenbo Luo, Jian Luan, and Ruihua Song, Think silently, think fast: Dynamic latent compression of Ilm reasoning chains, 2025. URL https:/ arx.v..rgg./////./........
abs/2505. 16552.
Xiangru Tang. Tianrui Qin, Tianhao Peng, Ziyang Zhou, Daniel Shao, Tingting Du, Xinming Wei, Peng Xia, Fang Wu, He Zhu, Ge Zhang, Jiaheng Liu, Xingyao Wang, Sirui Hong, Chenglin Wu, Hao Cheng. Chi Wang, and Wangchunshu Zhou. Agent kb: Leveraging cross-domain experience for agentic problem solving, 2025. URL https:tpp/sccm http:://aaavvvooe org/ abs abs/ 2507. 06229.
James Thorne, Andreas Vlachos, Christos Christodoullpoulls, and Arpit Mittal. Fever: a large-scale dataset for fact extraction and verification. arXiv preprint arXiv: 1803,05355 2018.
Peng Wang, Zexi Li, Ningyu Zhang, Ziwen Xu, Yunzhi Yao, Yong Jiang, Pengjun Xie, Fei Huang, and Huajun Chen. Wise: Rethinking the knowledge memory for lifelong model editing of large language models, 2024a. URL https: / wwwp:iioorraaaaaa
14768.
Ruoyao Wang, Peter Jansen, Marc-Alexandre Côté, and Prithviraj Ammanabrolu. Scienceworld: Is your agent smarter than 5th grader?, 2022. URL httppp////axxv.rr////aae/ 07540.
Xiaoqiang Wang, Suyuchen Wang, Yun Zhu, and Bang Liu. MeM Bridging memory reten- tion and retrieval via reversible compression. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar (eds.), Findings of the Association for Computational Linguistics: ACL 2025. pp. 4541-4557, Vienna, Austria, July 2025a. Association for Compu- tational Linguistics. ISBN 979-8-891766 191-8--5577---555 doi: 10.18653/v1/2025 1/205...11/.....n..n.g..... acl.235. URL https://ach anthelanntttoooo wtwphhhhllo..oo///222// org/2025. Findingg-eee 235/-
14

---

MemGen
Yu Wang, Yifan Gao, Xiusi Chen, Haoming Jiang, Shiyang Li, Jingfeng Yang. Qingyu Yin, Zheng Li, Xian Li, Bing Yin, et al. Memoryllm: Towards self-updatable large language models. arXiv preprint arXiv:2402.04622 2024b.
Yu Wang, Dmitry Krotov, Yuanzhe Hu, Yifan Gao, Wangchunshu Zhou, Julian McAuley, Dan Gut- freund, Rogerio Feris, and Zexue He. M+: Extending memoryllm with scalable long-term mem- ory, 2025b. URL https:://///.....p.
/arxivv..v.../../i...aa////aa.i. org/ htgg//bb///a 2502. 00592.
Zhenhailong Wang, Haiyang Xu, Junyang Wang, Xi Zhang, Ming Yan, Ji Zhang, Fei Huang, and Heng Ji. Mobile-agent-e: Self-evolving mobile assistant for complex tasks, 2025c. URL https: / wwtp://rggabb///aaeeeeee 117333.
Zora Zhiruo Wang, Jiayuan Mao, Daniel Fried, and Graham Neubig. Agent workflow memory, 2024c. URL hwt.h:ttt///rrtt
org/ abs/ 2409. 07429.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. Chain -f-thugghttt prompting elicits reasoning in large language models, 2023. URL https::////... /ttsss///rxiii http://wwwwwwwxa//////// hetrrg/ab///201100
Yifan Wei, Xiaoyan Yu, Yixuan Weng, Tengfei Pan, Angsheng Li, and Du. Autotir: Autonomous tools integrated reasoning via reinforcement learning, 2025. URL https: aaaxiiivivvv//..///aaaa/aaaaavva /ttpxv/.xx../////////....../ abs/2507 21836.
Ronald J Williams. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning, 8:229-256, 1992.
Mingyuan Wu, Jingcheng Yang, Jize Jiang, Meitang Li, Kaizhuo Yan, Hanchao Yu, Minjia Zhang, Chengxiang Zhai, and Klara Nahrstedt. Vtool-r1: Vlms learn think with images via rein- forcement learning on multimodal tool use, 2025a. URL https::::1 /ttp:///aavvvv/av/eaaeeeeeate org abs 19255.
Yaxiong Wu, Sheng Liang. Chen Zhang, Yichao Wang, Yongyue Zhang, Huifeng Guo, Ruiming Tang, and Yong Liu. From human memory to ai memory: A survey on memory mechanisms in the of Ilms, 2025b. URL https://aaxiii org/ www./bo/////aa/oo..ooooooo aas///bss//5..... 15965.
Zidi Xiong, Yuping Lin, Wenya Xie, Pengfei He, Jiliang Tang, Himabindu Lakkaraju, and Zhen Xi- ang. How memory management impacts IIm agents: An empirical study of experience-follee following behavior, 2025. URL https: htt///xxxvvv.r/////.o org/ abs 2505 16067.
Wujiang Xu, Kai Mei, Hang Gao, Juntao Tan, Zujie Liang, and Yongfeng Zhang. A-mem: Agentic memory for IIm agents, 2025a. URL //te:///rrrr http://wwwwwa/a/aaaaa//// /arg/abs// 22tp//2.s/22222122// 2200..2110.. 12110.
Yige Xu, Xu Guo, Zhiwei Zeng, and Chunyan Miao. Softcot: Soft chain-of-thought for efficient reasoning with llms, 2025b. URL htttts//aaxii..... wttporgg.abb.a.aaaae/eeeee 221..2234..........
Yige Xu, Xu Guo, Zhiwei Zeng, and Chunyan Miao. Softcot++: Test-time scaling with soft chain- of-thought reasoning, 2025c. URL htttts:///aaatt.. wwwparwwwwwwrorrraaaeeeeeeeeeeee
11484.
Zhangchen Xu, Yang Liu, Yueqin Yin, Mingyuan Zhou, and Radha Poovendran. Kodcode; di- verse, challenging, and verifiable synthetic dataset for coding, 2025d. URL htt.ttt/p///xiiii org/abs 22.3. 02951.
Zifan Xu, Haozhu Wang, Dmitriy Bespalov, Xuan Wang, Peter Stone, and Yanjun Qi. Latent skill discovery for chain-of-thougle reasoning. arXiv preprint arXiv:2312.04668 2023.
An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115, 2024a.
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, and Bo Zheng et al. Qwen3 technical report, 2025. URL https: ///rxiiv..v. orgg abs / 2500. 09388.
John Yang, Carlos E. Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, and Ofir Press. Swe-agent: Agent-computer interfaces enable automated software engineering, 2024b. URL https: // www.iv.ivvvvv...vv www.wv.ooooo/aooooooooooooo /240 157933 15793.
15

---

MemGen
Yi Yang, Yixuan Tang, and Kar Yan Tam. Investim: A large language model for investment using financial domain instruction tuning, 2023. URL httph:ttppccccc arxiv. org/ abs 209. 13064.
Weiran Yao, Shelby Heinecke, Juan Carlos Niebles, Zhiwei Liu, Yihao Feng. Le Xue, Rithesh Murthy, Zeyuan Chen, Jianguo Zhang, Devansh Arpit, Ran Xu, Phil Mui, Huan Wang, Caim- ing Xiong, and Silvio Savarese. Retroformer: Retrospective large language agents with policy gradient optimization, 2024. URL https: /ttpe:///axxiviii org/ aaaaabss230000 02151.
Da Yin, Faeze Brahman, Abhilasha Ravichander, Khyathi Chandu, Kai- Chang, Yejin Choi, and Bill Yuchen Lin. Agent lumos: Unified and modular training for open-source language agents, 2024. URL https::///. /aaxivivvv..////.///aaa//aaaavva org/ abs/ /2311. 05657.
Hongli Yu, Tinghong Chen, Jiangtao Feng, Jiangjie Chen, Weinan Dai, Qiying Yu, Ya-Qin Zhang, Wei- Ying Ma, Jingjing Liu, Mingxuan Wang, and Hao Zhou. Memagent: Reshaping long- IIm with multi-conv rl-based memory agent, 2025. httpttps: /arxivv///avvvaaa/aa org/ abs 2507. 02259.
Aohan Zeng, Mingdao Liu, Rui Lu, Bowen Wang, Xiao Liu, Yuxiao Dong, and Jie Tang. Agenttun- ing: Enabling generalized agent abilities for llms, 2023. URL https: /ttp://aaavv..//ii org/orrg///.ooo. /
2310.1282:
Guibin Zhang. Muxin Fu, Guancheng Wan, Miao Yu, Kun Wang, and Shuicheng Yan. G-memory: Tracing hierarchical memory for multi-agent systems, 2025a. URL htttttt//axiiv..om abs/2506. 07398.
Jianguo Zhang, Tian Lan, Rithesh Murthy, Zhiwei Liu, Weiran Yao, Ming Zhu, Juntao Tan, Thai Hoang, Zuxin Liu, Liangwei Yang, Yihao Feng, Shirley Kokane, Tulika Awalgaonkar, Juan Car- los Niebles, Silvio Savarese, Shelby Heinecke, Huan Wang, and Caiming Xiong. Agento- hana: Design unified data and training pipeline for effective agent learning, 2024a. URL atttss://arxie org/ abs 2202.155111
Yang Zhang, Wenxin Xu, Xiaoyan Zhao, Wenjie Wang, Fuli Feng. Xiangnan He, and Tat-Seng Chua. Reinforced latent reasoning for Ilm-based recommendation, 2025b. URL https:// a arxiv.org/ant abs 2205.555551 9092.
Zeyu Zhang, Xiaohe Bo, Chen Ma, Rui Li, Xu Chen, Quanyu Dai, Jieming Zhu, Zhenhua Dong, and Ji-Rong Wen. A survey the memory mechanism of large language model based agents, 2024b. URL https: http:///xxxvvv...
abs 21.2224441110
Andrew Zhao, Daniel Huang, Quentin Xu, Matthieu Lin, Yong-Jin Liu, and Gao Huang. Expel: Lim agents are experiential learners, 2024. URL https: /ttp//rrxvvvv// org/abss /tt///b///// /200..30.... 10144.
Boyuan Zheng, Michael Y. Fatemi, Xiaolong Jin, Zora Zhiruo Wang. Apurva Gandhi, Yueqi Song, Yu Gu, Jayanth Srinivasa, Gaowen Liu, Graham Neubig, and Yu Su. Skillweaver: Web agents can self-improve by discovering and honing skills, 2025. URL 2504. 07079.
Wanjun Zhong, Lianghong Guo, Qiqi Gao. He Ye, and Yanlin Wang. Memorybank: Enhancing large language models with long-term memory, 2023. URL https:://axiix// wwwp:iii.orr/aaaoo.o ars../////b....... 10250.
Zijian Zhou, Ao Qu, Zhaoxuan Wu, Sunghwan Kim, Alok Prakash, Daniela Rus, Jinhua Zhao, Bryan Kian Hsiang Low, and Paul Pu Liang. Meml: Learning to synergize memory and reasoning for efficient long-horizon agents, 2025. URL https: /aariviivvv.////e///aae/e/eeevva
http:p/:///a...ooorr/////////// wrg/abs//5055 15841.
Rui-Jie Zhu, Tianhao Peng, Tianhao Cheng, Xingwei Qu, Jinfa Huang, Dawei Zhu, Hao Wang, Kaiwen Xue, Xuanliang Zhang, Yong Shan, et al. A survey on latent reasoning. arXiv preprint arXiv:2507.06205 2025.
A USE OF LARGE LANGUAGE MODELS
In preparing this work, we employed large language models (LLMs) to assist with: polishing the writing, conducting literature reviews, and creating visualizations.
16

---

MemGen
B ADDITIONAL RELATED WORKS
LLM Decoding & RL. Two additional topics that relate to our work are LLM decoding and re- inforcement learning (RL). From the decoding perspective, MemGen dynamically generates and in- serts latent tokens, which shares similarity with speculative decoding where a drafter model receives the current decoding context and produces subsequent drafted tokens (Cai al., 2024; Fu et al., 2024; Li et al., 2025b; Goel et al., 2025). However, these methods primarily aim to accelerate LLM inference. whereas MemGen focuses on leveraging latent states as effective carriers of mem- ory. From the RL perspective, MemGen employs rule-based RL to train the memory trigger, which closely related to reinforcement learning with variable reward (RLVR), including GRPO from DeepSeek-R1 (DeepSeek-Al et al., 2025) and its various derivatives (Qian et al., 2025; Wu et al., 2025a; Wei et al., 2025; Fu al., 2025). While there exist efforts combining RL with agent memory, to our knowledge, most do not address self-improving memory; for example, MemAgent (Yu et al., 2025) and MEMI (Zhou et al., 2025) focus on handling long-context inputs rather than evolving memory mechanisms.
Latent Memory We further extend our discussion by incorporating related work on latent memory mechanisms: R3Mem (Wang et al., 2025a) introduces reversible, hierarchical memory architec- ture that integrates efficient long-term information retention with faithful retrieval, achieving state- of-the-art performance long-context modeling, retrieval-augmented generation, and prolonged interactive tasks.
C OPTIMIZATION ALGORITHM DETAILS
In this section, we provide detailed exposition of the optimization algorithms for training the memory weaver, W mentioned in $4.3. The core principle, established in Equation (10), to update only the weaver's parameters, denoted as 0', while keeping the reasoner me frozen. This modularity allows MemGen to be compatible with various optimization paradigms. We detail the specific implementations for Supervised Fine-Tuning (SFT) and a reinforcement learning approach, Group Relative Policy Optimization (GRPO).
C.1 COMBINING MemGen WITH SFT
The objective of Supervised Fine- MininTunng is to train the memory weaver to generate latent mem- ories that guide the frozen reasoner To to replicate the behavior observed in dataset of high- quality demonstration trajectories. We leverage the provided history of past experiences, H (<),,; where each T; is treated as an expert demonstration.
Each expert trajectory 7;" consists of a sequence of states and actions, T.' (3i,0, a 81,1, ...)....) Fach expert action of, is sequence of tokens, a;, = (z,,,,: "i, ,,,,,,, ). The goal is to maximize the conditional log-likelihood of generating these expert token sequences.
During the training of the weaver, both the reasoner me and the memory trigger T are held fixed. At each token generation step where the trigger activates, the weaver Wa takes the reasoner's hidden states H.,,<y input and produces latent memory M. The reasoner then generates the next token conditioned on this memory. The SFT objective is to adjust the weaver's parameters to maximize the probability of the ground-truth token "i,...
Formally, the optimization problem to minimize the negative log-likelihood of the expert trajec- tories, averaged over the dataset H. The loss function for the weaver's parameters 8' is defined as:
T--1 L,
LSFT(0') = -E H~H
Σ 1=1
log ne (z(
Si, 2i, Mias))
(11)
where the latent memory M.,.j is synthesized by the weaver at that specific step:
My Wg (H.,,))).
(12)
Note that the generation of M.r only occurs if the fixed trigger T determines an invocation is needed step j. In steps where memory is invoked, the conditioning term M is omitted. The
17

---

MemGen
gradients computed exclusively with respect the weaver's parameters 0' and used update them via gradient descent:
0' V
(13)
where oj is the learning rate. Through this process, the memory weaver learns to synthesize latent memories that effectively steer the frozen reasoner's generative process align with the training data's 's behavior, thereby internalizing the knowledge from the demonstration data without corrupting the general capabilities of the core LLM.
C.2 COMBING MemGen WITH GRPO
The memory weaver can also be trained using reinforcement learning objective. We specifically adapt the GRPO algorithm. The training process begins by sampling a batch of tasks from the ex- perience history H. For each task 2;, we use the policy 11g WT (composed of the frozen reasoner TT and the current memory weaver Wer) generate group of K distinct trajectories, denoted as G = [Tr.1, Ti,2; ... Ti.K) J. Each trajectory is generated by the agent's interaction with the environ- ment and results final reward ((T,,)), evaluated by the reward function. Going forward, GRPO computes group-relative baseline by averaging the rewards of all trajectories within the group G::
K
R(G.)
= K k=1
Σ R(TsK))
(14)
The advantage for specific trajectory Ti,k is then its reward relative to this baseline:
A(T.,)) = R(T,,)) R(G.).
(15)
This formulation allows the weaver to learn by differentiating between better and worse outcomes within a set its own generations, promoting policies that produce trajectories with above-average rewards. The final objective function, maximized with respect to the weaver's parameters 6', is:
JGRPO (0) EE m GI WoriT
Wor A(T,,)) log Wo (TiEe x.) 8 KL (IN (.. ax)) Herl mx))
K
1
(16)
where eer is a fixed reference policy and 8 is a coefficient controlling the strength of the KL regu- larization. The gradients are computed only for the weaver's parameters O', thus refining its ability to synthesize impactful latent memories while preserving the integrity of the core reasoner.
C.3 MEMORY TRIGGER TRAINING DETAILS
We now detail the training procedure of the memory trigger Tnngerr During this stage, the reasoner Ty and the memory weaver Wyr remain frozen, and only the trigger parameters & are updated. For each task x; with initial environment state 8o, we generate trajectories under the composed policy induced by ne. We, and T Tnggerr At each decoding step t and token position j where 2t,j E D, the trigger observes the hidden-state prefix Hi,s and samples binary action
T
dijj (a(d H H2,r)) Bernoulli(p..).
Pa. a (Tuigeer (H, (H,,)))),
(17)
where de,j corresponds to TNVOKE. If invoked, the decoder pauses and the weaver synthesizes a latent memory M, from H.,y after which generation resumes based on the augmented context.
Let I, denote the set of activated positions in trajectory т, and let d, = {dr, (e.)) be the trigger decisions within that rollout. After completing the episode, the environment returns scalar reward R(T,), which we assign uniformly all decisions in d,
To discourage excessively frequent invocation, we introduce pparsity-inducin penalty. For group of rollouts generated from the same task x;, identify the high-reward subset
Huigh = {i: R(;)) 2 median, R(Tr)),
(18)
and compute the reference activation level
de
(19)
Htigh
In
LE Meach
(t.j)I,
18

---

MemGen
which estimates the typical invocation frequency among successful trajectories. The trigger is trained to maximize the reward while penalizing activations above pr:
max E
R((T)
Σ (L,))Z,
max (0, (0, - -D)
(20)
Fo d, Tuiegee
where À > O controls the sparsity regularization strength.
We optimize this objective following GRPO-style procedure. For each task prompt, generate group of G trajectories (T,")) y==1 (we set G in practice; Table 3). We compute per-trajectory
rewards R(T,() and construct normalized advantages
A = R(T, M4
C G
Ass
1
2 R(T! 9=1
1 G
(R(T!
- pi)3,
9=1
thus measuring each rollout's performance relative to the group. The trigger parameters are then updated by maximizing GRPO-style surrogate objective:
G G 9=1 (L))Z!())
Σ
log N H (3) A! -BKL (T || Rousa).
max
G
After each update, we set Teai + Fe and repeat. This objective allows the trigger to learn to invoke memory only at critical points that improve group-relative performance while maintaining sparsity.
D EXPERIMENTAL DETAILS
D.1 TRAINING DATASET SETUP
For all datasets except PopQA, we utilize their official training sets to train both the memory weaver and memory trigger. Specifically, we first train the memory weaver in isolation. Since the trigger module is not yet available at this stage, latent memory tokens are randomly inserted at punctuation marks during training to provide diverse learning signals. Once the memory weaver adequately trained, we proceed to train the memory trigger, enabling it to learn when invoke memory (with the memory weaver fixed). For PopQA, which lacks dedicated training set, we instead leverage the trained model on TriviaQA and directly evaluate on PopQA. The dataset statistics summarized in Table 2.
Table 2:
Dataset Statistics.
Dataset Name
Train Size
Test Size
Note
ALFWorld
3.32K
134
From AgentBank-ALFWorde
TriviaQA
4.13K
7.9K
From AgentBank-Triviade
PopQA
14.3K
KodCode
2K
KodCode-Light-RI--NN
BigCodeBench
912
228
GPQA
448
198
GPQA- for testing
GSMSK
7.47K
1K
MATH
1.6K
500
19

---

MemGen
D.2 PARAMETER CONFIGURATIONS
Table 3: Hyperparameters used the training MemGen.
Settings
Hyperparameters
train batch learning_rate le-5
epochs =
Training(SFT)
warmup_ratio 0.1
optim = aadamw_tordd schedular cosine
rollout_batch_size 8
train_batch_size 8
epochs =
beta= 0.0
Training(GRPO)
num iiterattionn
learning_ rate le-5
warmup_ratio = 0.1
optim = dedamwttorch
schedular = cosine
r= 16
lora_appaa 32
paget-mmdulle = [q-proj, v-proj]
LoRA
lora.dropout = 0.1
bias
task_type CAUSAL.LM
-ofllodd
Optimization
flash deepspeed enable sleep
E
EXTRA RESULTS
Table Results on Qwen2.5-1.5B. All values represent the performance metric for each task (e.g.,
accuracy %). We highlight the best and second best results.
Method
ALFWorld
TrivialQA
PopQA
KodCode BigCodeBench
GPQA
GSM8K
MATH
Vanilla
22.54
32.10
16.08
24.55
40.35
11.62
39.51
36.63
CoT
18.30
28.67
18.39
32.32
38.59
15.67
56.79
45.22
SFT
36.57
63.84
39.20
55.83
37.72
11.11
54.83
38.84
GRPO
43.55
68.21
43.15
62.11
70.34
15.65
68.10
47,42
Owen2 5-1,5B
REINFORCE
43.25
66.50
41.87
60.20
67.80
12.50
67.40
46.89
REINFORCE+
43.66
66.90
44.69
63.33
69.50
13.80
69.04
47.33
Agent-FLaa FLAN
35.80
64.28
38,90
56.21
43.83
9.35
53.02
29.82
Expel.
28.96
25.20
20.20
31.15
39.78
8.12
45.12
38.12
Memory Bank
27.89
38.14
22.78
37.93
35.87
13.87
47.88
30.47
AWM
30.42
55.69
32.54
SoftCoT
33.07
62.22
38.78
55.13
36.10
9.31
$4.50
38.55
Co-processor
35.66
64.78
40.12
56.65
38.10
12.12
57.12
37.40
MemGen SFT
40.30
65.02
41.28
58.16
42.47
18.28
58.15
47.12
MonGen GRPO
54.27
73.42
49.28
65.43
72.81
18.18
73.39
53.36
20

---

MemGen
E. CONTINUAL LEARNING RESULT
The results in Table indicate three main findings. First, MemGen exhibits stronger knowledge re- tention and forward transfer than SFT and ExpeL. For example, when trained on GPQA, MemGen reaches 47 47.96% on GSM8K and 28.80% on KodCode, surpassing SFT at 43.74% and 18, 18,50% by margins of +..22% and +10.30% respectively. Similarly, when trained on KodCode, MemGen maintains 40.34% on AQuA and 20.09% on GPQA, whereas SFT yields 28.61% % and 2.53%. Sec- ond, MemGen demonstrates more balanced cross-task generalization. Unlike SFT, which primarily improves performance the most recent task (c.g., 54. 10% on KodCode but only 2.53% GPQA), MemGen achieves competitive results across tasks. After training on GSM8K, attains 38.43% on AQuA and 21.72% GPQA, both higher than SFT and ExpeL, suggesting that latent memory captures task-invariant reasoning. Third, MemGen effectively mitigates forgetting on earlier tasks. After sequential training on KodCode, it preserves 40.34% AQuA compared to 27.14% 14% of Expel. and 28.61 % of SFT, showing stronger robustness to catastrophic forgetting. These findings suggest that MemGen provides more stable and transferable mechanism for continual learning.
E.2 TRIGGER FREQUENCY VISUALIZATION
More visualizations of the trigger frequeney are displayed Figures and 8, where we paired Qwen2 5-1.66 5B or Smol LM3-3B with the GSM8K dataset and tested subsets of GSM8K, KodCode, and GPQA (each subset having the same number of samples), We then tallied the fre- quency of memory trigger INVOKE judgments each relative percentile position in the LLM output.
GSMBK (Avg: 73-17)
120
Memory Invocation Count
GPQA (Avg: 46.87)
100
KodCode (Avg: 51:70)
80
60
40
20
0
10
Relative Insertion Position (%)
Figure 7: Memory invocation frequency across benchmarks inference (trained on MemGen SFT+Qwen2. 5-1 5B +GSM8K).
E.3 FRAMEWORK ANALYSIS
E.3.1 ABLATION STUDY
To assess the effectiveness of the proposed memory trigger, we conduct an ablation study compar- ing different memory invocation strategies, summarized in Table 6. Specifically, Random denotes a naive baseline where latent memory tokens are inserted at arbitrary token positions with a fixed probability p. All delimiters activated represents a sentence-level strategy that invokes the memory weaver at every delimiter position without any selection mechanism. Finally, MemGen 's dedicated Trigger corresponds our standard approach, where the memory weaver activated by the trained trigger Tinggen. The results reveal several key observations. First, sentence-level intervention already improves performance compared to random invocation. For instance, activating the weaver at all delimiters yields 17.34%, 56. 66,, and 64.15% GPQA, Kodcode, and TriviaQA, respectively, consistently outperforming all random baselines (e.g., p = 0.5 achieves only 16.66%, 52.95%, and 57. 57.28%). This highlights the importance of aligning memory injection with semantic boundaries rather than distributing across the sequence. More importantly, our trained trigger achieves the best overall performance, reaching 18. 28.28% 58.16%. and 65.02% the three benchmarks. This
21

---

MemGen
160
GSM8K (Avg: 74-49)
Memory Invocation Count
140
GPQA (Avg: 48.59)
KodCode (Avg: 27.96)
120
100
80
60
40
20
0-
40
60
Relative Insertion Position (%)
Figure 8: Memory invocation frequency across benchmarks inference (trained on MemGen
SFT+Smo1LM3- +GSM8K).
Trained on GSM8K
GSM8K
KodCode
GPQA
Vanilla
SFT
MemoryBank
MATH
Mem
10
20
30
40
50
60
Figure 9: The generalization study of KemGen, We train MemGen SFT all four datasets.
GSM8K and evaluate it on
Trained on KodCode
Vanilla
GSM8K
SFT
Memory Bank
KodCode
MemGen
GPQA
MATH
10
20
30
40
50
60
Figure 10: The generalization study of MemGen. We train MemGen SFT KodCode and evaluate on all four datasets.
demonstrates that selective activation. i.e., deciding when and where to weave in memory, provides the most effective support for reasoning, as it balances memory utility and interference more pre- cisely than coarse-grained alternatives.
22

---

MemGen
Table 5: Continual learning results of Qwen2. 5-1 5B-Inst across four reasoning and programming datasets (AQuA, GPQA, GSM8K, KodCode). The model is sequentially trained on each dataset (AQuA - GPQA GSM8K - KodCode), and after each training stage, evaluation is conducted on all four benchmarks.
Method
AQuA
GPQA
GSM8K
KodCode
Trained On
Vanilla
41.34
11.62
39.51
24.55
AQuA
SFT
42.52
16.67
42.10
18.20
ExpeL
41.73
12.67
40.16
16.30
MemGen SFT
43.31
19.70
39.80
19.55
GPQA
SFT
38.55
17.17
45.74
18.50
Expel.
37.24
14.35
42.67
15.20
MemGen SFT
39.85
20.72
47.96
28.80
GSM8K
SFT
33.46
13.13
52.31
19.45
ExpeL
34.89
12.42
48.78
13.65
MemGen SFT
38.43
21.72
55.67
19.75
KodCode
28.61
2.53
24.14
54.10
ExpeL
27.14
6.23
31.44
48.35
MemGen
40.34
20.09
53.72
52.95
Table 6: Ablation study of different memory invocation strategies. Random denotes naïve baseline where latent memory tokens are inserted at arbitrary token positions with a fixed probability p. All delimiters activated represents sentence-level strategy that invokes the memory weaver at every de- limiter position without any selection mechanism. Finally, MemGen's 's dedicated Trigger corresponds to our standard approach, where the memory weaver is activated by the trained trigger Tuigger-
Memory Invocation Strategy
GPQA
Kodcode
TriviaQA
Random (p 0l.2)
15.66
54.55
63.55
Random (p 0.5)
16.66
52.95
57.28
Random (p = 0.8)
12.63
53.60
62.22
delimiters activated
17.34
56.20
64.15
MemGen's 's dedicated Trigger
18.28
58.16
65.02
E.3.2 ANALYSIS OF MEMORY WEAVER
We further investigate how the parameterization of the memory weaver influences downstream per- formance. As shown in Table 7, increasing the number of trainable parameters by adopting a full- parameter SFT setup surely enhances the weaver's memory and learning capacity, leading to higher task performance across benchmarks. Nevertheless, the LoRA-based instantiation already achieves competitive results while retaining high parameter efficiency, demonstrating that even lightweight adaptations can endow the weaver with sufficient capacity to generate effective latent memories.
E.3.3 EFFICIENCY ANALYSIS
Table 8 reports average inference time and task performance across three benchmarks. Both SFT and MemGen drastically reduce per-task inference time compared with vanilla models due to fewer tokens required to reach correct answers. For instance, on KodCode with Qwen2. 5 1-15--166 1.5B. MemGen SFT completes tasks in 2.94 s, a 75.4% reduction from 11.96 s for vamilla, while improving accuracy by 33.61%. On ALFWorld+Owen3 SFT (20.08 vs 19.76 s) but increases accuracy from 83.59% to 85.82%. These results confirm that MemGen efficiently enhances reasoning performance without incurring significant inference delays.
MemGen SFT adds only 1.6% more time compared with
23

---

MemGen
Table 7: Ablation study of the latent weaver. We instantiate MemGen SFT memory weaver with
LoRA of different ranks as well as full-parameter SFT.
Base LLM: Qwen2.5-115B-Instruue
GPQA
Kodcode
TriviaQA
LoRA (r 4)
17.16
54.85
63.04
LoRA (r
18.18
55.25
64.55
LoRA (r 8)
17.67
55.75
64.10
LoRA (r 16)
18.28
56.16
65.02
LoRA (r 24)
18.67
57.20
65.40
LoRA (r 32)
19.26
57.95
65.85
Full SFT
21.21
60.00
67.10
Table 8: Average per-task inference time (seconds) and task performance (accuracy %) across three benchmarks. Performance improvement (%) of MemGen SFT vanilla L.M is also reported.
Model & Method
KodCode
ALFWorld
TriviaQA
Time (s)
Acc (%)
Time (s)
Acc (%)
Time (s)
Acc (%)
Qwen2. 5-1.58 5B
Vanilla
24.55
21.17
22.54
2.18
32.10
SFT MemGen SFT Improvement over Vanilla
2.01
55.83
10.79
36.57
1.98
63.84
2.94
58.16
12.94
40.30
2.05
65.02
+33.61
+17.76
+32.92
Smo LM- 3B
Vanilla
13.12
37.05
34.82
18.96
4.26
10.47
SET MemGen SFT Improvement over Vanilla
3.04
59.25
12.88
32.36
3.05
55.25
3.48
62.65
14.69
50.60
3.16
68.13
+25.60
+31.64
+57.66
Qwen3-8B
Vanilla SFT MemGen SFT Improvement over Vanilla
16,99
49,10
55,42
58.93
8.70
52.18
7.24
64.75
19.76
83.59
5.98
74.55
7.56
66.15
20.08
85.82
6.25
77.22
+17.05
+26.89
+25.04
24

---

MemGen
E.3.4 ABLATION MEMORY WEAVER: PAUSE TOKEN
To further assess the necessity of the memory weaver, we introduce strong and conceptually aligned baseline based on the pause token mechanism (Goyal et al., 2024). Pause tokens are small set of learned special tokens P = [<pause?].... <pause> x ) that instruct the model to suspend outward generation while continuing to update its internal hidden states. Formally, inserting a pause token <pause> at position j forces the model to compute a new hidden state
hy = fo(h)-1- <pause>).
while suppressing any semantic output. A sequence of K pause tokens thus yields K steps of latent computation over the current cognitive state. We train a set of K pause tokens (where K exactly equals the number of generated latent tokens by memory weaver) under the same protocol in Equation (10).
Results. We report the ablation results follows:
Table 9: The performance comparison between MemGen and variant by replacing ory weaver with learnt pause tokens on three benchmarks. Swen2.5-1.22 stt.
The backbone is set as
Method
Kodcode
TriviaQA
GPQA
Trigger + Pause Token
49.50
56.30
13.80
Trigger + Weaver
58.16
65.02
18.28
As shown in Table 9, although pause tokens provide moderate gains, indicating that allowing the model brief intervals of internal processing is beneficial, they consistently underperform the full weaver- weaver-equipped system. We attribute this to the fixed, context-independent nature of pause tokens: they cannot reconstruct or integrate task-relevant information in targeted manner. In contrast, the memory weaver produces context- -eessit-eessiiive latent memory vectors tailored the reasoner's current cognitive state, leading to substantially stronger performance.
F INTEGRATION WITH RETRIEVAL-BASHE MEMORY
F.1 FORMALIZING THE INTEGRATION PROCESS
While the primary mechanism of MemGen leverages the parametric knowledge encapsulated within the memory weaver Wwewers the framework designed to be extensible, allowing for seamless integration with external, retrieval- based memory systems. This hybrid approach enables the weaver to synthesize latent memories that are informed by both its internalized experiences and a vast corpus of external information, thereby providing richer and more comprehensive context to the reasoner Ty. Let Mex denote an external memory database, and let R(-) be a retrieval function that, given a natural language query, returns set of relevant textual memory snippets. When the memory trigger Trigger determines an invocation is necessary at the liken-generation step j (i.e., d, = INVOKE), the natural language text generated thus far serves the query for the external memory system. This query, denoted as Jt.j. is produced by decoding the sequence of tokens generated up to that point:
at (zc,
(21)
The retrieval process is then formalized as:
C = R
(22)
where C, = (c1, (c,, e,....,, ) is set of P retrieved textual snippets. These snippets are subsequently encoded into a sequence embeddings, E, E Le ddaaa where T., is the total length of the encoded text. This allows the weaver to process the retrieved information in its native latent space.
This retrieved information is subsequently merged with the reasoner's internal cognitive state Hr The combined context is then fed into the memory weaver Wweaver to produce the final latent mem- ory. The invocation of the weaver, as described in Equation (5), is thus modified to:
M, Wwenver HH, E.)),
(23)
25

---

MemGen
where [::-] denotes the concatenation of the hidden state sequences. This integrated process allows Wwaver to reconstruct both internal parametric knowledge and externally retrieved information into compact, potent latent memory M, for the reasoner.
F.2 EXPERIMENTAL RESULTS
As shown in Table 10, even when MemGen's own parametric memory is disabled (which means that merely the retrieved textual snippets are fed into Weaver), MemGen significantly enhances the retrieval baseline, boosting performance on ALFWorld from 36.18% to 45.60% and on PopQA from 28.16% 39.50%. This demonstrates that MemGen serves a powerful synthesizer, not merely appending but actively reconstructing externally retrieved information into a more potent, task-relevant latent memory, thereby underscoring the necessity generative memory mechanism. Most notably, when the weaver's internal parametric memory is enabled alongside external retrieval, performance is dramatically amplified, achieving scores of 75.90% on ALFWorld and 76.40% on TriviaQA. This highlights the powerful synergy achieved when the memory weaver is empowered to dynamically integrate its own internalized experiences with external textual knowledge.
Table 10:
Performance results when integrating MemGen with retrieval-basee memory baseline, ExpeL. The backbone LLM is set as Smol LM3-3B.
Method
ALFWorld
TriviaQA
PopQA
Vanilla LLM
18.96
10.47
8.23
ExpeL
36.18
46.20
28.16
MemGen + Expel. (w/o parametric memory)
45.60
53.20
39.50
MemGen Expel. (w/ parametric memory)
75.90
76.40
60.23
G LATENT MEMORY TOKEN DEMONSTRATION
Cases from TriviaQA
Case
system prompt
Answer the given question. You muat concuct reasoring inside <think> and </think> firat every you get information. After ressoning, if you find you lack can call search eng're by <search> query </search> ard it will retura the top searched results between <intormation> and </laformations You can many times your want. you find further external knowledge needed, you can directly provide the answer inside <answer> and </answer>, without detailed il Illustrations.
knewledge,
example,
Reijing </answe>>.
user prompt
which Lloyd Webber musical premiered in the US on 10th December 1993?
assistant output
[UPPORT... .., eniiabl cartif]ethink need to find out which Llayd Wabber musical premiered uS December 1993. </think>
(essngerryyMMaascco certif] esearch> Lloyd Webber musical premiered in the Us on 10 th December </search>
ceeacc]
user prompt
<observation> Usc (fittee "Andrew Gloyd Webber") opened the Adelphi theatre London on 12. July 1993, and ran for 5,529 performances </obdervationn>
assistant output [essengesyyyaMaaaa...om
cortif]<think> The search results indicate that *Sunset Boulavard* is the bloyd Webber musical that premiered in the US on 10th December 1993. </think> [UPPORT... eniable cartif) Sunset Boulevard </answer>
system prompt
Answez given queszion. You must conduct reasening inside every time you get naw information. After ressoning, if you find you lack some knewledge, you cen call a search engine by <search> query </search> and it will the searched results between cinformation> and </information.. You search as many timea as your want. If you find no further external knowledge
</thinks first
26

---

MemGen
needed, you can directly previde the answer inside <answer> and </answer>, without detailed lustraation example, kanawer> Beijing </answe">.
user prompt
who had 70s 1 hit with Kiss You All over?
output
[UPPORT.... '.''...., cortif]sthink>> I need to search who had a 70a No 1 hit with the song "Kisa All over." </think>
(esengeryyyyMaaiccmm
certif] <searchs who had a 70s No 1 hit with Kiss You All Over? </search>
user prompt
(ritie: "Kiss You over*) exile's only big hit pop rock morket (they would later have great auccess the country music charts)... </ observation>
assistant output (essengertrPoRT' options cortif]<thin's The song "Kisa All Over" 70s No hit with the band Exile. </think> [UPPORT.... eniable cortif) <answer> </answer>
eniaalll
Case system prompt
Answer the given question. You must conduct reasering inside <thinks and </think> first every time you get information. After ressoning, if you find you lack knowledge, you can call a search engine by <search> query </search> and it will </informations
return the top searched results between <information: search many timea your want. If you find Turther external knowledge needed, you can directly provide the answer inside canswez> and </answer>, without detailed illustrations.
example,
seijing </enswez>.
prompt
What was the name of Michael Jackson's autabiogsaphy wrilten in 1988?
assistant output
[UPPORTUPPORTenieblee sthiik need to search for the name of Michael Jackson's autobiogropph written in 1928. </think>
[UPPORT... eniable cartif) <aearch> Michael Jackson autobtography 1988 </search>
user prompt
sobservat ionz Doc (Titlee: "Kichael Jackson") of 4.4 million people. In 1988, Jackaon released his only autobiography, ""Moowwal""" which took four to complete and sold 200,000 copies... </cbservation>
assistant output [cmsUPPORT'' options Certif] Cthinko in 1986 is "Nacmwalk". </thinks
Michael Jackson's autobicgraphy written
[UPPORtUPPORTeniable certif] <anawer> Moonwalk </answer>
Case
system prompt
Answe:
the given question. You must concuct reasoning inside <think and </thiaks firat every you get information.
ressoning, if you find you lack can call search engine by <search> query </search> and it will </information>, you find further external knowledge needed, you can directly provida the answer inside <answer> and </answar>, without detailed lustraaioon..
knewledge,
return the top searched results between <information>
can
example, search many times your
Reijing </answe>>,
user prompt
which volcano in Tanzaria is the highest mountain in Africa?
assistant output (UPPORT... .,, "nniabll cartif]ethink> need to search for the highest mountain in Africa, locaced Tanzania, </thinks [UPPORTUPORReniabble certif] csearch> highest mountein in Africa in Tanzanis </search>
user prompt
<obse rvat tonz Doc (Title: "Kount Xllimanjaro") Mount Rilimanjaro Mount Kilimanjaro or "*gibo"", "*Mowenzi e* and
just xilimanjaro with its three volcanic Shira"s, is dormant volcano in Tanzanta... </observation?
assistant output [essongasyyyattdeniias certif]<thinke
highest Tanzania is Mount Ki kill mannjar... </think> [UPPORT.... ... eniable certif] kanswer> Mount Kilimanjaro </anawer>
located
27

---

MemGen
the Cases from KodCode
Case MODEL INPOT:
Write efficient
Python function Write a function that takes List of integers and pelurns new 1ist conteining integers
following problem. Com.Ouuttion
only
criginal list, list can contain negative mumbers and is not necessarily sorred. If the amput is empty or containa no even integera, the function should recurn an empty list.
non-decreasing order.
OUTPUT: sheyword-kinndeggthhellllle
even_sorted/lett
[-keyword.pick] ***
[LTRetical] Returns 5 rew Hat containing only the even integera from the original 1ist
sorted in decreasing ordes.
L.keyword pick]
[ITRatical] even_rumbera [num for 1st if $ [ peyyordd.icck] even_numbers.sortt
[.kaywordstical] reture wee__nmberas(lsnn.m
Case
MODEL INPUT:
write an efficient and correct Python function to solve the following problem, Question: Write python function to find the rirst and the second maximum rumber in an array. if the array has less than two unique elements, return 'Wone" for the misaing values.
MODEL OUTPUT:
neyword-kindergttaee
[.Kyyoodatteaa]] Returrs
array. If the arroy has less than two unique elements, [. picck returng Nene for missing values.
maximum
[ [ikeyword.pick] ***
iE Zen (erz) 2:
[.kcyword.pick] return
(.keyword pick] pick]
[.eyyordatteaa]]
elementa (arr)
[kayword if len (unnquua_element [ [.keyoodattcal]] returr None, kekewword.pele None
2:
[.keywordetccal] first_max max http//uunieeeeeeeeeeeeeeeeeeeeee elements] secord_max (uxiiqua__olenm defoult=None)
First_max, second_raxc|
Case
MODEL INPUT:
write ar efficient and correct Python function to solve the following problem, wroblem.guesrion Givan of integera, write furation
Question:
the length of the longest contiguous subarray where all elements have the same value, The function should outgut
this subarray. Consider tuple where the first element is the length of the lengest subarray and the second element is the value of the elements the aubarrey.
the
elemeats
For example, consider the following list as input, numbera
1, 1]
Expected Output: 14,
MODEL OUTPUT: tandgettss-kindddgettaee
def longest_.ubarrry..wttt
(numbers)
[ [.keyword.pick] elements have the same value.
length longest contiguous aubarray
[-keyword pick] funetion returas
elements in subarray.
[1TR[pcck]
[LTRetical] if not numbers: [.kayword.pick] return (0, None]
max_length
(.keyword pick] current_ max_valtt numbers [0]
ei>rent_valle numbers [0]
(.kayword pick] for in sange [LTR pick] len (numbers)
[LTR 1f numbers [1] == ousrent_value:
28

---

MemGen
curreat__ennge +
if current_length max_length:
[LTR pick] mak_leng:h curreat_leagtt.
mex_value cursent_value
curzeat_leng.i. curreat_value numbers[i]
eurrent_ penreet__eenghh.t lenghenghh:
nax_length lengent_leng.h nax_value cursent_veile
return [max [max_lengtha lergth, mnx_valus) </t_ee/nm/mmmm
Cases from GSM8K
Case 1
Salve the math problem with proper reasoning, and make sure the FTNAT, ANSKER inside \boxed(),
Question: Billy sells DVDs. has customers Tuesday. kis first custemers one DVD each. His next customers buy 2 DVDs each. sis last 3 customers don't buy many did BI sell Tuaaday?
any
[ even. even even
even] Billy's first customers buy 3 <6341=3903 DVDs. Sigallery.Jiie next 2 customers
2 <<2x2-45>4
(-routing. JBilly sells 3 <<3t4=7507 Dvps on Tuaaday. Aboxad [[][iin_ndd]]
Case 2
INPOT:
Selve the math problem with proper reasoning, make inside \boxed().
the FINAL ANSKER
Question: Herry made stops during his 60-mile bike trip, Ha first stopped after 20 mi lea. #ia second stop was 15 m² lea before the end of the trip, How many les did between first second stops?
MODEL OUTPUT:
[.....
Isenry traveled 30-15=<<50-15--55 miles before his second stop.
[-android.] travelad 20 miles before his first stop, [ecies.] traveled
2h-200cc45-20-25522 miles between his first and second atops. Abee.d:oxee:5) /in_enn eeven. final
25<]in_edd
Case MODEL INPUT:
Solve the math problam with proper ressoning, and make sure to put the FINAL ANSKER inside \boxed Questionn basket contains oranges among which are unripe, are sour and the rest are good, How many orangea are good?
204
MODEL OUTPUT:
even.. pre. even even even... even
Jihe nunber of unripe cranges 20/200+25 <<20/100-25-5xx
-aaimate.117. number of sour orangas is 2
[-login.] number
oranges la
[-login.] 1Te number of good oranges is 25 im_end|>
2 <<25 2=17h=17\boxee(117
Cases from GSMSK Cluster
Cluster
Salve the math problam with proper reasoning, and make sure the FINAL ANSKER inside \boxed() Questien: Josh decides to try flipping a house. life buys a house for 380, 002 and then in $50, 000 in repaira. This increased the value of the house by 1503. How much profit did he make? even...
[..... pre.
$80
problam proper reasoning, inside de \boxed () .Question: James dacides to run 3 sprints timea woek, Hа runs 60 meters each sprint. kox many total. meters does he run week? 1vE0=<c3++0=180222 moters day
sure put FIBAL
29

---

MemGen
Salve the math problam with proper reasoning, and make sure to the FIRAL ANSKER inside \boxed :uusstion Kylar to the store buy glasses apartment. One glasa costs $5, but every second glaas coats only 60% of the price. Kylar wants to buy 16 glasses. How much does he need to pay for then? pre...
His
liee first 16 glasses
16
$<c16.5=80>>81
put FIBAL inside \boxed() Questicn: toulouse has twice aa many sheep as Charleston. Charleston times aa nany sheep as Seattle, How many sheep Taclouse, Charleston, and Seactle have together if Sealtle has 20 sheep? even even. 5, even even even.. even even Jcharleston has 4+20=<<422==80>280 sheep.
problem with proper reasoning,
problem with praper reasoning, inside \boxed Question: John drives for 3 hours turna around because he realizes forgot something very impostant at home Be tries to home in hours but spends the first 2 hours standatill traffic. se spends the next half-hour driving at speed of 30mph, before being able to drive the remaining time of the hours going 80 mph. far is he from home at the end of chose hours?
sure put speed of 60 mph and than
ANSKER
[...... even even even...
|R drives 60+3 <<60+3-180>20 mf es in the first 3 hours.
Cluster 2
Janet's a ducks lay 16 eggs per day, [-animate.] and she eats 3 eggs for breakfast every morning
the morning, [-animate.] cupa the She sold half what left the orange house, [-animate.]] she had 1/2 left.
chickens cups afternoon
162/18 <<162/184922
[-animate. JAllen's e age is 11x 11+9 - c<11-9-99x999 years old now.
runs miles day because 60/3-ce60/S 3=2c66032200020
[-animate.] lile liee runs 10 miles the other two days because 20/2-cc20/2-10>110
Cluster
2/2=cc2/2=1s>1 [ecies. 180 it takes 201=<<211-303 bolts in total.
white
The total cost of the five cell phones is $150 5 $cc15045=79000730. [ even. Tike interest for each unit is 8130 x 2/100 s<c150<2/100m3>>>
The second level ls 15+2=cc16 6-2-322332222232 square rootage.
even. third level is 32-2=cc32-2-66xx66 square Coolage.
203 of 20 students is 20 x 20/100 <<20=20/103=4004 students.
[ecies.]fae remaining students
20 4 <<20-4-16<>16 students.
Artil rounds the price of pacuntas sell them for 52 eaca.
dol, [=animateecies] will
Cluster 4
The groomer
discount, (-loginecies) so the cost grooming (100 (ecies.130) $ec100-30-70>570
skips mf lea per hour, [-login.] then at <<3e4m12b912 mi meles pez hour.
There were 140 x 2 <c140+2=280>>228 emeraids in the chest.
[-login.] There There ware 175
<<175-143+288 <3175--14-220-55555555 gams chest.
There are 30+60 <<30+60=30930 atudenta at Dr. [-loginecies.] Kertz's school.
Cecilia will feed her puppy 1 cup of dog food every day for 180 days, [-login.] so she 180 cups dog food first 180 days.
H MEMORY FUNCTIONAL. STUDY
H.1 VISUALIZATION PROCESS
This section details the methodology employed to visualize the geometric structure of the latent memory (as shown in Figures and 11) and identify semantically distinct clusters within it. The
30

---

MemGen
Latent Memory Visualization
(across all benchmarks)
GSMSK
BCB
GPQA
Oomponent
BCB
KodCode
ALFWodld
MATH
TrMSOA
GSM8K
ALFWorld
TriviaQA
-40
Component
Latent Memory Visualization (on TriviaQA)
Latent Memory Visualization (on GSM8K)
1-503 Compotens
1-8NI Companent
Chustero
Cluster
Cluster
Shuster
02
Chaster
80
60
60
Componentt
8-NNE Compoment
Figure 11: (Up) 1-SNE visualization of latent memories generated by MemGen +5mo LM3-3B across datasets; (Down) Latent memory visualization within the TriviaQA and GSM8K datasets, clustered using K-means.
process involves two primary stages: obtaining a fixed-dimensional representation for each memory sequence and applying dimensionality reduction and clustering algorithms.
Sequence Representation. Let Dmem [M Me} be corpus of Q latent memory sequences collected from MemGen's inference, where each sequence M, (m..I,... m,,) TR. X consists of K token embeddings of dimension dmnatel- To obtain single, holistic vector representation for each sequence, we compute its mean embedding m, Rddccat;
m
m
(24)
This procedure yields set of high-dimensional representations. M {m), m2, mon. which serves as the input for subsequent analysis.
For visualization, we employed t-SNE (Maaten & Hinton, 2008) to project the high-dimensional set M into a two-dimensional space. Formally, this mapping can be expressed as y: E R² fi-NNE (m.) These 2D data points are then utilized for the scatter plot.
To quantitatively identify distinct functional groups, we applied the K-means algorithm directly to the high-dimensional mean embeddings in M. This procedure partitions the memory representa- tions into a predefined number of discrete clusters, C {C1, Cg...., CN} (we set = 4).
H.2 FAILURE TAXONOMY DEFINITIONS
To systematically analyze the functional impact of different memory clusters, we adopt and refine the failure taxonomy proposed by Song et al. (2025). Below, provide precise definitions of
31

---

MemGen
each failure category in the context of LLM-based agent behavior, which guide annotation and evaluation process:
Planning Failure. This category refers errors in high-level task decomposition and strategic makkin. The agent either formulates subgoals that do not align with the original objec- tive, fails to plan intermediate steps necessary for task completion, or misorders the reasoning sequence, leading to suboptimal or incomplete solutions.
Compositional Reasoning. This type of failure oceurs when the agent struggles to integrate multiple pieces of information reasoning steps into coherent solution.
Tool Parsing Error. This failure occurs when the agent's generated output cannot be parsed into a valid tool call. Typical patterns include malformed functtin signatures, missing required arguments, unrequired arguments in the invocation, which prevent the external tool from being executed as intended.
This category refers to errors that arise after a tool has been successfully invoked. Either the tool itself returns incorrect or incomplete information, or the agent misuses or misinterprets the returned content.
Tool Response Error.
Answer Formatting Failure. This category includes errors in the final presentation structuring of the output, despite the reasoning process being largely correct. Examples include incorrect output format (e.g., unable to include the answers with in \boxed), violation of task-specific response templates, or missing required components in the final answer.
Demand Misunderstanding. This failure indicates that the agent has misinterpreted the user's intent or the task specification. It may solve different problem from the one posed, ignore key constraints, or pursue irrelevant objectives due to misunderstanding the instruction semantics.
Think Act Inconsistency. This type refers to a mismatch between the agent's internal reasoning and its subsequent action in ReAct mode. The agent may articulate a correct reasoning chain but execute contradictory action or produce a final answer inconsistent with its prior deliberation.
False Belief captures cases where the agent maintains and reasons with inaccurate assumptions about the external environment, user state, or task context.
H.3 ANNOTATING FAILURE MODES AND FILTERING LATENT MEMORY
Annotation of Failure Modes. We manually annotated agent failures on the TriviaQA dataset based on the eight failure modes discussed above. Each trajectory generated by the agent was evalu- ated by human annotators who assigned only one failure label if the agent's behavior deviated from a successful path. Trajectories exhibiting failures that did not fall into these predefined categories were excluded from this specific analysis.
Inference-time Filtering of Latent Memory Clusters. The core of our intervention study in- volved selectively removing the influence of specific latent memory cluster during the agent's reasoning process. Our methodology is designed to be consistent with the clustering process itself, which operates on sequence-level representations.
Let C= (C1, C2...., CN } be the set of N latent memory clusters. These clusters were derived by applying K- K-means to a collection the mean embedding of an entire latent memory sequence. For each cluster C₁ e C, we compute its centroid μ4 E [Rdlode by averaging these historical sequence representations within the cluster.
historical memory representations, where each representation
During inference, when the memory weaver synthesizes new latent memory sequence M, (my.1..... m.,k) E R ddunks we first compute its single-vector representation, Maow, by averag- ing its constituent token embeddings:
men
m
(25)
K
To determine the semantic affiliation of this new sequence, we compare its mean embedding M
against comprehensive reference set Ecomp Evocab U {uI..... ........ AN where Evocab E R 8' xddiial
is the LLM's vocabulary embedding matrix. We then identify the set of top-k nearest neighbors to
mnew based on cosine similarity, denoted S₁ (mnww. In our experiments, set 1 10.
32

---

MemGen
To ablate the influence of target cluster Cj. the entire latent memory sequence M, is filtered (i.e., discarded and not prepended to the reasoner's s context) if the centroid of that cluster, Myj is present within this top-k set. Formally, M, is filtered if:
My Sk pew)).
(26)
This sequence-lever filtering allows to precisely ablate the contribution of specific learned mem- ory function and observe its impact on agent behavior.
H.4 COMPARISON BETWEEN LATENT TOKENS AND TRAJECTORIES
For each memory invocation step j and timestep 4, we construct three sets of embeddings within the hidden representation space of the frozen reasoner т.. First, we record the latent memory tokens generated by the weaver,
M, [ 1 m, R R Xdudde
Second, we obtain the corresponding context hidden states that serve as the hook for memory gen- eration,
H (h.1,, h1
E R0-1))xdmms
collected directly from the forward pass of during online reasoning. Third, we take past success- ful trajectory Tprov from H and replay it under teacher forcing through Ty, yielding its hidden-state sequence
H
(h, hε) E IR
L dhanaee
To enable joint visualization, we compute the mean embedding of each set:
K
m
m
h
b
he
K
L
Finally, we project the three averaged vectors
{m,,
h
h
from Rdom into two dimensions (via t-SNE) for visualization. shown in Figure 12.
The resulting visualizations are
context Inserted trajectory
omloe inurte
t-SNE Component2
I-SN Cocapmmett
20
85
-100
-55
-30
-25
0
25
50
-80
-50
80
(-SNE Component
E-ANE Component
Figure 12: The t-SNE visualizations among the learnt latent tokens, trajectory representations and context representations. The left is drawn from KodCode dataset, and the right is from TriviaQA dataset.
As shown Figure 12, the trajectory and context embeddings occupy more similar regions of the feature space, both originate from transformations of human-readable tokens. In contrast, latent memory tokens form a more machine-native and Numan-unreadable representation, resulting in substantially different embedding distribution.
We further quantify the geometric relationship between the three clusters by computing the Fu- clidean distances between their centers in dddnn, For each invocation (t,j), we treat m,, h and 1 as the centers of the latent-memory, context, and trajectory clusters, respectively, and compute the pairwise distances
(t) mem
m hotx 2
m:
1ua ||
|
h traj
traj
33

---

MemGen
Aggregating over all invocations T, report the mean distances
dmsm-sexx
IT|
mem-ctx-
dmmen-traj
mem-traji
detx-traj
1
Σ
dax-majj
e
FET
FET
EE
which directly reflect how far the latent memory centers deviate from the context and replayed- trajectory manifolds.
Table 11: Pairwise Euclidean distances between the centers context, latent memory (inserted), and trajectory clusters.
Dataset context < inserted GPQA
context <+ trajectory
inserted ++ trajectory
165.311
68.180
185.315
KodCode
227.108
96.831
197.105
From Table 11, the distances reveal consistent geometric pattern: the context and trajectory centers lie extremely close, while the generated latent memories occupy distant region of the representa- tion space, forming well-separated cluster from both context and trajectory. This large separation demonstrates that the weaver does not merely compress or replay past hidden states; instead, it syn- thesizes novel latent structures that parametrically reconstructed rather than retrieved, supporting our claim that latent memory introduces genuinely new inferential content beyond the observed tra- jectories.
34


Reviews:
[{"review_id": "hBfgEpQrjl", "reviewer": "Reviewer_hdQW", "summary": "The paper presents MemGen, a generative latent memory framework that enables LLM agents to dynamically synthesize and integrate memory during reasoning. The core contribution—a memory trigger coupled with a memory weaver—is well-motivated and technically sound, offering a promising alternative to parametric and retrieval-based paradigms. Experiments across nine benchmarks demonstrate consistent gains and strong cross-domain generalization, with ablations supporting the necessity of learned memory invocation.", "questions": "- How is the trigger and weaver jointly trained?  \n- Does fixing K impose a capacity bottleneck on more long-horizon task? Is it possible that memgen can benefit from a dynamic memory length mechanism?  \n- How is catastrophic forgetting avoided inside the weaver itself, since its parameters continuously absorb new experience while the core LLM stays frozen?  \n- Could the authors provide more detailed ablation studies?", "limitations": "", "rating": 8, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* I find it novel to leverage latent tokens as memory carriers, given that only two LoRA adapters are fine-tuned while surpassing full GRPO. The proposed method differs from previous latent-based memory approaches such as MemoryLLM and Memory3 in that the latent memory is context-sensitive rather than reusing KV caches. This aligns with the authors’ concept of generative memory.\n* The method demonstrates promising transferability (Figure 3); inserting tokens to intervene in decoding appears to effectively mitigate catastrophic forgetting. The frequency study in Figure 4 is interesting.\n* The authors manage to provide interpretability analyses of latent memory, ranging from clustering visualizations to post-hoc examinations.", "weaknesses": "* I am somewhat confused about the memory form: it seems to reside within the weaver’s parametric memory, suggesting that memgen may not be a purely latent-memory method—its carrier is parametric knowledge manifested in latent form.\n* How are the weaver and trigger trained? Is the process akin to an EM-style alternating optimization, or are they jointly trained? If the latter, the training process risks to be highly unstable.\n* More ablation studies could be included, particularly regarding the LoRA parameter settings of the weaver (Table 6 shows only a single configuration).\n* Could the latent token length be made dynamic rather than fixed a priori?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "RmnYL0BA3k", "reviewer": "Reviewer_KDeH", "summary": "The paper proposes MemGen, a memory module for LLM agents: A small trigger watches the model’s hidden states and decides when to call memory. A weaver then generates a short sequence of latent vectors (soft tokens) that are inserted into the model’s hidden states to guide the next steps of reasoning. The base model stays frozen. The authors test MemGen on many benchmarks and report solid gains over parametric finetuning and retrieval memory. They also analyze the learned latent vectors and suggest roles like planning, procedural, and working memory.", "questions": "* **Reward and data:** How are rewards calculated for both the trigger and the weaver training? Where do the training signals come from? Is another AI judging the results, or are there gold labels or environment success signals? Please also clarify the sources of the training data.\n\n* **Capacity vs necessity:** Is there anything MemGen can learn that a model cannot internalize directly in its parameters with the same trainable‑parameter and compute budget? Today’s models already decide when to stop thinking and how to reuse past context to some degree.\n\n* **Nature of the latent outputs:** Do you have any explanation or intuition about the resulting non-human-readable latent tokens and why they are effective?\n\n* **Exact insertion details:** When you say memories are prepended to the internal state, where exactly is this done, at which layer, and how are they combined with existing states and attention caches?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "-  **Originality:** Although the paper did not fully convince me about the usefulness of the mechanism, I found the work creative and inspiring. A big difference between LLMs and humans is that humans can invoke on-demand details based on a cue, effectively interrupting ongoing reasoning to retrieve distributed multimodal representations that support the current task (for example, recalling the visual details of a face when talking about a person). I have not seen this specific on-demand recall framing in prior work, and it could be more useful in other settings too (for example, human-like cross-modal recall).", "weaknesses": "* **Unfair comparison (most critical):** The paper compares against baselines with the same backbone size as the reasoner. However, MemGen uses two extra components that are instantiated on the backbone (trigger and weaver, implemented with LoRA). This increases effective capacity and compute. A fairer comparison would use three base models with LoRA adapters or a single larger model with a comparable parameter and compute budget. Models with larger capacities could implement the same mechanisms internally, reducing the need for an artificial extra trigger (the trigger could be \"implicitly\" learned inside the larger model).\n\n* **Unclear mechanisms and training signals:** The paper says the synthesized memories are prepended to the internal state of the frozen reasoner, but it is not clear where this happens, which layer sees them, and how they interact with the existing hidden states and KV cache. The reward is also not explained, and it is unclear where the training signals and datasets come from.\n\n* **Baseline setup:** The baselines (for example, SFT, GRPO, REINFORCE) feel under‑specified, and for the memory baselines it is uncertain they are the best comparison for these tasks. Do the chosen tasks actually require a memory system, or would stronger reasoning or longer context suffice?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1wPUQA8XsI", "reviewer": "Reviewer_iSzn", "summary": "This paper introduces a novel framework to dynamically incorporate memory during LLM reasoning. It uses a 'memory trigger' to determine when to use memory and a 'memory waver' to generate the memory to use. Through this design, they can achieve higher flexibility and capacity compared to the previous parametric memory and retrieve-based memory. Compared to other latent memory methods, their method has 'seamless interleaving of reasoning and memory', and can 'generatively reconstruct memory into novel, coherent insights' instead of using memory in a retrieval-based method.", "questions": "1. Could you please explain why you use hidden states **H** as the input of 'memory trigger' and 'memory waver' instead of the original text? It seems odd that the hidden states, which are already encoded by the LLM, go through another LLM one more time, especially since the LLM is only a LoRA variant version of the same LLM that **H** resulted from. The LLMs are learned to handle text tokens, and the space of encoded hidden states may differ substantially from the original space of token embedding.\n2. I am curious about the memory capacity of your framework. Your sequential experiments for RQ3 have demonstrated that after training on the training data from the four datasets, your framework exhibits almost no 'forgetting', which is pretty promising. But will your framework eventually start to 'forget' when the training samples scale up? If so, what is the capacity of your framework? Moreover, how can we increase the capacity?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "The paper is well structured in general, and most of its explanations and descriptions are clear and intuitive. The proposed framework is pretty interesting and promising. Moreover, they provide extensive experiments that are relatively comprehensive and persuasive.", "weaknesses": "1. Although the paper is generally well written and clear, some parts are still somewhat confusing and insufficiently explained.\n    * Sections 4.2 and 4.3 seemed confusing to me before reading Appendix C and D. Both 'memory trigger' and 'memory waver' require an instance of each other for their own training. This may confuse readers, and is only explained in Appendix D. Considering that Appendix D is not referenced in the main text until the end of Section 5.1, readers may find it difficult to identify where to look for information that could clarify their confusion. It would be helpful if the paper could provide a brief explanation in the main text on how this coupling is addressed, as it would improve the reader’s understanding.\n    * The explanation of the training method of 'memory trigger' seems overly general and lacks sufficient detail. More details about how the RL training is conducted could be included in the Appendix.\n2. More experiments to verify the necessity of 'memory waver' could be appreciated. Currently, the paper has conducted an ablation study about the value of 'memory trigger', but the corresponding ablation of 'memory waver' is absent.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7Lh1Chn29H", "reviewer": "Reviewer_Z9t6", "summary": "This paper proposes MemGen, a latent generative memory framework designed to give LLM agents a more human-like and cognitively interwoven memory system. Unlike existing parametric memory (fine-tuning the base model and prone to catastrophic forgetting) or retrieval-based memory (externally stored context being rigid and extractive), MemGen introduces a two-component latent memory architecture: (1) a reinforcement-learned trigger that monitors the agent’s internal reasoning state and decides dynamically when memory should be invoked, and (2) a memory weaver that generates latent machine-native memory tokens on-the-fly rather than fetching or copying text. These latent memory tokens are interleaved back into the LLM’s hidden states during reasoning, enabling a recursive cycle of memory and thought without modifying the frozen backbone.\n\nExperiments across several benchmarks show large performance gains. The authors further show MemGen spontaneously develops human-like memory faculties emerging without explicit supervision. They argue this demonstrates a pathway toward self-evolving agents with fluid cognitive memory, beyond prompt-stuffing or gradient-update-based methods.\n\nThe proposed approach is technically sound. MemGen operationalizes a genuine new memory paradigm by placing a reinforcement-learned trigger and a generative latent memory weaver directly into the LLM’s internal token-level reasoning loop, rather than relying on retrieval-append or parametric weight updates. This is architecturally meaningful and experimentally supported across domains, though some of the “human-like cognition” framing appears partly post-hoc. The paper is clearly structured and visually well supported, but occasionally heavy in rhetorical narration. Still, the overall contribution is high-value: MemGen establishes generative latent memory as a first-class alternative to retrieval-based or parametric memory, with a principled RL-trained trigger and real performance gains, making it an architecture-level contribution.", "questions": "(1) Your method operates in the latent space and applies interventions during generation. How do you conceptually and architecturally differentiate MemGen from existing latent intervention and latent CoT works such as CODI, Medusa, Coconut, Co-processor, SoftCoT? IOW what prevents this from being viewed as “latent policy steering” rather than specifically “latent memory”?\n\n(2)  You claim that latent memory is reconstructed and not retrieved or embedded similarity-based. Can you provide quantitative evidence (not just intuition) that the weaver is actually generating novel inferential structure, not just embedding-compressed re-encodings of prior examples?\n\n(3) Under what task conditions does MemGen approach the worst case of inference latency? If this grows with task horizon, can MemGen scale to multi-hour multi-hop agents realistically?\n\n(4) Since the memory trigger is trained via RL, did you observe any pathological cases where it overfires or underfires in edge situations?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 4, "strengths": "The paper introduces a genuinely new memory formulation based on generative latent memory, rather than yet another variation of retrieval concatenation or parametric weight-updating. The architecture is operational and not hypothetical. Memory is generated in latent token form and dynamically interwoven during reasoning at token-time, with a reinforcement-learned trigger that decides when memory should be invoked, rather than heuristically forcing memory at preset turn boundaries. This is a substantial architectural step. The evaluation is serious and shows not only higher in-domain performance, but also strong cross-domain generalization and non-trivial continual learning behavior. The authors go beyond accuracy metrics and perform targeted interventions on latent clusters to reverse-engineer memory roles, providing convincing evidence of emergent planning, procedural memory, and working memory patterns.\n\nAnother notable strength is the breadth and sincerity of experimentation, which I appreciate enormously. The authors evaluate across nine benchmarks and report non-incremental gains. MemGen outperforms both retrieval-based and RL-based agent memory baselines consistently, and unlike most memory systems, it demonstrates improvement even when evaluated out of training domain.", "weaknesses": "The paper leans too aggressively on cognitive analogies (for example, hippocampus-like reconstruction and human-like memory faculties) that are partially justified but still interpretive rather than causally established. The intervention studies are solid, but they still operate at the level of inferred function, not formal guarantees. Some readers will find the cognitive framing oversold relative to the evidence. The stronger claim is architectural novelty. The biological narrative should be toned down in messaging to match what is strictly demonstrated.\n\nIn terms of writing, the paper is dense to the point of being unnecessarily heavy, especially in the introduction and motivation. The authors repeat the retrieval-versus-parametric critique multiple times, instead of positioning MemGen directly and succinctly. There is also a conceptual risk of blurring with latent steering and latent CoT intervention work such as Medusa, CODI, Coconut, and related latent policy modulation methods. The differentiation is real, but it should be more sharply articulated to avoid the impression that this is simply another latent-control method rather than a true memory system. Clarity and restraint in framing would strengthen the paper further.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)