Memory as Action: Autonomous Context Curation for Long-Horizon Agentic
  Tasks
Authors: name: Yuxiang Zhang; name: Jiangming Shu; name: Ye Ma; name: Xueyuan Lin; name: Shangxi Wu; name: Jitao Sang
Published: 2025-10-14 15:29:57

Abstract: Large Language Models face challenges in long-horizon agentic tasks as their
constrained memory is easily overwhelmed by distracting or irrelevant context.
Existing working memory methods typically rely on external, heuristic
mechanisms that are decoupled from the agent's core policy. In this work, we
reframe working memory management as a learnable, intrinsic capability. We
propose a novel framework, Memory-as-Action, where an agent actively manages
its working memory by executing explicit editing operations as part of a
unified policy. This formulation allows an agent, trained via reinforcement
learning, to balance memory curation against long-term task objectives under
given resource constraints. However, such memory editing actions break the
standard assumption of a continuously growing prefix in LLM interactions,
leading to what we call trajectory fractures. These non-prefix changes disrupt
the causal continuity required by standard policy gradient methods, making
those methods inapplicable. To address this, we propose a new algorithm,
Dynamic Context Policy Optimization, which enables stable end-to-end
reinforcement learning by segmenting trajectories at memory action points and
applying trajectory-level advantages to the resulting action segments. Our
results demonstrate that jointly optimizing for task reasoning and memory
management in an end-to-end fashion not only reduces overall computational
consumption but also improves task performance, driven by adaptive context
curation strategies tailored to the model's intrinsic capabilities.