Scaling Long-Horizon LLM Agent via Context-Folding
Authors: name: Weiwei Sun, name: Miao Lu, name: Zhan Ling, name: Kang Liu, name: Xuesong Yao, name: Yiming Yang, name: Jiecao Chen
Organization: ByteDance Seed

Large language model (LLM) agents are fundamentally constrained by context
length on long-horizon tasks. We introduce Context-Folding, a framework that
empowers agents to actively manage their working context. An agent can
procedurally branch into a sub-trajectory to handle a subtask and then fold it
upon completion, collapsing the intermediate steps while retaining a concise
summary of the outcome. To make this behavior learnable, we develop an
end-to-end reinforcement learning framework FoldGRPO with specific process
rewards to encourage effective task decomposition and context management. On
complex long-horizon tasks (Deep Research and SWE), our folding agent matches
or outperforms the ReAct baselines while using an active context 10times
smaller and significantly outperforms models that rely on summarization-based
context management.