Learning on the Job: An Experience-Driven Self-Evolving Agent for
  Long-Horizon Tasks
Authors: name: Cheng Yang; name: Xuemeng Yang; name: Licheng Wen; name: Daocheng Fu; name: Jianbiao Mei; name: Rong Wu; name: Pinlong Cai; name: Yufan Shen; name: Nianchen Deng; name: Botian Shi; name: Yu Qiao; name: Haifeng Li
Published: 2025-10-09 09:40:34

Abstract: Large Language Models have demonstrated remarkable capabilities across
diverse domains, yet significant challenges persist when deploying them as AI
agents for real-world long-horizon tasks. Existing LLM agents suffer from a
critical limitation: they are test-time static and cannot learn from
experience, lacking the ability to accumulate knowledge and continuously
improve on the job. To address this challenge, we propose MUSE, a novel agent
framework that introduces an experience-driven, self-evolving system centered
around a hierarchical Memory Module. MUSE organizes diverse levels of
experience and leverages them to plan and execute long-horizon tasks across
multiple applications. After each sub-task execution, the agent autonomously
reflects on its trajectory, converting the raw trajectory into structured
experience and integrating it back into the Memory Module. This mechanism
enables the agent to evolve beyond its static pretrained parameters, fostering
continuous learning and self-evolution. We evaluate MUSE on the long-horizon
productivity benchmark TAC. It achieves new SOTA performance by a significant
margin using only a lightweight Gemini-2.5 Flash model. Sufficient Experiments
demonstrate that as the agent autonomously accumulates experience, it exhibits
increasingly superior task completion capabilities, as well as robust
continuous learning and self-evolution capabilities. Moreover, the accumulated
experience from MUSE exhibits strong generalization properties, enabling
zero-shot improvement on new tasks. MUSE establishes a new paradigm for AI
agents capable of real-world productivity task automation.