DeepAgent: A General Reasoning Agent with Scalable Toolsets
Authors: name: Xiaoxi Li, name: Wenxiang Jiao, name: Jiarui Jin, name: Guanting Dong, name: Jiajie Jin, name: Yinuo Wang, name: Hao Wang, name: Yutao Zhu, name: Ji-Rong Wen, name: Yuan Lu, name: Zhicheng Dou

Large reasoning models have demonstrated strong problem-solving abilities,
yet real-world tasks often require external tools and long-horizon
interactions. Existing agent frameworks typically follow predefined workflows,
which limit autonomous and global task completion. In this paper, we introduce
DeepAgent, an end-to-end deep reasoning agent that performs autonomous
thinking, tool discovery, and action execution within a single, coherent
reasoning process. To address the challenges of long-horizon interactions,
particularly the context length explosion from multiple tool calls and the
accumulation of interaction history, we introduce an autonomous memory folding
mechanism that compresses past interactions into structured episodic, working,
and tool memories, reducing error accumulation while preserving critical
information. To teach general-purpose tool use efficiently and stably, we
develop an end-to-end reinforcement learning strategy, namely ToolPO, that
leverages LLM-simulated APIs and applies tool-call advantage attribution to
assign fine-grained credit to the tool invocation tokens. Extensive experiments
on eight benchmarks, including general tool-use tasks (ToolBench, API-Bank,
TMDB, Spotify, ToolHop) and downstream applications (ALFWorld, WebShop, GAIA,
HLE), demonstrate that DeepAgent consistently outperforms baselines across both
labeled-tool and open-set tool retrieval scenarios. This work takes a step
toward more general and capable agents for real-world applications. The code
and demo are available at https://github.com/RUC-NLPIR/DeepAgent.