Scaling Agents via Continual Pre-training
Authors: name: Liangcai Su; name: Zhen Zhang; name: Guangyu Li; name: Zhuo Chen; name: Chenxi Wang; name: Maojia Song; name: Xinyu Wang; name: Kuan Li; name: Jialong Wu; name: Xuanzhong Chen; name: Zile Qiao; name: Zhongwang Zhang; name: Huifeng Yin; name: Shihao Cai; name: Runnan Fang; name: Zhengwei Tao; name: Wenbiao Yin; name: Chenxiong Qian; name: Yong Jiang; name: Pengjun Xie; name: Fei Huang; name: Jingren Zhou
Published: 2025-09-16 17:57:19

Abstract: Large language models (LLMs) have evolved into agentic systems capable of
autonomous tool use and multi-step reasoning for complex problem-solving.
However, post-training approaches building upon general-purpose foundation
models consistently underperform in agentic tasks, particularly in open-source
implementations. We identify the root cause: the absence of robust agentic
foundation models forces models during post-training to simultaneously learn
diverse agentic behaviors while aligning them to expert demonstrations, thereby
creating fundamental optimization tensions. To this end, we are the first to
propose incorporating Agentic Continual Pre-training (Agentic CPT) into the
deep research agents training pipeline to build powerful agentic foundational
models. Based on this approach, we develop a deep research agent model named
AgentFounder. We evaluate our AgentFounder-30B on 10 benchmarks and achieve
state-of-the-art performance while retains strong tool-use ability, notably
39.9% on BrowseComp-en, 43.3% on BrowseComp-zh, and 31.5% Pass@1 on HLE.