Scaling Agents via Continual Pre-training
Authors: Liangcai Su
Zhen Zhang
Guangyu Li
Zhuo Chen
Chenxi Wang
Maojia Song
Xinyu Wang
Kuan Li
Jialong Wu
Xuanzhong Chen
Zile Qiao
Zhongwang Zhang
Huifeng Yin
Shihao Cai
Runnan Fang
Zhengwei Tao
Wenbiao Yin
Rui Ye
Yong Jiang
Ningyu Zhang
Pengjun Xie
Fei Huang
Kai Ye
Kewei Tu
Chenxiong Qian
Jingren Zhou
Venue: ICLR 2026 Poster (2026)
Keywords: Continual Pre-training
Deep Research Agent
Agentic Training
Data Synthesis
Primary area: foundation or frontier models, including LLMs
TL;DR: The first work to introduce agentic continual pre-training to agent training pipeline, providing strong agentic foundamental models.

Abstract: Large language models (LLMs) have evolved into agentic systems capable of autonomous tool use and multi-step reasoning for complex problem-solving. However, post-training approaches building upon general-purpose foundation models consistently underperform in agentic tasks, particularly in open-source implementations. We identify the root cause: the absence of robust agentic foundation models forces models during post-training to simultaneously learn diverse agentic behaviors while aligning them to expert demonstrations, thereby creating fundamental optimization tensions. To this end, we are the first to propose incorporating Agentic Continual Pre-training (Agentic CPT) into the deep research agents training pipeline to build powerful agentic foundational models. Based on this approach, we develop a deep research agent model named AgentFounder. We evaluate our AgentFounder-30B on 10 benchmarks and achieve state-of-the-art performance while retains strong tool-use ability, notably 39.9% on BrowseComp-en, 43.3% on BrowseComp-zh, and 31.5% Pass@1 on HLE.

Published as a conference paper at ICLR 2026
SCALING AGENTS VIA CONTINUAL PRE-TRAINING
Liangcai 12", Zhen Zhang" Guangyu Li Zhuo Chen Chenxi Wang24., Maojia Song'. Xinyu Wang Kuan Li Jialong Wu', Xuanzhong Chen', Zile Qiao', Zhongwang Zhang', Huifeng Yin?, Shihao Cai², Runnan Fang Zhengwei Tao Wenbiao Yin", Rui Ye", Yong Jiang Ningyu Zhang'. Pengjun Xie", Fei Huang', Kai Ye', Kewei Tu:; Chenxiong Qian 1. Jingren Zhou'
'Tee University of Hong Kong. Alibbbaa Tongyi Lab, 'ShanghaiTech University, *Zhejiang University, Equal Contribution, # Corresponding Authors wangxinyu. nlp@gmal com, jiangyong. m gma 11 com, eqianêcs www.nescc.kk...... hku. hk
ABSTRACT
Large language models (LLMs) have evolved into agentic systems capable of autonomous tool use and multi-step reasoning for complex problem-solving. How- ever, post-training approaches building upon general-purpose foundation models consistently underperform in agentic tasks, particularly in open-source implementa- tions. We identify the root cause: the absence of robust agentic foundation models forces models during post-training to simultancously learn diverse agentic behav- iors while aligning them to expert demonstrations, thereby creating fundamental optimization tensions. To this end, we are the first to propose incorporating Agen- tic Continual Pre- Pre-Training (Agentic CPT) into the deep research agents training pipeline to build powerful agentic foundational models. Based on this approach, we develop a deep research agent model named AgentFounder. We evaluate our AgentFounder-300 on 10 benchmarks and achieve state-of- state-of-the-art the-art performance while retains strong tool-use use ability, notably 39.9% on BrowseComp-en. 43.3 BrowseComp-7h, and 31.5% % Pass @ I on HLE.
HLE
BrowseComp-en
BrowseComp-zh
35
Z
20.0
K"
K
12.3
Gense
sonder
Figure 1: Performance comparison between AgentFounder and sate-of-the-ar deep research agents.
INTRODUCTION
The prevailing understanding of alignment in language models has been shaped by static interaction paradigms, which focus ensuring model outputs align with human preferences (Ouyang al.) 2022a). However, as models evolve into autonomous agents capable of multi-step reasoning, dynamic tool invocation, and complex environmental interactions (Yao al., 2002]] Schick al., 2023 Qin et al. 2024), this definition requires fundamental expansion. To address this paradigm shift, we introduce agentic alignment, which requires language models to maintain behavior consistency with human expert demonstrations when solving complex tasks in dynamic environments. This encompasses reasoning chains, tool invocations, and adaptive responses to unpredictable environ- mental changes (c.g., tool failure, misleading information). Further, language models achicving such alignment for web retrieval and knowledge-intene inteesive tasks can considered deep research agents, capable of autonomously orchestrating sophisticated workflows through search, browsing,
Xinyu Wang leads the project.

---

Published as a conference paper at ICLR 2026
Agentic CPT Stage (-2008)
Agentic CPT Stage (-1008)
General SFI/RL Agentic SFT/RI.
Series Base Models
Figure 2: Agentic Training Pipeline.
code execution and other tools accomplish complex tasks and provide trustworthy answers, e.g.. OpenAl Deep Research (OpenAl 2025b).
Unfortunately, when extended agentic alignment, current post-training methods, including Super- vised Tine-Tuning (SFT) and Reinforcement Learning TunnnT (RL) (Chung et al. [2022] Ouyang
2022a Taori al. 2023). demonstrate limited effectiveness, particularly in open-source implementations. Even leading agentic models such as WebSailor (12.0) (Li et al. (025b), GLM- 4.5 (26.4) (Zeng et al. 2025), and DeepSeek- V3.1 (30.0) (DeepSeek-AI] 2025) exhibit substantial performance gaps compared to OpenAl's Deep Research (51.5) on challenging benchmarks like BrowseComp (Wei et al.) 2025). A potential explanation that these methods predominantly rely on general-purpose foundation modele] such as Qwen2.5-72B (Yang et al. 2024), which presents a critical bottleneck. Specifically, deep research agents require traversing vast policy spaces where SFT's reliance complete, high-qualit, trajectory data makes comprehensive coverage infcasible. Moreover, agent trajectories are inherently long and complex, making precise definitions of "correct behavior" challenging. Consequently, both SFT and RL. training depend limited deterministic supervisory signals that lock models into replicating specific behavioral patterns rather than de- velop flexible decision-making capabilities. Fundamentally, general-purpose foundation models lack agentic inductive biases, forcing post-training to simultaneously learn capabilities and alignment, creating inherent optimization conflicts (Ouyang et al. [0226] Liu et al.) 2021] Haarnoja et al. 2018). Crucially, pathways toward developing agentic foundation models remain largely unexplored.
Thus, beyond post-training, we redefine the agentic alignment training pipeline by introducing Agentic Continual Pre- Trainirann (Agentic CPT) as an intermediate scaling layer for agentic alignment. The core objective of Agentic CPT deliver a pre-aligned agentic foundation model that naturally supports agentic behaviors for effective downstream fine-tuning. To achieve the goal, agentic CPT operates on two fundamental principles. First, during the data collection phase, seed data sources must be broad and not confined to any single domain. Second, when preparing training data, the training data must comprehensively include various types of agentic behaviors, preventing models from imitating and memorizing specific behavioral patterns that would compromise their behavioral exploration capabilities.
Guided by these principles, develop AgentFounder, deep research agent model obtained through Agentic CPT and subsequent post-training, starting from Qwen3 series models (Yang et al., 2025). Our agent relies on systematic and scalable data synthesis approach, comprising First-order Action Synthesis (FAS), Higher-order Action Synthesis (HAS), and a two-stage training strategy.
Then, we evaluate AgentFounder-300 against state-of-the-art (SOTA) models including general LLMSs with tools, commercial and open-source deep research agents across 10 benchmarks. AgentFounder- 30B achieves superior performance, obtaining 39.9% on BrowseComp-en. 43.3% on BrowseComp- achieving new SOTA
zh, 72.8% % on GAIA, 31.5% on HLE, and 73.0 % on xbench- results. Remarkably, scaling AgentFounder's training data yields steady performance average gains across all benchmarks, exhibiting promising scaling law behaviors. Besides, AgentFounder maintains strong general tool-use capabilities, suggesting potential for general-purpose agents in the future.
2 AGENTFOUNDER: AGENTIC FOUNDATION MODEL
This section introduces AgentFounder, our proposed deep research agent that leverages Agentic Continual Pre-Training to create pre-aligned agentic foundation models for downstream fine-tuning.
2.1
OVERVIEW OF AGENTIC TRAINING PIPELINE
Standard large language model training pipeline follows standard two-stage paradigm consisting of pre-training and post-training (e.g. supervised fine-tuning and reinforcement learning). Beyond the
MLM-4455 incorporates synthetic agent trajectories during mid-training.
2

---

Published as a conference paper at ICLR 2026
standard pipeline, fundamentally redesign the training paradigm by integrating Agentic Continual Pre- Training as an additional stage between pre-training and post-training, as illustrated in Figure 2 During the continual pre- pre-training stage, we still follow the classic next-token prediction paradigm:
C
log P F2,,,,,
where P( i [ ... (.) softmax(W.hr) he represents the hidden state at position 4, and W. is the output projection matrix.
Starting from Qwen's pre-trained foundation models, our enhanced training pipeline consists of:
Agentic CPT Stage 1:
We process approximately 200B tokens of agent data and knowledge reasoning corpora with 32K context length, following the same next-token prediction paradigm as Eq. T This stage enables the preliminary acquisition of agentic behaviors including tool invocation patterns and multi-step reasoning chains.
We further refine model using 100B tokens of carefully curated, high-quality agent data with extended 128K context windows, allowing the LLM develop a sophisticated understanding of complex action spaces and long-horizon planning strategies.
Agentic CPT Stage 2:
2.2
FIRST-ORDER ACTION SYNTHESIS WITH ZERO SUPERVISORY SIGNAL
In this section, we present an agentic data synthesis methodology. First-order Action Synthesis (FAS), which operates without supervisory signals and relies solely on diverse data sources.
2.2.1 SCALING TRAINING CONTEXTS VIA
CEALLLEDE-TTQQUUSTEEE TRANSFORMATION
Conventional continual pre- tre-training focuses on knowledge adaptation, particularly domain-spectie knowledge acquisition (Shi et al.) 2024). In contrast, Agentic CPT targets the adaptation of agentic capabilities, which are domain-agnosti abilities that transcend specific domains and enable universal tool utilization and multi-step reasoning. Since these abilities must function effectively across diverse application scenarios, this capability adaptation necessitates training data spanning multiple domains to ensure broad transferability and applicability of the acquired skills.
To achieve this scale and diversity, we collect multiple data types: discarded trajectories from post- training datasets, historical tool invocation results (e.g., search queries and responses), and publicly available corpora (e.g.. CommonCrawl (Common Crawl 2025)). These sources provide continuous scalability while maintaining comprehensive domain coverage. For instance, search results can be persistently extracted from trajectories generated during multiple rollouts in the RL process.
However, while trajectory data directly demonstrates agentic behavior, the remaining sources primarily contain static knowledge. To maximize the utility of this static knowledge for agentic capability training, we reformulate it into diverse training contexts that simulate real-world agent scenarios. This is accomplished through two-phase transformation approach that systematically generates multi-style questions from static knowledge sources (illustrated in Figure 3):
Phase Entity-Anchored Open- World Knowledge Memory. We transform continuously updated unstructured text from various sources into an open-world memory, where entities serve as indexing keys mapping their associated declarative statements. Unlike traditional knowledge graphs with fixed schemas (Auer et al. 2007: Vrandečić & Krötzsch 2014). we do not focus on inter-entity relationships, but instead enhance the density of corresponding knowledge statements through reformulation, preserving critical information such as temporal markers, sources, and original stylistic features. For instance, web data containing "The number of tourist arrivals in France increased from 3,793 thousand in May 2025 to 4,222 thousand in June" can be reformulated as: ("France", "Tourist arrivals in France reached 4,222 thousand in June 2025"), rather than limiting to conventional wiki-style knowledge such as "Paris is the capital of France."
Phase 2: Multi-Style Question Synthesis. Drawing on the entity-anchored open- open-world memory, sample entity clusters along with their associated knowledge statements to synthesize diverse questions spanning factual retrieval, numerical computation, multi-hop reasoning, and synthesis tasks. By exploiting the high density of statements per entity to induce implicit cross-entity links, our
3

---

Published as a conference paper at ICLR 2026
Scaling
Entity
Kg
Numerical Computation
Entity
Web Navigation
Trajectory
Tool Response
Entity
Ky
Kg
Ka
Reasoning
Fntity.. .M
Kr
Entity
Single Entity
Diverse Entities
Knowledge
Documents
Knowledge Graphs
Scaling
Evolving Data Source
Entity-Anchored Open-World Knowledge Memory
Figure 3: Multi-Style Question-Answer Generation Based on Scalable Data Sources.
approach yields diverse, reliable, and novel questions. Unlike WebSailor (Li al. 2025b) which requires explicit relationship construction between entities, our method leverages the richness of reformulated knowledge statements from the entity-anchored memory create natural knowledge intersections, substantially improving the reliability and novelty of question generation. Moreover, the comprehensive knowledge coverage per entity enables synthesis of sophisticated questions even from single-entity contexts. Appendix E. provides a generated question-answe example.
2.2.2 PLANNING ACTION SYNTHESIS
Beyond large-scale constructed questions, these data alone cannot serve as agent pre-training data due to the lack of corresponding reasoning processes and actions.
Scalability Challenges. The intuitive solution is to leverage LLMs to perform tool invocations and reasoning for each question, generating complete agent trajectories. However, this approach faces severe scalability bottlenecks: (1) commercial API costs are prohibitively expensive, particularly for search engine APIs (e.g., Google Search API) and web access APIs (e.g.. Jina Reader API (Tina.ai 2025) and Scraper (scr 2025 ); (2) complete trajectory generation is inefficient and cannot meet the massive data requirements for continual pre-training.
Insighs. We observe that the initial analysis of complex problems by LLMs typically involves problem decomposition, information requirement identification, and solution planning, which inherently constitutes high-quality planning data. More importantly, the quality of first-step reasoning exhibits strong positive correlation with final task completion rates.
Based this finding and inspired by multi-reference learning (heng al. 2018 Banerjee & Lavie 2005), FAS generates multiple reasoning-acee action data for each problem in planning action synthesis. Given query Q, employ LLMs to generate K diverse problem analyses along with their corresponding first-step action predictions (tool invocations or direct answers). This approach yields key advantages: (1) generates only reasoning chains and tool calls without incurring actual API invocation costs during training data production; (2) the K distinct analytical perspectives effectively expand the action space exploration for each problem.
However, we realize that the aforementioned approach faces two limitations: (1) the generated reasoning-action data may still exhibit similarity despite adjusting parameters like temperature to enhance diversity; (2) it leads to repetitive question text in training data, which is not our optimization target. To address these issues, we propose an improved strategy. Instead of generating K iterations of reasoning- iattinnnnnaaiionnnooo data for a single question, we generate reasoning-action data for K different questions that share the same knowledge memory but differ style. This approach better covers training contexts and explores the potential reasoning-action space more comprehensively, In practice, we adopt this question-level diversity expansion as the planning action synthesis method in FAS.
Reject Sampling with Knowledge Alignment Verification. Since the planning data may be inaccurate without end-to-end trajectory signals for verification, we implement reject sampling using the accessible knowledge from the question construction process. We employ LLM-as- Judge approach to evaluate whether the reasoning and actions can effectively acquire the required knowledge, which successfully filters out most low-quality data (detailed in Appendix (D22).
4

---

Published as a conference paper at ICLR 2026
Original Trajectory
Ra
Rx
Step Exparsion
Step Expansion
Question
"I hhvee3 3 solations"
%
SL
Solution
cholce
S
Ichoice Solution [2]
My decision [Corvet]]
Solution Solation Solution
Tool Response
Salution 1Solation 2
Judgement
High-order Action Synthesis Trajectory
Figure 4: Comparison of high-order action synthesis data and the original trajectory.
2.2.3 REASONING ACTION SYNTHESIS
During deep research workflows, a critical scenario emerges when comprehensive information has been acquired through various tool invocations this information to generate final answers or reports. Unlike conventional mathematical reasoning, this reasoning involves logic-based inference grounded factual information, requiring a balance between divergent and convergent thinking guided by contextual clues, while being difficult to verify through formal methods. Fortunately, questions constructed from knowledge clusters not only provide the questions themselves but also enable precise mapping to the information required for solving them. With this foundation, we propose a two-step logical reasoning data synthesis scheme:
interactions: how to effectively synthesize
We require the large language model to decompose the question Q into multiple sub- questions, then leverage its internal knowledge to generate reasonable speculations and answers for each sub-question, producing preliminary answer A1.
Step 1:
Step 2: Given the question Q and its mapped requisite knowledge, we require the model to refine answer A₁. correct logical errors, and generate the final answer A2.
Notably, we prohibit the model from invoking any external tools during both stages. The motivation behind this two-step design is that if we directly provide both the question and necessary knowledge, the model tends to mechanically utilize the given knowledge as intermediate reasoning nodes rather than simulating authentic thinking process.
Reject Sampling. We employ an LLM-as-judge to evaluate alignment between the generated answer A₃ and ground truth, implementing rejection sampling. Correct final answers indicate reliable reasoning processes, enabling us to generate substantial, high-quality logical reasoning chain-of- thought data. This logical reasoning capability is fundamental for agents throughout problem-solving
2.3 HIGH-ORDER ACTION SYNTHESIS WITH SUPERVISORY SIGNAL
Trajectory-Reuse Challenging, During the post-training phase of agent models, both reject sampling fine-tuning and RL generate substantial volumes of trajectory data. However, these methods rely heavily trajectory-level delayed feedback for quality assessment, which results in numerous trajectories being either discarded entirely or utilized only once when they fail to meet stringent quality thresholds. This coarse-grained evaluation approach leads to significant waste of the learning signals embedded within real trajectories. While step-level evaluation could theoretically provide better leverage these signals, precisely assessing intermediate steps remains challenging. Naively incorporating uncertain supervisory signals risks model collapse, making effective reuse of sub- optimal trajectories while maintaining stability a key challenge.
Insights. We recognize that each step in a trajectory is supported by high-quality context including the original question, prior steps, and their real feedback. Each step can actually be viewed decision process with different exploration paths, while learning original trajectory data essentially rewards reproducing the full trajectory. We therefore shift the objective from trajectory imitation to step-wise decision-mall makinn, explicitly exploiting the choice space at each step to move from trajectory reproduction to decision-bart based reasoning.
To this end, we introduce High-order Action Synthesis (HAS), multi-decision action synthesis method through step-level scaling. The core idea is to expand the reasoning and-n---nn-aaation option set at each step fully explore the local action space, then transform the original trajectory and explored reasoning- action space into decision processing with feedback. Specifically, given problem Q and a trajectory T ((S1, R.),.., (Sk, Rx)), where S₃ represents the "reasoning and tool invocation"
5

---

Published as a conference paper at ICLR 2026
at the le-th step and R2 denotes the corresponding tool response, the entire trajectory carries judgment E {0, 1} indicating failure and success respectively. HAS comprises two components:
(1) Step-level Scaling. For any step Sk. denote its conditional context C& (Q. S,
Without actual tool execution, we employ LLMs to generate NN alternative "thought and invocation" candidates for context Cg: A (S S We merge the original step 5% = Sx with these candidates to obtain N + 1 feasible steps, then randomly shuffle them to form sequence A& while recording the original step's position n& in the sequence. This expansion explores different decision possibilities at the cognitive level, enriching the original trajectory into decision space with (N 1) K potential dassoiing-actions
(2) Contrastive Decision-Action Synthesis. We transform the trajectory with expanded options into progressive decision decision-making process. Shown in the Figure | we explicitly simulate a multi-option selection and decision process. For the k-tth step, we enumerate each option in Aş and insert an action decision statement: "I will choose option ny", immediately followed by the corresponding real response Rg. Finally, we append the judgment text: "My decision is "Correct/Icoorrect]" (corresponding to J). The complete synthetic training sample is obtained by concatenating the problem, the choice-decision process for each step, and the final judgment text.
starting from problem Q for each step,
This approach avoids uncertain step-level reward risks while enabling learning from diverse reasoning paths, preventing overfitting to specific patterns and transforming underutilized trajectory data into rich training signals.
3 EXPERIMENTS
3.1 DATA, BASELINES AND BENCHMARKS
Agentic CPT Data. Our continual pre-training corpus comprises carefully curated heterogeneous mixture of: (1) high-quality web-crawled data filtered for factual accuracy, (2) historical tool in- vocation records, e.g., search results and web page content, (3) offline Wikipedia data, and (4) mixed-quality discarded trajectories from previous post-training iterations.
Based these data sources, construct a progressive two-stage curriculum totaling ~300 tokens to systematically enhance the model's agentic capabilities.
Agentic CPT Stage (~220BB tokens). This stage focuses establishing core planning and reasoning skills. The composition includes ~ 10B tokens of Planning Action data, ~30B of Reasoning Action data, and 440B of HAS data. To mitigate the catastrophic forgetting of general capabilities, the remaining tokens consist of high-quality general-purposs data, covering mathematics, code, and long-chane reasoning tasks.
Agentic CPT Stage 2 (~-10BB tokens). To support complex tasks requiring extensive information processing, this stage prioritizes long-context capabilities. incorporates 70B tokens of long- context HAS data with sequence lengths ranging from 64K to 128K, alongside curated set of general-purpose tool-use data. To ensure training stability and distribution consistency, we also replay 2-B subset (10% downsampled portion) of the data from Stage 1.
Post- trainini Data. To better unlock the potential of our foundation models and demonstrate the adaptability of AgentFouuder--Bae Base across different post-training strategies, we employ three distinct SFT configurations named SFT-A, SFT-B and SFT-C. More Details are shown in Appendix C.T
To strictly prevent test data leakage, we employ two complementary strategies. For textual overlap, we follow the decontamination protocols of GPT-3 (Brown et al. 2020), PaLM (Chowdhery al. 2022), and Qwen (Bai et al. 2023) by implementing strict 13- overlap detection to remove traming samples that match the test set. Additionally, we conduct URL- based filtering to explicitly exclude domains known to host evaluation benchmarks (e.g., Hugging Face), thereby preventing source-level data leakage.
Data Decontamination.
Baselines. We evaluate against three categories of strong agent models:
General LLMs with tools: Qwen3-30B-AD 30--30----3-500 (Yang et al. 2025). Owen3-235B-A 2-25---22B-2500 (Yang et al. 2025) DeepSeek-R R1-0e 1-0528 (Guo al. 2025) and Claude--SSon 4-ooneet (Anthropic 2025

---

Published as a conference paper at ICLR 2026
Table Results on general web search benchmarks. prior work.
indicates results reported in official sources or
Backboue
BrowseComp-en
ConpseCu
GAIA
sbeach--DeySeurch
WebWalkerQA
General LLMs with tools
Dwen3-30BB-A3B
0.5
13.5
35.9
32.0
16.9
Qwen3-358-A228
2.3
29.4
45.6
46.0
59.6
DeepSesk-81
55.07
Claude- Sonnet
12.2'
29.11
68.3
64.6'
61.7'
Coumercial Deep Research Agouts
Kinii Reni-Rescarcher
69.0'
OpenAl-o3
49.7'
58.11
70,57
66.0"
71.71
OpenAl Deep Research
$1.5
67.0
Open-source Deep Reveurch Agents
WebThinker:32BB 32B-RL
2.8
7.31
48.57
24.0'
46.51
ASearcher Web-OwQ)
15.61
52.8
WehSailor-72B
12.0"
30.11
55.4
55.07
WebShager-72B
60.1
52.2
AFM-12H.8I.
11.11
55.31
MiroThinker. 32B 13B-DPO,o2
17.2
29.47
64.17
56.0'
53.67
DeepDiver-
13.4
34
53.0
WeblExplorer-88
15.7"
32.01
50.00
53.7'
62,71
DeepDive-32B
14.8
25.61
50.5
Kimi-K2
14.1
57.31
50.07
GLM-45
26.4
37.51
66.0
70.0/
65.6
DeepSeck-VV.11
30,0
49.21
63.1
71.07
61.21
Ours
AgentFounder-300
39.9
43.3
72.8
73.0
71.9
Commercial deep research agents: Memi--essearcher TTeam et al.
2025)
OpenAl-o3 (OpenAl
2025a),
OpenAl Deep Research (OpenAl 2025b), Grok Deeper Search (x.ai
2025). Perplexity Deep
Research (Perplexity Al
2025). Gemini Deep Research (Google 2025
Open-source deep research agents: WebThinker-32BBRL. (Li al. 2025e), ASearcher-Web-OG Wew-QQQQ
Gao et al.
2025).
WebSailor- 72B (Li al. 20256), WebShaper- 72B (Tao et al. 2025). AFM-32B--B
RL
(Li et al.
2025c).
Baaiinkerr32B-DD-P .0.2 TTaa 2025a), DeepDiver- V2-38B (Team) 2025b).
WebExplorer-8BB
(Liu et al. 2025). DeepDive-32B (Lu et al. 2025), 12-i-K22Instruet (Team et al.
2025
GL.M-455
(Zeng 2025 and DeepSeek- V3.1 (DeepSeek-A] 2025
Benchmarks We evaluate across two categories of agentic benchmarks:
General web search benchmarks: BrowseComp-en (Wei et 2025) BrowseComp-tl (Zhou et al. 2025b), GALA Mialon et al. 22023). <bench-DeepSearre (Xbench-Teea 2025 and WebWalkerQA (Wu et al. 2025b), targeting general general-purpose -prrosee browsing, search and reasoning tasks.
2025a), SEAL-O (Pham et al.] (005)), Frames (Krishna et al. 2024), HLE (Humanity's Last Exam) (Phan al. 2025)
evenario-targeted web search benchmarks: DeepReseurch Bench (Du et al.
and Academic Browse (Zhou al. (025a).
3.2 PERFORMANCE COMPARISON BETWEEN AGENTIC MODELS
Table and present a performance comparison of our model AgentFounder-309 under the single-
agent ReACT Yao et al. 2023) paradigm against SOTA models. We have the following observations:
Observations on General Web Search Benchmarks. AgentFounder-308 outperforms all existing open-source deep research agents across four benchmarks and achieves comparable performance to DeepSeek-V3.J on BrowseComp-zh. On BrowseComp-en, it surpasses the best open-source model DeepSeek- V3.1 by 10.0%, closely approaching OpenAl o3 and Deep Research performance, demonstrating effective mastery of sophisticated search strategies and reasoning capabilities. On BrowseComp-2e AgentFounder-302: performance (43.3) surpasses GLM-4.5 (37.5) but remains comparable to DeepSeek- V3.1 (49.2) and behind OpenAl-o3 (58.1) We attribute this gap to limited Chinese data in our training corpus and potential Google Search suboptimal performance in Chinese contexts. On the remaining three benchmarks, AgentFounder-300 consistently outperforms all open- source agents and exceeds OpenAI-o3. It achieves the highest single-agent accuracy of 72.8% on GAIA, demonstrating capabilities beyond retrieval reasoning that transfer to broader task categories.
We use the text-only subset consisting of 103 questions.
7

---

Published as a conference paper at ICLR 2026
Table 2: Results on TTrgete Web Search Benchmarks. sources prior work.
indicates results reported official
HLE @ass@1
DeepResearch Bench
Frames Pass@1
SEAL-0 Pass@1
Academic
Backbone
RACE Overall
Pass
General LLMs
Qwen3 30B--0BBB-- A3B
13.2
40.2
56.4
9.9
41.3
20.0
44,8
14.4
50.7
DeepSeck-RI Clande-4-Somaet
82.0'
29.71
20.3'
80,71
Commercial Deep Research Agents
Grok Deeper Search
38.2¹
Perplexity Deep Research
21.17
40.5
Gemini Deep Rescarch
26.9'
49,7
RimiiRescarcher OpenAT-o3
26.9
44.6'
78.8
36.01
OpenAl Deep Reseurch
26.6
46,5
Open-source Deep Research Agents
ASearcher Web QwQ
12.5
70.9/
DeepDive-321
76.1
29.31
MinoThinker-32h 2BBDDP0as
17.8
74.8
WobExplorer-88
17.3"
75.7
Kimi-K2
25.2
GLM-4.5
21.2'
39.2
78.9
34.2
55.6
DeepSeck-V3.1
29.8'
35.4
83.7
42.61
65.0
Ourx
AgentFounder-308
31.5
47,9
89.6
43.9
75.3
Table 3: Adaptability Validation of the AgentFounder Base Model with Different Post- tea--Trannng Data.
Base Model
SFT Data
BrowseComp-en
BrowseComp-zh
GAIA
HLE
Qwen3-30B-A335 30B-A3B------
A.B--Base
SFT-A
26.9
29.8
67.0
23.5
AgemtFoumder-30tt BOsse
SFT-A
31.4
35.6
72.8
30.4
Relative A
SFT-A
+4.5
+5.8
+5.8
+6.9
Bwen3-30B-A3B-B2
SFT-B
28.6
35.6
71.8
27.0
AgemtFounder-300 30seBBBe
SFT-B
43.3
Relative &
SFT-B
+11.3
+7.7
+1.0
Qwen3-30B- -------0B--3-----
SFT-C
24.5
36.7
68.9
27.9
AgentFounder- 30B Base
SFT-C
38.8
44.3
71.8
28.9
Relative A
SFT-C
+14.3
+7.6
+2.9
+1.0
Observations on Scenario-Targeted Targeted Web Search Benchmarks. AgentFounder-300 demonstrates impressive performance across specialized tasks. On HLE, it becomes the first open-source model surpass the 30-point threshold at 31.5%, significantly exceeding all closed-source deep research products including Gemini-22.5-Pr Deep Research, Kimi-Researcher, and OpenAl Deep Research. AgentFounder-300 scores 75.3% Academic Browse, substantially outperforming existing open- source models as an academic assistant. It outperforms all models on Frames, demonstrating superior pulti-perspecttve reasoning and information synthesis. On Seal-O, it comprehensively outperforms open-source agents, indicating strong resistance to information interference. Finally, it achieves 47.9% on ReepResearch Bench RACE Overall, surpassing OpenAl Deep Research and all open-source source agents, confirming comprehensive, readable, and in-depth report generation.
3.3 ADAPTABILITY OF THE AGENTIC BASE MODEL TO POST-TRAINING
Experiment Design. We adopt the AgentFounder-30B 30B-Base as the pre-alignment agentic base model. We then validate the adaptability of our base model by conducting agentic alignment training with three distinct SFT datasets described in Appendix C.1 SFT-A, SFT-B, and SFT-C.
Observation Results. Table 3 presents the results, from which we make the following observations:
(1) Agentic CPT demonstrates consistent and substantial improvements. Specifically, models trained on SFT-A, SFT-B, and SFT-C datasets show average performance gains of 5.75%, 6.13,, and 6.45% respectively when built upon Bonnnrooonder--B---aa.
8

---

Published as a conference paper at ICLR 2026
(2) Post-training data remains erucial for unlocking base model capabilities. Despite sharing the same AgentFounder-300 30B-aee foundation, models exhibit significant performance variations across different yost-training datasets. For instance, BrowseComp- AgentFounder-30B with SFT-B outperforms variants with SFT-A and SFT-C by 8.5% and 3.0% respectively. This underscores the irreplaceable role of agentic post-training and highlights the ongoing challenge of fully realizing base model potential through optimal training strategies.
3.4 IMPACT ON GENERAL CAPABILITIES
Experiment Design. To mitigate potential catastrophic forgetting, we explicitly incorporated high- quality general- purpese data into training mixture. We evaluate the performance three widely recognized general benchmarks: MMLU Li et al. (003)), SuperGPQA Du et al. (2025b), and GPQA Rein al. (003)). In this analysis, we compare the original foundation model initialization before any agentic adaptation, denoted as Qwen3-30B-A. A3B-- 3B-Base, against our agentic base model, AgentFounder-B0e A38--33-40 Base (prior to Agentic SFT). quantify the changes in general ability.
Table 4: Comparison of general capabilities.
Benehmark
Owen33-0BBAABB-Bae
AgentFounder-30B-BB
A
MMLU
81.38
80.11
-1.27
SuperGPQA
35.72
34.27
-1.45
GPQA
43.94
42.58
-1.36
Results and Analysis. As presented in Table we observe a marginal regression in performance across the general benchmarks. Specifically, AgentFounddr:33B-A3BB points on MMLU, 1.45 on SuperGPQA, and 1.36 on GPQA compared to the Boen3-30B-ABB BBse. This slight degradation primarily attributable to the relatively small proportion of gencral-purpose data in the mixture compared to the massive volume of agentic data. However, considering the significant gains achieved in agentic capabilities, we deem this minor drop an acceptable trade-off. The results confirm that our strategy of mixing general data effectively mitigates severe catastrophic forgetting, maintaining the model's foundational knowledge at robust level.
exhibits decrease of 1.27
3.5
ABLATION STUDIES
3.5.1
IMPACT OF TRAINING STRATEGIES
We examine whether our proposed two-stage training paradigm can bring performance improvements.
Experiment Design. We set the training tokens to 50B, using Qwen3-30B-Ae A3B-Base the initial- ization checkpoint and employing SFT-A data. We compare two configurations: AgentFounder Stage which applies single-stage training on all data where some HAS data may be truncated due to length constraints, and AgentFounder Stage 1&2, which implements our complete two-stage training approach that specifically incorporates synthesized long-context agent data in the second stage.
As shown in Table 5 our analysis demonstrates that the two-stage training paradigm yields consistent performance improvements. Specifically, the AgentFounder Stage &&2 configuration achieves an average improvement of 3.3% on Pass and 3.7% on Pass@3 across the three evaluation tasks compared to the mingle-stage baseline. These results substantiate the necessity of learning complete long-context agent data rather than truncated sequences. While resource constraints preclude evaluation of single-stage training with extended context lengths (e.g., 128K), such approaches would incur substantially higher computational costs.
Observation Results.
3.5.2 IMPACT OF DATA TYPES
Experiment Design. We investigate the effectiveness of the two data types introduced in this work for agentic CPT. We conduct single-stage training experiments using approximately 50B tokens of FAS data and FAS+HAS mixed data respectively, followed by post-training with SFT-A data.
Observation Results, Toblos that both FAS and HAS data contribute meaningful performance improvements: (1) FAS data demonstrates clear efficacy. Training with FAS data yields substantial

---

Published as a conference paper at ICLR 2026
Table Evaluation of the effectiveness of the two-stage AgentFounder training strategy.
BrowseComp-en
BrowseComp-zh
Strategy
Puss@1
P
Pusc 1
Pus
Pass
Pax
AgentFounder Stage Only
31.4
49.9
34.3
50.5
69.9
81.6
AgentFounder Stage &
35.5
58.5
72.8
Relozive A
+4.1
+2.1
42.9
+8.0
42,9
+0.9
Table 6: Effect of data type (HAS and FAS).
CompeeComp-en
BrowseComp-zh
GAIA
Data
Tokens
Puss@
Pass@
Pass@1
Pass@3
Pass@1
Pass@3
Non CPT
0B
26.9
38.0
29.8
45.3
67.0
75.7
37.0
72.8
80.6
FAS+HAS
50B
31.4
50.1
40.1
54.7
69.9
82.5
Relative A
0.0
+0.2
+3.1
+0.4
-2.9
+1,9
gains, particularly in Pass@3 metrics. On BrowseComp-zhl FAS-based continual pre-training
achieves 9.0% improvement, establishing higher performance ceilings for post- pest-training. (2) HAS provides complementary benefits. FAS+HAS combination consistently delivers positive gains across benchmarks. While GAIA shows 2.9% Pass@ 1 decrease, the corresponding 1.9% Pass improvement suggests normal evaluation fluctuation rather than systematic degradation.
3.6 SCALING LAw EXPLORATION
Terformance Thand Curve
62.2
Average (56)
49%%
Aug. Pessi8
50.3
33%
56.0
37%
56.2
27%
15%
20
306-32
Model Size (Billion Parameters)
Training Tolens
(a) Model Scaling
(b) Data Scaling
Figure 5: Scaling Law Exploration for Agentic Capabilities.
Scaling Law Analysis. We systematically explore scaling laws across model size and training data volume. Our experiments reveal that agentic capabilities consistently improve with both model size +30BA3B) and data volume scaling (15B-3115 tokens). Specifically, data volume scaling improves average accuracy across eight benchmarks from 54.2% 62.2%, demonstrating effective scaling of agentic CPT data. However, whether the performance gains from model scaling stem from the MoE architecture (with reduced activation) warrants further investigation. More details are provided in Appendix D.I
scaling (IB-+B BBBB44B-30BBBB
4 CONCLUSION
In this work, we redefine the training pipeline for agentic alignment in deep research agents by introducing Agentic Continual Pre-Training (Agentic CPT) beyond traditional post-training for the first time. Furthermore, we present a systematic agentic CPT method comprising scalable agentic data synthesis and an adapted two-stage training strategy. Specifically for agentic data synthesis, we propose first-order action synthesis without additional commercial API calls, including planning action synthesis and reasoning action synthesis to enhance the model's 's planning and logical reasoning capabilities. Additionally, provide higher-order action synthesis method that remodels trajectories as multi-step decision-making problems, leveraging step-level expansion to thoroughly explore solution paths. Built large-scale offline synthesized HAS and FAS data, we develop powerful agentic model, AgentFounder-333 which surpasses current source models across 10 benchmarks, establishing state-of-the-art
10

---

Published as a conference paper at ICLR 2026
5 ACKNOWLEDGMENT
This work is partially supported by the RGC for Early Career Scheme 2N..27210224.. Any opinions, findings, or conclusions expressed in this material are those of the authors and do not necessarily reflect the views of RGC.
6 ETHICS STATEMENT
We confirm that all training data used in this work consists of legally available and permissible datasets. All authors have strictly adhered to ICLR's ethical guidelines and standards throughout the research process. Our work does not involve promote content related to bias, violence, privacy violations, or other ethically problematic materials.
REFERENCES
ScraperAPI:
Scale data collection with simple web scraping api, 2025. URL https: www. SC raperapi.com
Anthropic. Introducing the claude 4 model family, 2025. URL m/ ht/news/claude-i Accessed: 2025-09-11.
Sören Auer, Christian Bizer, Georgi Kobilarov, Jens Lehmann, Richard Cyganiak, and Zachary Ives. Dhpedia: A nucleus for web of open data. In international semantic web conference, pp. 722-735. Springer, 2007.
Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, Binyuan Hui, Luo Ji, Mei Li, Junyang Lin, Runji Lin, Dayiheng Liu, Gao Liu, Chenggiang Lu, Keming Lu, Jianxin Ma, Rui Men, Xingzhang Ren, Xuancheng Ren, Chuanqi Tan, Sinan Tan, Jianhong Tu, Peng Wang. Shijie Wang, Wei Wang, Shengguang Wu, Benfeng Xu, Jin Xu, An Yang. Hao Yang, Jian Yang, Shusheng Yang. Yang Yao, Bowen Yu, Hongyi Yuan, Zheng Yuan, Jianwei Zhang, Xingxuan Zhang, Yichang Zhang, Zhenru Zhang, Chang Zhou, Jingren Zhou, Xiaohuan Zhou, and Tianhang Zhu. Qwen technical report. CoRR, abs/2309.116600 2023.
Satanjeev Banerjee and Alon Lavie. Meteor:
An automatic metric for mt evaluation with improved correlation with human judgments. In Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization, pp. 65-72, 2005.
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Kerbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners. In NeurIPS, 2020.
ByteDance, Deerflow:
A community-driven deep research framework combining language models with tools like web search, crawling, and python execution. https:://qthhiii combbteed ance sacce/deer-fine flow] 2025.
Chen Chen, Xinlong Hao, Weiwen Liu, Xu Huang. Xingshan Zeng. Shuai Yu, Dexun Li, Shuai Wang, Weinan Gan, Yuefeng Huang, Wulong Liu, Xinzhi Wang, Defu Lian, Baoqun Yin, Yasheng Wang, and Wu Liu. Acebench: Who wins the match point in tool usage?, 2025. [RLhttps
//arxiv.org//abs/2500
Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung. Charles Sutton, Sebastian Gehrmann, et al. PaLM: Scaling language modeling with pathways. CoRR, hbs//200400231,, 2022.
Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. Scaling instruction finettunnd language models. CoRR, abs/2210.1 1416,, 2022.
11

---

Published as a conference paper at ICLR 2026
Common Crawl. Common crawl dataset, 2025. URL.Ltpps.///omooozaa.....r
DeepSeek-AL. DeepSeek- V3.1 Model Card. htttss//hunginng.ccc.../eeeeeee DeepSeek-V3. 2025.
Mingxuan Du, Benfeng Xu, Chiwei Zhu, Xiaorui Wang, and Zhendong Mao. Deepresearch bench: A comprehensive benchmark for deep research agents. arXiv preprint arXiv:2506. 1763, 2025a.
Xinrun Du, Yifan Yao, Kaijing Ma, Bingli Wang, Tianyu Zheng, King Zhu, Minghao Liu, Yiming Liang, Xiaolong Jin, Zhenlin Wei, et al. SuperGPQA: Scaling LLM evaluation across 285 graduate disciplines. arXiv preprint arXiv:2502. 14739, 2025b.
Runnan Fang, Shihao Cai, Baixuan Li, Jialong Wu, Guangyu Li, Wenbiao Yin, Xinyu Wang, Xiaobin Wang, Liangcai Su, Zhen Zhang, Shibin Wu, Zhengwei Tao, Yong Jiang, Pengjun Xie, Fei Huang, and Jingren Zhou, Towards general agentic intelligence via environment scaling, 2025a.
Tianqing Fang, Zhisong Zhang, Xiaoyang Wang, Rui Wang, Can Qin, Yuxuan Wan, Jun-Yu Yu Ma, Ce Zhang, Jiaqi Chen, Xiyun Li, al. Cognitive kernel-pro: A framework for deep research agents and agent foundation models training, arXiv preprint ar arXiv:2508.004
2025b.
Jiaxuan Gao, Wei Fu, Minyang Xie, Shusheng Xu, Chuyi He, Zhiyu Mei, Banghua Zhu, and Yi Wu. Beyond ten turns: Unlocking long-horizon agentic search with large-scale asynchronous rl. arXiv preprint arXiv:2508.079)) 2025.
Xinyu Geng, Peng Xia, Zhen Zhang, Xinyu Wang, Qiuchen Wang, Ruixue Ding, Chenxi Wang, Jialong Wu, Yida Zhao, Kuan Li, et al. Webwatcher: Breaking new frontiers of vision-language deep research agent. arXiv preprint arXiv:2508.05748, 2025.
Google. Gemini deep research your personal research assistant. 2025. Accessed: 2025-09-10.
goog geo
wwervieewdeeppreeeeee
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. DeepSeek-RI: Incentivizing reasoning capability in LLMs via reinforcement learning. arXiv preprint arXiv:2501. 2025.
Kshitij Gupta, Benjamin Thérien, Adam Ibrahim, Mats L. Richter, Quentin Anthony, Eugene Belilovsky, Irina Rish, and Timothée Lesort. Continual pre-training of large language models: How to (re)warm your model?, 2023.
wwtpa/s/rrr/////o//////o/.oosss/ 2308. 04014
Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, and Sergey Levine. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with stochastic actor. In International conference on machine learning, pp. 1861-1870. Pmlr, 2018.
Xisen Jin, Dejiao Zhang, Henghui Zhu, Wei Xiao, Shang- Wen Li, Xiaokai Wei, Andrew Arnold, and Xiang Ren. Lifelong pretraining: Continually adapting language models emerging corpora, 2022. URL
08534
Jina.ai. Jina, 2025. URL. http::tttt///ttt
Zixuan Ke, Yijia Shao, Haowei Lin, Tatsuya Konishi, Gyuhak Kim, and Bing Liu. Continual pre-training of language models. arXiv preprint arXiv:2302.03241, 2023.
Satyapriya Krishna, Kalpesh Krishna, Anhad Mohananey, Steven Schwarez, Adam Stambler, Shyam Upadhyay, and Manaal Faruqui. Fact, fetch, and reason: A unified evaluation of retrieval- augmented generation. arXiv preprint arXiv:2409. / /2941, 2024.
Haonan Li, Yixuan Zhang, Fajri Koto, Yifei Yang, Hai Zhao, Yeyun Gong, Nan Duan, and Timothy Baldwin. CMMLU: Measuring massive multitask language understanding in Chinese. CoRR, abs/2306.09212, 2023.
Kuan Li, Zhongwang Zhang. Huifeng Yin, Rui Ye, Yida Zhao, Liwen Zhang, Litu Ou, Dingchu Zhang, Xixi Wu, Jialong Wu, Xinyu Wang. Zile Qiao, et al. Websailor-v2: Bridging the chasm to proprietary agents via synthetic data and scalable reinforcement learning, 2025a.
12

---

Published as a conference paper at ICLR 2026
Kuan Li, Zhongwang Zhang, Huifeng Yin, Liwen Zhang, Litu Ou, Jialong Wu, Wenbiao Yin, Baixuan Li, Zhengwei Tao, Xinyu Wang, et al. Websailor: Navigating super-human reasoning for web agent. arXiv preprint arXiv:2507.02592, 2025b.
Weizhen Li, Jianbo Lin, Zhuosong Jiang, Jingyi Cao, Xinpeng Liu, Jiayu Zhang, Zhenqiang Huang, Qianben Chen, Weichen Sun, Qiexiang Wang, et al. Chain-of-agents: End-to-end agent foundation models via multi-agent distillation and agentic rl. arXiv preprini arXiv:2508. 13167, 2025c.
Xiaoxi Li, Guanting Dong, Jiajie Jin, Yuyao Zhang, Yujia Zhou, Yutao Zhu, Peitian Zhang, and Zhicheng Dou. Search-ol: Agentic search-enhanced large reasoning models. arXiv preprins 05t::2501.00366,, 2025d.
Xiaoxi Li, Jiajie Jin, Guanting Dong, Hongjin Qian, Yutao Zhu, Yongkang Wu, Ji-Rong Wen, and Zhicheng Dou. Webthinker: Empowering large reasoning models with deep research capability. arXiv preprint arXiv:2504.21776 2025e.
Zijian Li, Xin Guan, Bo Zhang, Shen Huang, Houquan Zhou, Shaopeng Lai, Ming Yan, Yong Jiang, Pengjun Xie, Huang Fei, Jun Zhang, and Jingren Zhou. Webweaver: Structuring web-scale evidence with dynamic outlines for open-ended deep research, 2025f.
Zhenghao Lin, Zhibin Gou, Yeyun Gong, Xiao Liu, Yelong Shen, Ruochen Xu, Chen Lin, Yujiu Yang. Jian Jiao, Nan Duan, and Weizhu Chen. Rho-1: Not all tokens are what you need, 2025.
URL
http://wwww...ga/////aa///// 07965
Evan Z Liu, Aditi Raghunathan, Percy Liang, and Chelsea Finn. Decoupling exploration and exploitation for meta-reinforcemmen learning without sacrifices. In International conference on machine learning. pp. 6925-6935. PMLR, 2021.
Junteng Liu, Yunji Li, Chi Zhang, Jingyang Li, Aili Chen, Ke Ji, Weiyu Cheng, Zijia Wu, Chengyu Du, Qidi Xu, Jiayuan Song, Zhengmao Zhu, Wenhu Chen, Pengyu Zhao, and Junxian He. Webexplorer: Explore and evolve for training long-horizon web agents, 2025. URL https: /ttpps//arxxvvorgg
abs 2009/ 06501
Rui Lu, Zhenyu Hou, Zihan Wang, Hanchen Zhang, Xiao Liu, Yujiang Li, Shi Feng, Jie Tang, and Yuxiao Dong. Deepdive: Advancing deep search agents with knowledge graphs and multi-turn arXiv preprint arXiv:2509. 104466,609 2025.
Grégoire Mialon, Clémentine Fourrier, Thomas Wolf, Yann LeCun, and Thomas Scialom. Gaia: benchmark for general ai assistants. In The Twelfth International Conference on Learning Representations, 2023.
Moonshot AI. Kimi-reseearcher: End-to-end rl training for emerging agentic capabilities. https
\/moonshotai. //monniiotai..iiiuubb..iibi Pooo/iimii hhs June 2025. Accessed: 2025-(99-10.
OpenAl. Introducing openai o3 and 04-mini, 2025a. URL
ntroducing 444 minii
OpenAl. Introducing deep research, February 2025b.
ntrrodcciingddee--resaaccc
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances neural information processing systems, 35:27730- 27744, 2022a.
Long Ouyang, Jeffrey WU, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F. Christiano, Jan Leike, and Ryan Lowe. Training language models to follow instructions with human feedback. In NeurIPS, 2022b.
Jupinder Parmar, Sanjev Satheesh, Mostofa Patwary, Mohammad Shoeybi, and Bryan Catanzaro. Reuse, don't 't retrain: A recipe for continued pretraining of language models, 2024. URL https 7/arxiv. httpv/o.ggbbbs/////////
07263
13

---

Published as a conference paper at ICLR 2026
Perplexity AL Introducing perplexity deep research. https: /ttpp://ww..errpeexttyy ai
/ttp:///ww..ccc.....///aaaaaaaaa 2025-09-10.
February 2025. Accessed:
Thinh Pham. Nguyen Nguyen, Pratibha Zunjare, Weiyuan Chen, Yu-Min Tseng, and Tu Vu. Sealqa: Raising the bar for reasoning in search-augmented language models. arXiv preprint arXiv:2506.0106 00.50000002, 2025.
Long Phan, Alice Gatti, Ziwen Han, Nathaniel Li, Josephina Hu, Hugh Zhang, Chen Bo Calvin Zhang, Mohamed Shaaban, John Ling, Sean Shi, et al. Humanity's s last exam. arXiv preprint arXiv:2501. 14249, 2025.
Zile Qiao, Guoxin Chen. Xuanzhong Chen, Donglei Yu, Wenbiao Yin, Xinyu Wang, Zhen Zhang, Baixuan Li, Huifeng Yin, Kuan Li, Rui Min, Minpeng Liao, Yong Jiang. Pengjun Xie, Fei Huang, and Jingren Zhou. WebResearcher: Unleashing unbounded reasoning capability in Long-Horizon Agents, 2025.
Yujia Qin, Shengding Hu, Yankai Lin, Weize Chen, Ning Ding, Ganqu Cui, Zheni Zeng, Xuanhe Zhou, Yufei Huang, Chaojun Xiao, al. Tool learning with foundation models. ACM Computing Surveys, 57(():1 1-00, 2024.
David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R. Bowman. GPQA: A graduate-level Google-proof Q&A benchmark. CoRR, abs/2311. 1.0022 2023.
Timo Schick, Jane Dwivedi- Yu, Roberto Dessi, Roberta Raileanu, Maria Lomeli, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. Toolformer: Language models can teach themselves to use tools. CoRR, abs/2302.04761, 2023.
Haizhou Shi, Zihao Xu, Hengyi Wang, Weiyi Qin, Wenyuan Wang, Yibin Wang, Zifeng Wang, Sayna Ebrahimi, and Hao Wang. Continual learning of large language models: A comprehensive survey. ACM Computing Surveys, 2024.
Huatong Song. Jinhao Jiang, Yingqian Min, Jie Chen, Zhipeng Chen, Wayne Xin Zhao, Lei Fang, and Ji-Rong Wen. R1-searcher: Incentivizing the search capability in llms via reinforcement learning. arXiv preprint arXiv:2503.0559) 2025.
Zhengwei Tao, Jialong Wu, Wenbiao Yin, Junkai Zhang, Baixuan Li, Haiyang Shen, Kuan Li, Liwen Zhang, Xinyu Wang, Yong Jiang, et al. Webshaper: Agentically data synthesizing via information-sceking formalization. arXiv preprinl arXiv:2507.15061. 2025.
Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B Hashimoto. Stanford alpaca: An instruction-folloving llama model, 2023.
Kimi Team, Yifan Bai, Yiping Bao, Guanduo Chen, Jiahao Chen, Ningxin Chen, Ruijue Chen, Yanru Chen, Yuankun Chen, Yutian Chen, et al. Kimi k2: Open agentic intelligence. arXiv preprint arXiv:2507.20534, 2025.
MiroMind AI Team. Mirothinker: An open-source agentic model series trained for deep research and complex, long-horizon problem inker 2025a.
rom woMi roTh
OpenPangu Team. Openpangu deepdiver- Multi-agent learning for deep information seeking,
2025b. URL http:ttss//a..ittooot www.ggitoooo..oooooooo com hat////ceenddtee www.endddrrrbe/eeeeeeeeeeeee open/ augu- Embedde 1e-DeepDDive ver
Denny Vrandečić and Markus Krötzsch. Wikidata: free collaborative knowledgebase. Communica- tions of the ACM, 57(10)778-85, 2014.
Jason Wei, Zhiqing Sun, Spencer Papay, Scott McKinney, Jeffrey Han, Isa Fulford, Hyung Won Chung, Alex Tachard Passos, William Fedus, and Amelia Glaese. Browsecomp: simple yet challenging benchmark for browsing agents. arXiv preprint arXiv:2504. /25/6, 2025.
14

---

Published as a conference paper at ICLR 2026
Jialong Wu, Baixuan Li, Runnan Fang, Wenbiao Yin, Liwen Zhang, Zhengwei Tao, Dingchu Zhang, Zekun Xi, Yong Jiang, Pengjun Xie, et al. Webdancer: Towards autonomous information seeking agency. arXiv preprint arXiv:2505.22648, 2025a.
Jialong Wu, Wenbiao Yin, Yong Jiang, Zhenglin Wang, Zekun Xi, Runnan Fang, Linhai Zhang, Yulan He, Deyu Zhou, Pengjun Xie, and Fei Huang. Webwalker: Benchmarking Ilms in web traversal, 2025b. URL httptth/taaaaitiia
org http//bb.2255..... 07572
Weiqi Wu, Xin Guan, Shen Huang, Yong Jiang, Pengjun Xie, Fei Huang, Jiuxin Cao, Hai Zhao, and Jingren Zhou. Masksearch: A universal pre-training framework to enhance agentic search capability, arXiv preprint arXiv:2505. 20285, 2025c.
x.ai. Grok beta - the age of reasoning agents, 2025. URL
xAI. Grok 3 beta - the age of reasoning agents, 2025, URL httph:tt///t/ http:://x.a//eewss nww..///iiw///oo-1--1
Xbench- Team. Xbench-deepsearch. 2025. URL http://s///ooocc..rrr///////////
An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2.5 technical report. arXiv preprint 1rXiv:2412.15115. 2024.
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint arXiv:2505.09388 2025.
Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. In International Conference on Learning Representations (ICLR), 2023.
Aohan Zeng, Xin Lv, Qinkai Zheng, Zhenyu Hou, Bin Chen, Chengxing Xie, Cunxiang Wang. Da Yin, Hao Zeng, Jiajie Zhang, al. Glm-4.5: Agentic, reasoning, and coding (are) foundation 200,22508.0677,,, 2025.
models. arXiv preprint
Dingchu Zhang, Yida Zhao, Jialong Wu, Baixuan Li, Wenbiao Yin, Liwen Zhang, Yong Jiang, Yufeng Li. Kewei Tu, Pengjun Xie, and Fei Huang. Evolvesearch; An iterative self-evolving search agent, 2025a. (Rthstts:///at /ttp///aavvvv///aa///i org/abs////eoooooooo/..ooo.. 25050554000 22501
Wentao Zhang, Liang Zeng, Yuzhen Xiao, Yongcong Li, Ce Cui, Yilei Zhao, Rui Hu, Yang Liu, Yahui Zhou, and Bo An. Agentorchestra: A hierarchical multi-agent framework for general-purpose task solving, 2025b. WRLLattps://aaxxe
org/ /td.bb/25555 12508
Renjie Zheng, Mingbo Ma, and Liang Huang. Multi-reference training with pseudo-references for neural translation and text generation. arXiv preprint arXiv: 09564, 2018.
Junting Zhou, Wang Li, Yiyan Liao, Nengyuan Zhang, Tingjia Miaoand Zhihui Qi, Yuhan Wu, and Tong Yang. Academicbrowse: Benchmarking academic browse ability of 1lms. arXiv preprint arXiv:2506.13788 13784, 2025a.
Peilin Zhou, Bruce Leon, Xiang Ying, Can Zhang, Yifan Shao, Qichen Ye, Dading Chong, Zhiling Jin, Chenxuan Xie, Meng Cao, et al. Browsecomp-zh: Benchmarking web browsing ability of large language models in chinese. arXiv preprint arXiv:2504. 19314 2025b.
Çağatay Yıldız, Nishaanth Kanna Ravichandran, Nitin Sharma, Mathhis Bethge, and Beyza Ermis. Investigating continual pretraining in large language models: Insights and implications, 2025. URL
https://arze org/ abs 2402. 17400
15

---

Published as a conference paper at ICLR 2026
A
THE USE OF LARGE LANGUAGE MODELS (LLMS)
We used LLMs solely for grammar checking and minor typo corrections in this paper. No "prompt injections" are present in the submission. Additionally, given that this paper aims to develop more powerful agentic LLMs, our experimental methodology (e.g,, using LLMs as judges) and illustrative examples necessarily involve LLM usage, which does not pose additional academic risks. All instances of L.L use have been clearly disclosed within the paper, and we confirm that our usage fully complies with the stated policy and academic ethics standards.
B RELATED WORK
B.1 DEEP RESEARCH AGENTS
Deep research agents are language models capable of autonomous tool use and multi-step reason- ing-such search, web access, code execution, and file parsing-to accomplish research-level tasks and produce reliable reports. OpenAl pioneered this field with a closed-source agent that analyzes hundreds of web pages and generates cited reports within minutes (OpenAl 2025b), inspiring similar products from Grok (xAI (222)), Perplexity (Perplexity Al 2025). Google Gemini (Google 2025). and Kimi (Moonshot All 2025).
Open-source deep research agents Open-source deep research agents have achieved strong re- sults on benchmarks such as BrowseComp-en (Wei et al., 2025), BrowseComp-zz (Zhou al.] 2025b), Xbench-Deege DeepSearch (Xbench-Team) (2025) and GAIA (Mialon et al.) 2023), with notable systems ranging from early works like Search-ol (Li et al., 2025d). RI-Searcher (Song et al. 2025 EvolveSearch (Zhang et al. 2025a), MaskSearch (Wu et al.) (025c)), and WebThinker (Li et al.) 2025c), to later agents tackling harder tasks, such as Mirokhinker (Team, 2025a). A common focus constructing challenging problems and trajectories for training. leading diverse data-generation strategies: iterative complexity growth (WehDancer (Wu et al. (025a)), ASearcher (Gao et al., 2025 )
knowledge graph-based synthesis (WebSailor (Li et al.) 2025b)), information-seeking task modeling (WebShaper (Tao et al.) 2025). Chain-of-Agemt pipeline (AFM (Li al., 2025c)), open knowledge graph sampling (DeepDive (Lu al. 2025 cross-page/riddle generation (Deepdiver [Team] 2025b)).
and autonomous exploration (WebExplorer (Liu et al. 2025)). While these methods enhance agent capability via supervised SFT or RL, they largely overlook continual pre-training. Recent open-source models such as Ki (Team et al. 2025), GLM-4.5 (Zeng et al. (025),, and DeepSeck-VS V3.1 (DeepSeekkAA) AI, 2025) have begun emphasizing agentic capabilities, but systematic exploration of continual pre-training for agents remains limited.
Multi Deep Research Agents and Multi-modal Deep Research Agents. Beyond training method- ologies, innovative inference paradigms have also emerged. Tencent's contribution extends be- yond their trained CK-Pro-SB model to include the open-sourced Cognitive Kernel-Pro multi-agent framework (Fang et al. 2025b). Tencent Youtu has developed multi-agent framework utilizing DeepSeck- V3.1, achieving 71.47% accuracy on the WebWalkerQA benchmark (Wu et al.) (025b). Additional notable contributions include SkyworkAl's DeepRescarchAgent (Zhang al. (2225) and ByteDance'ss 's deer-flow Dance 2025). which further explore the potential of agent models. In the multimodal domain, Alibaba Tongyi Lab has open-sourced Wathhrr (Geng et al. 2025). the first multimodal deep research agent.
B.2 CONTINUAL PRE-TRAINING
While LLMs are initially pretrained on large general corpora, research has shown that continual pre-training (CPT) can further boost performance through ongoing unsupervised training (Gupta et al. 2023 Lin et al., 2025) Jin et al. (022). Ke et al. (2023) introduce continual
pre-training, which mitigates catastrophic forgetting and enables knowledge transfer for better end- task results. Extending this, Çağatay Yildız et al. (2025) study
models, finding CPT consistently improves performance for models under 1,5B parameters and surpasses standard domain adaptation. Parmar al. (2024) offer guidelines
data distribution and learning-ras scheduling for CPT, showing gains when applied to a 15B- 3--parameter model over its original pre- pre-training baseline. Together, these studies position CPT promising method for enhancing L.LM capabilities.
16

---

Published as a conference paper at ICLR 2026
However, regarding CPT and nowadays agents development, existing work focuses primarily either on CPT on tasks without tool calling or on post-training (such as SFT and RL) for agent develop- ment. Integrating agentic capabilities directly via the continual pre- pre-training phase remains largely unexplored. This motivates our exploration of Agentic CPT as novel paradigm that embeds agentic reasoning and tool-use capabilities at the foundational pre-training level.
C EXPERIMENTAL SETUP DETAILS
C.1 SFT DATA CONFIGURATION
In this paper, we adopt three SFT data configuration as follows:
SFT-A: Employs two-stage training paradigm, first general conversational data, fol- lowed by specialized React-style agent trajectories with explicit reasoning chains.
SFT-B: An enhanced version of SFT-A that maintains the two-stage training paradigm but incorporates balanced mixture of general conversational data and React-style trajectories in each stage.
SFT-C: Employs a two-stage training paradigm with general conversational SFT data and React with summarized reasoning trajectories.
For all SFT configurations, the set of the challenging information-seekien questions is constructed following the methodology from WebSailor-V V2 (Li et al., 2025a), WebResearcher (Qiao et al. 2025 WebWeaver (Li et al. 2025fi and AgentScaler (Fang et al. 2025a).
C.2 TOOLS
In this work, we utilize five different tools for our AgentFounder model, namely Search, Visit, Python Interpreter, Google Scholar, and File Parser:
Search leverages the Google search engine for large-scale information retrieval. The tool accepts list of one returns the top- 10 ranked results, with each result comprising a title, a descriptive snippet, and its corresponding URL.
search queries to be executed concurrently. For each query, it
Visit is designed for targeted information extraction from web pages. The tool takes as input set of web pages, where each page is paired with a dedicated information-seeking goal. The process begins by employing Jina (Jina.ai 2025) to retrieve the full content of given web page. Subsequently, a summary model processes this content to extract only the information pertinent that page's specific goal.
Python Interpreter is used to execute Python code within a sandboxed environment. The input is string of Python code, which must be enclosed within <code> tags for proper execution. The tool runs the provided code and captures its standard output; therefore, any results values intended to be seen must be explicitly passed to the print() function. This capability enables dynamic computation, data manipulation, and the use of various Python libraries secure and isolated manner.
Google Scholar is used to retrieve information from academic publications. The input consists of a list of one or more search queries, allowing for multiple, distinct searches within single tool call. The tool leverages the Google Scholar search engine execute each query and gather relevant scholarly literature, such as articles, papers, and citations.
File Parser answers user queries by analyzing a mix documents, web pages, and multi- media files (e.g., PDF, DOCX, MP4) from local or URL sources. It works in two steps: first, it converts all input into plain text, transcribing audio/video content when necessary. Second, summary model reads this unified text generate a direct answer to the user's question.
C.3
When evaluating our AgentFounder models, we employ specific inference parameters to ensure stable and reproducible results: temperature 0.85, repetition penalty 1.1, and top-p 0.95. These settings
17

---

Published as a conference paper at ICLR 2026
recommended based extensive empirical validation to optimize the balance between creativity and consistency in agentic reasoning tasks. We set a maximum tool usage limit 128 calls per task and constrain the context length to 128K tokens.
D ADDITIONAL EXPERIMENTAL RESULTS
D.1 SCALING LAw EXPLORATION
D.1.1 SCALING WITH MODEL SIZE
Experiment Design. We investigate how model scale influences agentic capabilities by evaluating models with IB, 4B, and 30B parameters on deep research benchmarks. Additionally, we compare our models with two larger-scale baseline models (DeepSeek-Vai) V3.1 and Kimi-K2) to assess the efficiency of our agentic continual pre-training approach across different model scales.
Observation Results. Our experimental results reveal that model scale plays crucial role in agentic performance, with our approach demonstrating superior scaling efficiency: (1) Consistent scaling benefits. Model size shows strong positive correlation with agentic performance. Average accuracy increases from 20.4% for the 1B model to 32.7% for the 4B model and further to 48. 98%9% for the 30B model, indicating that larger models possess enhanced capacity for complex agentic behaviors such as effective tool use and multi-step reasoning. (2) Superior scaling efficiency. The AgendFounder-30B model achieves 48.9% accuracy, exceeding the performance of two larger baseline models, DeepSeek-Vas V3.1 (43.0%) and Kimi-K2 (29.6%), despite their greater size. This suggests that agentic continual pre-trae pre-training approach enables more effective utilization of model capacity for agentic tasks. These results demonstrate that combining agentic continual pre-training with appropriate model scaling provides strong foundation for building nigh-performance deep research agents.
D.1.2 SCALING WITH DATA VOLUME
We investigate the scaling properties of Agentic CPT and validate the effectiveness of our two-stage training strategy across varying data volumes.
Experiment Design. We train AgentFounder models with data volumes ranging from OB to 315B tokens, using Qwen3-30B-A 3---33------BB A3B-Base as the initialization checkpoint. We implement our two-stage training paradigm, where Stage 2 incorporates 128K context window training 65B and 315B token checkpoints. Performance is evaluated using the average Pass@3 metric across multiple agentic benchmarks to assess scaling behavior and training effectiveness.
Observation Results. As shown in the scaling curve, our analysis reveals three key findings:
(1) Logarithmic scaling law holds for agentic capabilities. The relationship between training tokens and performance exhibits logarithmic characteristics, with the most substantial improvements (3.8%) occurring within the initial 15B tokens, demonstrating that agentic behaviors can be efficiently acquired through targeted pre-training.
(2) Two-stage CPT with extended context provides consistent improvements. Stage CPT with 128K context windows delivers notable gains at both 65B (+1.8% over 50B) and 315B (+1.0% over 210B), indicating that long-context training effectively enhances complex reasoning capabilities even when the base scaling curve approaches saturation.
(3) Sustained improvements scale demonstrate robustness. AgentFounder maintains consistent performance improvements throughout the entire 315B token budget, achieving total gain of 8.0% (from 54.2% 62.2%), validating our training methodology's effectiveness in preventing premature convergence.
D.2 QUALITY ANALYSIS OF FAS DATA ON PLANNING ACTION
Experiment Design. We evaluate the quality distribution and filtering effectiveness of FAS planning action data. Starting with FAS synthesized trajectories, we apply a prompt- based weak supervision filter that analyzes the original question, generated trajectory, and metadata to produce binary
18

---

Published as a conference paper at ICLR 2026
Weakly-Supervised Filtering for First-Order Action Synthesis
Data Percentage (%)
100
Distribution (43.5 Total)
26.2%
Ci Content inconcitency
80
FAS Filtering
Detalled judgment Breakdown
Percentage (%)
Logic Diacontinuiyy PD: Parsmetar Devistion
9605 Joreuno
60
Pncorecy
15
Accuracy 52%
40
10
6.9%
5.7%
3.5%
20
Correct 46.45 49.4%
Cornect Incorrect Filered
1.2%
Before Filter
After Filter
Figure 6: Filtering performance and representative low-quality outputs for weakly supervised filtering in first-order action synthesis.
accept/reject decisions with detailed rejection reasons. We measure the impact on data quality by comparing pre- and post-filtering accuracy rates and analyzing error type distributions.
Observation Results. As shown in our analysis, the filtering mechanism significantly enhances FAS planning action data quality: (1) Filtering achieves substantial quality improvement. While initial FAS generation yields balanced correct/incorrect trajectories (50%/50%), our filter removes 43.5% of problematic samples, increasing retained trajectory accuracy from 50% 82%. This confirms FAS effectively generates diverse planning actions requiring quality control for high-fidelity selection. (2) Semantic errors dominate rejection patterns. Content Inconsistency accounts for 26.2% of rejections, followed by Search Necessity (6.9%) and Logic Discontinuity (5.7%). The concentration in semantic rather than syntactic errors (Invalid Tool: 1.2%) indicates FAS maintains structural validity while requiring refinement in semantic alignment. (3) Quality improvement justifies volume reduction. Though absolute correct data proportion slightly decreases (50% + 46.3%) the 82% accuracy among retained samples validates prioritizing precision recall for effective agentic CPT.
D.3 COMPREHENSIVE ANALYSES
D.3.1 TRAINING PROCESS
Comparative Convergence Identical Corpus
Agantiiunder 382-422 BAGI
1.4
avor SCRAVe pas HAs yeu
(2Magni
Last 100 Steps (Zoomed)
0.60
LM Cross-Entropy
0.82
2.2
3.1
1290
1283
1907
1300
3740
Steps
2.0-
0.8
400
600
800
1000
1200
Training Steps
Figure 7: Training loss evolution showing superior convergence of AgentFounder models compared to baseline.
19

---

Published as a conference paper at ICLR 2026
We validate whether agentic continual pre-training alleviate the dual-burden problem by endowing models with foundational agentic capabilities before post-training.
Experiment Design. We conduct comparative loss analysis between AgentFounder models and the bascline during SFT on identical downstream tasks. All models are trained for 1,340 steps using the same SFT-A data, with training efficiency measured through final loss, minimum achieved loss, and average loss over the last 100 training steps.
Observation Results. As shown in Figure 7 our analysis reveals that agentic CPT significantly enhances fine-tuning efficiency: (1) AgentFounder substantially reduces SFT loss. All Agent- Founder variants achieve markedly lower loss values compared the baseline across all metrics. While the baseline reaches a final loss of 0.8656, our best- best-performing AgentFounder-25 30B (315B) model achieves 0.7953, demonstrating that agentic CPT effectively endows models with founda- tional capabilities that facilitate adaptation to downstream agent tasks. (2) Scaling CPT data yields monotonic improvements. Loss values decrease progressively as CPT data volume increases from FAS-only to 315B tokens. The FAS+HAS mixture outperforms FAS alone, validating that reor- ganizing supervision signals from post-training into CPT format strengthens the model's agentic foundation.
D.3.2 TOOL CALL ANALYSIS
WebWalker vs. GAIA
HLE vs. BrowseComp(en)
WebWelker
0.200
HLE Browse Compten)
GAIA
Sample Proportion
Sample Proportion
0.175
0.150
0.10
0.100
0.075
0.05
0.050
0.00
(4.9)
18.
112.50
(76.17
138 20)
30)
136.37)
140, 41)
0,000
10.2
(0.5)
(8.00
nz
171
pd.
211
04
25
30
(22
301
(1%
30)
140
41)
(44 451
140
Number of Tool Calls
Number Tool calls
Figure 8: Tool call distribution comparison.
Experiment Design. We analyze AgendFounder's 's tool invocation distributions across four repre- sentative benchmarks understand how the model adapts its tool usage strategies to different task complexities. We examine tool call patterns on HLE, BrowseComp WebWalker, and GAIA-text benchmarks, categorizing them based on their task characteristics and required exploration depth.
Observation Results. As illustrated in Figure 8 AgentFounder demonstrates distinct adaptive behaviors across task types: (1) Complex research tasks exhibit intensive tool usage. BrowseComp- en and HLE show heavy-tailed distributions with high tool density. BrowseComp-en requires extensive invocations for persistent web browsing, while HLE's extended patterns reflect the need to combine complex reasoning with search-augmented inference. (2) Structured tasks employ conservative tool usage. WebWalker's distribution peaks sharply at low invocation counts for efficient text navigation, while GALA-text exhibits compact distribution suited for well rell-defined problems with clear solution paths. These contrasting patterns demonstrate AgentFounder's 's ability to calibrate tool usage based task complexity- intensivv exploration for open-ended research versus targeted invocations for structured problems.
D.3.3 GENERAL TOOL-USE ABILITIES
Beyond the tool-use capabilities of Deep Research, we construct agentic CPT data to address more general tool-use scenarios. As shown in Table we compare AgentFounder-33 with Qwen3-30B- A3B on ACEBench (Chen et al. 2025). a benchmark designed to evaluate the tool-use performance of LLMs in diverse, general settings. The results reveal that AgentFounder surpasses Qwen3- 30B-A3B in general tool-use ability, suggesting that the Agentic CPT framework can be effectively applied to broader range of agentic scenarios.
20

---

Published as a conference paper at ICLR 2026
Model
ACEBench
Qwen3-30B-A3B
67.2
AgentFounder-30h
70.0
Table 7: A comparison of general tool-use ability on ACEBench on overall performance.
GAIA @
Pass@3 Analysis by Difficulty Level
100%
pass@1
87.2%
84.6%
pass@3
80%
79.5%
76.9%
Pass Rate
60%
58.3%
40%
20%
Difficulty Level
Figure 9: Pass rate on different levels of the GAIA dataset.
D.4 PERFORMANCE OF GAIA ON DIFFERENT LEVELS
We show the Pass Rate of the GAIA dataset w.r.t different levels in Figure 9
Specifically, the model achieves its highest performance on level tasks, with 79.5% Pass rate and an 87 Pass@3 rate. more substantial performance degradation is evident at level where the Pass rate drops to 50.0% and the Pass @3 3 rate falls to 58.3%. This trend indicates that the model's efficacy significantly impacted by the complexity of the tasks.
Expert
Leyers
080S
Expert Activations Across (Ours)
Expert Activations Across 48 Layers (DIt).)
0000
0802
Loye
Figure 10: MоE activations illustration on BrowseComp-2l zh dataset. Top- Middlc-Bottom: Bascline, Our model, Difference.
21

---

Published as a conference paper at ICLR 2026
D.5 MOE ACTIVATIONS
We collect the model's 's router logits for the questions' last token and display the top 8 ones after softmax. The scores sorted and averaged over all samples in BrowseComp-zh. The top one shows the activations of the baseline model without applying CPT. The middle one shows the result of our model AgentFounder-30--A
The bottom one shows the difference after caleulating ours minus the baseline. It can be observed that, after undergoing CPT, the expert distribution within the MoE architecture becomes more balanced in the final layers of the model, rather than being concentrated. This potentially offers the benefit of enabling more diversified utilization of multiple experts, which empirically leads to greater training stability during the post-training phase. It also helps mitigate the risk of overfitting by preventing the phenomenon of "dead experts" exppets that cease to be activated and thus fail to contribute meaningfully to the model's performance.
E EXAMPLES
E.1 GENERATED QUESTION EXAMPLE
Example: Generated Questions with the entity "Paris"
Source Entity: Paris
Knowledge The Louvre welcomed 8.7 million visitors in 2024. Visitor numbers thus remained at 2023 levels (8.9 million visitors) in the rather unique context of the Paris 2024 Olympic and Paralympic Games
Knowledge In 2023, France's bedbug outbreak sparked a political row; Paris pushed pre-2024 Olympic action, and the transport minister summoned transit operators.
Knowledge At Paris 2025, Airbus announced 132 firm orders and up to 106 additional option- sfincreases AviLease Aviieess (40+37), Riyadh Air (25+25), ANA (27), LOT (ef-444)-nddrscoorin Puris as a global aviation deal hub.
At the biennial aerospace marketplace named after the city whose grramid-fronted museum recorded high single-digit millions of visitors during period of global athletic celebra- tion, and where the year before a citywide nuisance led authorities to convene transit operators, which buyer placed a perfectly balanced commitment with firm orders equal to options? Answer: Riyadh Air
Question:
This example synthesizes a question from the entity Paris using three news-sourced statements. Knowledge I offers an indirect locational anchor via visitor counts at the gyramid-fronted museum during a global sporting period, which localizes the marketplace without naming the city. Knowledge 2 adds a prior-year civic disturbance that supplies a relative timeline and helps isolate the 2025 iteration of the biennial event. Knowledge provides the order breakdown at that edition, establishes the criterion "firm orders equal options," and narrows the answer to Riyadh Air. Since these facts are recent and fluid, reliable resolution typically requires external retrieval with search tools.
E.2 REASONING ACTION SYNTHESSS DATA EXAMPLE
Example: An Overview Reasoning Action Synthesis Data
The Location): First, the question mentions .... which presents several potential candidates: the Louvre in Paris with its iconic glass pyramid entrance, However, the crucial filtering criterion is the high visitor numbers "during period of global athletic celebration." This could refer to various major international sporting events: Among these candidate cities, only Paris simultaneously possesses both pyramid- fronted museum and hosted a global athletic celebration, thus confirming this refers to the Louvre during the 2024 Paris Olympics. (Clue2: The Time): Next, the question describes "the year before citywide nuisance led authorities to convene transit operators," requiring consideration of events in 2023 that could have affected all of Paris and involved the transportation system. Possible candidates include massive public transit strikes, severe air pollution episodes, Having established the city as Paris, the "biennial aerospace marketplace named after the city" clearly points to the Paris Air
(Cluel:
22

---

Published as a conference paper at ICLR 2026
Show, which is held in odd-numbered years, thus locking in the timeframe the 2025 Paris Air Show.
(Clue3: The Order): Finally, the question asks which buyer placed "a perfectly balanced commitment with firm orders equal to options." Examining the major order announcements at the 2025 Paris Air Show reveals: ... Therefore, the answer Riyadh Air.
E.3 AGENT TRAJECTORY EXAMPLE
Example: Agent Trajectory
Question: At the biennial aerospace marketplace named after the city whose pyramid- gyrated-fronted museum recorded high single-digit millions of visitors during a period of global athletic celebra- tion, and where the year before citywide nuisance led authorities to convene transit operators, which buyer placed a perfectly balanced commitment with firm orders equal to options?
Step 1
<think>
So we need to identify:
-A biennial aerospace marketplace named after city.
-That city had a pyramid- pyramid-fronted museum that recorded high single-digit millions visitors during period of global athletic celebration (maybe Olympic Games?).
"High single-digit millions of visitors" probably means 7-9 million visitors, maybe 8 million. The museum has pyramid front. Which museum has a pyramid? There is the Louvre Pyramid in Paris: the Louvre Museum has glass pyramid entrance; it gets about 10 million visitors per year historically (oreDC.VID). But high single-digit millions? Actually the Louvre had around 9.5 million visitors in 2019 (maybe earlier?). Also "global athletic celebration" could be Olympics. The 2016 Olympies were in Rio Janeiro; but Rio's museum? The city might be Rio de Janeiro; it has the Museum of Tomorrow, but not pyramid. Another possibility: The city Beijing (the National Museum?), the Bird's Nest stadium for Olympics? No.
...Maybe they refer to the "Louvre Abu Dhabi"? has a dome, not pyramid. The "Kunsthis- torisches Museum"? Not. There's the "Museum of the Iraqi Ministry"" Hmm. Better to search. Let's s search web for "pyramid front museum high single digit millions visitors Olympic". </think>
<tool_call>>
"query': ['pyramid front museum high single digit millions visitors Olympies', 'pyramid front museum visitors 9 million', 'pyramid front museum visitor numbers Olympic Games', 'pyramid front museum recorded high single-digit millions visitors during Olympics"] @ttol_ccal>>
<tool Kkool_response"
A Google search for 'pyramid front museum high single digit millions visitors Olympics' found 10 results:
Web Results
[Louvre[[attss:: htt.pp::/ee...wiiii.aaa..o
received 8.7 million
visitors in 2024, 200,000 less than 2023...
A Google search for 'pyramid front museum visitors 9 million' found 10 results: Web Results
[Not just the Louvre: How Europe's museums face the (ttuee]tttps:://eee
com/
n/ //not--0 justtths-ttth-h-
www.reeewwwwwuooesseeeeeeeeeeeee areeuuus-ar---aiii-t-tteeee fut
e/a-71471 If you haven't 't visited the Louvre, then you haven't...
</tool resposeesp
Step 2
<think>
The museum recorded high single-digit millions of visitors during a period of global athletic celebration likely referring to the Paris Olympics in 2024 This refers to some incident in Paris year before something? "Citywide nuisance" possibly referencing noise something
23

---

Published as a conference paper at ICLR 2026
that caused public disruption. Let's search for "citywide nuisance led authorities convene transit operators Paris"
</think>
stool_call>
"query": l'citywide nuisance led authorities to convene transit operators Paris year before',
'Paris citywide nuisance transit operators convened'. 'Paris citywide nuisance transit operators meeting']
</tool_cal _aall>
<tool_ stoolrresponse
A Google search for cityyidd nuisance led authorities to convene transit operators Paris year before found results:
Web Results
[Innovative solutions for sustainable cities]
(https:/ //ww.. ecologie gouv. fr
ility_gb. pdf) Date published: From long time, France has built up tightly-meshed mass
transit system, upgraded and adapted new forms of mobility: coverage of mass transit in the...
Google search for 'Paris citywide nuisance transit operators convened' found 10 results:
Web Results
[Electric Rental Scooters to Banned in Paris ...]
(https: /www. nyt nyt ime s com/ 22/2/02333/0
c --scottrr--aan html
referendum emphasized how many residents...
</tool response>
Step 3
<think>
The "citywide nuisance" could be referring to the electric scooters, described as a nuisance and ...due to the scooter issue. Let's search for article referencing "authorities convened transit operators" after "citywide nuisance".
</think>
<tool_call
'query': ['"covvened transit operators" Paris "electric scooters"", ""convene" "transit operators"
Paris "scoottr""]
</tool_calls
<tool_response>
Google search for '"convened transit operators" Paris "electric scooters"" found 10 results:
Web Results
[Rented e-scooters cleared from Paris streets on eve of ban batthttsss/////ww.teeeee tww.guaaw.....aaaaaaa..
n. wtt.:o//ooolld/22////////1111111 http:////enccceeeeoooooeeeeeeeee
-reets--nn-eee-f--Dee Paris will become the first European capital to...
Google search for ""convene" "transit operators" " Paris "scooter" found 10 results:
Web Results [U.S. National Blueprint for Transportation Decarbonization ... h.. hww.:wnwwwwwywww....e/ee/..
waw-naaionnnnllll......o. ona wnwa--bleprrntttt----
#tzanppottatioon--Gccartooooltt pdf) Mobile applications can more seamlessly
integrate multiple...
</tool_response>
Step
Step 25
<think>
...a could be considered perfectly balanced commitment (firm orders equal to options) for Avelo Airlines. But question: "which buyer placed perfectly balanced commitment with firm orders equal options?" Let's read that article.
</think>
24

---

Published as a conference paper at ICLR 2026
<tool _all>
---ito-the-u----ee market fo foll Lowloo 50- orders purchase rig
mits-from-ucce avelo air nes ". 'Extract details about buyer and number of firm orders options', url info': [iix': query': "'immm orders" "options" "equal" "Paris Air url': http:///a.///a///aa/.//////aaaaa
finaall 50r oy 'title": 'Embraer E2 Finally Breaks c
Into The U.S. Market Following ...., snippet': options for five more at the 2025 Paris Air Show, with deliveries beginning in 2028. SAS followed with its largest aireraft order in nearly '
"hostlogo": None, 'dtte': "5days ago', 'source"; None, 'id": 166]
</tool </tool_call>
<tool reeo__eesssse>>
The useful httpmn/wwiitttaee////eeaaeeeee/a /ttp://999///mmmmrrr---2----
www.::owwwwwwwooo..o - 10-
rm ordersdderss
d-purchase-rights www.-ightsstooucc--------e------ aarloo-airines..
..SSumary: Avelo Airlines, an Ultra-Low-Cosi Carrier, has placed firm order for 50 Embraer E2-E195 aircraft and has 50 additional purchase rights. The aircraft are scheduled enter service with Avelo in mid-2027, with deliveries continuing until 2032.
</tool response>
Step
Step 49
<think>
Not helpful. However we have enough evidence answer. Thus answer: Riyadh Air. But perhaps the question expects "Riyadh Air" is the buyer. Let's verify the official naming: "Riyadh Air" (the new Saudi airline). The order was for 25 Airbus A350- 1000s with options for an additional 25, which is perfectly balanced. So answer: Riyadh Air...
</think>
<tool_call _all>>al>>>_-_
'query': ["'ASL Aviation" "firm orders" "options" "Paris Air Show""]
</tool_call call>
Google search for "'ASI Aviation" "firm orders" " "options" "Paris Air Show" found 10 results:
Web Results
[Fleet Renewal ASL Aviation
www. Aslaviationoolcing
com/ fleet renewal/) At ASL Aviation firm orders and 10 purchase rights, announced at the Paris Air Show Bourget June 2019...
respossess
Step 50
<think>
Let's examine context: "the year before a citywide nuisance led authorities to convene transit operators"... Now, the event where the buyer placed a perfectly balanced commitment with firm orders equal to options might at the same air show (Paris Air Show). In 2025, Riyadh Air placed perfect balance of 25+25.
Alternatively, in 2019 (previous Paris Air Show), ASL Aviation placed 10+10... So maybe the question expects us to answer about Riyadh Air because it's the However need to consider that "the year before a citywide nuisance led authorities to convene transit operators" indicates that the nuisance event happened the previous year relative the statement. Then ask: which buyer placed perfectly balanced commitment at the biennial aerospace marketplace? That suggests the buyer placed that order the Paris Air Show 2025 (recent). Therefore Riyadh Air correct.
Thus answer: Riyadh Air.
We need to provide answer clearly: The buyer is Riyadh Air, which placed a firm order for 25 Airbus A350- 1050000000000 aircraft with purchase rights for additional 25 (equal).
25

---

Published as a conference paper at ICLR 2026
Thus final answer: Riyadh Air.
</think> <answer>
The question references several interconnected events centered around Paris...:
1. Biennial Acrospace Marketplace: ...Parii Air Show, officially known as ... attracting manufacturers, airlines, buyers, and aviation professionals from across the world...
Pyramid-Fronter Museum and High Single-Digit Visitor Numbers During Global
Athletic Celebration: In 2024, during the period of the Paris Olympic Games-a major global athletic celebration the Louvre recorded visitor numbers in the high single-digit millions (approximately 8.7 million visitors in 2023 and similar figures in subsequent months)...
Citywide Nuisance and Convening of Transit Operators: In 2023, Paris experienced citywide debate electric scooters (e-scooters), which many residents and officials labeled "nuisance" due to safety concerns and congestion...
Recent Developments at the Paris Air Show
Among the many deals announced, Riyadh Air stood out for placing perfectly balanced commitment... This means the number of firm orders equals the number of options, making the commitment symmetrical and strategically flexible. Key details of Riyadh Air's order:..
Historical Context and Other Buyers While this demonstrated a similarly balanced commitment, the query appears to focus recent activity aligning with the context provided (i.e., after the citywide e-scooter nuisance debate and Olympic Games).
Buyer
Year Announced 2025
Aireraft Model
Firm Orders
Options
Riyadh
Airbus A350- 10000
25
ASL Aviation Holdings DAC
2019
Boeing 737-800BCF
10
10
Conclusion
In summary, the buyer that placed perfectly balanced commitment with firm orders equal to options at the recent Paris Air Show is Riyadh Air... <answer>
26


Reviews:
[{"review_id": "9CUDWY382X", "reviewer": "Reviewer_pAr1", "summary": "This paper proposes Agentic Continual Pre-Training, an intermediate training stage between pre-training and post-training to bridge the optimization conflict between pre-training and agent post-training. To test the effectiveness of the proposed method, the authors further synthesized large-scale data for model training. The resulting model, AgentFounder-30B, achieves state-of-the-art results on 10 benchmarks, including 39.9% on BrowseComp-en, 31.5% on HLE, and 72.8% on GAIA. Ablations show consistent improvements, and scaling analyses indicate steady gains with larger models and data. Key contributions include:\n\n(1) First-order Action Synthesis (FAS): reformulates static knowledge into complex questions and synthesizes planning/reasoning behaviors without API costs.\n\n(2) Higher-order Action Synthesis (HAS): reuses suboptimal trajectories by expanding each step into contrastive multi-choice decision processes.\n\n(3) Two-stage CPT strategy: progressive training with 32K and 128K contexts (200B + 100B tokens) to enhance long-horizon reasoning.", "questions": "The paper's core contribution relies heavily on the synthetic FAS and HAS data synthesis. Could you specify the data release plan, like whether the whole CPT stage training data will be released? If it's not, it's possible to release a sample subset for community inspection.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "(1) The data synthesis pipeline is scalable and efficient.  FAS enables large-scale agentic data generation through knowledge-to-question transformation without costly API calls, while HAS reuses suboptimal trajectories by converting them into step-level decision processes. Together, they improve data efficiency and coverage. The addition of reject sampling with knowledge alignment verification further enhances data quality.\n\n(2) The experiment results are promising. AgentFounder-30B achieves SOTA across ten benchmarks, outperforming all open-source agents and even surpassing some commercial systems. It is the first open-source model to exceed 30% on HLE (31.5%), with consistent gains across all SFT configurations, demonstrating strong robustness.", "weaknesses": "(1) Limited Theoretical Evidence for “Optimization Conflict” \nThe claimed conflict between capability learning and alignment is intuitively appealing but empirically unsupported. It would be great if the authors could provide more evidence for this claim, like gradient interference analyses (e.g., cosine similarity between CPT and SFT objectives), visualize loss landscapes, and probe capability retention during SFT with vs. without CPT.\n\n(2) Lack of analysis regarding the influence of the model on data synthesis.  It would be great if the authors could provide more details on the models used in data synthesis. Ideally, it would be great if the authors could further provide an oblation study regarding model performance against the quality of the model for data synthesis.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "6a0JRD6YQr", "reviewer": "Reviewer_cMZ8", "summary": "This paper introduces Agentic Continual Pre-Training as an intermediate training state between pre-training and post-training for building deep research agents. The authors propose a two-stage training pipeline: stage 1 involves standard next-token prediction training on synthetic data, while stage 2 focuses on long-context training. To support the training pipeline, authors propose two key data synthesis methods: First-order Action Synthesis (FAS) for generating planning and reasoning data and Higher-order Action Synthesis (HAS) for transforming wasted trajectories into multi-step multi-choice decision problems. \n\nThe trained model achieves strong performance across 10 benchmarks, including BrowseComp, HLE, GAIA, etc. However, the paper does not provide details of some of the claims and doesn't provide enough details about the generated data.", "questions": "- GLM 4.5 Comparison: Please add some details on how your strategy differs from the GLM 4.5 strategy. What is the difference between the agentic data sizes used for GLM 4.5 and the AgentFounder model?\n- FAS vs Trajectory Collection: What is the performance comparison between FAS-generated data and actual collected trajectories (e.g., from rollouts)? Is FAS primarily a cost-saving measure or does it provide qualitative benefits?\n- HAS Design Choices: In HAS data construction, you generate N alternatives. What is the value of N here? Please add details about the dataset construction to the paper.\n- In the first stage, is there any general-purpose reasoning corpus added to the data mix? If not, does the model suffer from forgetting? Some evaluations on general-purpose language reasoning benchmarks like MMLU would be appreciated and would make the claims stronger.\n- Can authors please provide error analysis of the trained model on the evaluation datasets? What type of errors are still prevalent after Agentic CPT? What type of errors are targeted in this training paradigm? This would add significant depth to the work.\n- It is not clear to me which data is used for what stage. Some clarification on this would be appreciated.\n- In Table 4, what happens if you skip Stage 1 training? \n- Evaluation Protocol: The Agentic CPT data is collected using the same set of tools as used in the evaluation. However, I am not sure if the other models were trained to use the same set of tools. This gives an inherent advantage to the AgentFounder model. If my understanding is correct, how does it affect the tool use for AgentFounder vs other models? Are the majority of the errors in baseline models because of incorrect tool use?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "- Strong Empirical Performance: The proposed model achieves state-of-the-art results on multiple challenging benchmarks and beats stronger and larger commercial models on most datasets.\n- Scalable Data Synthesis: The FAS approach for generating diverse question-answer pairs from entity-anchored knowledge without requiring expensive API calls or human supervision is practical and cost-effective.\n- Model Size and Data Scaling: The Authors show that their method scales with the model size and data volume effectively.\n- Adaptability to different SFT data: Table 3 shows that the authors' model consistently improves performance across different post-training dataset setups, suggesting the CPT does provide a better foundation.", "weaknesses": "- Unverified Claims: The Authors claim that general-purpose foundation models create \"inherent optimization conflicts\", but this is not verified in this work, and no supporting evidence in the form of a citation is provided. Similarly, authors also claim that \"the quality of first-step reasoning exhibits strong positive correlation with final task completion rates\". Although this statement may seem intuitive, no experimental validation is provided to support it.\n- Unclear distinction from domain adaptation: While the authors claim that the trained model provides a better foundation model for post-training, the paper doesn't verify this claim outside the domain of web-search-based benchmarks. This is understandable because the data was collected using web-search-specific tools; however, the authors should not claim it as a foundation model.\n- Limited Transparency on data composition: While the work mentions using 200B and 100B tokens for Stage 1 and 2, respectively, the actual composition of the FAS/HAS data or the data sources is never discussed in the work. How did the authors verify that there was no potential test-data leakage during the training data collection?\n- Zero Supervisory Signal: In section 2.2, the authors claim that the data is collected using a zero supervisory signal. However, in section 2.2.2, the authors employ rejection sampling using an LLM-as-a-judge. This is a type of supervision. Authors should clarify this claim.\n- Limited Error Analysis: The paper does not provide details error analysis showing what types of tasks/questions benefit most from Agentic CPT versus post-training alone, making it difficult to understand the method's limitations.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "d09lQmo3tZ", "reviewer": "Reviewer_UZu1", "summary": "This paper proposes Agentic Continued Pre-Training for having better base pre-trained models for capabilities requiring tool calls and multi-hop reasoning like DeepResearch. The authors develop a model named AgentFounder-30B using this continued pre-training setup by generating synthetic pre-training data using FAS (First-order Action Synthesis) and HAS (Higher-order Action Synthesis); and further apply SFT on top of the base model with agentic capabilities. They achieve SoTA scores (across open models/deepresearch agents) on various agentic benchmarks like BrowseComp, HLE, GAIA etc.", "questions": "I have asked most of my questions in the weaknesses sections above. I am willing to increase my score if the authors are able to address the issues.", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- The paper tackles an important problem of building better base models for agentic capabilities where standard pre-training data is not easily available for multi-hop reasoning, long trajectories, etc.\n- The paper's synthetic data generation pipeline is interesting and captures various insights like collecting discarded rejection sampling and historical tool use data, correlation of quality of first-step reasoning with final task completion rates.\n- The empirical results are impressive.", "weaknesses": "I feel there are quite a lot of points to address before this paper is ready for publication.\n\n- The entire paper reads more like a high-level blog rather than a technical white paper with concrete methodology, steps/details, and experiments.\n- The paper tries to present that they are the first to do agentic CPT, but a lot of mid-training/pre-training works do this step before post-training for example: https://arxiv.org/abs/2507.20534\n- Lines 65-67 \"Consequently, both SFT and RL training depend on limited deterministic\nsupervisory signals that lock models into replicating specific behavioral patterns rather than develop\nflexible decision-making capabilities\". The authors don't show any evidence for this and neither do they cite any works highlighting this phenomenon. SFT does mimic specific patterns, but I've not seen this case happening with RL.\n- A lot of key technical details are missing (at least in the main paper), like where are the questions sourced from, how many parallel steps are generated for the HAS strategy, what models are used for the rejection sampling, number of tokens for SFT and what's the source of the SFT data, etc.\n- How is the model performance affected on non-agentic benchmarks by adding the agentic CPT data? There's no analysis on that.\n- What's the effect of the model used for rejection sampling? Can weaker models help as well to eliminate the \"distillation\" effect of using more capable models to get the rejection sampling data?\n- The effect of HAS strategy is minimal on top of FAS, and the evaluation gains seem to be just noise.\n- Since I don't know the source of the questions used to generate the synthetic data, how did the authors ensure no contamination with the evaluation sets like HLE.\n\nAlthough not a weakness, there are a few typos here and there: AgentFoudner instead of AgentFounder in various places, line 224: high-orider instead of high-order.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)