WebDancer: Towards Autonomous Information Seeking Agency
Authors: Jialong Wu
Baixuan Li
Runnan Fang
Wenbiao Yin
Liwen Zhang
Zhenglin Wang
Zhengwei Tao
Ding-Chu Zhang
Zekun Xi
Xiangru Tang
Yong Jiang
Pengjun Xie
Fei Huang
Jingren Zhou
Venue: NeurIPS 2025 poster ()
Keywords: Web agent
Information seeking
Primary area: applications

Abstract: Addressing intricate real-world problems necessitates in-depth information seeking and multi-step reasoning. 
Recent progress in agentic systems, exemplified by Deep Research, underscores the potential for autonomous multi-step research. 
In this work, we present a cohesive paradigm for building end-to-end agentic information seeking agents from a data-centric and training-stage perspective.
Our approach consists of four key stages: (1) browsing data construction, (2) trajectories sampling, (3) supervised fine-tuning for effective cold start, and (4) reinforcement learning for enhanced generalisation.
We instantiate this framework in a web agent based on the ReAct format, WebDancer.
Empirical evaluations on the challenging GAIA and WebWalkerQA benchmarks demonstrate the strong performance of WebDancer, achieving considerable results and highlighting the efficacy of our training paradigm. 
Further analysis of agent training provides valuable insights and actionable, systematic pathways for developing more capable agentic models.

WebDancer: Towards Autonomous Information Seeking Agency
Jialong Wu" ! Baixuan Li", Runnan Fang", Wenbiao Yin" Liwen Zhang, Zhenglin Wang, Zhengwei Tao, Dingehu Zhang, Zekun Xi, Xiangru Tang, Yong Jiang, Pengjun Xie, Fei Huang, Jingren Zhou
Tongyi Lab Alibaba Group
Correspondence wujiallongal@gmailccoo
fyinuenbias ao yub, jangg-jnnnyyl wtw.://aababbbaiccc.oom
Abstract
Addressing intricate real-world problems necessitates in-depth information seeking and multi-step reasoning. Recent progress in agentic systems, exemplified by Deep Research, underscores the potential for autonomous multi-step research. In this work, we present cohesive paradigm for building end-to-end agentic information seeking agents from a data-centric and training-stage perspective. Our approach consists of four key stages: (1) agentic data synthesis, (2) trajectory sampling, (3) supervised fine-tuning for effective cold start, and (4) agentic reinforcement learning for enhanced generalization. We instantiate this framework in web agent based on the ReAct, WebDancer. Empirical evaluations on the challenging information-sseking benchmarks, GAIA and WebWalkerQA, demonstrate the strong performance of WebDancer, achieving considerable results and highlighting the efficacy of our training paradigm. Further analysis of agent training provides valuable insights and actionable, systematic pathways for developing more capable agentic models."
1 Introduction
Web agents are autonomous systems that perceive their real-world web environment, make decisions, and take actions to accomplish specific and human-like tasks. Recent systems, such ChatGPT Deep Research [1] and Grok DeepSearch [2], have demonstrated strong deep information-secking capabilities through end-to-end reinforcement learning (RL) training.
The community's previous approaches for information seeking by agentic systems can be categorized into two types: (i) Directly leveraging prompting engineering techniques to guide Large Language Models (LLMs) or Large Reasoning Models (LRMs) [3-5] to execute complex tasks. (ii Incor- porating search browser capabilities into the web agents through supervised fine-tuning (SFT) or RL [6. 5, 7-10]. The first training-free methods are unable to effectively leverage the reason- ing capabilities enabled by the reasoning model. Although the latter methods internalize certain informatioon-seeking capabilities through SFT or RI. training, both the training and evaluation datasets relatively simple and do not capture the real-world challenges, for instance, performance on the 2Wiki dataset has already reached over 80%. Morcover, the current SFT or RL training paradigm does fully and efficiently exploit the potential of informmaiion-seeking behavior. Building autonomous information seeking agency involves addressing set of challenges that span web environment perception and decision-making: making: (1) acquiring high-quality, fine-grained browsing data that reflects
Equual contribution. Correspondence.
The codes and demo are released in https: /g/ github. wwt.thhh......o.teeeeeeeeeeeeeee wwtmm/Allabbbac. Wtwp:aaaaaaiDDDePPPeeeeee Dbt.bpabbab-..//Deeeeeseeerra
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

diverse user intents and rich interaction contexts, (2) constructing reliable trajectories that support Long-horizon reasoning and task decomposition, and (3) designing scalable and generalizable training strategies capable of endowing the web agent with robust behavior across out-of- ditriitibbttion web environments, complex interaction patterns, and long-term objectives.
To address these challenges. our objective is to unlock the autonomous multi-turn information- seeking agency, exploring how to build a web agent like Deep Research from scratch. An agent model like Deep Research produces sequences of interleaved reasoning and action steps, where each action invokes a tool interact with the external environment autonomously. Observations from these interactions guide subsequent reasoning and actions until the task is completed. This process is optimized through end-to-end -uggmented training. The ReAct framework [11] the most suitable paradigm, as it tightly couples reasoning with action to facilitate effective learning and generalization in interactive settings.
We aim to provide the research community with a systematic guideline for building such agents from a data-centric and training-steg stage perspective.
From a data-centric perspective, constructing web QA data is crucial to building web agents, re- gardless of whether the training paradigm is SFT or RL. Widely used QA datasets are often shallow, typically consisting of problems that can be solved with a single or a few-turn search. Previous works often filter the difficult QA pairs from open-sourced
datasets using prompting techniques [7]. Additionally, challenging web-based QA datasets typically only have test or validation sets, and their data size is relatively small. For example, GAIA [12] only has 466, WebWalkerQA [3] contains 680 examples, and BrowseComp [13] has 1,266, making them insufficient for effective training. Therefore, the automatic synthesis of high-quality datasets becomes crucial. [14, 15]. We synthesise the datasets in two ways: 1). By crawling web pages to construct deep queries, referred to as CRAWLQA, enabling the acquisition of web information through click actions. 2). By en- hancing easy-to-hard QA pairs synthesis to incentivize the progression from weak-to-strong agency, transforming simple questions into complex ones, termed E2HQA.
From a training -tagge perspective, prior work has explored SFT or off-policy RL, but these approaches often face generalization issues, particularly in complex, real- weal-world search environments. Other methods adopt on-policy RI directly [6], but in multi-tool settings, early training steps tend to focus primarily on learning tool usage via instruction following. To enable more efficient and effective training, we adopt a two-stage approach combining rejection sampling fine-tuning (RFT) with subsequent on-policy RL. For the trajectory sampling, restrict the action space to two commonly effective web information-sseking tools as action: search Q and click & Building on this setup, we employ rejection sampling to generate trajectories using two prompting strategies: one with a strong instruction LLMs for Short-CoT and another leveraging the LRMs for Long-CoT. These yield high-quality trajectories containing either short or long thought, respectively. In the RL stage, we adopt the Decoupled Clip and Dynamic Sampling Policy Optimization (DAPÓ) algorithm [16]. whose dynamic sampling mechanism can effectively exploit QA pairs that remain underutilized during the SFT phase, thereby enhancing data efficiency and policy robustness.
Our key contributions can be summarized as follows:
we abstract the end-to-end web agents building pipeline into four key stages: Step I: Construct diverse and challenging deep information seeking QA pairs based on the real-world web environment ($2.1); Step II: Sample high-quality trajectories from QA pairs using both LLMS and LRMs to guide the agency learning process ($2.2); Step III: Perform fine-tuning adapt the format instruction following to agentic tasks and enviromments ($3.1); Step IV: Apply RL to optimize the agent's 's decision-makee maainn and generalization capabilities in real-world web environments ($3.2). We offer a systematic, end-to-end pipeline for building lon--term information -eeioonnn-eiinggg web agents.
Extensive experiments on two web information seeking benchmarks, GAIA and WelWalkerQA, show the effectiveness of our pipeline and WebDancer ($4). We further present a comprehensive analysis covering data efficiency, agentic system evaluation, and agent learning ($5).
WebDancer represents the first end-to-end trained deep research agent within the Tongyi DeepRe- search series [17]. It is among the earliest works in the community to propose fully end-to-end training pipeline for deep research agents, pioneering studies on both agentic data synthesis for deep research tasks and agentic reinforcement learning.

---

CRAWLQA
whire gome nanked fourth in the Entot Eare Fetruaryy 2025 but was not featured In the 2824 Sodoe Bbmes Shownsss?
Koziah's
Page
Explorstion
Synthasis
E2HQA
sigriflcant contributions to fuzty particulanly reals of intelliget digital tachnclogies and industrial processes, utilizing systes noted for complex reasoning apposed to traditional sinary logte, and differret froe the Waadant aperoach
technology
In 2010, accollas
WTo received the BEEE Frank
nomed aitter a plonzen in
artsFicisl siteellgence RnS mAtrded Mho was recipient this award?
Rosenblatt Award
Qn
Qn+1
Nichio Sugano
Figure Two web data generation pipelines. 1 For CRAWLQA, first collect root url of knowlageable websites. Then we mimic human behavior by systematically clicking and collecting subpages accessible through sublinks the root/... ... page. Using predefined rules, we leverage GPT4o to generate synthetic QA pairs based on the gathered information. @ For E2μQA, the initial question Q1 is iteratively evolved using the new information C, retrieved from the entity E, at iteration i, allowing the task to progressively scale in complexity, from simpler instances to more challenging ones. We use GPT-4o rewrite the question until the iteration reaches n.
Deep Information Seeking Dataset Synthesis
2.1 QA Pairs Construction
To enable longer-horizon web exploration trajectories, it is essential curate substantial corpus of complex and diverse QA pairs that can elicit multi-step reasoning, goal decomposition, and rich interaction sequences. The main requirements for these QAs are: (i) diversity of question types, and (ii) increased task complexity as measured by the number of interaction steps required for resolution. In contrast prior datasets that predominantly involve shallow queries solvable in 2-3 steps, objective is to scale both the volume and the depth of multi-hop reasoning. To achieve this, we primarily develop the below datasets: CRAWLQA and E2HQA.
CRAWLQA Constructing QA pairs based information crawled from web pages represents effective paradigm for scalable knowledge acquisition [3]. We begin by collecting the root URLs of official and knowledgeable websites spanning arxiv, github, wiki, etc. [12] To emulate human browsing behavior, we recursively navigate subpages by following accessible hyperlinks from each root site. We employ GPT-4o synthesize QA pairs from the collected content. To ensure specificity and relevance of questions, inspired by Sen et al. [18], we prompt LLMs to generate questions of designed types (e.g., COUNT, MULTI-HOP INTERSECTION)
in-context learning [19].
E2HQA Similar the reverse construction strategy [13, 20], begin from large QA pairs in SimpleQA style [21] where each answer is a concise, fact-seeking entity. We first select an entity En from the question Qn. where n represents the number of refinement iterations. Then, we use the LLMs to construct query based on this entity in order to search via search engine S' for information C, related to E,, After that, we LLMs restructure the obtained content into a new query R, to replace the original entity in the question. The process can be signaled as: R,, π (S(C..)). This have constructed before finding
way, the new question Qn+1 requires solving the sub-problem the answer to the original question. Moreover, it ensures that the answer does not change during the question refinement, thereby preserving the validity of the QA pairs. By continuously searching, we can gradually rephrase an initially simple question into a more complex multi-step one. Moreover, the number of steps needed to solve the problem can controlled by adjusting the number of rephrasing times.
3

---

2.2 Agent Trajectories Rejection Sampling
Agent Setup Our agent framework is based on ReAct [11], the most popular approach to language agents. A ReAct trajectory consists of multiple Noought-Ationn-Dseervaa
rounds, where an LM generates free-form Thought for versatile purposes, and structured Action to interact with environments (tools) and receive Observation feedback. We assume that the agent execution loop at time can be denoted as (Tr, Olt, 0t), where denotes Thought, a signifies Action, and o represents Observation. a can he further expressed as (a"", aa"), where a is the name of the action, and of the parameters required to perform the action, a.t E (search, visit, answer), which corresponds to the two most important agentic tools in the deep information seeking. For search Q acti on, a? consists of query and filter_year, while for visit & action, a consists of goal and url url_link. The observation of search action includes the Top-10 titles and snippets, whereas the observation of the visit action is the evidence and summary, generated by a summarizer model Mg. The iteration terminates when the action is answer.
Then the historical trajectory can be signaled as:
He = (To, 0o 00: .I, 04-1,
At time step 1, the agent receives an observation or from the web environment and generates thought Tr taking an action 04, following poliy x or/He).
The Chain-of-Thougee (CoT) method has significantly enhanced the inferential capabilities of LLMs through a step-by-step reasoning process [22], corresponding to the thought component in agentic systems. This process is critical for agentic execution, enabling high-level workflow planning, self-reflection, information extraction, adaptive action planning, and accurate action (tool usage).
Short and Long CoT Construction Agent models internalise the CoT generation capability an active behavioral component of the model. [23, 24] The length of CoT and the associated thinking patterns play a crucial role in performance [25-27] We propose two simple yet effective methods for constructing the short CoT and long CoTT, respectively. For short CoTSs, we directly leverage the ReAct framework to collect the trajectories using a powerful model, GPT-4o. For long CoTs, we sequentially provide the LRMss QwQ-Plus, with the historical actions and observations oanroiittons at each step, enabling decide the next action autonomously, Notably, we exclude the previous thought during further inference, as the LRM, QwQ-Plus, has not been exposed to multi-step reasoning inputs during training. However, we retain the thought at each step the generated trajectory, as they serve as valuable supervision signals. The LRM's intermediate reasoning process, denoted as, denoted as "<reasoning content>". ", is recorded as the current thought of the current step. Each constructed QA instance undergoes rejection sampling up to N times to ensure quality and coherence.
Trajectories Filtering We adopt three-stage funnel-based trajectory filtering framework consist- ing of validity control, correctness verification, and quality assessment.
For validity control, direetly prompting LLMs generate responses in the ReAct format under long-content conditions may result in non- non-compliance with instructions. In such cases, we discard these data points.
For correctness verification, we only retain correct results. We follow the evaluation method- ology proposed by Phan et al. [28] and Wei et al. [13] and use GPT-4o for accurate judgment. For quality assessment, we first apply rules to filter out trajectories with more than two actions, ensuring that there are no hallucinations and no severe repetitions. Subsequently, we filter the trajectories based on prompting to retain those that meet the following three criteria: Information ron-redund,ncy, Goal Alignment, and Logical Reasoning and Accuracy.
The QA pairs that are not present in the SFT dataset can be utilized during the reinforcement learning stage effectively."
3 Multi-Step Multi-Tool Agent Learning
After obtaining high-quality trajectories in ReAct format, we seamlessly incorporate them into our agent SFT training stage. Specifically, Thought segments are closed by <<hinkk>] and </think>] Action
The details of training datasets and shown in App. D.

---

segments by <lool
and
</tool calb>
Observation segments by <tuol. responee and </tool response>
The final Action segment corresponds to the final answer, enclosed by <answer> and clanswer> In addition, the QA data without trajectories. which those filtered during earlier stages, can be effectively leveraged during the RL phase. We first train policy model To via agent SFT for cold start, followed by agent RL for generalization. The overall training framework is illustrated in Figure
(I)SFT
(II)RL
KL
Base Model
0.T.4
CRAMLQA E2HQA
Brd4y
LLM Policy
5/2
Group Computation
Ag
R
Reward Calculation
Rollout
Advantage
Task input
<teal resporses
coogle search "Gedot respanse)
<cool call>
"arguments"...
thought
Long COT
to & Tool Call
stool rew.opeepssss....oooo wttoos...ttooorosooooo
Which game ranked Fourth in the Sodot Xt Gane Jow February 2025...?
s/
cchtokothoughtt//ee c/htnnk
LLM Policy
<cool ("naac": "visit", "arsuments"...
ctooL.call collbacttonc/toolooooooo
stool Lool_rrsponee s0bs4/tool__ rtoprr/eeseeeeeseee
<tool _cool_call>>>>
<Lhānkolo c/
<taol, pesporree. rrsporsss
</tonl
ethinko thigghh/hhhhhhhhhii conserraanseerc//nsser
</tool_coll>
("rase": "search", "arguments"...
canswers gare that...//nnswrn
G Dostput
The overview of training framework. (I) The SFT stage for cold start utilizes the reformatted ReAct datasets, where the thought includes both short and long CoT, respectively. (II) The RL stage performs rollouts with the tool calls on the QA pairs that are not utilized during the SFT stage, and optimizes the policy using the DAPO algorithm.
Figure 2:
3.1 Agent Supervised Fine Tuning
To capture complete agentic trajectories, we train the policy model 0 via supervised fine-tuning on obtained decision-makin, trajectories. The cold start enhances the model's capability couple multiple reasoning and action steps, teaching it a behavioral paradigm of alternating reasoning with action, while preserving its original reasoning capabilities as much as possible. Following the empirical findings of Chen et al. [29, 6). Zhang et al. [23], to avoid interference from external feedback during learning, we mask out loss contributions from observation
the agentic world modelling task, which has been shown to generally improve performance and robustness. Given the task context to and the complete agentic execution trajectory = (Io, I1, In-1, where each E {T, (T,0,)). the loss function L is computed as follows:
L
[s. # o) log ( tt, Fee)
12,
Here, 1|2; * o] filters out tokens corresponding to external feedback, ensuring that the loss is computed over the agent's autonomous decision steps. The SFT stage offers strong initialization for the subsequent RL stage [30].
3.2 Agent Reinforcement Learning
The agent RL stage aims to internalize the agency capability into the reasoning model, enhancing its multi-turn, multi-tool usage capacity with outcome- rewards. [31] Building on the SFT stage, RL employs Decoupled Clip and Dynamic Sampling Policy Optimization algorithm to refine and incentivize the policy model na 's ability to interleave
sequences. DAPO Decoupled Clip and Dynamic Sampling Policy Optimization (DAPO) algorithm an RL algorithm that optimizes policy to produce higher-reward outputs under reward model R [16, 32]. For each question-answer pair (g, a) from the data distribution D, DAPO samples a set of candidate agentic executions (o, 1G The policy is then updated to maximize the following objective:
5

---

[DAPO(0) Do N (cooteezt))
lo.1
min (ri,())Aai,, clip clip(ri.(0), (Ti,(0)) Elown + Ehigh Ar.))
Ei=1 |oi|
s.t. 0 < (os is_equivalently, o:) <G,
where agentic execution o, refers solely to the tokens generated by models, excluding any tool responses. In contrast, contezt, including both the model outputs and tool responses, is used to construct the imput trajectory for computing To However, the optimization is applied only to the model-generated portion og, aligning with the SFT. is the clipping range of the importance sampling ratio ri, (0). And Ai, e is an estimator of the advantage of the i-th agentic executions time step t:
To 05 g: Oi,ct)
R² - --eanman ({R:))
Ti. (0)
A
Toaid Oi g: 0i,ct)
std (
The dynamic sampling mechanism over-sample and filters out prompts with accuracy equal to 1 and O. It is crucial in data-training pipeline, the remaining QA pairs, being synthetically generated-may contain invalid or noisy instances that could otherwise degrade policy learning. Such unreliable samples can be effectively ignored, ensuring the agent focuses learning from high-quality signals.
Agentic Action Rollout Within the ReAct framework, each round of agentic execution begins by generating a thought, closed by [think>> and [</hink>>]. followed by action name am and
corresponding parameters a", enclosed by <stool calls and calls operation, all conditioned the iteration history H. These components are iteratively used to interact with the real-world search
environment, producing observation as feedback, bounded by responses and </tool response
upon the <tooo_response is detected. The round of interaction spans from <thinks to </tool response>
The rollout concludes with the generation of cunswer>> and [clanswerz]] following the final thought.
Reward Design The reward design plays a critical role during the RL training process [26]. Our reward system mainly consists of two types of rewards, scoreforaa and scoreanswer- Given that format consistency has been largely addressed during the initial RFT stage, the scoreformat in the overall reward. The scoreformat is binary: it is set to 1 only if the entire output strictly conforms the required format and all tool calls in json format are valid. Considering that the QA answers are inherently non- non-verifiable, cannot be reliably evaluated using rule-based F FI/EM metrics, despite the brevity of the responses, and that the final evaluation relies LLM-as-Judah -J-dddge [33] which the judge model is Mj. we opt to employ model-based prompt evaluation as the answer reward signal [34-36], The eaorea is also binary, assigned as only when the response is judged as correct by the LLMs. The final reward function is:
assign small weight to
R(g.. y) 0.1 scoTeteeaat + 0.9 scor Casser where ū denotes the model prediction and y the reference answer.
(5)
4 Experiments
4.1 Experimental Setup
We evaluate our approach on two established deep information benchmarks: GALA and Walkerr. In this work, we adopt the (LMM-a-JJdges paradigm evaluate both tasks using the Pass@1 metric, following [4]. The details of the datasets and basclines are introduced in App. E. and App. E.2, respectively. The implementation details are shown in App. E.3. Qwen-7B and Qwen-32B are trained on Short-CoT datasets, while QwQ-32B is trained on Long-CoT datasets. Further analyses are shown in Sec. 5.
4.2 Experimental Results
Main Results shown Table 1, frameworks without agentic capabilities (No Agency) perform poorly on both the GAIA and WebWalkerQA benchmarks, highlighting the necessity of active

---

Table Main results on GAIA and WebWalkerQA benchmarks. We discuss the reported results of baselines and concurrent works in App. C.I. "." means results that are either not reproducible or not reported. The best results among all frameworks are bolded.
GAIA
WebWalkerQA
Backbone
Framework
Level
Level
Level
Avg.
Eaay
Medium
Hard
Avg.
No Ageney
Qwen- 2.5-BB
Base
12.8
3.8
6.8
1.25
0.8
0.7
0.8
Base
20.5
9.6
8.3
13.6
3.8
2.5
3,3
3.1
Owen-2.5-32B
RAG
12.8
11.8
8.3
11.8
23.1
14.3
11.3
15.3
Qwen-2.5-72B
Base
20.5
13.5
0.0
14.6
9.4
7.1
3.3
6.3
GPT-4o
23.1
15.4
17.5
6.7
5.5
Base
30.8
15.4
25.0
22.3
7.5
2.1
4.6
4.3
QwQ-32B
RAG
33.3
36.5
32.0
36.9
26.1
33.5
31.2
DeepScck-R1-671B
Base
43.6
26.9
8.3
31.1
5.0
11.8
11.3
10.0
Clove-Soureed Agentic Frameworks
OpenAl DR
74.3
67.4
Open-sourced Agentic Frameworks
Search-ol
23.1
17.3
0.0
17.5
Qwcn-2.5-7B
R1-Seurcher
28.2
19.2
8.3
20.4
Qwen-2.5-32B
Scarch-ol
33.3
25.0
28.2
Scarch-ol
53.8
34.6
16.7
39.8
43.1
35.0
27.1
34.1
QwQ-32B
WebThinker-Be
53.8
44.2
16.7
44.7
47.2
41.1
41.9
WebThinker-LL
56.4
50.0
16.7
48.5
58.8
44.6
40.4
46.5
Simple DS
50.5
ReAct Agentic Frameworks
Vanilla ReAct
28.2
15.3
18.4
28.1
31,2
16.0
24.2
Qwen-2.5-7B
WebDancer
41.0
30.7
31.0
40.6
44.1
28.2
36.0
Vanilla ReAct
46.1
26.9
0.0
31.0
35.6
38.7
22.5
31.9
Qwen-2.5-32B
WebDancer
46.1
44.2
8.3
40.7
44.3
46.7
29.2
38.4
QwQ-32B
Vanilla Reáct
48.7
34.6
16.6
37.8
35.6
29.1
13.2
24.1
WebDancer
61.5
50.0
25.0
51.5
52.5
59.6
35.4
47.9
GPT-4o
Vanilla ReAct
$1.2
34.6
8.3
34.6
34.6
42.0
23.9
33.8
information-seeking and agentic decision-making for these tasks. The closed-source agentic system, OpenAl DR, through end-to-end RL training achieves the highest scores. Among Open-sourced frameworks, agentic approaches built on top of native strong reasoning models like QwQ-32B consistently outperform their non-agentic counterparts, demonstrating the effectiveness of leveraging bessoing-sspciaaiizee models in agent construction. Importantly, under the highly extensible ReAct framework, our proposed WebDancer shows substantial gains over the vanilla ReAct baseline across different model scales. Notably, it even surpasses the performance of GPT-4o in the best-case scenario. This demonstrates that even within a lightweight framework, our method significantly enhances agentic capabilities over the underlying base model, validating the strength and generality of our approach. WebDancer also
Results on More Challenging Benchmarks We evaluate our approach two more challenging datasets, BrowseComp (En.) [13] and BrowseComp-zhh (Zh.) [20], which are designed better reflect complex information- seeking scenarios using PASSC1/PASSON As shown Table 2, WebDancer demonstrates con- sistently strong performance across both datasets, high- lighting robustness and effectiveness in handling difficult reasoning and information-seeking tasks.
Table 2: Results on BrowseComp (En.) and BrowseComp-zhe (Zh.).
Framework
Browsing
En.
Zh.
x
0.6
6.2
GPT-4o
1.9
QwQ-32B
x
11.1
WebDancer
3.8/7.9
18.0/31.5

---

-02
Msuge 3mn
Carago-
1022
Boome
78
328
owQ
78 328
0w0
78
328
Qwa
Figure Detailed evaluation results using Pass@1, Pass@3 and Cons@3 metric on GAIA benchmark.
Analysis
Detailed Results We conduct detailed analyses on the GAIA datasets. Given the dynamic and complex nature of agent environments, as well the relatively small and variable test set, further conduct a fine-grained analysis of Pass@3 and Cons@3 in Figure 4. The Cons@3 metric is computed by evaluating the number of correct responses out of three independent attempts: achieving one correct answer yields a score of 1/3, two correct answers yield 2/3, and three correct answers result full score of For non reasoninn models, RL leads substantial improvements in both Pass@3 and Cons@3. Notably, the Pass@1 performance after RL is comparable to the Pass@3 of the SFT baseline, consistent with previous findings [37, 38] suggesting that RL can sample correct responses more efficiently. For LRMs, while the improvements in Pass01, Pass@3, and Cons@3 after RL are marginal, noticeable gain in consistency observed; this may be due to sparse reward signals caused by excessively long trajectories [39, 40]. This suggests that continued on-policy optimization may yield limited benefits for LRMs in agentic tasks. Our best-performing model achieves Pass@3 score of 64.1% % on GAIA and 62.0% % on Web WalkerQ.A
High-quality trajectory data crucial for effective SFT of agents. We propose two data construction strategies, re- sulting in the creation of datasets CRAW and ε2нQA. After applying trajectory rejection sampling to the QA data, we further perform filtering enhance data qual- ity. In Figure 3, we conduct ablation studies on the QwQ and evaluate the effectiveness the constructed datasets. In long-Coo CoT, hallucinations often arise when the model attempts to answer by simulating observations, primarily due to its exclusive reliance on internal reasoning mech- anisms. [5] Final performs better than under low-data regimes, emphasizing the value of robust filtering.
(130))
(4442)
(2455)
(17744
(8558)
Figure 3: Results on data efficiency us- ing GAIA benchmark. Open-only refers to using only challenging QA datasets from open- sources.
SFT for cold start is essential, as the agent tasks demand strong multi-step multi-tool instruction- following capabilities. We empirically investigate this by comparing performance under a single reinforcement learning setting using QwQ The results show that the Pass@3 performance is signifi- cantly limited, achieving only 5% % on the GAIA. For the RL phase, both Pass@3 and Cons@3 show consistent improvements the number training steps increases, as illustrated Figure 5a.
The thinking pattern knowledge used by strong reasoner models struggle transfer- able to those of small instruction models. As shown in Table reasoning models trained on trajectories synthesized by reasoning mod- els significantly enhance their reasoning perfor- mance [41]. For non-reasoning models, Long-
Table 3: Results CoT knowledge transfer. Inv. denotes invalid rate, R. refers to whether the model is reasoning model.
Model
R.
Pazuit3
Short-Cot Conwit
Fan.
PazxR3
Long-Cot Conae3
Frer
Owen2.5 7B
33.98
12:35
0.68%
35.92
21.00
21.36%
Owen2.5 32B
CoT also demonstrates good performance, but it introduces additional issues, such as higher invalid rate, often manifested as repeti iitiinn.o leading to exceeding the model's context length, particularly in smaller-scale models. These reasoning patterns do not easily transfer to instruction-tuned models, which are generally optimized for task- following behavior rather than deep reasoning. This observation aligns with the findings in [42, 43],
42.72
24.35
4.20%
45.63
30.00
13.59%
QwQ-328
28.33
39.66
13.27%

---

51-
Neater Langii
Condttto aoy Ky
Pass03
Pofomasee
P26003
40
00
50
Dem
2FF
Tergeekkre
(a) Performance across training steps
(b) Evolution of thought length and number of actions.
(c) Passë1 and Passê3 results dif- ferent temperatures.
using the DAPO algorithm.
Figure 5: Analysis on RI algorithm, emergent agency, and agent environments using GAIA bench- mark.
which highlight the brittleness cross-model reasoning knowledge transfer. As such, direct transfer of reasoning capabilities from reasoner models to instruction models remains a non-trivial challenge.
RL enables longer reasoning processes and supports more complex agentic action. As demon- strated by the results on Qwen-32B in Figure 5b, we observe that SFT leads to more frequent action generation and extended reasoning sequences, largely due to the nature of our training data (App. E.1). RL frameworks facilitate the emergence of more sophisticated reasoning strategies by allowing models optimize sequences of decisions, rather than single-step outputs. This enables models to learn from delayed rewards and engage in deeper exploration action spaces, leading to more coherent and longer reasoning trajectories. Moreover, RL encourages agentic behaviors where models autonomously decide intermediate steps, subgoals, or tools to achieve final objectives, as shown in App. F. Such capabilities are particularly useful in complex environments where straightforward task-following fails to generalize.
Web agent executes in dynamic, evolving environment that inherently resists stabilization. As shown in Figure 5c, adjusting the decoding temperature had minimal impact on final performance, indicating that decoding variability alone does not account for agent instability. Instead, attribute much of the performance fluctuation to changes in the web environment itself, highlighting the non-stationary and open-ended nature of real-world agent deployment. Unlike static datasets with fixed distributional properties, real-world environments evolve over time, requiring agents to remain robust under changing contexts and partial observability. Additionally, further investigate potential overfitting, we conduct memorization stress test: we fine-tuned a Qwen-7B model on 69 correctly sampled trajectories from the GAIA development set for 10 epochs, and subsequently evaluate its performance on the same set. Despite this, greedy decoding only achieved 37.4% %, suggesting the difficulty of stabilization on the open-domained agentic tasks.
Related Works
Information Seeking Agents and Benchmarks. Recent advances in information-seeking agents aim to integrate web interaction into LLMs' reasoning. [44] WebThinker [4] and Search-ol [5] use tool-augmented LLMs that actively retrieve evidence mid-inference. Some works like R1- Searcher [7], ReSearch [6] and Search-R1 [8] focus on reinforcement learning to teach search behavior from outcome- based rewards. DeepResearcher [10] extends this by operating in real web environments with online RI, while SimpleDeepSSeacheer [9] shows that a small number of distilled demonstrations can train effective agents without full RL. These works demonstrate promising capabilities but often rely on limited or simplistic data. In parallel, benchmarks like GAIA [12] and WebWalkerQA [3] test reasoning and browsing, but many are single-turn domain-limited BrowseComp [13] and BrowseComp- zh [20] increase task complexity, requiring multi-hop search and multilingual reasoning, yet still lack diversity and scalability. Our work addresses these gaps by proposing automatic synthesis QA datasets designed to challenge agents across domains and task types in more realistic web environments.
WWe also experiment with mixing short-CoTs and long- CoTs, but observe significant performance improvements.

---

Agents Learning. Agent learning has evolved from in-context learning towards training-bass based methods [45-47]. Recent studies [48-50] have primarily focused on leveraging SFT with curated tolving trajectories following the ReAct paradigm. However, empirical evidence suggests that pure SFT-based agents often exhibit limited generalization performance when confronted with adaptive operational contexts [10, 51-53]. Building upon these limitations, RL-based methods [7, 10, 10, 51, 54] have demonstrated remarkable potential in developing sophisticated search strategies through learned exploration policies. Despite their theoretical advantages. practical implementations face persistent challenges in training stability and sample efficiency. WebDancer implements a two- stage framework: an initial cold-start phase employing trajectory-basee SFT to establish fundamental agency patterns, followed by targeted RL to cultivate adaptive long-term neggttemm& agency capabilities.
Conclusion
In this work, we propose a systematic framework for building end-to-end multi-step information- seeking web agents from scratch. By introducing scalable QA data synthesis methods and a two- stage training pipeline combining SFT and on-policy RL, our WebDancer agent achieves strong performance on GAIA and WebWalkerQA. These findings underscore the significance of our proposed training strategy and provide valuable insights into the critical aspects of agent training. Moving forward, this research offers actionable and systematic pathways for the community to advance the development of increasingly sophisticated agentic models capable of tackling complex real-world information-sceking tasks.
References
[1] OpenAL Deep research system card, 2025.
URL httpp://cdn. com//
deep- heaperreeerrhyyyeeeeeeee card. pdf.
[2] x.ai. Grok beta the age of reasoning agents, 2025. URL
[3] Jialong Wu, Wenbiao Yin, Yong Jiang, Zhenglin Wang, Zekun Xi. Runnan Fang, Deyu Zhou, Pengjun Xie, and Fei Huang. Webwalker: Benchmarking ilms in web traversal, 2025. htRpa:/tta///aiiii org/ abs/ 1501.07572.
[4] WebThinker Team. Webthinker: Empowering large reasoning models with deep research capability, 2025. URL https://github. cttcchu..commRR/ www//UC--/L....cc SLLPIRR/ebbhinkee Github.
[5] Xiaoxi Li, Guanting Dong. Jiajie Jin, Yuyao Zhang, Yujia Zhou, Yutao Zhu, Peitian Zhang, and Zhicheng Dou. Search-ol: Agentic scarch-enhanced large reasoning models. arXiv preprint arXiv:2501 .053666, 2025.
[6] Mingyang Chen, Tianpeng Li, Haoze Sun, Yijie Zhou, Chenzheng Zhu, Fan Yang, Zenan Zhou, Weipeng Chen, Haofen Wang. Jeff Z Pan. et al. Learning to reason with search for llms via reinforcement learning, arXiv preprint arXiv:2503.1 19470, 2025.
[7] Huatong Song, Jinhao Jiang, Yingqian Min, Jie Chen, Zhipeng Chen, Wayne Xin Zhao, Lei Fang, and Ji-Rong Wen. R1-searcher: Incentivizing the search capability in lmms via reinforcement learning. arXiv preprint arXiv:2503.00 03555....5........ 2025.
[8] Bowen Jin, Hansi Zeng, Zhenrui Yue, Dong Wang, Hamed Zamani, and Jiawei Han. Search-rl: Training Ilms to reason and leverage search engines with reinforcement learning. arXiv preprint 2rXiv:2503.095/1 2025.
[9] Shuang Sun, Huatong Song, Yuhao Wang, Ruiyang Ren, Jinhao Jiang, Junjic Zhang, Lei Fang, Zhongyuan Wang, Wayne Xin Zhao, and Ji-Rong Wen. Simpledeepscarcher: Deep information seeking via web-powered reasoning trajectory synthesis. 2025. URL https:://tttthiee cwm/RCCa Btt///RAAAAAAOOOAAS BWWCCA/BB//immmeeeeeeeeeraa
[10] Yuxiang Zheng, Dayuan Fu, Xiangkun Hu, Xiaojic Cai, Lyumanshan Ye, Pengrui Lu, and Pengfei Liu. Deeprescarcher: Scaling deep rescarch via reinforeement lcarning in real-world environments, 2025. URL https://arxiv. org/ 220. 0211//00000.0......
[11] Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. In International Conference on Learning Representations (ICLR), 2023.
[12] Grégoire Mialon, Clémentine Fourrier, Thomas Wolf, Yann LeCun, and Thomas Scialom. Gaia: a benchmark for general ai assistants. In The Twelfth International Conference on Learning Representations. 2023.
10

---

[13] Jason Wei. Zhiqing Sun, Spencer Papay. Scott McKinney, Jeffrey Han, Isa Fulford, Hyung Won Chung. Alex Tachard Passos, William Fedus, and Ame lia Glaese. Browsecomp: A simple yet challenging benchmark for browsing agents.
[14] Runnan Fang, Xiaobin Wang, Yuan Liang, Shuofei Qiao, Jialong Wu, Zekun Xi, Ningyu Zhang, Yong Jiang, Pengjun Xie, Fei Huang, et al. Synworld: Virtual scenario synthesis for agentic action knowledge refinement. arXiv preprint arXiv:2504.03561 2025.
[15] Yuxin Zuo, Kaiyan Zhang. Shang Qu, Li Sheng, Xuckai Zhu, Biqing Qi. Youbang Sun, Ganqu Cui, Ning Ding, and Bowen Zhou. Ttrl: Test-time tist-time reinforcement learning, 2025. URL https://arxiv: org/ abs/2504. 16084.
[16] Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, al. Dapo: An open-source IIm reinforeement learning system scalc. arXiv preprint arXiv:2503. /4476, 2025.
[17] Tongyi DeepResearch Team. Tongyi deepresearch: new era of open-source researchers. https: //github www.thhu.ooeeee.eee wttp//AAaaaaaNa//eeaaaaae
NLP WLL//eepResearch, 2025.
[18] Priyanka Sen, Alham Fikri Aji, and Amir Saffari. Mintaka: A complex, natural, and multilingual dataset for end-to-end question answering. In Nicoletta Calzolari, Chu-Ren Huang, Hansaem Kim, James Pustejovsky, Leo Wanner, Key-Sun Choi, Pum-Mo Ryu, Hsin-Hsi Chen, Lucia Donatelli, Heng Ji, Sadao Kurohashi, Patrizia Paggio, Nianwen Xue, Seokhwan Kim, Younggyun Hahm, Zhong He, Tony Kyungil Lee, Enrico Santus, Francis Bond, and Seung-Hoon Na, editors, Proceedings of the 29th International Conference on Computaatonal Linguistics, pages 1604- 1619, Gyeongju. Republic of Korea, October 2022. International Committee on Computational Linguistics. URL https: aa//a/clllmml http///aaaaaatloooco/o org/2022. cwwiiooo.nn100 138//
[19] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33: 13:1177-1901, 2020.
[20] Peilin Zhou, Bruce Leon, Xiang Ying, Can Zhang, Yifan Shao, Qichen Ye, Dading Chong, Zhiling Jin, Chenxuan Xie, Meng Cao, al. Browsccomp-2h: Benchmarking web browsing ability of large language models in chinese. arXiv preprint arXiv:2504. 1931 2025.
[21] OpenAL Introducing simpleqa, 2025. URL
com/ http://ppwa..ccooo//////////dodd htt.aa/rrrouccciinnsssleeeeeeeee simpleqa/.
[22] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Fid Chi, Quoc Le, Denny Zhou, al. Chain-of-thought prompting elicits reasoning in large language models. Advances neural information processing systems, 35:24824 24837, 2022.
[23] Yuxiang Zhang, Yuqi Yang, Jiangming Shu, Xinyan Wen, and Jitao Sang. Agent models: Internalizing chain-of-action generation into reasoning models. arXiv preprint arXiv:2503.0655 2025.
[24] Xinji Mai, Haotian Xu, Xing W. Weinong Wang. Yingying Zhang, and Wenqiang Zhang. Agent scaling law: Agent rl with spontaneous code execution for mathematical problem solving, 2025.
[25] QwQ Team. Qwq-32b: Embracing the power of reinforcement learning, 2025. URL https:///uenlll github. ho/bllo/////////////o wllog//gg-3// http//qq33222b 32b/.
[26] Daya Guo, Dejian Yang, Haowci Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseck-rl: Incentivizing reasoning capability in Ilms via reinforcement learning. arXiv preprint arXiv:2501. 12948, 2025.
[27] Siwei Wu, Zhongyuan Peng. Xinrun Du Tuney Zheng. Minghao Liu, Jialong Wu, Jiachen Ma. Yizhi Li. Jian Yang. Wangchunshu Zhou, Qunshu Lin. Junbo Zhao, Zhaoxiang Zhang, Wenhao Huang, Ge Zhang, Chenghua Lin, and J. H. Liu. comparative study on reasoning patterns of openai's ol model, 2024. URL https://arxix www.aawwwwwa..aaaaaaeeeeeeeo aaaaaba/2444 /2100. 13639.
[28] Long Phan, Alice Gatti, Ziwen Han, Nathaniel Li, Josephina Hu, Hugh Zhang, Chen Bo Calvin Zhang, Mohamed Shaaban, John Ling, Sean Shi, et al. Humanity's 's last exam, arXiv preprint arXiv:2501. 14249, 2025.
[29] Baian Chen, Chang Shu, Ehsan Shareghi, Nigel Collier, Karthik Narasimhan, and Shunyu Yao. Fireact: Toward language agent fine-tuning, arXiv preprint ar 2023.
[30] Chong Zhang, Yue Deng, Xiang Lin, Bin Wang. Dianwen Ng. Hai Ye, Xingxuan Li, Yao Xiao, Zhanfeng Mo. Qi Zhang, et al. 100 days after deepseek-rl: A survey on replication studies and more directions for reasoning language models. arXiv preprint 20555 2025.

---

[31] Komal Kumar, Tajamul Ashraf, Omkar Thawakar, Rao Muhammad Anwer. Hisham Cholakkal, Mubarak Shah, Ming-Hsuan Yang. Phillip HIS Torr Fahad Shahbaz Khan, and Salman Khan. Lim post-training: A deep dive into reasoning large language models. arXiv preprint arXiv:2502.21322. 2025.
[32] Mohamed Amine Ferrag, Norbert Tihanyi, and Merouane Debbah. Reasoning beyond limits: Advances and open problems for llms. arXiv preprint arXiv:2503.22733 2025.
[33] Lianmin Zheng. Wei-Lin Chiang. Ying Sheng. Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing. et al. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in Neural Information Processing Systems, 36:46595 46623, 2023.
[34] ByteDance Sced, Yufeng Yuan, Yu Yue, Mingxuan Wang, Xiaochen Zuo, Jiaze Chen, Lin Yan, Wenyuan Xu, Chi Zhang, Xin Liu, al. Sced-thinking-vl. 5: Advancing superb reasoning models with reinforcement learning. arXiv preprint ar Xiv::2504. 13914, 2025.
[35] Wenyuan Xu, Xiaochen Zuo, Chao Xin, Yu Yue, Lin Yan, and Yonghui Wu. A unified pairwise framework for rlhf: Bridging generative reward modeling and policy optimization. arXiv preprint arXiv:2504.0 04950, 2025.
[36] Zijun Liu, Peiyi Wang, Runxin Xu, Shirong Ma, Chong Ruan, Peng Li, Yang Liu, and Yu Wu. Inference- time scaling for generalist reward modeling. arXiv preprint arXiv:2504.0244: 2025.
[37] Yang Yuc, Zhiqi Chen, Rui Lu, Andrew Zhao, Zhaokai Wang, Yang Yuc, Shiji Song, and Gao Huang. Daes reinforcement learning really incentivize reasoning capacity in Ilms beyond the base model?, 2025. URL https://arxis: org/ aaaaaaa//aa 25044.54..1 13837.
[38] Gokul Swamy, Sanjiban Choudhury, Wen Sun, Zhiwei Steven Wu, and J. Andrew Bagnell. All roads lead likelihood: The value of reinforcement learning in fine-tuning, 2025. URL 01067.
org/ wwtp://////aaaa/eeeeeeee abs/2503.
[39] Lang Feng, Zhenghai Xue, Tingcong Liu, and Bo An. Group-in-group policy optimization for llm agent training, 2025. URL. httts:///aaiiiiii wrg/abs/250.. httppe///axxxv..orr/aaaa////
10978.
[40] Zhepei Wei, Wenlin Yao, Yao Liu, Weizhi Zhang, Qin Lu, Liang Qiu, Changlong Yu, Puyang Xu, Chao Zhang, Bing Yin, Hyokun Yun, and Lihong Li. Webagent-rl: rl: Training web agents via end-to-end multi-tun reinforcement learning, 2025. URL https: //arxiv org/ abs/ abs/2505... 16421.
[41] Zhibin Gou, Zhihong Shao, Ycyun Gong, Yelong Shen, Yujiu Yang, Minlie Huang. Nan Duan, and Weizhu Chen, Tora: A tool-integrated reasoning agent for mathematical problem solving. ArXiv, 174522 2023. http::// semanticschollla. org/ CCrrppaIID 263310365.
[42] Yuetai Li, Xiang Yue, Zhangchen Xu, Fengqing Jiang. Luyao Niu, Bill Yuchen Lin, Bhaskar Rama- subramanian, and Radha Poovendran. Small models struggle to learn from strong reasoners. arXiv preprint 2025.
[43] Huifeng Yin, Yu Zhao, Minghao Wu, Xuanfan Ni, Bo Zeng, Hao Wang. Tianqi Shi, Liangyingg Shao, Chemyang Lyu, Longyue Wang, Weihua Luo, and Kaifu Zhang. Towards widening the distillation bottleneck for reasoning models, 2025.
[44] Zekun Xi, Wenbiao Yin, Jizhan Fang, Jialong Wu, Runnan Fang. Ningyu Zhang, Jiang Yong, Pengjun Xie, Fei Huang, and Huajun Chen. Omnithink: Expanding knowledge boundaries in machine writing through thinking. arXiv preprint arXiv:2501.0972 2025.
[45] Bang Liu, Xinfeng Li, Jiayi Zhang, Jinlin Wang, Tanjin He, Sirui Hong, Hongzhang Liu, Shaokun Zhang, Kaitao Song, Kunlun Zhu, et al. Advances and challenges in foundation agents: From brain brain-inspired intelligence evolutionary, collaborative, and safe systems. arXiv preprint arXiv:2504.01 01940,099,,..0 2025.
[46] Wangchunshu Zhou Yixin Ou, Shengwei Ding, Long Li, Jialong Wu, Tiannan Wang. Jiamin Chen, Shuai Wang, Xiaohua Xu, Ningyu Zhang, Huajun Chen, and Yuchen Eleanor Jiang. Symbolic learning enables self-evolving agents. 2024. URL https: / /axxvv.vivvv..c orgg/v.orrr/aeeo. abs/ /206. 18532.
[47] Wangchunshu Zhou, Yuchen Eleanor Jiang, Long Li. Jialong Wu, Tiannan Wang. Shi Qiu, Jintian Zhang. Jing Chen, Ruipu Wu, Shuai Wang, Shiding Zhu, Jiyu Chen, Wentao Zhang, Xiangru Tang, Ningyu Zhang, Huajun Chen, Peng Cui, and Mrinmaya Sachan. Agents: An open-source framework for autonomous language agents. 2023. URL https: // arxiv.
org/ wwwpiiiww...o..oe.ooo..oe.oooeee
[48] Shuofei Qiao, Ningyu Zhang. Runnan Fang. Yujie Luo, Wangchunshu Zhou, Yuchen Jiang, Chengfei Lv, and Huajun Chen. Autoact: Automatic agent learning from scratch for qa via self plannilan..n. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages
3003 3021, 2024.
12

---

[49] Aohan Zeng. Mingdao Liu, Rui Lu, Bowen Wang, Xiao Liu, Yuxiao Dong, and Jie Tang. Agenttuning: Enabling generalized agent abilities for Ilms. In Findings of the Association for Computational Linguistics ACL 2024, pages 3053-3077, 2024.
[50] Zehui Chen, Kuikun Liu, Qiuchen Wang. Wenwei Zhang, Jiangning Liu, Dahua Lin, Kai Chen, and Feng Zhao, Agent-flan: Designing data and methods effective agent tuning for large language models. Findings of the Association for Computational Linguistics ACL 2024, pages 9354-9366, 2024.
[SI] Shaokun Zhang, Yi Dong, Jieyu Zhang, Jan Kautz, Bryan Catanzaro, Andrew Tao, Qingyun Wu, Zhiding Yu, and Guilin Liu. Nemooroo-reeseach--tooe preprint 4rX0v:2505.000244 2025.
Tool-using language models with reinforced reasoning. arXiv
[52] Cheng Qian, Emre Can Acikgoz, Qi Hc, Hongru Wang, Xiusi Chen, Dilck Hakkani- Tür, Gokhan Tur, and Heng Ji. Toolrl: Reward is all tool learning needs, arXiv preprint arXiv:2504.1399 2025.
[53] Yuanqing Yu, Zhefan Wang, Weizhi Ma, Zhicheng Guo, Jingtao Zhan, Shuai Wang, Chuhan Wu, Zhiqiang Guo, and Min Zhang. Steptool: A step-grained reinforcement learning framework for tool learning in Ilms. arXiv preprint 10000224,, 2024.
[54] Dingehu Zhang, Yida Zhao, Jialong Wu, Baixuan Li, Wenbiao Yin, Liwen Zhang, Yong Jiang. Yufeng Li, Kewei Tu, Pengjun Xie, and Fei Huang. Evolvescarch: An iterative self-evolving search agent, 2025, URL attps://arxiv/ http///iiii.oo.aa/////// abs/ / 2505. 22501.
[55] Yuxiang Wei, Olivier Duchenne, Jade Copet. Quentin Carbonneaux, Lingming Zhang. Daniel Fried, Gabriel Synnaeve, Rishabh Singh, and Sida I. Wang. Swe-rl: Advancing Ilm reasoning via reinforcement learning on open software evolution, 2025.
[56] Shiyi Cao, Sumanth Hegde, Dacheng Li, Tyler Griggs, Shu Liu, Eric Tang, Jiayi Pan, Xingyao Wang, Akshay Malik, Graham Neubig. Kourosh Hakhamaneshi, Richard Liaw, Philipp Moritz. Matei Zaharia, Joseph E, Gonzalez, and Ion Stoica. Skyrl-vO: Train real-world mong-oorizon agents via reinforcement learning. 2025.
[57] Chaoyun Zhang, Shilin He, Liqun Li, Si Qin, Yu Kang, Qingwei Lin, and Dongmei Zhang, Api agents vs. gui agents: Divergence and convergence. arXiv preprint arXiv:2503. 11069, 2025.
[58] Yijia Shao, Yucheng Jiang, Theodore A Kanell, Peter Xu, Omar Khattab, and Monica S Lam. Assisting in writing wikipedia-like articles from scratch with large language models. In NAACI-HI.T, 2024.
[59] An Yang, Anfeng Li. Baosong Yang. Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang. Chenxu Lv, Chujie Zheng. Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge. Haoran Wei, Huan Lin, Jialong Tang, Jian Yang. Jianhong Tu, Jianwei Zhang, Jianxin Yang. Jiaxi Yang. Jing Zhou, Jingren Zhou, Junyang Lin, Kai Dang. Keqin Bao, Kexin Yang. Le Yu, Lianghao Deng, Mei Li, Mingfeng Xue, Mingze Li, Pei Zhang. Peng Wang, Qin Zhu, Rui Men, Ruize Gao, Shixuan Liu, Shuang Luo, Tianhao Li, Tianyi Tang, Wenbaao Yin, Xingzhang Ren, Xinyu Wang. Xinyu Zhang, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yinger Zhang. Yu Wan, Yuqiong Liu, Zekun Wang. Zeyu Cui, Zhenru Zhang. Zhipeng Zhou, and Zihan Qiu. Qwen3 technical report. arXiv preprint arXiv:2505.00 09388, 2025.
[60] Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36:53728 53741, 2023.
[61] Mengkang Hu, Yuhang Zhou, Wendong Fan, Yuzhou Nic, Bowei Xia, Tao Sun, Ziyu Ye, Zhaoxuan Jin, Yingru Li, Zeyu Zhang, Yifeng Wang, Qianshuo Ye, Ping Luo, and Guohao Li. Owl: Optimized workforce learning for general multi-agent assistance in real-world task automation, 2025. URL https: / g/g github.cc. com/ camel-at aa/owll owl.
[62] Guohao Li, Hasan Ahed Al Kader Hammoud, Hani Itani, Dmitrii Khizballin, and Bemard Ghanem. Camel: Communicative agents for "mind" exploration of large language model society. In Thirty-seventh Conference on Neural Information Processing Systems, 2023.
[63] Xinbin Liang, Jinyu Xiang, Zhaoyang Yu, Jiayi Zhang, Sirui Hong, Sheng Fan, and Xiao Tang. Openmanus: An open-source framework for building general ai agents, 2025. URL hhttp:::////ot org/10 10. 5281/zemode 15186407.
[64] anthropic. Meet claude, 2025. URL https: http:///wwwaaataeee www.aapic...ccom//// cttp:/pppp.ccoclaaaaaaaaa claude.
[65] He Zhu, Tianrui Qin, King Zhu, Heyuan Huang, Yeyi Guan, Jinxiang Xia, Yi Yao. Hanhao Li, Ningning Wang, Pai Liu, et al. Oagents: An empirical study of building effective agents, arXiv preprint 250. 2025.
13

---

[66] Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. Musique: Multihop questions via single-hop question composition, 2022. URL https://arxiv wwteziiv.orggaaae httg//bb/2108..... 00573.
[67] Ofir Press, Muru Zhang. Sewon Min, Ludwig Schmidt, Noah A. Smith, and Mike Lewis. Measuring and narrowing the compositionality gap language models, 2022.
[68] Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, and Hannaneh Hajishirzi. When to trust language models: Investigating effectiveness of parametric and non-parametric memories, 2022.
[69] Xanh Ho, Anh-Khoa Duong Nguyen, Saku Sugawara, and Akiko Aizawa. Constructing multi-hop qa dataset for comprehensive evaluation of reasoning steps, 2020. URL https: :ttp:p::///riiiii org/abs////a11aaooa /t1.///..000000000.....
[70] Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W. Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. Hotpotqa: A dataset for diverse, explainable multi-hop question answering, 2018. URL htttt://aaxiiv org/ abs/aaa//// 1809. 09600.
[71] An Yang, Baosong Yang. Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115. 2024.
[72] OpenAL. Gpt-4 system card, 2022. URL https: //tp///////////eee http:///cc...o.icc//////////aa.t wwt.///o//apcceeeccc http://ppaaaaaaeaa wttp://ggtt----yteeeeeeee card. pdf.
[73] Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin
Lin, and Chuan Wu. Hybridflow: flexible and efficient rihf framework. arXiv preprint arXiv: 2409. 19256, 2024.
[74] Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principlex, 2023.
14

---

NeurIPS Paper Checklist
The checklist is designed encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No], or [NA].
[NA] means either that the question Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justification right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will published with the paper.
The reviewers of your paper will be asked use the checklist as one of the factors their evaluation. While "[Yes] is generally preferable to "[No]" ", it is perfectly acceptable to answer "[No] providedd proper justification is given (e.g.. "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering binary way, we
"[No] or "[NA]" is not grounds for rejection. While the questions are phrased acknowledge that the true answer is often more nuanced, please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS paper checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Abstract and Sec.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: App.
15

---

Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, c.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification:
This work does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: Sec. 4 and App. E.3
Guidelines:
The answer NA means that the paper does not include experiments.
16

---

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g.. with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors welcome describe the particular way they provide for reproducibilityy In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: Sec. 4 and App. E.3, and we will provide samples of datasets in the supple- mental material. The full code will be released upon acceptance of the paper.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: /ttpsscccppbblcc////eeccppcpppcp wwwubblccguuideee guwdee/ /CodeSubmisi ssionPolicy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
17

---

Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification: App. E.3
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented in the core the paper to a level detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: We adopt pass@3 and consi stency33 metries evaluate the stability and statistical significance of agent performance, which we find more appropriate than error bars for our task setting.
Guidelines:
The answer NA means that the paper does include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of exccution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: App E.3.
Guidelines:
The answer NA means that the paper does not include experiments.
18

---

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethies
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics http::/nerrrss.cccubll////// ctw.e/tithhcieeelnneeeneeaceee ines?
Answer: [Yes]
Justification: The research conducted in the paper conforms with the NeurIPS Code of Ethics in every respect.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (c.g.. there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
App. B
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied partieular applications, let alone deployments. However, if there is direct path any negative applications, the authors should point it out. For example, it is legitimate point that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (c.g., pretrained language models, image generators, scraped datasets)?
Question:
Answer: [Yes]
19

---

Justification: App. B
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model. for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging. and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: All assets used in the paper are credited and the license and terms of use are properly respected.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re- re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/mooel as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
20

---

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowd-sourcing or research with human subjects.
Guidelines:
The NA answer means that the article does not involve crowd-sourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
Justification: The paper does not involve crowd-sourcing or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
21

---

A Limitations
Although our proposed framework has demonstrated promising results, several limitations remain, which point to ongoing efforts and potential directions for future work.
Tool Number and Type Currently, we integrate only two basic informatiinnseekinng tools. To enable more advanced and fine- iinegrrined retrieval capabilities, plan incorporate such as browser modeling by abstracting browser functionalitics into modular tools, and a Python sandbox environment for interacting with external APIs [55-57]. This allows the agent to perform more human-lin like and efficient interactions, paving the way not only for tackling more challenging benchmarks but also for progressing toward more general and autonomous agency.
sophisticated tools,
Task Generalization and Benchmarks Our current experiments focus on two short-answer ioormattion-seeinng tasks. However, comprehensive web agent should also be capable document- level research and generation [58]. Extending to such open-domain, long-form writing poses signifi- cant challenges in reward modeling agentic tasks, which we are actively investigating, particularly how to design more reliable and informative reward signals for long-form generation in open-ended settings [36].
Data Utilization While we have accumulated large corpus of QA pairs and corresponding trajectories, effectively scaling learning remains a challenge, particularly in the RL stage, where only small subset (e.g., 5,000 pairs) can be utilized due to computational and stability constraints of RL in agentic tasks. This underscores the need for more efficient data utilization strategies to fully exploit the richness of the collected dataset.
High Rollout Cost The RL phase incurs substantial computational and time overhead, as each rollout involves multiple rounds of tool invocations and LLM completions. This high cost not only limits scalability but also slows down iterative development and experimentation. A promising direction is to develop more efficient mechanisms for integrating tool calls with model completions, which can reduce rollout time and cost without sacrificing learning policy.
Hybrid Thinking We consider two types of datasets characterized by short and long CoTs. Cur- rently, our models are trained on single dataset type. In future work, we plan to develop hybrid reasoning agent model capable dynamically controlling the reasoning length of the agent. [59]
Thinking Pattern In tool invocation, hallucinations may occur. For example, when dealing with mathematical problems, one might crroneously invoke a "calculate" tool that does not actually exist. Additionally, over-action may arise during the reasoning process, where redundant actions performed even after the answer has been confirmed.
B Broader Impacts
Building open-source, autonomous web agents capable of long-term information seeking has the potential greatly benefit scientific research, education, and productivity by democratizing access to complex web-based reasoning tools. However, such systems also raise concerns, including the risk of misinformation propagation if agents rely unreliable sources, and the possibility of misuse in automated content extraction or surveillance, emphasize the importance of transparency, source attribution, and responsible deployment practices to mitigate potential harms.
Discussions
C.1
Concurrent Work
Comparison with the Training-based Methods We primarily compare our approach with two training-based methods: WebThinker and SimpleDeepSearcher, highlighting the key differences. WebThinker also adopts an SFT followed by RL. setu,, but employs an off-policy RI. algorithm [60]. Furthermore, WebThinker triggers actions and observations within the <thinking_conneet whereas
22

---

our approach adopts a native ReAct style architecture, executing each action after completing its corresponding reasoning step. In contrast. Simple DeepSearcher relies solely on supervised fine- tuning over carefully curated dataset. Our approach similarly follows an SFT-then-RI. paradigm, but crucially leverages on-policy RI. via DAPO. Our core contribution lies in building scalable end-to-end pipeline, from data construction to algorithmic design, that supports native ReAct reasoning. This framework is compatible with both instruction LLMs and LRMs, enabling seamless integration and improved generalization.
Comparison with the Prompting-basse Methods Recent efforts in the community have ex- plored building more autonomous and general- agent systems, such OWL [61, 62], and OpenManus [63], by leveraging foundation models with strong native agentic capabilities, such as Claude [64]. These systems typically rely carefully engineered agent frameworks and prompting workflows, often involving multi-step tool usage and human-curated task structures. In contrast, we advocate for open-source models with emergent ageney, crucial for democratizing agentic Al and advancing fundamental understanding of how agency can arise and scale in open systems. Our native RAct framework embraces simplicity, embodying the principle that less is more. Training native agentic models is fundamentally valuable.
C.2 Post-train Agentic Models
Agentic models refer foundation models that natively support reasoning, decision-make making, and multi-step tool use in interactive environments. They exhibit emergent capabilities such as planning, self-reflection, and action execution through structured prompting alone. Recent systems like DeepSearch and Deep Research illustrate how powerful foundation models can serve as agentic cores, enabling autonomous web interaction through native support for tool invocation and iterative reasoning. However, since web environments are inherently dynamic and partially observable, reinforcement learning plays crucial role in improving the agent's adaptability and robustness. In this work, we aim to elicit autonomous agency in open-source models through targeted post- training.
C.3 Action Space
In our current design, the agent operates with "search" and "visit" actions, which are considered fundamental primitives in the information-seeking process [4, 65]. In principle, these two actions are sufficient to access and retrieve any information available on the web. Moreover, our framework is designed to be modular and extensible. It supports seamless integration with both browser modeling (e.g., scrolling, form filling) and Python sandbox environments, enabling more complex interactions when needed. Given the challenges of sample efficiency in RL settings, we chose focus on "search" and "visit" as a strong starting point. These tools already demonstrate substantial capabilities across our benchmark tasks.
C.4 Reward Reliability
In our ctrly-stage experiments, we extensively compared several commonly used reward designs in the search-agent setting, including recall, F1, and model-based rewards. Our findings motivated the use of LLM-as-Judge. Specifically, both recall- and F1 Fi-based rewards suffered from reward hacking:
For recall, the model often learned to include large numbers of candidate answers in order to boost recall, leading to verbose and unreliable outputs.
For F1, the model tended output only partial answers that scored well but lacked com- pleteness and readability.
As an alternative, we adopt a model-based reward using LLM-as-Judge. Our prompts are adapted from established benchmarks (e.g., HLE [28], BrowseComp [13]). To evaluate the robustness of this judge, we tested two strong LLMs: Qwen2.5-72B (open-source) and GPT-4o (closed-source). The results demonstrated high consistency between the two. Furthermore, we manually audited 100 samples judged by Qwen2.5-72B and found only one judge error, suggesting strong reliability in practice. While a full quantitative evaluation of judge accuracy remains interesting direction for
23

---

future work, our empirical evidence indicates that the LLM-as-Judge is significantly more aligned with the QA task and more robust than standard metric-based rewards in this setting.
C.5 Task Generalization
Our current work focuses short-answer QA tasks, primarily because they offer well-defined evaluation protocols and allow reward computation to be more reliable, contrast, long-form QA tasks pose significant challenges in both RFT data construction and RL training due difficulties in evaluation. Interestingly, we observe that our model, trained on short QA tasks, demonstrates strong generalization to long-form settings in terms of information--sekiigg behavior. This suggests that the model has learned effective strategies for decomposing complex queries and locating relevant evidence, which naturally transfers to long-form tasks. We will include illustrate this generalization behavior in the revised version. However, long-form QA involves not only information retrieval but also generation quality, which remains an open challenge. We plan to explore this direction in future work.
study to
D Training Dataset
We collect 40K samples of E2HQA and 60K sam- ples of CRAWI.QA. These data samples are used to generate trajectories via either QwQ or GPT-4o, followed by a multi-stage filtering process ensure quality, as described in Sec. 2.2. Table 4 separately reports the statisties for SFT data generated using Long-CoT and Short-CoT reasoning. We plan to scale this high-quality dataset further to investigate whether increasing the data volume leads to signif- icant performance gains in future work.
Table 4: Statistics of training datasets. The thinking length is the average of the tokenized length of the thoughts.
CoT Type
Num.
Action Count
Thinking Length
Short Long
7.678
4.56
$10.03
6,550
2.31
1599,39
Filtering Criterion: Regarding the trajectory filter employed in Sec. 2.2, it is important to note that, during the quality assessment phase, we mitigate the presence of repetitive patterns by identifying and constraining the maximum occurrence of n-grams (z=10) within each trajectory to a threshold of 4. The purpose of this to prevent the model from internalizing detrimental patterns, thereby safeguarding the integrity of the inference process.
Open-only Datasets: We select set of widely-used QA datasets, including MuSiQue [66], Bam- boogle [67], PopQA [68], 2Wiki [69], and HotpotQA [70]. To ensure question difficulty, we apply a simple RAG-based filtering process remove easy questions.
E
Experimental Details
E.1
Benchmarks
GAIA is designed to evaluate general AI assistants on complex information retrieval tasks, while WebWalkerQA focuses specifically deep web information retrieval. Our experiments 103 questions from GAIA's text-only validation split and 680 questions from the WebWalkerQA test set.
E.2 Baselines
We compare WebDancer against the following frameworks:
No Agency: which denotes direct use base ability of models and simply uses retrieval- augmented generation (RAG). Includes 4wwnn25577527728-nnnsrrr [71], QwQ-32B [25], DeepSeek-R116711 [26], GPT-4o [72].
Close-Sourced Agentic Frumeworks: OpenAl Deep Research (DR) use end-to-end reinforce- ment learning to complete multitask research tasks.
Open-Sourced Agentic Frameworks: WehThinker equips an LRM with a Deep Web Ex- plorer to autonomously search and browse web pages nei--reasnning. interleaving tool use
24

---

with chain-of of thought. For a fair comparison, we reproduced the results using Google Search and further replicated both the Base and RL versions of the method. Search-ol [5] performs information-seehte and then using an L.M to answer based on the retrieved content, without optimizing the search process itself. R1-Searcher [7] trains LLM to learn when and how to search using outcome-based reinforcement learning, without any supervised demonstrations.
by first generating search queries, retrieving web documents,
E.3 Implements Details
We train using the multi-turn chatml format, structuring each dialogue such that tool responses are represented as user messages, and both thoughts and actions generated by the model are represented as assistant messages.
Dataset Construction:
The number of reject samplling N = 5. The summarizer model Mg is Owen-2.5-72B3 We build our system using the widely adopted ReAct framework, implemented on top of the Qwen-Agents 5.
Training and Inference: We construct the judge model Mj based on ---nn-22--Instu and design the reward prompt following [28]. For RL, we implement verl [73, 74] to support the RL. algorithm and rollouts. The rollout number in RL 16. We set the inference parameters as follows: temperature = 0.6, top, = 0.95. For the LRM, we use repetition penalty of 1.1, while for the LLM, the repetition penalty is 1.0. the RL, the temperature of rollout is 1.0 and lopp = 1.0.
We conduct all experiments using 32 nodes with NVIDIA H20 (96GB).
E,4
Prompts for Agent Trajectories Sampling
Traditional ReAct for LLMs
Prompts for ReAct
Answer the following questions best you can.
Use the following format:
Question: the input question you must answer
Thought: you should always think about what to do
Action: the action take, should be one of [tool names]]
Action Input: the input to the action, use ISON Schema with explicit parameters Observation: the result of the action (this Brigght/Actoo/Acctii
Input/Observation can be repeated many times)
Thought: you should always think about what to do
Action: Final Answer: the final answer to the original input question
## Execution Framework
Thinking phase
Mandatory components:
(a). Evidence chain completeness assessment
(b). Tool selection rationale
2. Action Phase
Allowed tools: Only use tools listed in "(tool_descs'' or can be Final Answer, which returns the answer and finishes the task.
You may only provide the 'Final Answer' when you can confidently confirm the answer.
You must also ensure that the 'Final Answer' is accurate and reliable.
To output the Final Answer, use the following template: Final Answer: [YOUR Final Answer]
25

---

3. Observation phase
Return information from the tool: The result of the action, you can use the result to think about the next step. You have access to the following tools:
{tool_desces
Begin!
You are likely to use the given tools to gather information and then make the final answer.
Solve the following question using interleaving thought, action, and observation steps. You may take many steps as necessary.
Question: {query]
Figure 6: Prompts for ReAct using LLMs.
Modified ReAct for LRMs
Case Trajectory in GAIA
Answer the following questions best you can.
Allowed tools: Only use tools listed in cooo_desss]' or can be Final Answer: You must also ensure that the Final Answer is accurate and reliable.
You have access to following tools:
[tool_desce
Begin!
Output Format:
Action: the action take, should be of [[tool_ samooo__amss]] Action Input: the input to the action, use JSON Schema with explicit parameters, when the action is 'Final Answer', do not have Action Input, directly return the answer You may take as many steps as necessary. Always use the tools to gather information before making final answer.
When you want to make tool call, please output complete "Action: and "Action Input: to make the tool call successful and then output "Observation: make the tool call successful. Question: {query]
Figure 7: Prompts for ReAct using LRMs.
F
Case Study
As shown in Figure 8, several sophisticated thinking patterns are demonstrated as below:
Step-by-step Decomposition WebDancer breaks down complex problem into smaller, manageable steps with "First Then Finally".
Hypothesis Testing WebDancer proposes hypotheses and verifies their validity. It assumes that "Nemo" from Finding Nemo is the orange clownfish (Amphiprion ocellaris). It validates this assumption by consulting the USGS database for non-native occurrences.
Handling Information Gaps When certain information is missing, WebDancer takes action
to fill the gap actively.
Iterative Reflection After failing to find the 7IP code in the USGS database, WebDancer realizes it needs to search for the specific address separately. Through second search,
26

---

successfully retrieves the ZIP code. This demonstrates the Agent's 's flexibility and adaptability when facing uncertainty.
Case Trajectory in GAIA
I'm researching species that became invasive after people who kept them as pets released them. There's a certain species of fish that was popularized as pet by being the main character of the movie Finding Nemo. According the USGS, where found as a nonnative species, before the year 2020? need the answer formatted as the five-digit zip codes of the places the species was found, separated by commas if there is more than one place.
Question:
this fish
Step 1
<think>
Okay, let's tackle this question step by step. The user wants to know the zip codes where the fish from Finding Nemo, which became invasive after being released by pet owners, was found as a non-native species according to USGS before 2020. First, I remember that the main fish in Finding Nemo is Nemo, a clownfish. So the species question is likely the orange clownfish, Amphiprion ocellaris, But should confirm that. Maybe I should start by searching for člownfish invasive species USGSio
there's any info. Wait, but the user mentioned the USGS specifically. Let me check what USGS says about
</think>
<tool_call>
("name": "search", "arguments": ("query": "USGS Amphiprion ocellaris nonnative locations
before 2020"))
ettol_calls
<tool_ reo_rresssss>>
Google search for "USGS Amphiprion ocellaris nonnative locations before 2020' found
results:
[clown anemonefish (Amphiprion ocellaris)
Species
Pro-
flle/hhttss//aaseer.usg wttps//ucceeeeaaaSeettaaa.aaaea with nonindigenous occurrences, the earliest and
Table 1. States
2. [clown anemonefish (Amphiprion ocellaris)] httpp::/haa..cruuss...oovvv
Citation information: U.S. Geological Survey. [2025]. Nonindigenous Aquatic Species Database. Gainesville, Florida. Accessed [5/9/2025]. Contact us if you are...
</tool_ resposees
Step 2
<think>
Okay, so I need to find the absolute difference in tens of thousands between the population of chinstrap penguins from the Wikipedia page as of the end of 2018 and the population from the 2020 Nature.com article. The user also wants to assume two penguins per breeding pair. Let me start by going through the information provided.Firg First, looking at the search results for the Wikipedia part. The sixth result mentions the Chinstrap penguin's 's conservation status, stating that 2018, the IUCN estimated around 8 million specimens... </think>
<tool_ccal>>
("name": "visit". "arguments": ("url": "https://nass.c www.........................e... www.ussg..o.uuu............. wwwp//ueriis/aaacSheee..aaa ?speciesID =3243", "goal": "Identify states with nonindigenous occurrences of Amphiprion ocellaris before 2020")}
<ttool_call> call>
<tool response>
I" l"evidence": "The provided webpage content includes table listing nonindigenous occurrences of Amphiprion ocellaris. The table shows that one individual was collected in Florida in July 2018 at Fred Howard Park, Pinellas Co., which is before 2...
</tool responses
Step 3
<think>
27

---

Okay, let's tackle this question step by step. The user is asking about the zip codes where the clownfish (Amphiprion ocellaris), made famous by Finding Nemo, was found as nonnative species in the US before 2020 according to USGS. First, I did search using the provided tools. The initial search led me to the USGS Nonindigenous Aquatic Species database page for Amphiprion ocellaris. Visiting that page showed table listing occurrences. The key point here is that the only entry before 2020 is Florida, specifically Pinellas County, collected in July 2018 Fred Howard Now, the next step is converting that location into a zip code
</think>
<tool_call>
I"name": "visit", "arguments"; ("url": "https::/nas.er watihaasser..sg...... er.usgs gov/queries/collectionnoo aspx? SpeciesID=3243", "goal": "Find specific collection locations with zip codes for Amphiprion ocellaris the US before 2020") )
</tool_call _aal>>
<tool response>
"evidence": "1468741 FL Pinellas Gulf of America, Florida, Fred Howard Park 2018 03 03100207 Crystal--thllachascotee eradicated", "rational": "The webpage provides collection record for Amphiprion ocellaris (clown anemonefish) in the United States, specifically Pinellas County, Florida. The record indicates that specimen was collected at Fred Howard Park in 2018. However, the webpage does not provide zip codes for the collection locations."
@tooo_responss
Step 4
<think>
So now I need to figure out the zip code for Fred Howard Park in Pinellas County, Florida. Since the USGS data doesn't provide zip codes, might need look up the location's address. Let me think if there's another way using the available tools. The allowed tools are search and visit. Maybe can perform another search to find the zip code of Fred Howard Park. Wait, the user's instructions say to use the tools provided. Since can't 't just know the zip code offhand, should use the search tool again to find the zip code Fred Howard Park, Pinellas County, Florida. Let me try that. So next action: search for "Fred Howard Park Pinellas County Florida zip code". But need to structure it properly. Let me check the parameters for the search tool. The query can be anything, can input "What is the zip code Fred Howard Park, Pinellas County, Florida?" After getting that info, can then present the zip code
</think>
<tool_call
("name": "search", "arguments": ("query": "Fred Howard Park Pinellas County Florida zip code"))
@/tool_calls
<tool response>
A Google search for 'Fred Howard Park Pinellas County Florida zip code' found 9 results: [Fred Howard Park Pinellas
Fred Howard Park is 155 acres of park and beach located off the Gulf of Mexico in Tarpon Springs.
There 1-mile causeway from the park to the white sandy
2.
[Fred Howard Park, 1700 Sunset
Dr. Tarpon Springs, -araadd-.arr-42217).....
FL 34689,
reooo_esspons>>>
Step 5
<think>
The first link is the species profile for Amphiprion ocellaris, which lists states with nonindigenous occurrences. The summary from visiting that page showed that Florida had recorded occurrence in 2018 8 at Fred Howard Park in Pinellas County. Next, needed to confirm the exact location details. The second visit to the collection info page provided record from 2018 in Pinellas County, Florida, specifically Fred Howard Park. However, the zip code wasn't listed there, and the specific location's 's zip code 34689. Since the question asks for zip codes, and this is the only location mentioned in the provided data,
28

---

the answer should be 34689. need to ensure there aren't It other locations, but based on the search results, only Florida's Pinellas County was cited. Thus, the final answer is the zip code 34689. </think>
<answer>34688 <answer> V
Figure 8: Case Study.
29


Reviews:
[{"review_id": "4mYz5fY5gb", "reviewer": "Reviewer_4wHM", "summary": "This paper presents WebDancer, a systematic and scalable framework for training web-based autonomous information-seeking agents. The framework follows a four-stage pipeline: (1) constructing diverse and challenging QA datasets (CRAWLQA and E2HQA), (2) generating agentic trajectories using rejection sampling under both short and long Chain-of-Thought (CoT) prompting strategies, (3) supervised fine-tuning (SFT) to provide a strong initialization, and (4) reinforcement learning (RL) with the DAPO algorithm to improve generalization. WebDancer is instantiated as a ReAct-style web agent, and evaluated on two challenging benchmarks (GAIA and WebWalkerQA), demonstrating strong performance compared to both open-source and closed-source agentic frameworks.", "questions": "See Weaknesses part", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n\n1. The direction of building autonomous information-seeking agents is both timely and important for advancing agentic AI capabilities. \n \n2. The proposed four-stage framework (data construction, trajectory sampling, SFT, RL) is clear and well-structured.\n\n**Weaknesses**\n\n1. The paper would benefit from improved exposition. Some essential context is missing, such as a basic introduction to the unique characteristics and challenges of web agent tasks for readers not already familiar with this area. Adding more illustrative case studies could further enhance readability and practical relevance.  \n2. The experiments focus primarily on the Qwen series as backbone models. It remains unclear how generalizable the proposed pipeline is to other LLM backbones.  \n3. The agent’s action space is restricted to “search” and “click/visit,” with no support for more complex interactions such as scrolling or system key events. This restriction should be discussed more explicitly.  \n4. The RL reward relies on LLM-as-Judge, but the paper does not provide an explicit evaluation of this judge’s accuracy. Since the judge could become a bottleneck, an analysis or discussion of its reliability is essential.  \n5. Could the authors provide more detailed case studies highlighting which types of web tasks or interaction patterns benefit the most from the SFT or RL stages? Conversely, what are the common failure modes or limitations in the current system, and how might they be addressed?", "was_revised": true, "final_justification": "I thank the authors for the responses. I maintain my initial positive rating of this paper.", "rebuttal": "We sincerely thank you for your insightful comments and constructive suggestions, which are highly valuable for improving our work. Below, we provide point-by-point responses to the raised concerns.\n## Q1: Exposition Improvement\nSearch-based web agent tasks present a set of unique challenges that differentiate them from traditional web-based QA or single-turn retrieval tasks. Specifically:\n\n1. **Lack of High-Quality, High-Difficulty Training Data**:\nUnlike simpler retrieval tasks, search-agent benchmarks such as *Gaia* and *BrowserComp* involve complex multi-hop reasoning and multi-step interaction. However, there is a significant scarcity of training data that reflects this level of difficulty. The few existing training datasets are typically not well-aligned with the test distributions of these benchmarks. These benchmarks feature long-horizon tasks, which require capabilities beyond what standard SFT data can teach.\n\n2. **Limited Support for Multi-Step Search and Deep Research**:\nCurrent agent models often struggle with multi-step retrieval and iterative refinement. This limitation hampers their ability to perform *deep research*, i.e., cases where information must be verified, cross-checked, or gathered from multiple heterogeneous sources.\n\n4. **Lack of Systematic Guidance for Deep Research Behavior**:\nBefore our work, there has been no systematic methodology for training or evaluating agents on “deep research” tasks. Most existing approaches focus on shallow interactions or fail to generalize to tasks requiring flexible search strategies, adaptation, and reflection.\n\nOur work aims to address these gaps by providing both a practical framework and empirical insights into training agents capable of sophisticated web-based research behaviors.\n\nWe will enhance the Introduction and Related Works sections with a clearer and more comprehensive introduction to the unique characteristics and challenges of web agent tasks, to better support readers who may not be familiar with this area. Additionally, while we have already included a case study in Appendix F, we appreciate the suggestion to make the practical relevance more prominent. We will incorporate a more illustrative and motivating case study directly in the Introduction to help guide the reader and highlight the real-world significance of our setting.\n\n## Q2: Backbone Generalization\n\nDue to computational constraints, we primarily conducted experiments using the Qwen2.5 series as backbone models. \nWe selected Qwen2.5 for its competitive performance and open availability, which allowed for a thorough and reproducible evaluation of our proposed pipeline. \nImportantly, **we have validated our approach across multiple sizes of Qwen2.5 (e.g., 7B, 32B), consistently observing performance improvements**, which supports the generality of our method across model scales.\nIn future work, we plan to extend our study to other backbone families, such as LLaMA and Mixtral, to further verify the robustness and generalizability of our approach across diverse architectures.\n\n## Q3: Agent Action Space\n\nWe agree that the restricted action space deserves a more explicit discussion. In our current design, the agent operates with \"search\" and \"visit\" actions, which are considered **fundamental primitives in the information-seeking process**[1,2].\nIn principle, these two actions are sufficient to access and retrieve any information available on the web.\n\nMoreover, our framework is designed to be **modular and extensible**. \nIt supports seamless integration with both browser modeling (e.g., scrolling, form filling) and Python sandbox environments, enabling more complex interactions when needed.\nGiven the challenges of sample efficiency in RL settings, we chose to focus on \"search\" and \"visit\" as a strong starting point.\nThese tools already demonstrate substantial capabilities across our benchmark tasks.\n\nWe will add further discussion of this design choice and its trade-offs in the revised version.\n\n## Q4: Reward Reliability\n\nIn our early-stage experiments, we extensively compared several commonly used reward designs in the search-agent setting, including ***recall***, ***F1***, and ***model-based rewards***. Our findings motivated the use of LLM-as-Judge.\n\nSpecifically, both recall- and F1-based rewards suffered from reward hacking:\n\n- For recall, the model often learned to include large numbers of candidate answers in order to boost recall, leading to verbose and unreliable outputs.\n\n- For F1, the model tended to output only partial answers that scored well but lacked completeness and readability.\n\nAs an alternative, we adopted a model-based reward using LLM-as-Judge. Our prompts are adapted from established benchmarks (e.g., HLE[3], BrowseComp[4]). To evaluate the robustness of this judge, we tested two strong LLMs: Qwen2.5-72B (open-source) and GPT-4o (closed-source). The results demonstrated **high consistency** between the two.\n\nFurthermore, we manually audited 100 samples judged by Qwen2.5-72B and found only one judge error, **suggesting strong reliability in practice**. While a full quantitative evaluation of judge accuracy remains an interesting direction for future work, our empirical evidence indicates that the LLM-as-Judge is significantly more aligned with the QA task and more robust than standard metric-based rewards in this setting.\n\nWe will include a detailed discussion of this aspect in the revised version.\n## Q5: Interaction Patterns and Agent Behavior\n**Current agent models lack robust multi-step reasoning and iterative retrieval, limiting their effectiveness in deep research tasks requiring cross-verification and multi-source synthesis.**\nAs shown in Appendix F, we have conducted a case-based analysis of agent behaviors in the GAIS task. We agree that a more detailed analysis of interaction patterns is valuable. Our current findings indicate that:\n\n- In the SFT stage, the model already learns **effective decomposition and high-level planning patterns**, such as structured sequences like *\"First ... Then ... Finally ...\"*.\n\n- In the RL stage, the agent further acquires more advanced behaviors such as **cross-validation, reflection, and adaptive replanning**. For the case in Appendix F, in a complex task involving ZIP code lookup, the agent first fails to find the result via the USGS database, but then adapts by issuing a second targeted search, ultimately succeeding, demonstrating robust handling of uncertainty.\n\nRegarding failure modes and current limitations, we acknowledge that longer interaction turns introduce long-context challenges in our proposed WebDancer, which can affect coherence and memory. As part of future work, we are exploring Long-CoT to Short-CoT condensation and summarization techniques to mitigate these issues.\n\nWe will expand this discussion and include additional case studies in the revised version to further clarify the evolution of interaction strategies and the impact of training on agent sophistication.\n\nWe sincerely appreciate your valuable suggestions, which are highly helpful for improving our work. We will incorporate all of them in the revised version. Please feel free to raise any additional concerns.\n\n[1]. OAgents: An Empirical Study of Building Effective Agents \n\n[2]. WebThinker: Empowering Large Reasoning Models with Deep Research Capability\n\n[3]. Humanity's Last Exam\n\n[4]. BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agents"}, {"review_id": "SPn9104yT7", "reviewer": "Reviewer_Q7WQ", "summary": "This paper presents a pipeline for training information seeking agents that can answer closed-form answers through multi-step interactions with the web environment.\nSpecifically, the proposed pipeline involves four steps: 1) collecting high-quality QA pairs for training; 2) sample trajectories using both LLMs and LRMs for QA pairs; 3) performing SFT on the filtered trajectories; and 4) performing RL on the QA pairs for enhanced generalization.\nAlong with the pipeline, this paper collects two large-scale QA pairs for training, i.e., CrawlQA (60K) and E2HQA (40K).\nThis paper applies the training pipeline to fine-tune Qwen-2.5-7B, Qwen-2.5-32B, and QwQ-32B. Experimental results demonstrate the effectiveness of the training pipeline on both GAIA and WebWalkerQA.", "questions": "1. Do you plan to release CrawlQA and E2HQA to the public, including open-source the code for data curation?\n2. In CrawlQA, how to make sure the navigation is meaningful and does not have a corresponding shortcut? For example, you may get the answer page after 10 clicks, but there might be a shortcut to directly go there or that page might be indexed by google search already.\n3. In addition to the dataset and trained models, what major insights do you think are important to convey to readers?\n\nClarification:\na. Is it possible that for the same question, there's both a short CoT version trajectory and a long CoT version? If so, would combining them for training result in some conflicts?\nb. Based on appendix D, only less than 10% questions were retained during RFT?", "limitations": "1. The most important use case of information seeking agents typically extend beyond closed-form QA. For example, there can be more open-ended needs such as finding a products with certain criteria or even generating a report for a science question. It might be an interesting future direction. That said, I agree this is not within the scope of this paper. So this is just a general suggestion.\n2. This paper may reveal more details regarding the constructed dataset for people to better assess it.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "### Strengths:\n1. The curated QA pairs, in particular CrawlQA, can be a valuable source of verifiable rewards for training information seeking agents.\n2. The experiments are comprehensive. Necessary ablations are conducted to justify the designs.\n\n### Weaknesses:\n1. Important details regarding the dataset construction are not revealed in this paper, which might make it difficult to assess the quality of the work. This is important because the techniques used for training in this paper are mostly existing ones, while the dataset is the crucial differentiating point.\n2. Related to the first point, the space arrangement of this paper could be improved. Currently, it spend majority of the space on the training techniques, while things like ReAct, GRPO, or the reward design with both format and correctness are something people already familiar with.\n3. (Minor) This paper may benefit from polished writing. Some sections appear to be verbose (e.g., Sec 2). In addition, there are several minor issues: a) Figure 1 seems not referred in the main text? b) line 274 is struggle transferable, grammar error", "was_revised": true, "final_justification": "I decide to raise my overall evaluation after reading the authors' response.", "rebuttal": "We sincerely thank you for your insightful comments and constructive suggestions, which are highly valuable for improving our work. Below, we provide point-by-point responses to the raised concerns.\n\n## Q1. Dataset Open-Sourcing\n**We are committed to open-sourcing all code used for data generation and curation.** Furthermore, we will release the full CrawlQA and E2HQA datasets on open-access platforms such as HuggingFace and ModelScope. By doing so, we aim to contribute to and support the broader open-source research community.\n\n## Q2. Quality of CrawlQA\nIn CrawlQA, our construction strategy follows the construction methodology introduced in WebWalkerQA[1], which demonstrates **that information buried deep within websites is generally not easily accessible via search engines**.\nHowever, we acknowledge that some queries may still have shortcuts due to search engine indexing.\nTo address this, we adopt a filtering step inspired by BrowseComp[2]: we employ a basic agent built on Qwen-72B with search capabilities to execute. **Queries that can be resolved in a single-step search are removed.**\nThis ensures that the retained QA pairs require meaningful multi-step navigation and better reflect realistic web exploration challenges. \nWe will include further details about this filtering process in the revised version.\n\n## Q3. Space Arrangement and Writing\nOur core contribution lies in presenting a systematic, end-to-end, and actionable reproduction of a Deep Research agent. \nThanks for your advice.\nGiven the complexity of the full pipeline, we will allocate substantial space to ensure clarity and completeness. \nThat said, we will revise the space arrangement to better balance novelty and background content, and refine the writing to be more concise and focused.\n\n## Q4. Insights and Novelty\n**Our core contribution lies in proposing a reusable and end-to-end framework for reproducing a Deep Research agent.**\nThis framework consists of four stages: (1) browsing-based data construction, (2) trajectory sampling, (3) supervised fine-tuning for effective cold-start capabilities, and (4) reinforcement learning to enhance generalization.\n\nThroughout this pipeline, we investigate several fundamental questions, such as **how to construct high-quality QA data, how to represent intermediate thoughts, how to design data formats for SFT and RL under deep research scenarios, and how to enable multi-step reasoning**.\nTo the best of our knowledge, this is **the first work** that attempts to systematically reproduce Deep Research behavior and provide actionable implementation strategies.\n\n## Q5. CoT Styles and Filtering Efficiency\nFor each query, we can collect both a short CoT and a long CoT version of the reasoning trajectory.\nImportantly, we train the two styles separately to avoid interference: as shown in Table 3, the instruct model is trained with short CoTs, while the reasoning model, like QwQ, uses long CoTs.\n\nWe also conducted transfer experiments across styles, and the results support the effectiveness of our current setup. \nAdditionally, we experimented with combining both CoT styles during training, but as you anticipated, this led to performance degradation due to conflicting reasoning patterns between short and long CoTs. \nThis confirms that treating them separately is a more effective strategy.\n\nRegarding the filtering efficiency, for queries used in RFT training, we apply multiple layers of filtering, including *format validity* control during rejection sampling, *correctness verification* of final answers, *quality assessment* of thought steps, and constraints on *the number of actions taken*.\n\n**Our QA construction pipeline is designed to be scalable**: as long as high-quality RFT-style trajectories can be generated, the corresponding QA examples are meaningful and useful. While high-quality QA data with detailed thoughts is crucial for RFT, other QA instances without such strict requirements can still be effectively used in the RL stage.\n\nWe will supplement and clarify this part in the revised version. \n\n## Q6. Task Generalization\nOur current work focuses on short-answer QA tasks, primarily because they offer well-defined evaluation protocols and allow reward computation to be more reliable. In contrast, long-form QA tasks pose significant challenges in both RFT data construction and RL training due to difficulties in evaluation.\n\nInterestingly, we observe that our model, trained on short QA tasks, **demonstrates strong generalization to long-form settings in terms of information-seeking behavior**. This suggests that the model has learned effective strategies for decomposing complex queries and locating relevant evidence, which naturally transfers to long-form tasks. We will include a case study to illustrate this generalization behavior in the revised version.\n\nHowever, long-form QA involves not only information retrieval but also generation quality, which remains an open challenge. We plan to explore this direction in future work.\n\nIf you have any further questions or suggestions, we would be happy to address them. We kindly ask you to consider our responses when reassessing the paper, and we appreciate your constructive feedback.\n\n[1]. WebWalker: Benchmarking LLMs in Web Traversal\n\n[2]. BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agents"}, {"review_id": "jwzelaYG4Y", "reviewer": "Reviewer_SVnS", "summary": "In this paper, the authors propose WebDancer, an end-to-end framework for building autonomous web agents for multi-step information seeking. The proposed approach consists of four main stages: (1) data synthesis to generate a large corpus of challenging question-answer pairs; (2) high-quality trajectory sampling to generate both short and long CoT reasoning steps; (3) a supervised fine-tuning (SFT) stage to provide the agent with a \"cold start\" for instruction and tool-use following; and (4) an on-policy RL stage to enhance the agent's decision-making and generalization capabilities. In experiments, extensive results demonstrate the effectiveness of WebDancer on the GAIA and WebWalkerQA benchmarks.", "questions": "1. Why do not compare other different RL training methods, such as GRPO?\n1. Do the Long-CoT trajectories exhibit more complex planning, self-correction, or deeper exploration, beyond just being longer? Could you provide some cases on Short-CoT and Long-CoT for better understanding and comparison?", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths:**\n\n1. This paper addresses a challenging problem of constructing a autonomous  information-seeking agents via a end-to-end paradigm. \n1. The proposed four-stage pipeline is comprehensive and systematic. The hybrid training method is practical for achieving efficient agent learning.\n1. The experiment design is solid and results show that the proposed method is superior.\n1. The paper is well-written, clearly structured and easy to follow.\n\n**Weaknesses:**\n1. The training of proposed method is resource-extensive.\n1. There may be reward hacking issue, since the evaluation relies on an LLM-as-Judge (72B) for both the RL reward signal and the final performance metric.", "was_revised": true, "final_justification": "My main concerns are addressed.", "rebuttal": "Thank you very much for your recognition of our work and for the constructive suggestions. We truly appreciate your thoughtful feedback and will address each of your comments in detail below.\n\n## Q1: Resource-extensive\n\nWe acknowledge that our method is resource-intensive during both training and inference. However, our goal is to build a **Deep Research agent** capable of solving high-value problems that often approach or exceed human-level complexity. In such settings, the trade-off between computational cost and reasoning capability is justified.\n\nNotably, even for advanced research agents like those developed by OpenAI, solving a single problem may require **30 minutes to an hour of computation**—a timeframe comparable to what human experts would spend on the same task. We believe that investing computational resources to automate such high-effort, high-reward tasks is both meaningful and impactful for augmenting human productivity.\n\nWe will clarify this motivation in the revised version.\n\n## Q2: : Reward Reliability\n\nIn our early-stage experiments, we extensively compared several commonly used reward designs in the search-agent setting, including ***recall***, ***F1***, and ***model-based rewards***. Our findings motivated the use of LLM-as-Judge.\n\nSpecifically, both recall- and F1-based rewards suffered from reward hacking:\n\n- For recall, the model often learned to include large numbers of candidate answers in order to boost recall, leading to verbose and unreliable outputs.\n\n- For F1, the model tended to output only partial answers that scored well but lacked completeness and readability.\n\nAs an alternative, we adopted a model-based reward using LLM-as-Judge. Our prompts are adapted from established benchmarks (e.g., HLE[3], BrowseComp[4]). To evaluate the robustness of this judge, we tested two strong LLMs: Qwen2.5-72B (open-source) and GPT-4o (closed-source). The results demonstrated **high consistency** between the two.\n\nFurthermore, we manually audited 100 samples judged by Qwen2.5-72B and found only one judge error, **suggesting strong reliability in practice**. While a full quantitative evaluation of judge accuracy remains an interesting direction for future work, our empirical evidence indicates that the LLM-as-Judge is significantly more aligned with the QA task and more robust than standard metric-based rewards in this setting.\n\nWe will include a detailed discussion of this aspect in the revised version.\n\n## Q3: GRPO Performance\nIn our early experiments, we also evaluated GRPO on the Qwen-32B backbone and observed that it achieved around 35 points on GAIA, which was lower than DAPO under the same conditions. The performance gap may stem from DAPO’s more effective dynamic sampling of agent trajectories, which better aligns with our task setting.\n\nGiven the computational cost and efficiency considerations of RL training, we chose to focus on DAPO as the primary RL algorithm in this work.\n\n## Q4: Short vs. Long CoT\nThank you for the insightful question. We agree that simply being \"longer\" does not necessarily imply better reasoning. In our work, the **Long-CoT trajectories** are not just extended versions of the Short-CoTs; rather, they demonstrate **richer reasoning behaviors**, including:\n\n* **Step-by-step decomposition**\n* **Hypothesis testing**\n* **Handling information gaps**\n* **Iterative reflection and self-correction**\n\nAs illustrated in **Appendix F**, Long-CoTs often break down complex queries into subgoals, assess intermediate evidence, and revise earlier assumptions based on new findings. In contrast, **Short-CoTs**, while still logical, tend to follow a more direct and surface-level path with limited internal deliberation.\n\nFor example, in response to a question about the presence of a nonnative species (clownfish) in the US before 2020, a **Short-CoT** may look like this (with tool outputs omitted for brevity):\n\n> *To find out where this species was found as a nonnative species in the United States before 2020, I will need to search for information from the USGS or related sources that track invasive species. I will start by searching for the clownfish and its status as an invasive species in the US.*\n\n> *The search results provide several resources related to the clownfish and its status as an invasive species. The most promising sources seem to be from the USGS database, which specifically tracks nonindigenous aquatic species. I should visit the relevant USGS webpage to gather detailed information about clownfish occurrences in the United States as a nonnative species.*\n\n> *The Wikipedia page did not provide any additional information regarding clownfish being found as nonnative species in the US before 2020.*\n\nWhile this Short-CoT is coherent and goal-oriented, it **lacks deeper exploration**, such as verifying conflicting sources or revisiting assumptions when evidence is missing.\n\nWe will **include a side-by-side comparison between Short-CoT and Long-CoT** examples in the revised version to better illustrate these distinctions.\n\nOnce again, we sincerely thank you for your recognition of our work and your thoughtful feedback. If you have any further questions or suggestions, we would be more than happy to address them."}, {"review_id": "0O77iqEtoZ", "reviewer": "Reviewer_PfE7", "summary": "This paper introduces WebDancer, an end-to-end framework for training autonomous web agents to perform complex, multi-step information seeking tasks. The proposed system integrates four stages: (1) QA data generation via CRAWLQA and E2HQA, (2) trajectory sampling using ReAct-style reasoning (Short/Long CoT), (3) supervised fine-tuning (SFT) for cold-start behavior, and (4) reinforcement learning (RL) using the DAPO algorithm for robustness and generalization. WebDancer is evaluated on GAIA, WebWalkerQA, and BrowseComp, achieving strong results that rival or surpass some existing open-source baselines.", "questions": "- Line 35 says \"unlock the autonomous multi-turn information36 seeking agency\", but the interactions seem to be single-turn only, with potentially multiple steps.\n- The data quality filtering does not present sufficient details. For example, the heuristic of hallucination detection is based on number of actions, could you explain why?", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n- The paper presents a systematic framework that combines high-quality data synthesis and filtering, SFT, and RL, all within a tool-augmented ReAct agent loop.\n- The paper provides detailed studies on the impact of SFT, RL, CoT length, data rejection, and training steps, which reveal rich insights about agent behavior under different setting.\n- CRAWLQA and E2HQA introduce diversity and complexity, and rejection sampling and multi-stage filtering ensures high data quality.\n\nWeaknesses:\n- The proposed method heavily relies on data distillation from advanced LLMs. Both QA pairs and trajectories are distilled from powerful LLMs (GPT-4o, QwQ), which is a standard practice and limits the novelty.\n- Tasks are limited to single-turn QA. The agent solves short-answer queries without any multi-turn interaction, clarification, or open-ended task structure.\n- Only two tools (search and click) are used, which limits the tool diversity and task complexity.", "was_revised": true, "final_justification": "The rebuttal from the authors addressed some of my concerns about the novelty, hence the score update.", "rebuttal": "We sincerely thank you for your insightful comments and constructive suggestions, which are highly valuable for improving our work. Below, we provide point-by-point responses to the raised concerns.\n\n\n## Q1. Novelty\n**Our core contribution lies in proposing a reusable and end-to-end framework for reproducing a Deep Research agent.**\nThis framework consists of four stages: (1) browsing-based data construction, (2) trajectory sampling, (3) supervised fine-tuning for effective cold-start capabilities, and (4) reinforcement learning to enhance generalization.\n\nThroughout this pipeline, we investigate several fundamental questions, such as **how to construct high-quality QA data, how to represent intermediate thoughts, how to design data formats for SFT and RL under deep research scenarios, and how to enable multi-step reasoning**.\nTo the best of our knowledge, this is **the first work** that attempts to systematically reproduce Deep Research behavior and provide actionable implementation strategies.\n\nWhile distillation from advanced LLMs has become a common practice, we would like to clarify that **effective distillation is neither trivial nor devoid of research value**. Our work systematically explores several under-explored but important aspects of this process, including which models to distill from, how to construct and filter high-quality distillation data, and how to best utilize this data for different stages (SFT and RL) of training.\n\nRather than treating LLM distillation as a plug-and-play component, **we view it as a central research challenge and contribute actionable solutions and insights to make the distillation process effective and scalable in the context of deep research agents.**\n\n## Q2. Task Generalization\nThank you for your careful reading and thoughtful comments. We would like to clarify that our current work focuses on **multi-step single-turn QA tasks**. As shown in Appendix F (case study), solving each query often requires the agent to perform **multiple steps of information retrieval, verification, and reasoning**. And **the agent interacts with the user in a single-turn setting**, meaning the user provides only one initial query and does not participate in the intermediate reasoning steps.\n\nWe will revise and clarify this point in the main text to avoid misunderstanding.\n\nAs for **multi-turn interactions and human-in-the-loop dialogue**, we agree that they are important directions for building more interactive research agents, and we plan to explore them in future work.\n\nRegarding open-ended task structures, our current focus on short-answer QA tasks is due to their well-defined evaluation protocols and relatively reliable reward computation. In contrast, open-ended and long-form tasks present challenges in both RFT data construction and RL training, mainly due to the lack of robust evaluation metrics.\n\nInterestingly, we observe that our model, trained on short QA tasks, **demonstrates strong generalization to long-form settings in terms of information-seeking behavior.** This suggests that the model has learned effective strategies for decomposing complex queries and locating relevant evidence, which naturally transfers to long-form tasks. We will include a case study to illustrate this generalization behavior in the revised version.\n\nWe appreciate your suggestion and will consider these broader task settings in future work.\n\n## Q3: Agent Action Space\n\nWe agree that the restricted action space deserves a more explicit discussion. In our current design, the agent operates with \"search\" and \"visit\" actions, which are considered **fundamental primitives in the information-seeking process**[1,2].\nIn principle, these two actions are sufficient to access and retrieve any information available on the web.\n\nMoreover, our framework is designed to be **modular and extensible**. \nIt supports seamless integration with both browser modeling (e.g., scrolling, form filling) and Python sandbox environments, enabling more complex interactions when needed.\nGiven the challenges of sample efficiency in RL settings, we chose to focus on \"search\" and \"visit\" as a strong starting point.\nThese tools already demonstrate substantial capabilities across our benchmark tasks.\n\nWe will add further discussion of this design choice and its trade-offs in the revised version.\n\n## Q4.  Data Quality Filtering\nThank you for pointing this out. We acknowledge that our current phrasing may be ambiguous. Specifically, the sentence “*we first apply rules to filter out trajectories with more than two actions, ensuring that there are no hallucinations and no severe repetitions*” might be misinterpreted as suggesting that the number of actions alone is used to detect hallucinations.\n\nTo clarify: the filtering pipeline operates **on top of** trajectories with more than two actions. We then apply additional filtering steps to ensure quality. In particular:\n\n* **Hallucination detection** is based on inspecting the *thought* steps to check whether the agent generates hallucinated tool outputs (i.e., fabricating a tool return and reasoning based on it, without actually invoking the tool).\n* **Severe repetition** is detected using **n-gram overlap heuristics**, ensuring that repetitive patterns across thoughts and actions are filtered out.\n\nWe will revise the manuscript to make this process clearer and avoid confusion. Additionally, we commit to **open-sourcing the full data filtering pipeline**, so that the community can inspect, reproduce, and build upon our approach.\n\nIf you have any further questions or suggestions, we would be happy to address them. We kindly ask you to consider our responses when reassessing the paper, and we appreciate your constructive feedback.\n\n\n[1]. OAgents: An Empirical Study of Building Effective Agents\n\n[2]. WebThinker: Empowering Large Reasoning Models with Deep Research Capability"}]

Author rebuttal:

Meta-review: **Recommendation:** Need Discussion

### (a) Summary of the Paper

This paper introduces WebDancer, a comprehensive, four-stage framework for building autonomous information-seeking web agents capable of "deep research." The proposed pipeline consists of: (1) constructing new, challenging QA datasets from web data, (2) sampling high-quality agentic trajectories using LLMs, (3) using these trajectories for supervised fine-tuning (SFT) to establish a strong baseline agent, and (4) further improving the agent's generalization and decision-making capabilities through reinforcement learning (RL). The authors instantiate this framework to train Qwen-based agents and demonstrate strong performance on challenging benchmarks like GAIA and WebWalkerQA.

The paper received a positive reception from all reviewers, with final scores leaning toward acceptance (5, 4, 4, 4). The authors provided a thorough and effective rebuttal, addressing most of the initial concerns and leading multiple reviewers to raise their scores. Despite this positive consensus on the paper's quality and empirical strength, a "Need Discussion" is warranted to deliberate on the nature and significance of the contribution, particularly regarding its novelty.

### (b) The Case for Acceptance

The primary argument for acceptance, shared by all reviewers, is that the paper presents a **valuable, systematic, and reproducible framework** for a highly relevant and challenging problem. Instead of proposing a single algorithmic tweak, the authors provide a complete, end-to-end blueprint for creating a sophisticated web agent. This systems-level contribution is a significant strength. Key points in favor of acceptance include:

*   **A Comprehensive and Actionable Pipeline:** The four-stage process is well-structured, logical, and provides clear, actionable guidance for researchers looking to build similar "deep research" agents. Reviewers praised this as a "comprehensive and systematic" approach (SVnS) and a "clear and well-structured" framework (4wHM).
*   **Strong Empirical Results:** The paper demonstrates impressive performance on difficult, multi-step information-seeking benchmarks, validating the efficacy of the proposed pipeline. The solid experimental design and thorough ablations were also commended.
*   **Significant Data Contribution:** The creation and planned release of two new large-scale QA datasets, CrawlQA and E2HQA, is a valuable contribution to the community in its own right, addressing a known scarcity of high-quality training data for this task (Q7WQ).
*   **Effective Rebuttal:** The authors successfully addressed numerous concerns during the rebuttal, providing clarifications on their data filtering process, validating the reliability of their LLM-as-Judge through a manual audit, and committing to open-sourcing all data and code. This responsiveness strengthened the paper and convinced several reviewers to raise their scores.

### (c) The Case for a More Cautious Stance

While empirically strong, the paper's primary weakness, noted by multiple reviewers (PfE7, Q7WQ), is its **limited conceptual novelty**. The contribution lies in the effective integration and systematization of existing techniques, rather than the invention of new ones. This raises a valid question about the paper's fit for NeurIPS. Key points for a more critical evaluation include:

*   **Reliance on Existing Components:** The core components of the pipeline—data distillation from powerful LLMs, the ReAct agent format, SFT for cold-starting, and on-policy RL algorithms like DAPO, are all established techniques in the agent literature. 
*   **Data-Centric Focus:** A significant portion of the paper's contribution is data-centric (the new datasets and the trajectory sampling process). While valuable, this positions the work more as an empirical or resource paper, which can sometimes be seen as less impactful than a work with fundamental methodological advances.
*   **Scope Limitations:** The agent's action space is limited (primarily search and click), and the tasks are constrained to single-turn QA. While the authors provide reasonable justifications, this narrows the scope of the demonstrated capabilities.

### (d) Key Points for AC Discussion

The consensus is that this is a high-quality empirical paper that provides a valuable service to the community. The core point of discussion is whether this type of contribution, a systematic, reproducible, and highly effective integration of existing methods for a complex task—meets the novelty and impact bar for NeurIPS. The discussion points are:

1.  **Nature of Contribution: Systematization vs. Algorithmic Novelty:** How should we weigh a strong, novel *system* against a work with more fundamental algorithmic novelty? This paper provides a clear blueprint for reproducing a complex agentic behavior. Is this "how-to" guide, backed by strong results, a sufficiently impactful contribution for the applications track, even if the individual components are not new?

2.  **Evaluating the Data Contribution:** The paper's new datasets are a significant asset. How much weight should be given to this resource contribution in the final decision? Reviewer Q7WQ initially pointed out a lack of detail on the data construction, which the authors addressed in the rebuttal. Does the promise to release the data and the full pipeline sufficiently validate this aspect of the work?

3.  **Reliability of the Evaluation:** The RL stage relies on an LLM-as-Judge. While the authors provided a manual audit in the rebuttal showing high reliability, this was not part of the original submission. Is this post-hoc validation sufficient to alleviate concerns about potential reward hacking or evaluation bias in a core component of the training pipeline?

Overall, this is a strong paper that the community would likely find useful. The discussion should center on whether its strengths as a practical, reproducible framework and a valuable data resource are sufficient to overcome the concerns about its limited originality in terms of core techniques.
Decision: Accept (poster)