Title: WebDancer: Towards Autonomous Information Seeking Agency
Venue: NeurIPS 2025
Accepted: yes
Keywords: Web agent

Information seeking
Abstract: Addressing intricate real-world problems necessitates in-depth information seeking and multi-step reasoning. 
Recent progress in agentic systems, exemplified by Deep Research, underscores the potential for autonomous multi-step research. 
In this work, we present a cohesive paradigm for building end-to-end agentic information seeking agents from a data-centric and training-stage perspective.
Our approach consists of four key stages: (1) browsing data construction, (2) trajectories sampling, (3) supervised fine-tuning for effective cold start, and (4) reinforcement learning for enhanced generalisation.
We instantiate this framework in a web agent based on the ReAct format, WebDancer.
Empirical evaluations on the challenging GAIA and WebWalkerQA benchmarks demonstrate the strong performance of WebDancer, achieving considerable results and highlighting the efficacy of our training paradigm. 
Further analysis of agent training provides valuable insights and actionable, systematic pathways for developing more capable agentic models.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces WebDancer, an end-to-end framework for training autonomous web agents to perform complex, multi-step information seeking tasks. The proposed system integrates four stages: (1) QA data generation via CRAWLQA and E2HQA, (2) trajectory sampling using ReAct-style reasoning (Short/Long CoT), (3) supervised fine-tuning (SFT) for cold-start behavior, and (4) reinforcement learning (RL) using the DAPO algorithm for robustness and generalization. WebDancer is evaluated on GAIA, WebWalkerQA, and BrowseComp, achieving strong results that rival or surpass some existing open-source baselines.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- The paper presents a systematic framework that combines high-quality data synthesis and filtering, SFT, and RL, all within a tool-augmented ReAct agent loop.\n- The paper provides detailed studies on the impact of SFT, RL, CoT length, data rejection, and training steps, which reveal rich insights about agent behavior under different setting.\n- CRAWLQA and E2HQA introduce diversity and complexity, and rejection sampling and multi-stage filtering ensures high data quality.\n\nWeaknesses:\n- The proposed method heavily relies on data distillation from advanced LLMs. Both QA pairs and trajectories are distilled from powerful LLMs (GPT-4o, QwQ), which is a standard practice and limits the novelty.\n- Tasks are limited to single-turn QA. The agent solves short-answer queries without any multi-turn interaction, clarification, or open-ended task structure.\n- Only two tools (search and click) are used, which limits the tool diversity and task complexity.\n\nQUESTIONS:\n- Line 35 says \"unlock the autonomous multi-turn information36 seeking agency\", but the interactions seem to be single-turn only, with potentially multiple steps.\n- The data quality filtering does not present sufficient details. For example, the heuristic of hallucination detection is based on number of actions, could you explain why?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-23"}

comments: SUMMARY:
In this paper, the authors propose WebDancer, an end-to-end framework for building autonomous web agents for multi-step information seeking. The proposed approach consists of four main stages: (1) data synthesis to generate a large corpus of challenging question-answer pairs; (2) high-quality trajectory sampling to generate both short and long CoT reasoning steps; (3) a supervised fine-tuning (SFT) stage to provide the agent with a "cold start" for instruction and tool-use following; and (4) an on-policy RL stage to enhance the agent's decision-making and generalization capabilities. In experiments, extensive results demonstrate the effectiveness of WebDancer on the GAIA and WebWalkerQA benchmarks.

STRENGTHS AND WEAKNESSES:
**Strengths:**

1. This paper addresses a challenging problem of constructing a autonomous  information-seeking agents via a end-to-end paradigm. 
1. The proposed four-stage pipeline is comprehensive and systematic. The hybrid training method is practical for achieving efficient agent learning.
1. The experiment design is solid and results show that the proposed method is superior.
1. The paper is well-written, clearly structured and easy to follow.

**Weaknesses:**
1. The training of proposed method is resource-extensive.
1. There may be reward hacking issue, since the evaluation relies on an LLM-as-Judge (72B) for both the RL reward signal and the final performance metric.

QUESTIONS:
1. Why do not compare other different RL training methods, such as GRPO?
1. Do the Long-CoT trajectories exhibit more complex planning, self-correction, or deeper exploration, beyond just being longer? Could you provide some cases on Short-CoT and Long-CoT for better understanding and comparison?

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 4
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-01

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a pipeline for training information seeking agents that can answer closed-form answers through multi-step interactions with the web environment.\nSpecifically, the proposed pipeline involves four steps: 1) collecting high-quality QA pairs for training; 2) sample trajectories using both LLMs and LRMs for QA pairs; 3) performing SFT on the filtered trajectories; and 4) performing RL on the QA pairs for enhanced generalization.\nAlong with the pipeline, this paper collects two large-scale QA pairs for training, i.e., CrawlQA (60K) and E2HQA (40K).\nThis paper applies the training pipeline to fine-tune Qwen-2.5-7B, Qwen-2.5-32B, and QwQ-32B. Experimental results demonstrate the effectiveness of the training pipeline on both GAIA and WebWalkerQA.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths:\n1. The curated QA pairs, in particular CrawlQA, can be a valuable source of verifiable rewards for training information seeking agents.\n2. The experiments are comprehensive. Necessary ablations are conducted to justify the designs.\n\n### Weaknesses:\n1. Important details regarding the dataset construction are not revealed in this paper, which might make it difficult to assess the quality of the work. This is important because the techniques used for training in this paper are mostly existing ones, while the dataset is the crucial differentiating point.\n2. Related to the first point, the space arrangement of this paper could be improved. Currently, it spend majority of the space on the training techniques, while things like ReAct, GRPO, or the reward design with both format and correctness are something people already familiar with.\n3. (Minor) This paper may benefit from polished writing. Some sections appear to be verbose (e.g., Sec 2). In addition, there are several minor issues: a) Figure 1 seems not referred in the main text? b) line 274 is struggle transferable, grammar error\n\nQUESTIONS:\n1. Do you plan to release CrawlQA and E2HQA to the public, including open-source the code for data curation?\n2. In CrawlQA, how to make sure the navigation is meaningful and does not have a corresponding shortcut? For example, you may get the answer page after 10 clicks, but there might be a shortcut to directly go there or that page might be indexed by google search already.\n3. In addition to the dataset and trained models, what major insights do you think are important to convey to readers?\n\nClarification:\na. Is it possible that for the same question, there's both a short CoT version trajectory and a long CoT version? If so, would combining them for training result in some conflicts?\nb. Based on appendix D, only less than 10% questions were retained during RFT?\n\nLIMITATIONS:\n1. The most important use case of information seeking agents typically extend beyond closed-form QA. For example, there can be more open-ended needs such as finding a products with certain criteria or even generating a report for a science question. It might be an interesting future direction. That said, I agree this is not within the scope of this paper. So this is just a general suggestion.\n2. This paper may reveal more details regarding the constructed dataset for people to better assess it.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents WebDancer, a systematic and scalable framework for training web-based autonomous information-seeking agents. The framework follows a four-stage pipeline: (1) constructing diverse and challenging QA datasets (CRAWLQA and E2HQA), (2) generating agentic trajectories using rejection sampling under both short and long Chain-of-Thought (CoT) prompting strategies, (3) supervised fine-tuning (SFT) to provide a strong initialization, and (4) reinforcement learning (RL) with the DAPO algorithm to improve generalization. WebDancer is instantiated as a ReAct-style web agent, and evaluated on two challenging benchmarks (GAIA and WebWalkerQA), demonstrating strong performance compared to both open-source and closed-source agentic frameworks.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\n1. The direction of building autonomous information-seeking agents is both timely and important for advancing agentic AI capabilities. \n \n2. The proposed four-stage framework (data construction, trajectory sampling, SFT, RL) is clear and well-structured.\n\n**Weaknesses**\n\n1. The paper would benefit from improved exposition. Some essential context is missing, such as a basic introduction to the unique characteristics and challenges of web agent tasks for readers not already familiar with this area. Adding more illustrative case studies could further enhance readability and practical relevance.  \n2. The experiments focus primarily on the Qwen series as backbone models. It remains unclear how generalizable the proposed pipeline is to other LLM backbones.  \n3. The agent’s action space is restricted to “search” and “click/visit,” with no support for more complex interactions such as scrolling or system key events. This restriction should be discussed more explicitly.  \n4. The RL reward relies on LLM-as-Judge, but the paper does not provide an explicit evaluation of this judge’s accuracy. Since the judge could become a bottleneck, an analysis or discussion of its reliability is essential.  \n5. Could the authors provide more detailed case studies highlighting which types of web tasks or interaction patterns benefit the most from the SFT or RL stages? Conversely, what are the common failure modes or limitations in the current system, and how might they be addressed?\n\nQUESTIONS:\nSee Weaknesses part\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-06"}