Explore to Evolve: Scaling Evolved Aggregation Logic via Proactive
  Online Exploration for Deep Research Agents
Authors: name: Rui Wang; name: Ce Zhang; name: Jun-Yu Ma; name: Jianshu Zhang; name: Hongru Wang; name: Yi Chen; name: Boyang Xue; name: Tianqing Fang; name: Zhisong Zhang; name: Hongming Zhang; name: Haitao Mi; name: Dong Yu; name: Kam-Fai Wong
Published: 2025-10-16 08:37:42

Abstract: Deep research web agents not only retrieve information from diverse sources
such as web environments, files, and multimodal inputs, but more importantly,
they need to rigorously analyze and aggregate knowledge for insightful
research. However, existing open-source deep research agents predominantly
focus on enhancing information-seeking capabilities of web agents to locate
specific information, while overlooking the essential need for information
aggregation, which would limit their ability to support in-depth research. We
propose an Explore to Evolve paradigm to scalably construct verifiable training
data for web agents. Begins with proactive online exploration, an agent sources
grounded information by exploring the real web. Using the collected evidence,
the agent then self-evolves an aggregation program by selecting, composing, and
refining operations from 12 high-level logical types to synthesize a verifiable
QA pair. This evolution from high-level guidance to concrete operations allowed
us to scalably produce WebAggregatorQA, a dataset of 10K samples across 50K
websites and 11 domains. Based on an open-source agent framework, SmolAgents,
we collect supervised fine-tuning trajectories to develop a series of
foundation models, WebAggregator. WebAggregator-8B matches the performance of
GPT-4.1, while the 32B variant surpasses GPT-4.1 by more than 10% on GAIA-text
and closely approaches Claude-3.7-sonnet. Moreover, given the limited
availability of benchmarks that evaluate web agents' information aggregation
abilities, we construct a human-annotated evaluation split of WebAggregatorQA
as a challenging test set. On this benchmark, Claude-3.7-sonnet only achieves
28%, and GPT-4.1 scores 25.8%. Even when agents manage to retrieve all
references, they still struggle on WebAggregatorQA, highlighting the need to
strengthen the information aggregation capabilities of web agent foundations.