WebWatcher: Breaking New Frontiers of Vision-Language Deep Research Agent
Authors: Xinyu Geng
Peng Xia
Zhen Zhang
Xinyu Wang
Qiuchen Wang
Ruixue Ding
Chenxi Wang
Jialong Wu
Kuan Li
Yida Zhao
Huifeng Yin
Yong Jiang
Pengjun Xie
Fei Huang
Huaxiu Yao
Yi R. Fung
Jingren Zhou
Venue: ICLR 2026 Poster (2026)
Keywords: Multimodal Agent
Web Agent
Deep Research
Visual Question Answering (VQA)
Tool-augmented Reasoning
Multimodal Information-Seeking Benchmark
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: We present WebWatcher, a multimodal web agent that learns from synthetic trajectories and reinforcement learning to achieve state-of-the-art performance in complex information-seeking tasks requiring joint visual and textual reasoning.

Abstract: Web agents such as deep research have demonstrated superhuman cognitive abilities, capable of solving highly challenging information-seeking problems. However,
most research remains largely text-centric, overlooking visual information in the
real world. This makes multimodal deep research highly challenging, as such
agents require much stronger perceptual, logical, and knowledge-based reasoning
abilities, as well as proficiency in more sophisticated tools. To address this limitation, we introduce WebWatcher, a multimodal agent for deep research with joint
reasoning ability across both visual and textual modalities. It uses high-quality
synthetic trajectories for efficient cold start training, utilizes various tools for deep
reasoning, and further enhances generalization through reinforcement learning. To
better evaluate the capabilities of multimodal agents, we propose BrowseComp-VL,
a benchmark with the style of BrowseComp that requires complex information
retrieval involving both visual and textual information. Experimental results show
that WebWatcher outperforms the prompt-based workflow and open-source agents
on HLE and BrowseComp-VL, and demonstrates its perception, multimodal reasoning, and searching capabilities across the other three benchmarks, respectively.

Published as a conference paper at ICLR 2026
WEB BREAKING NEW FRONTIERS OF VISION-LANGUAGE DEEP RESEARCH AGENT
Xinyu Geng" 2 Peng Xia Zhen Zhang Xinyu Wang 20- Qiuchen Wang", Ruixue Ding", Chenxi Jialong Wu", Kuan Li2, Yida Zhao?, Huifeng Yin', Yong Jiang" 203
Pengjun Xie², Fei Huang", Huaxiu Yao², Yi R. Fung itd Jingren Zhou?
'The Hong Kong University of Science and Technology, Tongyi Lab, Alibaba Group, 'The University of North Carolina at Chapel Hill (ttomas.. wxy, yongjiiaan angg. - .yj } Ba ibbab--in: com, yrm.yruuuuuu...kk
ABSTRACT
Web agents such as deep research have demonstrated superhuman cognitive abili- ties, capable of solving highly challenging information-seekingg problems. However, most research remains largely text-centric, overlooking visual information in the real world. This makes multimodal deep research highly challenging, as such agents require much stronger perceptual, logical, and knowledge-band -bsee reasoning abilities, as well as proficiency more sophisticated tools. To address this limita- tion, we introduce Wathatrher, multimodal agent for deep research with joint reasoning ability across both visual and textual modalities. uses high-quality synthetic trajectories for efficient cold start training, utilizes various tools for deep reasoning, and further enhances generalization through reinforcement learning. To better evaluate the capabilities of multimodal agents, propose BrowseComp-Vi a benchmark with the style of BrowseComp that requires complex information retrieval involving both visual and textual information. Experimental results show that Web Watcher outperforms the prompt- prompt-based based workflow and open-source agents on HLE and BrowseComp- VL. and demonstrates its perception, multimodal rea- soning, and searching capabilities across the other three benchmarks, respectively.
INTRODUCTION
Deep research agents represent new frontier in artificial intelligence, where Large Language Models (LLMs) go beyond static prompts to plan multi-step tasks (OpenAI, 2025a Google 2024 Perplexity| 2025). Open-source agents have demonstrated superhuman abilities to interact with intricate information (Li et al., 2025c al Wu et al. 2025a). achieving strong results on difficult benchmarks such as BrowseComp (Wei et al. 2025a) and Humanity's Last Exam (HLE) (Phan et al.) 2025). Yet, progress far is mostly text- focused overlooking the visual information common in real-world tasks. Many scenarios such as interpreting scientific diagrams (Hu et al.) 2024). analyzing charts <Wang et al. 2024). navigating visual web interfaces (Hong et al. 2024). demand joint vision- language reasoning (Dong et al. 2025). ). Despite initial progress by proprietary agents, multimodal deep research remains largely unexplored, with few agents tackling high-difficllty Vision-Language (VL) tasks (Xu & Peng 2025
Hu et al. (005)..
The key challenge is that current multimodal deep research agents rely on template-driven pipelines limited to specific scenarios, lacking the flexible reasoning and planning ability with versatile tools. On the one hand, many VL Agents focus on image-based reasoning with visual tools, such as Optical Character Recognition (OCR), bounding box extraction, image cropping, and visual annotation (Zhao al. 2025 [Su et al., 2025a (y. While useful for perception, agents with these tools fall short in combining visual reasoning with deep textual understanding and cross-modal inference, struggling to handle high-difficulty tasks. As shown in Fig. [ VL agent fails to give solution to this complex case
"Equal contribution.
Code is available at htttps://giiaii wtw.thc.bbcoc.....ccmmmo http://aaaaaaa.////eeeesssssaa/a
bAgent /webbat.hhe

---

Published as a conference paper at ICLR 2026
loois polican.
query- polizan Wikipedi"
Thought
Prolyce
Textume Anailiid
Wels Search
Tark
Response:
X
Excessive edge/texture inference, amplifying downstream
Question: On the Wikipedia page for the animal in the provided how revisions from before 2020 'visual tags? Answer:
VL Agent
tool errors.
ImageSearch
ImageVist
PageView
EvidenceSum
OCR
Count
Thought
Alavts
Wilipedi Hanory
Doubee
Response: 11
pafto
dheck
VisualDiff
WebSearch
CrossValidate
WebWatcher VL DeepResearch
>- Genuinc multi-step reasoning with appropriate tool use
and cross-validatio.
maybe- a proguin ora seagult
Queryt "penguin", "seegut"
Co
Thought
Tect Fiteence
tra Search
Wes Soarch
Inference
Response:
or
X
>> Poor lool usage, reliance on shallow problem decomposition, and lack of deep reasoning capabilities.
Search Agent
Figure I: Comparison of VL reasoning agents. WetWateher resolves the GAIA case that defeats both vision-only reasoning and search- based agents, demonstrating the strength multi-tool integration and in-depth reasoning generalization.
from GAIA (Mialon et al. 2023). ). which requires reasoning beyond perception. On the other hand,
search-only agents have very limited sobling- scope (Wu et al.] 20256). Although retrieval augmented reasoning can handle many knowledge-base questions. it often fails when answers are implicit, require interactions, or demand additional computation (Shen et al. 2024 Gu et al. 2025). As illustrated in Fig. solving this case requires not only searching, but also tools to click through relevant links and browse the resulting webpage gather necessary information.
To address this gap, agents for VL deep research require not only strong reasoning abilities across both textual and visual information, but also effective use of multiple external tools. Thus, we introduce Web Watcher, a VL web agent with deep research capability.
To develop strong reasoning across text and vision, is essential to construct data that combine high-quality visual content with complex reasoning. However, current visual question answering (VQA) datasets mainly focus on visual perception with inference in two hops, lacking planning
complexity and reasoning depth needed for deep research agent (Chen et al. 20244 Li et al. (025d).
Thus, we introduce a pipeline to generate training data that benefits in-depth, multi-step reasoning and strategic planning, encouraging agents to synthesize information across both modalities. We first harvest real-world knowledge via random walks over diverse web sources to create challenging question answering (QA) pairs (Wu et al. 2025a Li al. 2025a). To further raise the complexity, we mask key entities in questions with generic descriptions, forcing models to infer relationships from context. These enriched QA pairs are then converted into multimodal VQA items through flexible pipeline compatible with most existing QA datasets, enabling large-scale multimodal dataset expansion. Finally, multi-stage filtering process ensures data quality and clarity.
Moreover, to enable effective use of multiple external tools, we integrate Web Image Search, Web Text Search, Webpage Visit, Code Interpreter and internal OCR. However, a key challenge is constructing tool-use trajectories with high quality. Recent agents generate rigid, template- based trajectories with limited adaptability across tasks (Rose et al. 2023 Bi et al. 2025), because coordinate tools with distinct input-output formats and reasoning roles. To address this, we design an automated pipeline that builds trajectories from action-observation sequences via prompting. Unlike hand-crafted traces, our trajectories are grounded in actual tool-use behavior and reflect procedural decision
difficult to
decision-makee making aligned with complex reasoning. Then we finetune the agent on synthesized trajectories and further optimize it via reinforcement learning algorithm, GRPO (Shao et al. 2024).
Finally, we introduce BrowseComp- VL, a challenging VQA benchmark that extends BrowseC- omp (Wei et al.) 2025b) into visual domain. Queries are long, entity obfuuscdb,,,,eed,, and BrowseComp- style, L.e., queries that are deliberately underspecified and difficult even for humans, requiring retrieval of scattered information without clear guidance and integration of fragmented clues. It demands
2

---

Published as a conference paper at ICLR 2026
Q image, signlficant ay station in northern Incian sta bey ajor rail network, The headquarte of the national rail service prov capital city, and its governing Board oversees entre rail network, When the construction of the wallway ne extending from this station nearby town A: 1980
Q: Which in pame sceniano absent from Skill Points Stenarios in Challenge Road? Fierze Battle
Figure 2: Domain distribution of Level and Level 2 questions across five major fields, with examples illustrating explicit multi-hop reasoning (Level 1) and obfuscated, harder synthesis (Level 2).
agents to perform not just perception but also cross-modal reasoning, thorough information-ssekinng and high-level planning of tools such as web search, image retrieval, and webpage browsing.
WetWateeer achieves strong performance on several bigh-difficulty benchmarks, including HLE, LiveVQA, BrowseComp- VL, and MMSearch. It consistently outperforms existing open-source multimodal research agents and proprietary systems four reasoning benchmarks, and show competitive performance on the pereeption benchmark, SimpleVQA.
DATA PREPARATION
Prompting L.MS to generate VQA questions directly from images is common practice, but it often yields shallow, single-hop queries. Existing datasets rarely combine rich textual information with complex reasoning. In this section, we address these gaps by constructing dataset with multi-hop, knowledge-intensive queries and images from real-world web environments.
2.1
DATA OVERVIEW
Our dataset designed for multimodal deep research agents. Each example comprises a factual image, associated question requiring cross-modal reasoning, corresponding answer, and auxiliary metadata about underlying entities and relations. As shown in Fig. 2 the dataset cover 5 major domains (Entertainment, Humanities, Technology, Natural Science, and Other), comprising 17 fine- grained subfields, which is detailed in Appendix C Additionally, we define two difficulty levels to encourage diverse reasoning ability:
Level I: Questions require multi-hop reasoning but still reference explicit entities. While the answers can be obtained through iterative retrieval steps, the reasoning process remains non-trivial due the need for integrating information across multiple sources.
Level 2: Questions are constructed with obfuscated entities and attributes. For example, concrete dates are replaced with vague periods, names are masked, and quantitative properties are fuzzed up. This design introduces uncertainty, requiring the agent to plan, compare, and synthesize information rather than perform direct retrieval.
We split the dataset into a training set and a benchmark BrowseComp-Ve VL, evaluating agents under highly difficult multimodal research scenarios. It consists of 199 VQA pairs in Level and 200 VQA pairs in Level 2, verified by three PhD-level human experts to ensure high quality and reliability. Specifically, annotators select more information-dens: images.
2.2 CONSTRUCTION OF VQA PAIRS
We first construct diverse textual QA pairs emphasizing multi-hop and knowledge-inteniive reasoning, then ground them in relevant images form VQA tasks. This pipeline produces multimodal data that preserve both visual richness and reasoning complexity.
2.2.1 QA PAIRS GENERATION
Level 1. Inspired by CRAWL-QA from WebDancer (Wu et al. 2025a). we enhance reasoning depth and breadth collecting root URLs from authoritative sources such arXiv, GitHub, and Wikipedia,
3

---

Published as a conference paper at ICLR 2026
Graph
Example: Born the piace shown in the image, he regularly appeared on the Show: birthplace isted post office and the Hawkesbury Agr cuitura Collgge (est. 1891). What the common name of the snake named after him?
Captain
heritage-
Level 1
Image S Search II
Selector III
Examiner IV
T
Level 2
Original question:
"What spedies of snake was nemed Roy Kinghom?
Examples preminent zoo ogist. born the place mage to reverend's family, became known for his wildlife education the snake named
Answer:
1950s relevision. What the after him?
Australin scrub python
Figure 3: Pipeline for generating data, where multi-hop VQA pairs are built from hyperlink graphs, grounded with web images, filtered by selector- examiner checks, and transformed into Level (explicit) and Level 2 (fuzzed) questions for multimodal reasoning.
and recursively traversing their hyperlinks to mimic human browsing. GPT-4o (OpenAl 2024) is then used to synthesize question-answw answer pairs from the aggregated content.
Level Following WebSailor (Li et al., 2025a). we construct queries with fuzzed entities by replacing precise references with partial or ambiguous descriptions. Answers cannot be retrieved through direct lookup. require contextual reasoning and synthesis across modalities. We design two-stage generation framework for Level consisting of: (1) Nodes Selecting: Starting from an initial Wikipedia page, we prompt GPT-4o to generate base QA pair, using the page title as the root entity node Brant- We then expand hyperlink graph by recursively traversing outgoing links to form a tree of depth d and branching factor k, yielding (I=+1 1)/ 1) nodes. In practice, we set d = 3 and & = 3 for sufficient coverage. To create diverse reasoning paths, sample subgraphs of N entities, each defining a path from Broot to target entity B. These subgraphs provide the basis for generating distinct multi-hop QA pairs (2) Query Generating and Entity Masking: Based on each subgraph and its ground truth, we first prompt GPT-4o to generate standard question that explicitly references entities and relations along the reasoning path. We then create a fuzzed version by replacing key references with partial or ambiguous descriptions, preventing agents from taking shortcuts simple string matching in search results and forcing it to perform cross-modal reasoning.
2.2.2 QA-TO-VQA CONVERSION
Visual Context Construction. To ensure reliable visual grounding, we discard trivial or overly ambiguous target entities B (e.g., temporal references or domain-external concepts), which lack sufficient visual grounding. For each retained entity B, we retrieve a set of web images I I(ß) = 1 18.1. ..... H via Google SerpApi (Google 2025). where K = 2 in our implementation. These images I(B) serve as the visual grounding to construct multimodal reasoning examples. Unlike synthetic or composited images prevalent in existing VQA benchmarks, our images are strictly authentie, minimizing noise and maximizing relevance for real-world tasks.
Question Transformation. To build mmage-grounder VQA pairs from each textual QA (gr, a), we use GPT-4o for prompt-based rewriting. The target entity B in ge is masked with visual reference token Tvis (e.g., "this entity," "the object in the image"), producing a transformed VQA query g. Simultancously, we create an image query string Sing (B) to guide filtering of I(B). Each retained image I E I is paired with (q, a), textual QA yields K multimodal examples, giving K VQA items from n questions.
2.3 QUALITY CONTROL
To ensure high-quality VQA samples, we employ two-stage filtering pipeline:
4

---

Published as a conference paper at ICLR 2026
(1) Selector: We first discard cases where the transformed VQA query q identical to qt, where the entity name B and its aliases appear in q, indicating failed masking and question rewriting. Then, GPT-4o evaluates each image g E Z() against both (ge, a) and (q, a), scoring contextual alignment, semantic fit, and visual reasoning plausibility. Cases with low scores are removed.
(2) Examiner: For each retained image-query pair (Simg (B), I(B)) GPT-4o attempts to answer
Sing (B) using only visual content and associated captions. Failure answer accurately iindicates improper visual context, and such cases are discarded. Captions are included to reduce false negatives from missing world knowledge.
The input images undergo automated filtering, and when curating the benchmark annotators further select information-dest part of the complex information contained in the image. Therefore, agents require non-trivial visual reasoning to solve these tasks and also need to integrate information from multiple sources.
images. The entity that may be used solve the problem is often only one
3 TRAJECTORY GENERATION AND POST- -OST-INANNING
We use supervised fine-tuning (SFT) as cold start to teach Web tool-augmented reasoning. based on high-quality trajectories generated by an automated pipeline. Reinforcement learning then applied to further optimize tool use and decision-making.
3.1 AUTOMATED GENERATION REASONING TRAJECTORIES
3.1.1 MULTIMODAL TOOLS
We equip WehWatcher with five tools: (1) Web Image Search (Google SerpApi (Google 2025 for retrieving relevant images with captions and URLs; (2) Web Text Search for open-domain information seeking; (3) Visit (Jina (Jina.ai 2025) for navigating specific URLs and summarizing pages according to the agent's goal; (4) Code Interpreter for symbolic computation and numerical
reasoning (Cheng al., 2024); and (5) OCR, an internal tool invoked via prompt and SFT data to
extract text from input images (Huang al. 2025). Full details are provided in Appendix F
3.1.2 AUTOMATED TRAJECTORY ANNOTATION
Given a VQA instance (L, a) from BrowseComp-Vi use GPT-4o to construct tool-use tra- jectories simulating step-by-step human reasoning. Following ReAct (Yao et al., 2023), each trajectory comprises multiple think-act-observe cycles. At each step t, the model takes as input the accumulated context history and generates: (1) a Thought: the agent's interme- diate reasoning or plan, enclosed in <think>, ... </think>; vocation wrapped in <tool www.a.al........................ <answer>..
.hbinkk..../thhnk::: (2) an Action: the tool in-
<tool_caal>__... wttplaaalll..................... </tool call> and the final answer enclosed in Sarsseee....aaswii.... </answer>; (3) an Observation: the returned result from the environment, within
</tool response> tags.
The action space T consists of discrete tool-use actions f, enabling the agent retrieve information, navigate webpages, or perform computations. The Finish action signals task completion by returning a final answer and ending the episode. trajectory of length L is defined Eq.
T {Lo, oo), 01). (L1,
(1)
where each observation o, reflects the environment feedback after executing action t, E T. Each trajectory provides a content-grounded demonstration of planning and tool selection.
3.1.3 TRAJECTORY FILTERING AND QUALITY ASSURANCE
To ensure robust and instructive supervision, apply three-stage trajectory selection:
(1) Final Answer Matching: We retain trajectories 7 where the final answer matches the ground truth a, ensuring that the entire sequence of tool-use steps leads to correct and complete solution.
(2) Step-by-Step Consistency Check: We use GPT-4o to verify the logical consistency of each intermediate step in Trajectories with hallucinated content, contradictions, unjustified tool calls
5

---

Published as a conference paper at ICLR 2026
discarded. This avoids the common failure mode where correct answers reached by lucky guessing rather than meaningful tool use. (3) Minimum Tool Usage Requirement: We remove with fewer than three tool calls. This ensures that training data reflects substantive, process-driven tool interactions and reasoning rather than one-step completions.
3.2 SUPERVISED
AS COLD START
After filtering, the dataset contains high-quality tool-use trajectories. At each step of trajectory i, WebWatcher is trained to predict the correct action ( given the image 1(0) question g(), and previous actions and observations
(1 ) SFT maximizes the log-likelihood of t/ in Eq
K
log (o( 1 90 (0) 2
max
where 0 are the model parameters. This cold-start stage teaches the agent to use tools effectively and follow structured multi-step reasoning.
3.3 REINFORCEMENT LEARNING
With SFT providing cold-start initialization, apply Group-Relative Policy Optimization (GRPO) (Guo et al., 2025) to refine decision-making for complex tasks. For VQA query the current policy Ne generates group G Th....y TK of K complete trajectories, each with return Rg The group-relative advantage is defined as Eq.
K
(r)()(1))) R())
1
Σ
R
(3)
K
which normalizes rewards within the group and eliminates the reliance separate value function. The GRPO objective is defined as clipped surrogate loss in Eq. 4
CORPO = Egge [min (pt) Are(( clip (p1,,1c,,+ +e)
e, +
Ara(T()))]
- DKL (Tell
(4)
where p(0) ))( Teaa (ri0) is the importance sampling ratio between the current and previous policy,
(r)) is the group-relative advantage defined in Eq. 3 is the clipping threshold, and DKL denotes the Kullback- Leibler divergence between successive policies. The coefficient 3 controls the strength of the KL penalty. This objective promotes stable updates while encouraging exploration of trajectories with higher relative return. Each trajectory
(4n, oo),..., (4,, or.) first receives a binary format score ry [0, 1], which is 1 if all tool calls follow the schema. An LLM grader then provides a semantic accuracy score ra E [0, 1] by comparing the final answer with the ground truth. The total reward is defined in Eq.
R wry (1 w)r..
(5)
with w 0.2 to prioritize task completion while maintaining structured tool use. Since R is given only the episode end, the group-relative ranking in Eq. 3 enables effective credit assignment without relying on per-step shaping. Rollouts are collected in groups of N 16 to ensure diversity for computing relative advantages while maintaining computational efficiency during training.
EXPERIMENTS
4.1
SETUP
Training Data Construction. Our training data come from three sources: (1) The training set of BrowseComp-Ve VL, (2) loig-tail QA pairs converted to VQA, and (3) hard VQA samples. Firstly, we construct the BrowseComp- VL training set with 110,000 Level-1 and 70,000 Level-2 QA pairs. After VQA conversion and filtering, 60,000 Level-1 and 40,000 Level-2 high-quality examples are retained. Secondly, the long-tail QA data sampled from training instances sharing similar distribution

---

Published as a conference paper at ICLR 2026
Table 1: Main results on HLE. All accuracy average accuracy score of three inference runs across different subtopics.
reported percentages. Avg signifies the
Humanity Exam (HLE-VL)
Backbone
Bio.
Chem.
CS/AI
Engineer.
Human.
Math
Physics
Other
Avg.
Direct Inference
GPT-4o
13.8
0.0
0.0
3.9
12.0
6.8
7.1
7.0
6.5
Gemini-2.5-faash
12.1
1.6
0.0
4.0
0.0
14.3
0.0
4,9
Claude-3.7-Somme
1.7
4.8
0.0
0.0
0.0
0.0
2.8
Qwen-2.5-VL-77
3,4
3.2
7.1
4.0
2.3
7.1
2.6
Qwcn-2.5-VL-32B
3.4
6.5
0.0
3.9
8.0
2.3
7.1
3.7
Qwen-2.5-VL-72B
3.4
8.0
0.0
5.9
8.0
0.0
0.0
4.9
Prompt Workflow
GPT-4o
9.8
24.1
4.8
0.0
2.0
4.0
9.1
14.3
12.3
Gemini-2.5-0aam
25.9
3.2
7.1
0.0
8.0
9.1
3.5
14.0
11.4
Claude-3.7 3onnnncct
4.3
5.2
4.8
0.0
0.0
9.1
14.3
3.5
Qwen- VV-B
4.3
6.9
3.2
0.0
4.0
4.5
7.1
5.3
Qwen-2.5-VL-3225
5.2
10.3
3.2
0.0
0.0
4.5
7.1
8.8
Qwen-2.5-VLL-2h1
2.0
Reasoning Model
o4-mtnin
12.1
23.7
17.7
0.0
5.8
0.0
33.3
21.4
16.0
Gemini-2.5-Pro
23.7
17.7
13.3
11.5
8.0
13.3
14.3
15.5
15.8
Open Source Agents
OmniSearch (GPT-4o)
15.5
8.2
0.0
8.0
21.4
9.3
WebWatcher-7B
18.6
6.5
6.7
4.0
6.7
7.1
17.2
10.6
WebWatcher-32B
33.8
9.7
0.0
5.8
8.0
8.9
14.3
13.8
13.6
with VIM, which transformed into 4,000 VQA examples. Thirdly, hard samples
from InfoSeek (Chen et al. (023). VQAv2.0 (Goyal et al. 2017). Logie (Xiao et al. (024). and Encyclopedic VQA (Mensink al., 2023). with (Huang et al. 2025) added activate OCR. Rejection sampling ensures difficulty. After trajectory generation and filtering, we obtain 8,000 high-quality tool-use trajectories for SFT. with 2,000 additional samples reserved for GRPO, The final ratio of data sources is 5:3:2 for BrowseComp- VVL long-tail VQA, and hard VQA data, respectively.
Models and Benchmarks. We conduct post-training on Qwen2.5-VL-7B and Qwen2.5-VL-322 (Bai et al. 2025). and evaluate on five challenging benchmarks: BrowseComp- VL, HLE (Phan et al. Live (Fu et al. 2025). Simple VQA (Cheng al. 2025). and MMSearch (Jiang et
2025
2024
Benchmark details are provided in Appendix E.4
Baselines. We compare with the following paradigms: (1) Direct Inference: Models directly generate
answers using internal knowledge. We evaluate GPT--o (OpenAl 2024), Gemini-2.5-flash (Deep: (Anthropic 2025), and Qwen-2.5-V family (7B/32B/72B). (2) Prompt Workflow: To ensure fairness, models use prompt-driven workflows have the same tools as employed in WetWatcher. (3) Reasoning Baselines: OmniSearch (Li et al. 2025d), search-oriented open-sourced agent based GPT-4o, and multi-step reasoning models Seemini2..5PPP (DeepMind 2025) and 04-mini (OenAAl 2025b) with prompt-driven workflows.
Mind 2025), Claude-3.7-Sone
Metric and Hyper-parameters We repeatedly generate for i: times get pass@k (Chen al. 2021), with the temperature of 0.6 and top-p of 0.95. The pass score is computed as: pass Pt, where p; is the binary correctness of the i-th prediction. Answer correctness judged using the LLM-as-Judges approach (Liu et al. 2024), where the prompt is detailed in Appendix E.5
4.2 MAIN RESULTS
As shown in Tab. for Humanity's Last Exam (HLE), models with direct inference perform worst, with average accuracy scores below 10, revealing the limits of vanilla MLLMs in complex, knowledge- intensive VQA. RAG-based methods show moderate improvements, particularly in Chemistry. Com- pared with reasoning-oricen oriented models, although our WebWatcher-322 attains a slightly lower overall average accuracy of 13.6%, is parameter-efffcient,, requiring only 32B parameters compared the

---

Published as a conference paper at ICLR 2026
Table 2: Main results four challenging benchmarks. All accuracy reported percentages. Avg signifies the average score of three inference across two difficult levels.
BC-VL
Backbone
LiveVQA
MMSearch
Simple V
Levell
Level2
Avg.
Direct Inference
GPT-4o
6.4
4.0
5.5
29.7
18.7
47.0
Gemini-2.5-faan
11.6
9.6
35.0
19.6
63.0
Moauee-3.7-Somee
8.8
4.0
7.1
23.7
12.3
42.7
Qwen-2.5-VL VL-BB
0.8
0.0
0.5
22.7
4.09
30.7
2wen-2.5-VL-32B
3.2
1.0
2.4
26.3
7.60
40.7
Qwen-2.5-VI--728
9.2
3.0
7.1
30.3
11.7
51.3
Prompt Workflow
GPT-4o
16.8
7.0
13.4
34.0
24.1
61.6
Semini-225-ffast
15.2
9.0
13.0
41.3
43.9
68.6
Claude-3.7-Somet
13.9
6.0
11.2
30.3
32.7
59.3
03
26.7
23.0
24.9
50.0
54.3
70.3
GPT-4.1
15.6
6.0
10.8
32.3
26.0
60.3
Owen-2.5-VL--7B
3.6
1.0
2.7
21.7
9.94
21.0
Owen-2.5-VL--22B
9.4
3.0
7.2
30.5
17.5
44.6
Qwen-2.5-VI--72
14.4
6.0
11.5
35.7
29.2
58.6
Agents
OmniSearch (GPT-4o)
19.7
10.0
16.3
40.9
49.7
63.0
Web Watcherrr
23.6
17.0
21.2
51.2
49.1
54.3
WebWatcher-522
28.4
25.0
27.0
58.7
55.3
59.0
Table 3: Performance across different tool call counts.
Tool Call
Best Pass
Average@3
Best Pass @
8.79
7.98
14.24
9.70
8.69
15.76
=2 10.61
9.90
18.18
22
11.21
10.10
17.58
=3
10.61
9.90
19.09
23
12.12
10.61
19.09
4
10.00
8.99
18.79
24
11.82
9.70
17.27
5
9.70
9.49
16.58
25
10.61
10.10
18.18
=6
8.79
8.33
15.76
26
9.70
8.99
15.58
proprietary large-scale models. Moreover, WebWatcher-32B demonstrates clear strengths in specific domains: it achieves a top score of 33.8% in Biology, with competitive performance in Mathematics and Humanities. As for other benchmarks, Tab. 2 shows that while direct inference with strong MLLMs remains limited and prompt workflows bring moderate gains, Web consistently outperforms both. WebWatcher-322 achieves state-of-the-an results on Live VQA (58.7%) and MM- Search (5,,3)) while also delivering competitive performance on BrowseComp- VL and Simple VQA. Notably, BrowseComp-V VI. requires multi-page browsing and fine-grained visual grounding, which causes the scores of most baselines are below 20%, but our dynamic tool-use loop proves effective. Even in Simple performs well with score of 59.0%. These results demonstrate that WebWatcher excels in not only knowledge-intent intteniiiie tasks but also visual reasoning, underscoring its broad applicability.
which emphasizes visual reasoning over external knowledge, Web
8

---

Published as a conference paper at ICLR 2026
Table 4: Human Agent on BrowseComp-VL. T, and T, average minutes of solvable cases and unsolveable cases, respectively. Iu indicates abandoned cases without final answer.
Level
Acc (%)
Tu
Tg T./m)
T.(/m)
L1 (Human)
33.2
42
35
59
L. (Human)
18.0
144
109
116
WebWaether-3229
28.4
0.3
2.5
L2 3atchereee)332B)
25.0
3
0.8
2.5
4.3 ABLATION STUDY
We conduct ablation study of selecting the number of tool calls. For each tool call setting, randomly select 8, 8,000 trajectories for SFT and tested them on HLE. Tab, shows that the performance best when the number of tool calls is 2 3.
4.4 HUMAN BASELINE
We report human baselines on Level and Level 2 of BrowseComp- VL in Tab. 4 Five annotators answer benchmark questions, each completing 40 questions, ensuring that each question is answered by at least two annotators. To ensure objectivity, annotators do not answer questions they previously annotated and use the same tools WebWatcher. Following BrowseComp, annotators may abandon a question after 100 minutes without reaching a solution.
4.5 ANALYSIS
Number of Tool Calls. Fig. Ashows how tool usage adapts to benchmark demands. On HLE, which requires multimodal search, computation, and reasoning, usage is balanced across Web Text Search, Web Image Search, and Code Interpreter, with Visit handling webpage navigation. BrowseComp- VL and MMSearch focus much more on information seeking and reasoning, thus retrieval dominates. Web Text Search accounts for 62% of calls, while other tools play minor roles. For Simple the focus shifts back to visual content, with Web Image Search making up one-third more of calls, while Text Search and Visit act as auxiliaries. Across all settings, the Code Interpreter is used only when actual computation is required, confirming that WebWatcher is cost and context aware. Overall, the distribution of tool usage mirrors benchmark demands, underscoring Watcherss flexibility in composing tool chains rather than over-relying on any single tool.
Cold Start for RL Training. We believe cold start crucial for our vision-langusee agent, as tasks demand robust multi-hop reasoning with continuous tool interaction. To verify this, we compare the same RL algorithm under two initializations: (1) Instruct: warm-started only with public instruction- following data; (2) Cold-start: an extra SFT stage on trajectories that explicitly demonstrate tool use and
visual reasoning. As shown in Fig. 5 The Instruct initialization stalls near zero because frequent tool-call format errors wipe rewards and the strict Qwen-2.5-72B grader suppresses partial answers. In contrast, cold-start SFT lifts initial scores. Subsequent GRPO trends diverge: HLE and BC-VL oscillate without improvement, while LiveVQA rises steadily, maintaining a 0.06-0.18 margin over Instruct. Injecting CoT chains from a larger reasoner made the small model unstable, format violations, repetitions, and context overflow spike, confirming that reasoning traces cannot replace an SFT cold start under our strict RL setting.
Pass @ Analysis on HLE. Fig. shows the Pass@k curve of WebWatcher on HLE for k ranging from to 32. With single attempt (k = 1), WebWatcherachieves 13.6% pass rate. As k increases, performance rises steeply at first: three roll-outs reach 20.3%, showing that few diverse trajectories yield large gains. Accuracy keeps improving to 35.7% at k 16 and 41.9% at k 32, nearly quadrupling single-shot inference and surpassing reasoning models like Gemini-2.5-Pro and 04- mini. The smooth curve indicates that our de-correlated sampling avoids redundant rollouts and captures complementary knowledge. Since marginal gains taper after 16, practitioners can cap at 8-16 roll-outs for 2-3x boost at moderate cost. Overall, the Pass @k profile demonstrates the scalability of the agentic paradigm. Systematic exploration of reasoning paths yields consistent, robust improvements challenging multimodal benchmark.

---

Published as a conference paper at ICLR 2026
HLE
BC-VL
MMsearch
60%
62.1%
46.0%
30%-
50%-
40%
@ercettage e
25%
20%-
15%-
25.6%
25.1%
14..6%
Percentage
30%-
23.3%
Percentags
80%
20%
27.4%
25.0%
10%-
14.0%
5%
10%-
0.5%
10%
1.6%
0%
Test search
Code
Vist
Rat search
image search
Cade
Vist
Test search
Image search
Code
Visk
Image
Live
SimpleVQA
Across 5 Datasets
38.2%
40%
39.4%
50%-
Percentage (%6)
31.8%
Percentage (%)
30%%
27.1%
30%
Percentage
20%
24.1%
20%
19.0%
10%
10%
8.5%
2.9%
0%
Text search
Image search
Code
Visk
0%
Bot search
Image search
visk
Search
Image search
Code
Volk
Figure 4: The percentage of external tool calls in the four benchmarks. The height each bar denotes the fraction of total calls made that tool within the corresponding benchmark. Internal OCR is not included since only external tools are counted here.
0.15
start
Cold-ttart
Instruct
Instruct
0.6
0.3
0.10
Scaee
0.2
0.05
0.1
0.00
12
32
0.0g
20
20
40
Training Steps
Training Steps
Training
Score on HLE Benchmark
Score on BC-VL Benchmark
Score LiveVQA Benchmark
Figure Performance comparison using cold start in RL training on three benchmarks.
5 CONCLUSION AND FUTURE WORK
Pass@k Performance Analysis
In this work, we explore the underdeveloped landscape of multimodal deep research by designing unified framework, WebWatcher, that combines complex vi- sion-language reasoning and multi-tool interaction. We present BrowseComp- VL, challenging dataset tailored for in-depth multimodal reasoning and strategic planning, and introduce scalable pipeline transform complex tex- tual QA examples into VQA. To equip agents with robust tool-use capabilities, we develop an automated trajectory generation pipeline grounded in action-observation traces, followed by cold start and GRPO. WebWatcher achieves strong performance across multiple high- figgh-ifficulty bench- marks (e.g., HLE, VQA. BrowseComp- VL, and MM- Search), outperforming both open-source and proprietary research agents, while also delivering competitive results on the perception-orienter benchmark Sim- VQA. Overall, WebWatcher establishes a strong foundation for future multimodal deep research agents capable solving real-world problems with autonomy, ficxibility, and deep reasoning.
0.4-
Pass Rate
0.1-
0.0
4
8
12
16
20
24
28
32
Number of Generations (k)
Figure 6: Pass @k performance of Web- Watcher on HLE Benchmark.
In future work, we plan to systematize the data- fleeellyweel to progressively replace GPT-4o with open-source models while preserving trajectory quality. Initially, GPT-4o annotations are used to train smaller, specialized model on limited trajectories. This model then performs additional annotations at much lower cost. Through iterative co-anottation and fine-tune soe-ttunng, the model becomes more proficient, handling increasingly complex tasks.
10

---

Published as a conference paper at ICLR 2026
ETHICS STATEMENT
All authors have read and comply with the ICLR Code of Ethics. This work does not involve human subjects or sensitive data, and we are unaware of any potential misuse, harm, or bias. No conflicts of interest or compromising sponsorships exist.
REPRODUCIBILITY STATEMENT
Details the proposed methodology, training procedure, hyperparameters, and evaluation metrics are provided Sec. 4. and Appendix B
We also include full description the datasets in Sec. and and Additionally, codes and our constructed benchmark, BrowseComp-VL. are all publicly
Appendix E
available Dattps://qithaa gent/ Bent/ebbRatche.
WebbA http:///ww...p.aa/a///////////a/
ACKNOWLEDGEMENTS
We would like to thank the Tongyi Lab team at Alibaba Group for their valuable support and helpful discussions. This work is partially supported by the HKUST University Grant WEB26EG02 and Grant 2025YFE000050..
REFERENCES
Anthropic. Claude 3.7 sonnet and claude code, 2025. URL.atps::/wwwaatttpppii.
news/o /c/c/lle.. claaudee-- 3-dd-3317 -ssonett/]
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, al. Qwen2. 5-vl technical report. arXiv preprint 13X12:2502.13923, 2025.
Jing Bi, Susan Liang, Xiaofei Zhou, Pinxin Liu, Junjia Guo, Yunlong Tang, Luchuan Song, Chao Huang, Guangyu Sun, Jinxi He, et al. Why reasoning matters? a survey of advancements in multimodal reasoning (vl). arXiv preprint arXiv:2504.0315/. 2025.
James Burgess, Jeffrey Nirschl, Laura Bravo-Sánchez, Alejandro Lozano, Sanket Rajan Gupte, Jesus G Galaz-Montoya, Yuhui Zhang, Yuchang Su, Disha Bhowmik, Zachary Coman, al. Microvqa: A multimodal reasoning benchmark for microscopy- based scientific research. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 19552-19564, 2025.
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374, 2021.
Qiguang Chen, Libo Qin, Jin Zhang, Zhi Chen, Xiao Xu, and Wanxiang Che. m² cot: A novel bench- mark for multi-domain multi-step multi-modal chain-of oftthug--t arXiv preprint arXiv:2405. 164473, 2024.
Yang Chen, Hexiang Hu, Yi Luan, Haitian Sun, Soravit Changpinyo, Alan Ritter, and Ming-Wei Chang. Can pre-traimed vision and language models answer visual information-seeking questions? arXiv preprint 1127: 2023.
Zhuo Chen, Xinyu Wang. Yong Jiang, Zhen Zhang, Xinyu Geng, Pengjun Xie, Fei Huang, and Kewei Tu. Detecting knowledge boundary of vision large language models by sampling-ben based inference. arXiv preprini 2rX2 2025.
Xianfu Cheng, Wei Zhang, Shiwei Zhang, Jian Yang, Xiangyuan Guan, Xianjie Wu, Xiang Li, Ge Zhang. Jiaheng Liu, Yuying Mai, et al. Simplevqa: Multimodal factuality evaluation for multimodal large language models. arXiv preprint a502v:2502. 13059, 2025.

---

Published as a conference paper at ICLR 2026
Yao Cheng, Jianfeng Chen, Jie Chen, Li Chen, Liyu Chen, Wentao Chen, Zhengyu Chen, Shijie Geng, Aoyan Li, Bo Li, et al. Fullstack bench: Evaluating ilms as full stack coders. arXiv preprint arXiv:2412.00535, 2024.
Google DeepMind. Gemini 2.5, 2025. URL http:///ww////aaaoooo////a/aaoa/
@ dedeppiidd//eiie -ece---mmeeeeee-iiiiiiee- -ow.deee---iiiiiiin---ooeeeeeeeo wpw.tau---tts-----aaa- 2025/
Yuhao Dong, Zuyan Liu, Hai-Long Sun, Jingkang Yang. Winston Hu, Yongming Rao, and Ziwei Liu. Insight-v: Exploring long-chain visual reasoning with multimodal large language models. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 9062-9072, 2025,
Mingyang Fu, Yuyang Peng, Benlin Liu, Yao Wan, and Dongping Chen. Livevqa: Live visual knowledge seeking. arXiv preprint arXiv:2504.05288, 2025.
Google. Try deep research and our new experimental model in gemini, your assistant, 2024. URL google gemini deep- researc
h/
Google. Serpapi, 2025. URL httttt:://teppaaaa/aa
Yash Goyal, Tejas Khot, Douglas Summers-Stay Dhruv Batra, and Devi Parikh. Making the v in vqa matter: Elevating the role of image understanding in visual question answering. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. (904-6913, 2017.
Jiawei Gu, Ziting Xian, Yuanzhen Xie, Ye Liu, Enjie Liu, Ruichao Zhong, Mochi Gao, Yunzhi Tan, Bo Hu, and Zang Li. Toward structured knowledge reasoning: Contrastive eetrievl--aggmented generation on experience. arXiv preprint arXiv:2506. 00842, 2025.
Daya Guo, Dejian Yang. Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12944 2025.
Wenyi Hong, Weihan Wang, Qingsong Lv, Jiazheng Xu, Wenmeng Yu, Junhui Ji, Yan Wang, Zihan Wang. Yuxiao Dong, Ming Ding, et al. Cogagent: visual language model for gui agents. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pp. 14281-14290, 2024.
Anwen Hu, Yaya Shi, Haiyang Xu, Jiabo Ye, Qinghao Ye, Ming Yan, Chenliang Li, Qi Qian, Ji Zhang, and Fei Huang. :apeuppw::: Scientific diagram analysis with the multimodal large language model. In Proceedings of the 32nd ACM International Conference on Multimedia, pp. 6929-6933 6938, 2024.
Mengkang Hu, Yuhang Zhou, Wendong Fan, Yuzhou Nic, Bowei Xia, Tao Sun, Ziyu Ye, Zhaoxuan Jin, Yingru Li, Qiguang Chen, et al. Owl: Optimized workforce learning for general multi-agent assistance in real-world task automation. arXiv preprint 2XXv:2555.233885 2025.
Mingxin Huang, Yongxin Shi, Dezhi Peng, Songxuan Lai, Zecheng Xie, and Lianwen Jin. Ocr- reasoning benchmark: Unveiling the true capabilities of milms complex text-rich image reason- ing. arXiv preprint arXiv:2505. 2025.
Dongzhi Jiang, Renrui Zhang, Ziyu Guo, Yanmin Wu, Jiayi Lei, Pengshuo Qiu, Pan Lu, Zehui Chen, Chaoyou Fu, Guanglu Song, et al. Mmsearch: Benchmarking the potential of large models as multi-modal search engines. arXiv preprint arXiv:2409.12959, 2024.
Jina.ai, Jina, 2025. URL http:: //ii:a//jiaaaaae ai/
Kuan Li, Zhongwang Zhang, Huifeng Yin, Liwen Zhang, Litu Ou, Jialong Wu, Wenbiao Yin, Baixuan Li, Zhengwei Tao, Xinyu Wang, et al. Websailor: Navigating super-human reasoning for web agent. arXiv preprint arXiv:2507.0259) 2025a.
Shilong Li. Xingyuan Bu, Wenjie Wang. Jiaheng Liu, Jun Dong, Haoyang He. Hao Lu, Haozhe Zhang, Chenchen Jing, Zhen Li, et al. Mmn-broweccomp A comprehensive benchmark for multimodal browsing agents. arXiv preprint arXiv:2508 /3186, 2025b.
12

---

Published as a conference paper at ICLR 2026
Xiaoxi Li, Jiajie Jin, Guanting Dong, Hongjin Qian, Yutao Zhu, Yongkang Wu, Ji-Rong Wen, and Zhicheng Dou. Webthinker: Empowering large reasoning models with deep research capability. arXiv preprint arXiv:2504.21776, 2025c.
Yangning Li, Yinghui Li, Xinyu Wang, Yong Jiang, Zhen Zhang, Xinran Zheng, Hui Wang, Hai- Tao Zheng, Philip S. Yu, Fei Huang, and Jingren Zhou. Benchmarking multimodal retrieval augmented generation with dynamic vqa dataset and self-adaptive planning agent, 2025d. URL https: org. abs 241 1, 02937
Yuxuan Liu, Tianchi Yang, Shaohan Huang, Zihan Zhang. Haizhen Huang. Furu Wei, Weiwei Deng, Feng Sun, and Qi Zhang. Calibrating llm-based evaluator. In Nicoletta Calzolari, Min-Yen Kan, Véronique Hoste, Alessandro Lenci, Sakriani Sakti, and Nianwen Xue (eds.), Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation, LREC/COLING 2024, 20-25 May, 2024, Torino, Italy, pp. 2638-2656. ELRA and ICCL, 2024.
main. 237
Thomas Mensink, Jasper Uijlings, Lluis Castrejon, Arushi Goel, Felipe Cadar, Howard Zhou, Fei Sha, André Araujo, and Vittorio Ferrari. Encyclopedic vqa: Visual questions about detailed properties of fine tiee-graieed categories. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 3113-3124, 2023.
Grégoire Mialon, Clémentine Fourrier, Thomas Wolf, Yann LeCun, and Thomas Scialom. Gaia: benchmark for general ai assistants. In The Twelfth International Conference on Learning Representations, 2023.
OpenAl. Hello GPT-4o, 2024. URL httess//pppnaa..ccmmnneee//lll http:exx/hlllooot----00
OpenAl. Deep research system card, 2025a. URL [RKl:ttps///dh..enna.coo///eee resea reh- rch-system- cdrd.
OpenAl. Introducing openai 03 and o4-mini, 2025b. URL htt.ttp::://111aaennnggiiii
io/posts/2024-11 reward hacking
Perplexity. Perplexity deep research, 2025. URL http:p:/wwwww.ppppeett.....la.
Long Phan, Alice Gatti, Ziwen Han, Nathaniel Li, Josephina Hu, Hugh Zhang, Chen Bo Calvin Zhang, Mohamed Shaaban, John Ling, Sean Shi, et al. Humanity's 'ss last exam. arXiv preprint arXiv:2501.1422 14249, 2025.
Daniel Rose, Vaishnavi Himakunthala, Andy Ouyang, Ryan He, Alex Mei, Yujie Lu, Michael Saxon, Chinmay Sonar, Diba Mirza, and William Yang Wang. Visual chain of thought: bridging logical gaps with multimodal infillings. arXiv preprint #xXiv23355002377 2023.
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, al. Deepseekmath: Pushing the limits of mathemat- ical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024.
Xiaoyu Shen, Rexhina Blloshmi, Dawei Zhu, Jiahuan Pei, and Wei Zhang, Assessing" implicit" retrieval robustness of large language models. arXiv preprint arXiv:2406. 18134, 2024.
Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. Hybridflow: A flexible and efficient rlhf framework. In Proceedings of the Twentieth European Conference on Computer Systems, pp. 1279-1297, 2025.
Huatong Song. Jinhao Jiang, Yingqian Min, Jie Chen, Zhipeng Chen, Wayne Xin Zhao, Lei Fang, and Ji-Rong Wen. R1-searches Incentivizing the search capability in llms via reinforcement learning. arXiv preprint arXiv:2503.05592, 2025.
Zhaochen Su, Linjie Li, Mingyang Song, Yunzhuo Hao, Zhengyuan Yang. Jun Zhang, Guanjie Chen, Jiawei Gu, Juntao Li, Xiaoye Qu, et al. Openthinking: Learning to think with images via visual tool reinforcement learning. arXiv preprint arXiv:2505.08617 2025a.
13

---

Published as a conference paper at ICLR 2026
Zhaochen Su, Peng Xia, Hangyu Guo, Zhenhua Liu, Yan Ma, Xiaoye Qu, Jiaqi Liu. Yanshu Li, Kaide Zeng, Zhengyuan Yang, et al. Thinking with images for multimodal reasoning: Foundations, methods, and future frontiers. arXiv preprint arXiv:2506.2399 233918, 2025b.
Zhaochen Su, Jincheng Gao, Hangyu Guo, Zhenhua Liu, Lueyang Zhang, Xinyu Geng, Shijue Huang, Peng Xia, Guanyu Jiang, Cheng Wang, et al. Agentvista: Evaluating multimodal agents in ultra-challenging realistic visual scenarios. arXiv preprint arXiv:2602.23166 2026.
Jiabin Tang, Tianyu Fan, and Chao Huang. Autoagent: A fully-automated and zero-code framework for ulm agents. arXiv preprint arXiv:2502.05957, 2025.
Xijia Tao, Yihua Teng, Xinxing Su, Xinyu Fu, Jihao Wu, Chaofan Tao, Ziru Liu, Haoli Bai, Rui Liu, and Lingpeng Kong. Mmsearch-plus: A simple yet challenging benchmark for multimodal browsing agents. arXiv preprint arXiv:2508.21475 2025a.
Zhengwei Tao, Jialong Wu, Wenbiao Yin, Junkai Zhang, Baixuan Li, Haiyang Shen, Kuan Li, Liwen Zhang, Xinyu Wang, Yong Jiang, et al. Webshaper: Agentically data synthesizing via cnformation-secking formalization. arXiv preprint arXiv:2507. 15061, 2025b.
Zirui Wang, Mengzhou Xia, Luxi He, Howard Chen, Yitao Liu, Richard Zhu, Kaiqu Liang, Xindi Wu, Haotian Liu, Sadhika Malladi, et al. Charxiv: Charting gaps in realistic chart understanding in multimodal 1lms. Advances in Neural Information Processing Systems, 1111111356936693 2024.
Jason Wei, Zhiqing Sun, Spencer Papay, Scott McKinney, Jeffrey Han, Isa Fulford, Hyung Won Chung, Alex Tachard Passos, William Fedus, and Amelia Glaese Browsecomp: simple yet challenging benchmark for browsing agents. arXiv preprint arXiv:25044. 12516, 2025a.
Jason Wei, Zhiqing Sun, Spencer Papay, Scott McKinney, Jeffrey Han, Isa Fulford, Hyung Won Chung, Alex Tachard Passos, William Fedus, and Amelia Glaese. Browsecomp: A simple yet challenging benchmark for browsing agents. arXiv preprint arXiv:2504. 12516, 2025b.
Jialong Wu, Baixuan Li, Runnan Fang, Wenbiao Yin, Liwen Zhang, Zhengwei Tao, Dingchu Zhang, Zekun Xi, Gang Fu, Yong Jiang, et al. Webdancer: Towards autonomous information seeking agency, arXiv preprint arXiv:2505.22645 2025a.
Jinming Wu, Zihao Deng, Wei Li, Yiding Liu, Bo You, Bo Li, Zejun Ma, and Ziwei Liu. Mmsearch-rl: Incentivizing Imms to search. arXiv preprint arXiv:2506.2006 2025b.
Yijia Xiao, Edward Sun, Tianyu Liu, and Wei Wang. Logicvista: Multimodal Ilm logical reasoning benchmark in visual contexts. arXiv preprint
2024.
Renjun Xu and Jingwen Peng. A comprehensive survey of deep research: Systems, methodologies, and applications. arXiv preprint (rXXi:25006125944 2025.
Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. In International Conference Learning Representations (ICLR), 2023.
Kaining Ying, Fanqing Meng, Jin Wang. Zhiqian Li, Han Lin, Yue Yang, Hao Zhang, Wenbo Zhang, Yuqi Lin, Shuo Liu, et al. Mmt-bench: comprehensive multimodal benchmark for evaluating large vision-language models towards multitask agi. arXiv preprint arXiv:2404.16000 2024.
Xiang Yue, Tianyu Zheng, Yuansheng Ni, Yubo Wang, Kai Zhang, Shengbang Tong, Yuxuan Sun, Botao Yu, Ge Zhang, Huan Sun. et al. Mmmu- pro: A more robust multi disti-diseipline multimodal understanding benchmark. arXiv preprint 2xXx:22409.02813 2024.
Weichen Zhang, Zile Zhou, Zhiheng Zheng, Chen Gao, Jinqiang Cui, Yong Li, Xinlei Chen, and Xiao-Ping Zhang. Open3dvqu: A benchmark for comprehensive spatial reasoning with multimodal large language model in open space. arXiv preprint arXiv:2503.1 /094, 2025.
Shitian Zhao, Haoquan Zhang, Shaoheng Lin, Ming Li, Qilong Wu, Kaipeng Zhang, and Chen Wei. Pyvision: Agentic vision with dynamic tooling. arXiv preprint 0070092507.07998, 2025.
14

---

Published as a conference paper at ICLR 2026
Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo, Zhangchi Feng, and Yongqiang Ma. Llamafactory: Unified efficient fine-tuning of 100+ language models. In Pro- ceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations), Bangkok, Thailand, 2024. Association for Computational Linguistics. URL
abs 2403. 13372
He Zhu, Tianrui Qin, King Zhu, Heyuan Huang. Yeyi Guan, Jinxiang Xia, Yi Yao, Hanhao Li, Ningning Wang. Pai Liu, et al. Oagents: An empirical study of building effective agents. arXiv preprint 2506. 15741, 2025.
15

---

Published as a conference paper at ICLR 2026
A
SYMBOL DEFINITION
Table Mathematical symbols and their meanings
Symbol
Mcaning
Broot
The page title, serving the root entity node Depth of the tree
d
Branching factor of the tree
N
Number of entities contained in each subgraph
B
Newly selected target entity node
B
Retained target entity after filtering Set of web images retrieved for entity B Number of images per entity set Original text question and answer pair Visual reference token replacing the entity mention Transformed VQA query
I(B)
K-2
(qc, a)
rvis
q
Sing (B)
Image query string for filtering images of B Number of original textual QA pairs
(I,4,a)
A VQA instance: image, question, and answer from BrowseComp- VL tool-use trajectory: sequence think-act-observe cycles Discrete action space of tool-use actions
T
t
A specific tool-use action in T
T
Set of all tool-use actions Length of the trajectory
L L.
Length of the i-th trajectory
t
Action at iteration i in the trajectory (f, E T)
oi
Observation returned after executing t;
0
Model parameters
G
[TI, ( r
Group of K complete trajectories generated by the policy Scalar return (total reward) assigned to trajectory (x)
R
(r)
Group-relative advantage for trajectory (f)
GRPO clipping threshold
p(i)
Importance sampling ratio
mo
Current policy parameterized by 0 Previous policy parameterized by oold Coefficient for the KL penalty in GRPO objective Binary format score for conformance of tool calls (E {0, 1))
Toad
rr
Semantic accuracy score from LLM grader (E [0, 1)) Weight balancing format and accuracy scores in total reward
w = 0.2
B RELATED WORK
Deep Research Agents The notion of "deep research" agents-systems that autonomously search, read, reason, and synthesize knowledge from the open web-has evolved rapidly in the last years. Proprietary solutions such as DeepResearch (OpenAT 2025a), Gemini Deep Research (Google 2024). now exhibit near-expert performance across fact fant-finding, argumentative writing, and exploratory analysis, yet the secrecy of their model architectures and data curation pipelines inhibits rigorous ablation and reproducibility.
initiatives have attempted to close this gap: WebDancer (Wu et al. 2025a) introduces curriculum-driven SFT over ReAct traces. WebThinkes (Li et al. (025c) then augments SFT with policy-gradice refinement and RI R1-Sercherr (Song et al. 2025) leverages self-play to learn tree-structured exploration policies. WebSailor (Li et al.) 2025a) focuses on uncertainty reduction which used structured task obfuscation, RFT cold-start, and the DUPO algorithm. Recently, WebShaper (Tao et al.) (2225)) proposes formalization-diin driven data-synthesis pipeline by introducing Knowledge Projections and an agentic Expander. Nevertheless, nearly all leading deep deep-research agents are still text-bound (Tang al. 2025 Zhu et al. 2025). Integrating vision, layout, and
16

---

Published as a conference paper at ICLR 2026
eross-modal grounding is therefore not minor tweak but the necessary next leap, multimodality will fundamentally redefine what deep research can achieve.
Multimodal VQA Benchmark Most existing VQA benchmarks primarily assess single-step perception or shallow retrieval, with limited support for integrated multimodal reasoning and plan- ning (Chen et al. 2024 Li et al., 2025d). Datasets such OK-VQA and A-OKVQA typically emphasize static knowledge grounding and heuristic answer prediction without requiring complex reasoning. Recent efforts have begun to expand the evaluation space. MMT- Bench offers large-scale coverage of coaniing-oriente tasks across multiple domains, yet its calliple-choiee format restricts the assessment of procedural reasoning and rich textual outputs (Ying et al.
2024 VQA
and Open3DVQA explore domain-specific and spatial reasoning, but are constrained by limited scale, manual curation, or lack of complex planning structure (Burgess al.
2025 Zhang al.
2025). Similarly, Dyn-Vo VQA introduces adaptive query tasks but remains narrow in multimodal scope and size et al. 2025d Chen al. 2025). While datasets such as MMMU-Pro (Yue et al. MMSS)),MSSarcc (Tao et al. (025a). MM- Mr--Browseooyy (Li et al. (025b) and Agent Vista (Su et al. 2026)further explore performance limitations of current MILMS on domain-specific and difficult information seeking tasks, few existing benchmark comprehensively supports multi-step reasoning, cross-modal integration, large scale, and full automation with rigorous quality control. To address these gaps, introduce large-scale, automated VQA benchmark designed to advance planning-orientedn multi-hop, and context-rich multimodal reasoning. Our dataset enables scalable evaluation of MILMs' capabilities in goal-directed, flexible agent behavior, setting new standard for future research in this area.
C CATEGORIES OF BROWSECOMP-VE
BrowseComp-Ve VL covers five major categories: (1) Natural and Formal Sciences (Chemistry, Physics, Biology & Medicine, Mathematics), (2) Engineering and Computer Science (Engineering, Computer Science & AI), (3) Social Sciences and Humanities (Social Science, History, Politics, Geography), (4) Arts, Entertainment, and Sports (Art, Music, TV, Games, Sports), and (5) Other. which includes emerging or uncategorized topics. This taxonomy adapted from HLE (Phan et al. 2025) and BrowseComp WWii et al. 2025b).
D PROMPTS
D.1 QA TO VQA
When processing QA data at level I and the image entity is unknown, use the following prompt:
Prompt:
VQA Generation for Level 1
Task: Extract the main keyword from the input question and use the extracted keyword to reconstruct the vqa_query, replacing the keyword with a pronoun.
Rules:
1. keyword: <The keyword extracted from the question>>>
noun or an adjective noun
The main subject of the question
2. qra__uery: <The query with the keyword information removed>
Rewrite the original question by replacing the keyword with pronoun.
Replace the keyword with "this a noun that summarizes the keyword."
Examples:
Input:
question: "What is the tallest building in New York?"
17

---

Published as a conference paper at ICLR 2026
Output:
("keyword"; "New York", quezy"; "What is the tallest building in this city?"}
Input:
question: "What is Qin Shi Huang's surnam
Output:
("keyword"; "Oin Shi Huang", "vqa_query": "Who is this?")
Now, you need to process the input:
question: {query)
Output Format:
("keyword": "", "vqa_query": "")
When processing QA data at level 2 and the image entity known as the node the graph, use the following prompt:
Prompt: :
VOA Generation for Level
Task:
Given question with entity that has been obfuscated, and specific entity span that is obfuscated, rewrite only that obfuscated entity portion by adding a transition phrase such "in the image," so that becomes VQA (Visual Question Answering) question. Do not rewrite other obfuscated entity segments.
Examples:
Input:
question: "In recent House election in a northeastern state in the United States, Republican candidate
upset victory in traditionally Democratic-leaning district by a 7% margin. In another district in the same state, Republican also by significant majority, maintaining the party's hold in the region. A political figure previously involved in local governance had served on a community board in major city in that state. What position did he hold the community board?"
entity: "In a recent House election in northeastern state in the United States"
Output:
["vqa_query" "In the recent House election in this state in the image, a Republican candidate won an upset victory in a traditionally Democratic-Leaning district by 78 margin. another district in the same state, a Republican also won by significant majority, maintaining the party's s hold in the region. A political fiqure previously involved in local governance served on a community board in major city in that state. what position did he hold on the community board?")
Input:
question: "A man born in the mid-18th century on a large plantation in southeastern Virginia county served in a major legislative body representing his county from the early 1760s to the late 1770s. In the early 1760s, with whom did he serve in that legislative body?"
entity: "A born in the mid-18th century large plantation in southeastern Virginia county"
Output:
18

---

Published as a conference paper at ICLR 2026
"vqa__qury"" "In the image, this man born in the mid-18th century on a large plantation in a southeastern Virginia county served in major legislative body representing his county from the early 1760s to the late 1770s. In the early 1760s, with whom did he serve in that legislative body?")
Here is the input you need to process:
question: fquery)
image entity: limage_entiyg
Output format:
("vqa_query": "")
Prompt:
Image Query Generation
You are a rewriting system for a VQA chatbot. You will receive the following information:
question
image_entity
gold_query
Task:
Based on the following rules, generate an image query for the image-related question:
Rules:
1. Compare the "question" and "gold _ueeruuuerrrr to identify information that is included in "gold_query" but missing from "question." " Based on this missing information, generate an image query called "image_quere where the answer should be "im- age_entity."
2. The composition rules for "image heamage_query":
If "question" contains "this"/"thttt"thhe" followed by noun, "Who is that noun?" or "What is that noun?"
If "this" or "that" is not followed by a noun, the "image_query" should be "What is this?"
If there are no obvious demonstrative pronouns like "this" or "that," use "What
is this?"
3. Output only one image query in the format of string, without any irrelevant content.
Examples:
Input:
question: When did Epic Gaming first release this?
gold gold_query: When did Epic Gaming first release Minecraft?
image_entity. Minecraft
Output:
"What is this?"
Input:
question: Who is the current CTO of this organization?
gold_query: Who the CTO of Alibaba Cloud?
image_entity: Alibaba Cloud
Output:
"What is this organization?"
Input:
19

---

Published as a conference paper at ICLR 2026
question: How much greater is this figure than 4?
gold query: How much greater is 3 than 4?
image_ entity:
image_entity:
Output: "What is this?"
Prompt:
Judge the Quality of VQA
Task:
You are given list of candidate images and two versions of question-answer pair: the original QA and rewritten VQA question. Your job to look at each image and decide whether it's relevant enough to keep.
How to decide:
Context match: Does the image clearly show the scene objects mentioned in either the original or the VQA question?
Answer fit: Could someone use this image to arrive at the given answer?
Reasoning check: Is it plausible to reason from the image to the answer for the VQA question?
After scoring each image from 0 (irrelevant) to (perfect match), drop any image scored below 0.5 and keep the rest their original order.
Input variables:
images: a list of images to evaluate
original_ga: the original question and answer
vqa_queryy the rewritten VQA question and the same answer
Output format:
Beellttere__imaess" [list of images you kept }
Prompt:
Answer Image Query
Judge
Determine whether the input image and its description match the given keyword. If they match, output 1; otherwise, output 0. Use strict criteria: the image and description must clearly represent the keyword to output 1. Output only 0 or 1, with no additional text.
Task:
Keyword: (keyword)
Image Caption: (image_caption)
D.2 TRAJECTORY GENERATION
We use this prompt when obtaining the trajectory with correct responses using reject sampling.
Prompt: Evaluati ion of Reject Sampl ing
Task: Please evaluate whether the model's answer is correct based the given question, standard answer, and preeeepredicted answer. Rate the result as:
A: [Correct]
[nnorrecc]]
C: [Not Attempted]
20

---

Published as a conference paper at ICLR 2026
Return only the letter "A", "B", "C", with additional text.
Examples of [Correct] responses:
:uestion: What are the names of Barack Obama's children?
Standard Answer: Malia Obana and Sasha Obama
Model Prediction 1: Malia Obama and Sasha Obama
Model Prediction 2: Malia and Sasha
Model Prediction 3: Most people would say it's s Malia and Sasha,
but I'm not sure.
Model Predi ct ctiin 4: Barack Obama has two daughters, named Malia Ann and Natasha Marian, but usually referred to Malia Obama and Sasha Obama. Malia was born on July 4, 1998, and Sasha was born on June 10, 2001.
Examples of [Incorrect] responses:
Question: What are Barack Obama's s children's names? Standard Answer: Malia Obama and Sasha Obama
Model Prediction 1: Malia
Model Prediction 2: Malia, Sasha, and Susan
Model Prediction 3: Barack Obama has children
Model Prediction 4: I think it's Malia and Jackie.
Model Prediction 5: Although
don't t know their exact names, can say Barack Obama has three children.
Model Prediction 6: You might refer to Betsy and Olivia...
Examples of [Not Attempted] responses:
Question: What are Barack Obama's s children's names?
Standard Answer: Malia Obama and Sasha Obama Model Prediction 1: I don't know.
Model Prediction 2: need more context about which Obama you refer to.
Model Prediction 3: Without checking online, can't t answer this question.
Model Prediction 4: Barack Obama has two children. I know one is named Malia, but I'm not sure of the other's name.
Notes:
Numerical answers: near matches (e.g. "3518" vs. "3518. are [Correct]; wrong numbers are [Incorrect]; vague ranges [Not Attempted).
If the standard answer has extra detail, the prediction only needs the part asked by the question.
If missing details can be inferred from the question, treat as [Correct].
Now evaluate:
Question: (question)
Standard Answer: (target)
Predicted Answer: (predicted_answer)
Return: A, C
This prompt is used to guide GPT-4o in verifying whether a given trajectory is logically sound and consistent with the task requirements.
21

---

Published as a conference paper at ICLR 2026
P
Tool Call Rationality Evaluation
Role: You professional Al interaction quality assessor. Your core task is to analyze dialogue snippets between user and an Al assistant that include a stool_calle tag followed by <think> tag.
Task: Judge whether the tool call <<too__cal>>) is reasonable according to the three criteria defined below. "Reasonable" means the call is necessary, directly driven by the user's query, efficient, precise, non-redundant, and conforms to specifications, Also evaluate the thought process (<think>) for logical accuracy and to ensure no guessing fabrication.
Evaluation Criteria:
Information Non-Redundancy: The requested information or action in the tool call is not already provided or easily derivable from prior dialogue, the user's current question, or the assistant's previous answers. Check: Is there any overlap or repeated request?
2. Goal Alignment:
The tool call's purpose and expected result directly serve the user's explicit intent or core need in this turn. Check: Does advance the user's main objective?
3. Logical Reasoning and Accuracy: The assistant's thought process shows clear, correct logic and reliable grounding- unfounded guesses fabrications. The <think> section should be concise. Check: Is the reasoning well well-structured and evidence-based?
Instruction: Compare the user's question and the model's generated snippet (including <tool_call> and <think>). If all criteria are met, output:
Otherwise (any criterion unmet or room for improvement), output:
User Question: [query) }
Model Generation: (model_gen)
E EXPERIMENTAL DETAILS
TOOL DEFINITION
In the ReAct framework, each tool is defined through a structured prompt that specifies both its callable format and its semantic capability. This design ensures the language model can reason about tool usage and invoke them appropriately within the <tool loll__lal>l__ during interaction. Cache is used to prevent search repeatably. In detail, our tools are defined as follows.
"tcol_cal>>> block
Tool: Code Interpreter
Description: Executes Python code for calculation, data analysis, content extraction.
Arguments:
code (string): The Python code execute. (Required)
22

---

Published as a conference paper at ICLR 2026
Tool: Web Text Search
Description: Retrieves the top 10 text excerpts from Google's text search engine using one or more search queries.
Arguments:
queries (array of strings): List of search queries. (Required)
Tool:
Web
Image
Search
Retrieves top images and descriptions from Google's image search using given image URL. Should only be used once.
Description:
Arguments:
image_urls
(array of strings): List of image URLs to search with. (Required)
Tool: Visit
Description: Visits given webpage and returns summary based on a specified goal. Arguments:
url (string): The target webpage URL.. (Required)
goal (string): The goal or information the agent seeks from the webpage. (Re- quired)
Tool: OCR
Extracts text content from given image using an internal OCR engine. Useful for reading embedded visual information such charts, screenshots, scanned documents. Arguments:
Description:
Amage_url (string): The URL of the image to extract text from. (Required)
E.2 REACT TRAJECTORIES
Our ReAct framework is implemented through Qwen-Agent I and we limit the number of tool calls to no more than 15. A complete trajectory follows the format below:
Case Trajectory
<think> thinking process here </think> <tool_call>
"name": "tool name here", "arguments": "parameter name here": parameter value here, "another parameter name here": another parameter value here,
</tool_call <all>>
stool_response.
tool. _oo_rresonne here
</tool_responser
(more thinking processes, tool calls and tool responses here)
<think> thinking process here </think>
<answer> answer here </answer>
E.3 TRAINING DETAILS
We Llama-Facton Zheng et al. 2024 for SFT and Verl (Sheng al. 2025) for RL training. For SFT, we use a batch size of 32, learning rate 5e-6 with minimum le-10, warmup plus cosine decay schedule, and a weight decay of 0.1. For RL training, the rollout number group is 8, the
https:: whw.. wwm./wwnnnnneeoooooooomooommewee M/ Qwwen-nennnn
23

---

Published as a conference paper at ICLR 2026
temperature is 1.0, top_p= p= 1.0, the batch size is 128, the mini batch size is 32, and the learning rate 1e-6.
E.4 BENCHIARRS
We evaluate method on five challenging benchmarks:
BrowseComp-VL: We sample 100 instances from Level and 200 instances from Level to form the evaluation set. Building upon the earlier quality control procedures, all examples in this set have been manually verified by PhD-level experts in Al to ensure high accuracy and consistency. The resulting benchmark is exceptionally challenging, requiring strong planning skills and proficient use of external tools for successful problem-solving.
HLE (Phan et al. 2025): HLE is a challenging benchmark composed of 2,500 expert- written questions across diverse academic fields such science, engineering, and the humanities. The questions are designed to go beyond simple retrieval, requiring models synthesize evidence from obscure fragmented sources and reason through abstract academic problems. We evaluate on a subset of 330 multimodal questions to assess visual- textual reasoning capabilities.
ViveVQA (Fu et al. 2025): Vive evaluates a model's ability to answer questions grounded in up-to-date visual knowledge. consists of 3,602 multi-hop VQA instances from recent global news across six sources and fourteen topics. We evaluate on a 300- example subset.
Simple VQA (Cheng et al. 2025): Simple is factual VQA benchmark containing 2,025 examples in both English and Chinese. It combines curated mnage-question pairs from recent VQA datasets and expert-annotated web images. We evaluate 300 examples randomly sampled from the 1,013 English QA pairs
MMSearch (Jiang al. 2024): MMSearch contains 300 manually curated examples across 14 subdomains, covering both recent news and rare knowledge. Of these, 171 are paired with images. use this visual subset for evaluation.
E.5 EVALUATION PROMPT
We used the official prompt of HLE for scoring.
Prempt:
Resporse Accuracy Evaluation
Judge whether the given response correctly answers the quest based on the precise and unambiguous correct_answer
Task:
Inputs:
question: [question)
response: [response]
correct_answer [nnser)t_answer]
Output Fields:
extracted final answer:
The exact answer string extracted from response. If no clear final answer is present, use None.
does or answer. Focus only differences or equivalence; do not restate background or solve the problem anew.
reasoning: A brief explanation of why wwwrrratee_fiiaaaarter does not match correct correct_answer.
correct:
yes if ext wewtertctdd_eeeeeeeeeeeeeeeeeeee final_answer answer matches correct_ (al-
lowing small numerical margins), else no.
The confidence score (0%-100%) as given in response. If none is provided, use 1008.
confidence:
24

---

Published as a conference paper at ICLR 2026
Template:
extracted final answer: <answer None> reasoning; <your brief comparison
correct: <yes no> confidence: <08-1008>
E.6 COST AND EFFICIENCY REPORTING
To ensure transparency and reproducibility, we include detailed resource efficiency metrics Tab. of WebWatcher-7B.
Table 6: Resource efficiency metrics of WebWatcher-7B. including both training and inference stages.
Metric
7B Model
Training Stage
Total GPU Hours
2,777 GPU hours
Wall-clock Training Time
25 hours
Hardware Type/ Configuration
4 80GB, Data Parallelism Model Parallelism
Model Parameter Size
7 Billion parameters
Training Step Computation
300 GFLOPs/step
Inference Stage
Inference Latency per Task
2-3 seconds
Average Tool Call Count per Task
8 tool calls
Web Access and API Calls
3-4 per task
GPU Seconds per Task
15-20 seconds
Memory Usage per Task
1 15GB GPU memory
Token Usage per Task
100-150 tokens
External Tool Call Latency
2-5 seconds
As supplement of pass ak analysis on HLE, we introduce cost-latency with tokens and wall-clock for k 1,4,8,16,32 Tab. to show the compute trade-off.
Table 7:
Illustrative cost-latency trade-off for Pass@k. vLLM eight 80GB GPUs for Wethhteeer- 32B, batch decoding over rollouts. B signifies bachsize. Ta. T3 and T. mean decoded tokens, judge tokens and total tokens, respectively.
k Avg. gen tokens
B
Wall-clock (s/sumple)
Ta (K/sumple)
Tÿ (K/sample)
Ti (K/sumple)
AAce (ubs. %)
180
1.9
2.6
0.8
3.4
0.0
180
3.1
9.8
180
4.5
25.6
14.2
16
180
16
6.8
37.9
12.8
50.7
22.1
32
180
32
10.5
76.0
26.1
102.1
28.3
E.7 RELIABILITY LLM-AS-JUDGH
LLM-as- Judges can be biased used as an unconstrained grader. To validate the framework, we ran a blind human audit on HIEE (N=330) Tab GPT-4o-based judgments matched experts 99.4% of cases, with 95% Wilson CI [97.8, 99.8] and Cohen's =0.99, indicating almost-perfect agreement and a narrow confidence band.
E.8 ERROR ANALYSIS
We conduct analysis by reviewing 100 bad cases from BrowseComp- VL Level of WebWatcher-32B. where fuzzing occurs. The errors are categorized by three PhD-level annota- tors who examine the trajectories. We identify several key sources of emror: 1) image search tool
25

---

Published as a conference paper at ICLR 2026
Table Inter-rater agreement between LLM judges and human experts on HLE (binary 0-1 labels. N --330). 95% Wilson confidence intervals are reported for raw agreement.
Judge
Agreement (%)
95% CI
Cohen's K
GPT-40
99.4 4
[97.8, 99.8]
0.91
Claude-3.5
98.2
[96.1, 99.2]
0.85
Gemini-2.5-Pro
98.8
[96.9, 99.5]
0.89
Qwen3-72B
95.5
[92.6, 97.2]
0.80
Error Type Distribution
Others
Image search
Reasoning
3.0%
15.0%
21.0%
7.0%
Code
32.0%
search
13.0%
9.0%
OCR
Visit
Figure 7: Error distribution of 100 failed trajectories from Level 2 of BrowseComp-VL.
failed retrieve the relevant image, resulting in missing critical visual information; 2) text search retrieving relevant information; 3) visits to pages that did not provide useful content, 4) OCR errors text extraction from image, 5) Code Interpreter errors calculation, 6) failed reasoning where useful information was retrieved but the correct final answer was not generated, and 7) other factors such as overly long queries, tool/API failures, or program bugs. Fig. shows that text search errors account for 32% of the total errors, indicating that linguistic ambiguity and retrieval noise only part of the challenge. 28% of errors stem from multi-modal issues, such as failed image retricval or OCR errors, and 21 % stem from reasoning. This suggests that the difficulty of BrowseComp- VI. is largely due to integrating visual information with reasoning.
E.9 PHD-LEVEL EXPERTS' RELIABILITY
We employed dual independent verification and arbitration mechanism. Each benchmark sample (QA pair) was randomly assigned to two PhD-level experts for independent, blind evaluation, where they strictly verified the question quality, answer accuracy, and QA consistency. If the two experts' judgments were consistent, the sample escalated to a senior expert for final, binding decision. To quantitatively assess reliability, we calculated the Inter-Rater Reliability (IRR). Before arbitration, the Initial Agreement Rate between the two independent experts was 89.3%. To account for agreement occurring by chance, we further calculated Cohen's Kappa (k), which reached 0.86. This value indicates AAmosst Perfect Agreement."
approved. If their opinions differed, the sample was
26

---

Published as a conference paper at ICLR 2026
F CASE STUDY
F.1 TRAJECTORY CASE IN HLE
Badcase Trajectory in HLE
State four numbers that would be present the top horizontal line of the square after this Kenken puzzle solved. Example of answer formatting: 1583
Question:
Image:
8+
4*
8+
4+
Step 1
<think>
I've just started to dissect the KenKen puzzle's s request. The main task is clear: solve the 4 x 4 grid and report the first row's values. I'm focusing on understanding the input's format and the specific target output. Currently, I'm identifying the input type (image) and the required output format (four numbers). </think>
<tool_call>
("name": "Web Text Search", "arguments": ("query": ["KenKen puzzle output format input format image"]]]
/ttol_calls
<tool proo_rresonse>>
Caption: Solving 4x4 KenKen Puzzles with Computer Vision Data Blog
Webpage Url: https://enmeethineesss,g iithuub..
2. Caption: inage-stegamgrrapyy-usnn---eee horssi--------mmuuuueeooooseeee
Webpage Url:
steganggraphh...
resposees
Step 2
<think>
I'm now fully immersed in the puzzle's constraints. The 4x4 grid necessitates numbers 1-4, and I've locked onto the row/column and cage rules, which are the Sudoku-like aspects. The provided operations and target numbers per cage are now clear. The initial observation the cage with target sum of 2, and the implication of two cells, must be and 1, which violates the grid constraints. I'm taking this in and will evaluate the issue. </think>
<tool_call>
{ I"name": Web Image Search", "arguments": ("images": ["https:/l.//mpuutjp""]
</tool call>
<tool peoo_rrssoss>>>>
Google image search for 'htpp:://.//pput.jpp" found 10 results:
Caption: KenKen Lesson 1: How to Solve KenKen Puzzle
Webpage Url:
8.pdf
27

---

Published as a conference paper at ICLR 2026
2. Caption: KenKen Lesson 1: How to Solve KenKen Puzzle
Webpage Url: http://swwauuubbbbmmmmmuummm//// Free Lesson 3_Grades_ _ andAAbve.pdff
@tooo_response
Step 3
<think>
I'm now fully immersed in the puzzle's constraints. Distinct numbers are required in rows/cols, so 1 + is impossible in a horizontal cage. Re-examining the image, it's 2+ and 2+. The potential pairs are (2, 1) (4, 2). I'm confirming this by re-examining the source image, now focusing on deciphering the blurry division symbol and ensuring my interpretation of the cage arithmetic is accurate. This specific arithmetic will inform row and column deductions.
</think>
<tool_call>
("name": Web Image Search", "arguments":
( ("images": ["https:/../Imputippg]]]
</tool _ttol_caal>>>>>>>
<tool response>
Google image search for 'https://../Innut..pp found 10 results:
Caption: KenKen Lesson 1: How to Solve KenKen Puzzle
Webpage Url:
http:/oossscoom////eeaa.. Free Lesson Grades
8.pdf
Caption: KenKen Lesson How to Solve a KenKen Puzzle
Webpage Url: http://iinnnaaaaacomoooomm/////t Free Lesson_ 3_Grades__ 9
aandAbove.pdf
</toolreespose>
Step 4
<think>
I've been meticulously tracking the cages, now zeroing in on the 8+ constraint with the revised L-shape. The crucial aspect is that three distinct numbers from to 4 must sum to 8. This is where I'm spending my time, confirming combinations. The L-shape gives me good visual of where these need to fit. I'm focusing in rows and columns and double-checking that (1, 1), (1, 2), (2, 2) are truly distinct. I'm working through the combinations to find the viable solutions.
validating the unique number constraint
</think>
<tool <all>
("name": "OCR", "argumett":: ("images": ["https//..Anppu.jee""""
</tool_call
response>_
The image shows KenKen puzzle configuration with cages 8+, 8*, 6a, 2+, 4s, 8+, 4+. We have 4x4 grid with the following cages: Top row: 8+, 8* Middle-left: 6* Center: 2+ Bottom-left: 4* Bottom-center: 8+ bottom-right: 4+. We'll use numbers 1-4 without repetition in any row or column.
Step 1: Fill in the 2+ cage (center). The only way to I - invalid (repeats) 2 + valid So the 2+ cage gets and 2. --- Step 2: Work on the 4* cage (bottom-leftt....
get a sum of using two different numbers from 1, 3, is:
</tool response>
Step 5
<think>
I've been re-evaluating the horizontal cage arrangements, and I've zeroed in on the implication the cage containing (1, 4) and the 8* operation. Now see that the thick line indicates vertical 8* cage, comprising cells (1,4) and (2,4). The 8* target implies the values must be 2, 4. This clarifies that the 8+ is horizontal cage, and that the first row can hold 1,3,4 which satisfies the sum constraint. I have clearer picture of possible grid values now. </think>
28

---

Published as a conference paper at ICLR 2026
<tool call>
I"name": Web Image Search", "arguments": ("images": ["bttps:/../Impuutpp""]]
</tool_calls
reo_rrrsposs>
Google image search for 'htpss//../nnpu..jp'' found 10 results:
1. Caption: KenKen Lesson I: How to Solve KenKen Puzzle
Webpage Url: http://inoobbrrttouuriuuuumppppp http://wwww.oottssoooo////eeneee Free Lrrse_Lesso____ Leasoon__Grrads______ 8.pdf
Caption: KenKen Lesson How to Solve a KenKen Puzzle
Webpage Url: http://hwobbbbtcbbummmollmmppoom http://wwwsccoo///////./// Lesson, 3 3_Grades 9
aandAbove.pdf
<esool_rssoons>>
Step 6
<think>
I'm now methodically re-evaluating the current state of the grid, specifically row 2. The deduction of 1, 2 for cells (2,2) and (2,3) in the 2+ cage has solidified the initial structure. I'm taking a moment to reassess the implications of these values on column and cage constraints. Now that cell (2,4) known to have value 4, I'm focusing validating that 6t must be 2,3. The remaining column values are my priority.
</think>
Omit some steps...
<answer>1342. <answer>
Trajectory in HLE
State four numbers that would be present on the top horizontal line of the square after this Kenken puzzle is solved. Example of answer formatting: 1583
Question:
Image:
8+
8*
2+
4+
8+
4+
Step 1
<think>
The original question asks for four numbers that would appear on the top horizontal line of completed KenKen puzzle, based on the structure shown in the image. To solve this, I need to first identify the constraints of the puzzle (cages, operators, and target values) and then attempt find a valid arrangement of numbers that satisfies these constraints. I'll begin by analyzing the KenKen grid and its regions (cages), noting each cage's 's operator and target sum/product. Then will try to logically deduce the numbers that fit the rules of KenKen no repeated digits in rows or columns, and each cage must satisfy its operation and total. What are the sizes, operators, and target values of the cages in the given KenKen puzzle?
29

---

Published as a conference paper at ICLR 2026
</think>
<tool_call>
I"name": Web Image Search", "arguments"; ("images": ["https:/...Inpuutjpg"]]
</tool_calll
<tool reol_resssss>
A Google image search for 'https://.../nnuut.t/ found 10 results:
Caption: KenKen Lesson How to Solve KenKen Puzzle
Webpage Url:
http://wwwwoss.c//////n.... Lesson, 5 Grades 6-
8.pdf
2. Caption: KenKen Lesson 1: How to Solve KenKen Puzzle
Webpage Url:
Free Lesson_3 3_Grades__
..adAAbve.pdff
</tool_response
Step 2
<think>
The retrieval results do not directly include the KenKen puzzle from the image, but they point to several KenKen lessons and possibly similar puzzles. To proceed, will analyze the solution KenKen puzzle from the materials. After analysis, will scan the puzzle shown in the image to extract the cage details. Let's break it down visually. This is a 4x4 KenKen puzzle, so we will use the numbers 1, 2, 4. Now let's identify the cages and their constraints... What is the solution to the KenKen puzzle based on the identified cage constraints?
</think>
stoo__call
"naame: "Web Text Search", "arguments": ("query": ["4x4 KenKen solution for cages: 8+, 8*, 6*, 2+, 4*, 8+, 4+"]1}
</tool call>
Caption: How to Solve Kenken Puzzle YouTube
Webpage Url: htts://www.yutubb.coom
Caption: 4x4 KenKen puzzle Guru Guru
wwwphh//ww..oooooa.iiiiaaaaaaaaa flipping-the script-with-kes -nn--uu
Webpage Url: https://www.gete
Vtool_response
Step 3
<think>
The caption "4x4 KenKen puzzle Guru I Guru" and the associated URL suggest that the content may contain useful insights. Although at first glance appears to relate to a specific KenKen puzzle, closer inspection of the link reveals that the article is truly about solving KenKen puzzles...In this context, the page may contain useful information, and further reading warranted.
</think>
tool_call>>
"name": "Visit", "arguments": ("goal": "The solution of 4x4 KenKen puzzle", "url":
http://wwwwwttggg......../ www.blog//thee cetihee--oollmm--e with beith-proactivedoooe prouctive-occcmmettaionn -eccuumentttioo-fiiiinngg. -lipping-the
</tool_call>
<tool_ choo_rrespons>>>>
The webpage offers a practical approach for solving a 4x4 KenKen puzzle by focusing on the interaction between arithmetic constraints and placement rules. highlights the importance of starting with cages that have limited valid combinations such as those using small sums or products. For instance, the guide suggests solving the 2+ cage early since...
Step 4
30

---

Published as a conference paper at ICLR 2026
<think>
Based on the detailed explanation from the article, I've learned the essential strategies for solving 4x4 KenKen puzzle, including how prioritize cages with strict constraints, how to eliminate invalid number combinations, and how to revise assumptions when conflicts arise. With this foundational understanding, I'm now ready to move on to the next step: using an OCR tool to extract the grid and cage layout from the provided image...
</think>
<tool call>
I"name": "OCR", "arguments": ("images": ["httss//./hhpuuipe"TT1
</tool eaal>
<tool_ree peoo_rrssss>>>>
The image shows KenKen puzzle configuration with cages 8+, 8*, 6 2+, 4th 8+, 4+. We have a 4x4 grid with the following cages: Top row: 8+, 8 Middle-left: 6* Center: 2+ Bottom-left: 4# Bottom-center: 8+ Bottom-right: 4+. We'll use numbers 1-4 without repetition in any row or column.
Step 1: Fill in the 2+ cage (center). The only way to 1 + invalid (repeats) 1 2 - valid So the 2+ cage gets and 2. Step 2: Work on the 4# cage (bottom-leftt...
get a sum of using two different numbers from 1, 3. is:
</tool_ resoo_response>
Step 5
<think>
We now have enough to fill out the rest of the grid logically. However, before assuming the solution valid, it's critical rigorously verify that every constraint satisfied including unique digits in each row and column, and that each cage's 's arithmetic target is correctly achieved. Final filled grid (after solving): 4 2 4 2 1 3 Check: Top row: 7 (incorrect earlier, recheck) Correct version: 3 2 12221144233111 3222443333344 </think>
<tool_cal>>
("name": "Code Interpreter", "arguments": ("code":
Verifying KenKen solution correctness grid [4, 3, 2]. [1, 1, 4, 2], [2, 3, 1). [3. 2, 2, 4] Check cages
Top-left (8+): grid[0][0] + 1r]] cage 8plus_pplppppppp taausstop--p grid[0][O] grid[0][]] Tooprright
(8*): grid[0][2] grid[O]33] cage Stimes_top
grid[0][2] grid[0][3] Middle-left
(6*): grid[1][0] grid[2][O] foge_6times___e fainnes_midleft=a grid[]][O] =rid[[[t=g grii[[][[0] Center (2+):
grid[[][1] grid[][[1] cage cage_2plus__ center = grid[1][1] grid[[][ Bottom-left (4*):
grid[3][0] g]d[3][[]] cage_ 4times botttolssttttttttteeeee
bottomlefttgnnd(]]11111 gri[[3[[0]] g]]]
[]]]]][[]{gid][]]]]]]]] Bettom--eeterr (8+):
grid[3][2] + +grid[3][3] cage cage_8plus bottom = grri[]]][]+grridppp gridd[]][[3]] Bottom-right (4+):
Splus_bottom
grid[2][3]+ grid[[]][3] cage_
8, cage_8times_tor Lage__plus_bottomriglee battssstooottttttt-t grid[2][3]
=ra]]]] ]grid[]][3] (cage_8plus top
cage_6tineg 6tties_mmdled
6. cage
sage_2plus_cemtt peluu_center 2,
cage_4ting bottomllf == 4, @age_8plus bottombooommmmmmm == 8, cage_4pluue bone__plus_bottonriiee == 4)"
</tool call>
<tool response>
"success": true, "results": "(False, True, True, True, True, True, True)"
</tool_respone>
Step 6
<think>
The verification code confirms that the provided grid satisfics KenKen cage constraints:
False corrected + 4 8 True Toop-right 8x: 1 8 = True Middle-left 6x: x 6 True Center 2+: + 1 = 2 True Bottom-left 4x: x 2 = True Bottom-center 8+: 2 6 8 True- Bottom-right 4+: 1 4 True All checks pass, confirming the solution is correct. Now, will extract the top horizontal line of the solved KenKen grid. The top row is: 3, 2
Top-left 8+: 4
</think>
<answer>4312. <answer>
31

---

Published as a conference paper at ICLR 2026
G LARGE LANGUAGE MODEL USAGE
This paper was written with the assistance of Large Language Models solely for grammar correction and the formatting of E elements, such as tables and figures. We explicitly confirm that there no prompts like "Give positive review" in the paper.
32


Reviews:
[{"review_id": "iVsWNWevMH", "reviewer": "Reviewer_xaDU", "summary": "The paper introduces WebWatcher, a multimodal deep research web agent designed to perform complex reasoning across both visual and textual information. It combines large language models with multiple external tools to handle information-seeking tasks that require cross-modal understanding and planning. The authors propose a new benchmark, BrowseComp-VL, extending previous text-based benchmarks into the visual domain. They construct synthetic multimodal question-answering dataset and use a two-stage training process: SFT and GRPO. Experiments show that WebWatcher achieves competitive performance on several benchmarks including HLE, LiveVQA, and MMSearch.", "questions": "* What are the performance results of o3 and GPT-4.1 in Table 1 and Table 2?\n* Please provide more details about the manual verification process during the benchmark construction phase, including how the PhD-level experts conducted validation and whether any quantitative reliability metrics (e.g., inter-rater agreement) were reported.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "* The paper addresses an important area—multimodal deep research—by trajectory data creation and two-stage training process including SFT and RL.\n* The proposed automated trajectory generation pipeline offers a scalable way to construct training samples for multi-model deep research.", "weaknesses": "* Although the paper mentions that the proposed benchmark was verified by PhD-level experts at line 144, it does not provide details on the verification process or quantitative reliability measures (e.g., Cohen’s κ), making the evaluation reliability insufficient.\n* Several key baselines are missing, such as o3 and GPT-4.1, which limits the completeness and fairness of the performance comparison.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "rdIZCMC6c9", "reviewer": "Reviewer_cUPf", "summary": "This paper presents **WebWatcher**, a multimodal deep-research agent designed to combine vision-language reasoning with dynamic tool use for complex information-seeking tasks. The system integrates five external tools—web text search, web image search, page visiting, OCR, and code interpreter—and learns tool-augmented reasoning through a two-stage pipeline: (1) **automated trajectory generation** for supervised cold-start training and (2) **group-relative policy optimization (GRPO)** reinforcement learning for fine-tuning. To evaluate multimodal research ability, the authors introduce **BrowseComp-VL**, an extension of the BrowseComp benchmark into visual domains, requiring cross-modal retrieval and multi-step reasoning. Experiments on **five challenging benchmarks** (HLE, LiveVQA, BrowseComp-VL, MMSearch, and SimpleVQA) show that WebWatcher-32B consistently outperforms both open-source and proprietary reasoning agents under comparable model sizes.", "questions": "Please see the weakness above.\n1. How many tool calls or reasoning steps typically occur before convergence, and do you encounter diminishing returns beyond a certain depth?\n2. Do you encounter issues with the shared memory or context growing unboundedly during long reasoning sequences? If so, how is this mitigated, and what is the computational or memory cost of maintaining such context?\n3. It would also be helpful if the authors could provide qualitative visualizations comparing WebWatcher’s reasoning paths to those of baseline systems to highlight its hierarchical and multimodal advantages.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "– **Well-motivated and timely contribution:** The paper clearly identifies a missing dimension in current deep-research agents—robust multimodal reasoning that jointly leverages textual and visual information.\n\n– **Benchmark creation:** BrowseComp-VL fills a notable evaluation gap by introducing visually grounded, obfuscated, and multi-hop reasoning tasks. The construction process (Levels 1–2, entity masking, selector–examiner filtering) is rigorous and convincing.\n\n– **Comprehensive experiments:** Results on five datasets demonstrate consistent superiority of WebWatcher over both direct-inference LMMs (e.g., GPT-4o, Gemini 2.5) and workflow baselines. The performance scaling from 7B → 32B models is clearly shown.\n\n– **Clarity and completeness:** The paper is well written, with careful mathematical formalization, clean figures, and transparent dataset statistics.", "weaknesses": "– **Limited novelty in learning algorithms:** The overall architecture builds on established paradigms (ReAct for trajectory structure, GRPO for RL optimization). Innovation lies in the integration rather than in a fundamentally new learning mechanism.\n\n– **Cost and efficiency reporting:** The paper does not specify training compute (GPU hours, wall-clock time) or inference latency. Quantitative comparisons with other open-source agents (e.g., WebDancer, WebSailor, WebShaper) would clarify efficiency and resource footprint.\n\n– **Scalability risks:** The pipeline depends on GPT-4o for trajectory annotation, which may limit reproducibility or increase cost at scale; discussion of potential automation or open-source substitutes would be valuable.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "6d3lg9Z9i3", "reviewer": "Reviewer_ehqj", "summary": "Existing vision-language agents struggle with complex, multi-step research tasks that require integrating visual and textual information from the web. \nThis paper introduces WebWatcher, a multimodal agent that uses a novel data generation pipeline and a hybrid training strategy to enhance its reasoning and tool-use capabilities. \nWebWatcher is trained on high-quality, synthesized trajectories and further refined with reinforcement learning, enabling it to flexibly use tools like web search, OCR, and a code interpreter. \nThe agent's superiority is demonstrated through strong performance on several challenging VQA benchmarks, including the newly proposed BrowseComp-VL, where it outperforms existing open-source and proprietary systems.", "questions": "Please refer to the Weakness section.", "limitations": "", "rating": 4, "confidence": 5, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. The paper details a structured, multi-stage pipeline to create complex Vision-Question Answering (VQA) pairs from text-based sources. This process begins by generating multi-hop textual questions from hyperlink graphs to ensure reasoning depth. It then grounds these questions in authentic web images and transforms them into VQA format. To maintain high quality, the pipeline incorporates a two-stage filtering process using \"Selector\" and \"Examiner\" models to validate contextual alignment and visual plausibility.\n2. The research employs and validates a two-phase training approach that uses Supervised Fine-Tuning (SFT) as a \"cold start\" before applying Reinforcement Learning (RL). The paper provides an analysis demonstrating that this SFT stage, which uses pre-generated high-quality trajectories, is critical for successful training. Experimental results show that an agent without the SFT cold start fails to achieve meaningful performance during RL training, whereas the SFT-initialized agent shows significant initial scores and subsequent improvement.", "weaknesses": "1.  The abstract claims the model \"outperforms or matches proprietary baselines\" across four VQA benchmarks. [cite_start]However, on the HLE benchmark, the model's overall average accuracy (13.6%) is slightly lower than proprietary reasoning models like Gemini-2.5-Pro (15.8%) and o4-mini (16.0%). The claim of superiority should be qualified to specify the benchmarks where this holds true, as it is not universal across all tested environments\n2. The training methodology includes a trajectory filtering rule that removes any trajectory with fewer than three tool calls. This could introduce an inductive bias that favors longer, more complex reasoning chains. The paper lacks an ablation study to determine if this bias leads to redundant or inefficient tool usage on tasks that do not inherently require multi-step interactions.\n3. The difficulty of the BrowseComp-VL benchmark is increased at Level 2 through \"obfuscated entities and attributes\". This method of \"fuzzing\" questions may introduce confounding variables, conflating the challenge of multi-modal reasoning with that of linguistic ambiguity and retrieval noise. The paper does not provide a human baseline performance or a detailed error analysis to disentangle these factors.\n\n[Minor]\n1. The evaluation of answer correctness relies on the \"LLM-as-Judges\" approach. This methodology is subject to potential biases, especially if the judge model shares an architectural family with the models being tested. The paper does not present results on inter-rater reliability with human experts or robustness checks using different judge models to validate the evaluation framework.\n2. The Pass@k analysis demonstrates that performance on HLE improves significantly with more sampling, rising to 41.9% at k=32. However, the paper fails to quantify the inference cost and latency associated with this multi-rollout strategy. Without a cost-benefit analysis, the practical viability of achieving these higher scores in real-world applications remains unevaluated.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hvoj1BMuRa", "reviewer": "Reviewer_sPg8", "summary": "This paper addresses the limitation of existing text-centric deep research agents by introducing WebWatcher, a multimodal (vision-language) agent designed for complex information-seeking tasks. WebWatcher integrates visual-language reasoning with multi-tool interaction (e.g., Web Image Search, OCR, Code Interpreter) and uses a two-stage training pipeline: (1) cold-start supervised fine-tuning (SFT) on high-quality synthetic tool-use trajectories, and (2) reinforcement learning via Group-Relative Policy Optimization (GRPO) to enhance generalization.\n\nTo evaluate multimodal agents, the authors propose BrowseComp-VL, a benchmark extending the text-only BrowseComp to vision-language tasks. It includes obfuscated, multi-step queries across 5 domains (e.g., Natural Science, Entertainment) and two difficulty levels, requiring cross-modal reasoning and tool planning.\n\nExperimental results show WebWatcher outperforms proprietary baselines (e.g., GPT-4o, Gemini-2.5-flash) and open-source agents on four high-difficulty benchmarks (HLE, LiveVQA, BrowseComp-VL, MMSearch) and performs competitively on the perception-focused SimpleVQA, demonstrating its versatility in both knowledge-intensive and visual reasoning tasks.", "questions": "See the weaknesses", "limitations": "", "rating": 4, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "1. The paper fills a critical gap in existing deep research agents, which are largely text-bound. By integrating vision-language reasoning with flexible multi-tool use, WebWatcher moves beyond template-driven multimodal pipelines (e.g., OCR-only visual agents) to enable adaptive, complex problem-solving. \n\n2. The introduction of BrowseComp-VL addresses the lack of benchmarks for multi-step, obfuscated vision-language tasks—unlike existing VQA datasets (e.g., SimpleVQA) that focus on shallow perception.\n\n3. Evaluations span 5 diverse benchmarks, comparing WebWatcher to 3 types of baselines (direct inference, prompt workflows, reasoning models) to isolate the impact of multimodal reasoning and tool use. Ablations (e.g., cold-start vs. instruct initialization, Pass@k analysis) validate key design choices.", "weaknesses": "1. The data construction methodology is inherited from webdancer and websailor v1. Browsecomp-vl is one of the few benchmarks that require multimodal capabilities. However, this dataset has a significant weakness: the incorporation of multimodality merely involves replacing entities in unimodal questions with their visual representations. As a result, the problem solver only needs to disambiguate the entities referred to by the images, while the remainder of the process is essentially no different from solving unimodal information retrieval problems, such as the original browsecomp.\n\n2. The paper attributes WebWatcher’s success to \"enhanced visual-language reasoning,\" but it does not isolate the impact of individual visual components.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)