WebSeer: Training Deeper Search Agents through Reinforcement Learning with Self-Reflection
Authors: Guanzhong He
Zhen Yang
Jinxin Liu
Bin Xu
Lei Hou
Juanzi Li
Venue: ICLR 2026 Poster (2026)
Keywords: lm agent
retrieval-augmented generation
Reinforcement Learning
Primary area: foundation or frontier models, including LLMs

Abstract: Search agents have achieved significant advancements in enabling intelligent information retrieval and decision-making within interactive environments.
Although reinforcement learning has been employed to train agentic models capable of more dynamic interactive retrieval, existing methods are limited by shallow tool-use depth and the accumulation of errors over multiple iterative interactions.
In this paper, we present WebSeer, a more intelligent search agent trained via reinforcement learning enhanced with a self-reflection mechanism. Specifically, we construct a large dataset annotated with reflection patterns and design a two-stage training framework that unifies cold start and reinforcement learning within the self-reflection paradigm for real-world web-based environments, which enables the model to generate longer and more reflective tool-use trajectories. 
Our approach substantially extends tool-use chains and improves answer accuracy. Using a single 14B model, we achieve state-of-the-art results on HotpotQA and SimpleQA, with accuracies of 72.3\% and 90.0\%, respectively, and demonstrate strong generalization to out-of-distribution datasets.

Published as a conference paper at ICLR 2026
WEBSEER: TRAINING DEEPER SEARCH AGENTS THROUGH REINFORCEMENT LEARNING WITH SELF- REFLECTION
Guanzhong He, Zhen Yang, Jinxin Liu, Bin Xul, Lei Hou, Juanzi Li Department of Computer Science and Technology, Tsinghua University he-gz24emails. teg.2emaiiss.ssinnuuaa..... edw..aaaeeddcooom on (xubin, 1i lijuanzi, houlei)etsinghua.edu.
ABSTRACT
Search agents have achieved significant advancements in enabling intelligent infor- mation retrieval and decision-making within interactive environments. Although reinforcement learning has been employed to train agentic models capable of more dynamic interactive retrieval, existing methods are limited by shallow tool- use depth and the accumulation of errors over multiple iterative interactions. In this paper, present WebSeer, more intelligent search agent trained via rein- forcement learning enhanced with self-reflection mechanism. Specifically, we construct large dataset annotated with reflection patterns and design two-stage training framework that unifies cold start and reinforcement learning within the self-reflection paradigm for real-world web-based environments, which enables the model to generate longer and more reflective tool-use trajectories. Our approach substantially extends tool-use uue chains and improves answer accuracy. Using a single results on HotpotQA and SimpleQA, with
14B model, we achieve state-of-the accuracies of 72.3% and 90.0%, respectively, and demonstrate strong generalization to out-of-distribution datasets. The code is available at GithubbWebbeer
INTRODUCTION
Large language models (LLMs) have demonstrated remarkable performance across a wide range of natural language processing tasks, including question answering, summarization, and dialogue generation (Hendrycks et al. 2021 Rein et al. 2024). However, relying solely on the parametric knowledge of language models poses fundamental limitations: it is static, often outdated, and prone to hallucinations (Sardana 2025). To overcome these challenges, retrieval- -uiieeeed generation (RAG) (Lewis et al. 2020) approaches have been developed to enable models to access and retrieve external documents dynamically.
With the continuous advancement of model capabilities, agentic RAG (Trivedi et al. 2023 Li et al. 2025 Jin et al.] 2023) has emerged as a powerful paradigm. This design empowers models to follow more complex reasoning trajectories. Unlike traditional RAG systems (Asai al. 2023 Trivedi et al., 20233 Yu et al. 2024), agentic RAG can freely browse vast knowledge sources available on the internet and leverage tools such as code execution extend their skills, enabling them to tackle a more diverse range of tasks. While agentic RAG greatly extends the scope of tool use and demonstrates strong potential, existing approaches exhibits several notable limitations in practice. In particular, when faced with complex or open-domain tasks, current systems often struggle to maintain coherent reasoning chains and robust retrieval. Errors introduced intermediate steps can easily accumulate, while the lack of effective coordination across components makes it difficult to achieve reliable end-to-end performance. Thus, despite its promise, agentic RAG still confronts a set of fundamental challenges that must be addressed.
1) Insufficient Search Calls. The most common issue when comes to model invocation tools to solve problems is Insufficient API Calls (Kokane al. 2025), which is also often seen in the RAG scenario, as shown in the Appendix [ This may be due to models being biased toward synthesizing the currently available information into plausible answer rather than actively seeking

---

Published as a conference paper at ICLR 2026
Step
Search-ri
Step 4
Relevant
Step 2
Step 3
Think
Irrelevant
Wreey zub
Insufficient Search Depth
Give up
The problem requires hope solve, the model gives up
Local Search
Step 2
Step 3
Answer
Error accumulation
The previous wrong path problem
Step 1
Step 2
WebSeer
Search
Search
WebSeer
WebSeer
Gross voll donionn
Up to 50 steps
Answer
Read Webpage
Sub-answer
refiection
Code runner
Comparasion between different search agent: Search-rl (Jin et al. 2025) faces shallow search and error buildup, while WebSeer trained using our self reff-reflectin paradigm significantly improve tool calls and maintain rigorous reasoning.
Figure 1:
new or complementary knowledge. Existing work typically exhibits short tool-use chains (Jin et al. 2025 Song al., 2025). suggesting that models may fall into suboptimal behaviors- prenst-rrlltlllly producing an answer instead of verifying its correctness or exploring further evidence.
2) Lack of Spontaneous Self-Reflection Mechanisms. Current search agents lack spontaneous reflection steps in RAG scenario: models neither actively cross-verify information nor autonomously rewrite queries or backtrack retrievals when uncertain. As result, if initial retrieval incomplete, the generation phase expands answers based on flawed or partial context, amplifying early errors.
3) Neglect of Real-World Web Scenarios. Most existing work focuses on retrieval from local vector databases, with limited attention to more complex and open-ended web agent scenarios.
In this paper, we introduce Webseer, novel search agent designed tackle complex real-world multi-hop question answering tasks. Different from prior approaches (Jin et al. (2025] Zheng et al.) 2025). WebSeer explicitly encourages deeper exploration and integrates a build-in self-reflection mechanism, enabling the model to backtrack, reformulate queries, and iteratively refine its reasoning process. As shown in Figure previous agentic RAG primarily rely on short tool-use chains and often terminate once superficially plausible answer is formed. In contrast, WebSeer actively prolongs the search trajectory and incorporates reflection steps, and revises its queries when uncertainty is detected. This design enables Webseer gather
comprehensive evidence, mitigate the accumulation of errors, and improve robustness in open-domian multi-hop reasoning. Specifically, we design two-stage training framework that unifies cold and reinforcement learning within the self-reflection paradigm. A central component of this framework is Self-Reflective Reinforcement Learning (SRRL), which leverages answer correctness signals during multi-turn interactions to more effectively encourage reflective behavior. To ground the model in realistic web scenarios, we equip it with three complementary tools: web search API external knowledge acquisition, a webpage reader for lightweight comprehension of web content, and code executor for precise computation.
To support this framework, we construct a high-quality dataset of long- reasoning trajectories through rejection sampling. These trajectories contain multiple rounds of answer refinement and substantially longer tool-use chains compared to conventional dialogue datasets. This training framework significantly increases the average length of tool invocation chains while maintaining rigorous reasoning quality. Compared to prior work (Zheng et al.) (025), all decisions and tool interactions handled by single model, eliminating the need for auxiliary agent controllers or stronger backbone models.
Overall, contributions three-fold:
2

---

Published as a conference paper at ICLR 2026
(1) SFT Construction
Eq
Rejection Sompling
Prompt to wriff the cnswer Retoin the correct verifiostion process
Last iter
Think
Tool
Result
Verify Answer
Act Qpp
Incorrect
Verify Answer
Sota Act
Incorrect
Think
Tool
Result
Verify Answer
Correct
Next iter
Think
Submit Answer
Question
Act
Verify S
Judge
y
Final answer
(2) Self-reflective Reinforcement Learning (SRRL)
Correct!
Answer Ground truth)
Feedback (text signal)
Incorrecti The Fl score is (If Answer Ground truth)
Rollout 1
Feedback
Act
Feedbock
Reward Calculation
Question
Rollout 2
Act
Feedbock
Act
Feedback
Act
Feedback
Rollout
Act
Feedbock
.Up to 20 submission attempts..
Act
Feedbock
Figure Overview of our two-stage training framework. In the first stage, we construct relf-reflective, tool-augmented reasoning trajectories. In the second stage, apply SRRL, allowing the model to iteratively refine and resubmit answers with token-level F--besed reward.
We propose the first unified two-stage training framework with a self-reflection paradigm, enabling more intelligent search agent that improves search depth, breadth, and accuracy.
We propose novel SFT data synthesis method that encourages proactive tool invocation, resulting significantly longer and more complex tool-use chains;
Through extensive experiments. demonstrate notable improvements in answer accuracy, achieving SOTA on HotpotQA with a 72.3% accuracy and on simpleQA with 90.0% accuracy. Our proposed training framework shows explicit self-verification behaviors and strong generalization to OOD datasets.
2 WEBSEER
We introduce WebSeer, a search agent for multi-hop question answering in real-world web settings. The model is trained with our proposed unified two-stage framework based on self-reflection as shown in Fig 2 To obtain diverse high-quality data, we propose multi-tum rejection sampling for collecting positive trajectories used in supervised fine-tuning, enabling the model to learn reflective reasoning patterns. Building on this, develop Self- Reflecefee Reinforcement Learning (SRRL), which incorporates feedback and preserves reflective context during RL, guided by effective reward design to achieve robust and optimal performance.
2. TASK FORMULATION
For given problem, our objective is to construct a reasoning chain augmented by tool invocations. Specifically, the reasoning chain consists of N sequential steps, each comprising model-generated output, a tool invocation, and the resulting observation. At each step, the model initially generates reasoning outputs based on the current context, then decides to invoke one or more external tools. The tool invocation parameters are extracted by the decoder, and subsequently, the invoked tools are executed, returning observations integrated into the reasoning context. The reasoning chain terminates if the model either abstains from further tool invocations invokes special submit-answer tool, signaling completion. Additionally, a predefined maximum step limit Tmux is enforced to prevent indefinite chaining, thus constraining the length of the reasoning trajectory.
3

---

Published as a conference paper at ICLR 2026
2.2 TooL DESIGN
In this work, we design three specialized tools to support external knowledge acquisition and reasoning execution. Additionally, we treat the Answer Submission step itself as a tool, ensuring that the model explicitly decides when to terminate reasoning and output the final answer.
Search Engine This tool receives a set of keywords and performs Google search. It returns the top page of search results in a structured format, including the title, URL, and snippet for each result.
Webpage Reader Due the prohibitive input length of raw HTML, we design the webpage reader to act as a lightweight, model-mediated summarization interface. Given a specific URL and a question, the system fetches the HTML content of the page and prompts the same language model to answer the question based on the page content. The tool then returns the model-generane answer.
Code Executor This tool accepts Python code snippet, executes the code in controlled environment, and returns the standard output as the tool result.
2.3 SELF-REFLECTIVE REASONING CHAIN CONSTRUCTION
Deepseek RI (DeepSeck-AA al. 2025) has highlighted the importance of cold-start in reinforcement learning training. Lee et al. (2025) provided several trajectories for training models to solve multi- hop QA problems. However, these trajectories only included cases where the correct answer was successfully found, without teaching the model how to handle situations in which an incorrect answer might arise. To address this limitation, we propose multi-turn rejection sampling method to collect reasoning paths that incorporate reflective patterns. To encourage trajectory diversity, we impose no restrictions on the form of reflection itself, retaining only those reasoning paths that ultimately converge to the correct solution.
Let D (xi, yf )a= denote a multi-hop QA dataset with ground-truth answers y We consider two models: (i) reasoner G, which, given an instance and its interaction history, generates a tool-augmented reasoning path until producing a final answer; and (ii) independent verifier V. which assesses the factual correctness of proposed answer by invoking tools and ultimately returning judgment. The verifier and the reasoner use the same model, tool interfaces, sampling parameters. and execution environment; the only difference is the prompt, which is provided in the Appendix
At reflection step E (1....., for instance 24, the reasoner receives the concatenated history H = (Pi, RI...., Pt-1, R,-1), and generates tool-augmented path P. that culminates in an answer proposal 0 +- G(v,, H2-1). To evaluate this proposal, we query the verifier up to budget of K, yielding verification outcome Rd V(. xx Pt, u6
Each R2 consists of (i) binary judgment Je E {CORRECT, INCORRECT] regarding ((()) and (ii) tool-augmented path that may be appended to the full path.
Validity predicate. We define a predicate Vi Re, (1) .ut) E {0, 1} that evaluates whether the judgment in Rg matches factual correctness, i.e.,
1, (J. CORRECT 11 = vi) or (J, INCORRECT A 91 + vt),
(1) V Re, vz
0, otherwise.
So, there exists = we accept the verifier's feedback and update the history via concatenation, H, H1-1U (P,, R. Otherwise, re-query the verifier to sample the next outcome
R!!") V(- 24, Pt, v.. ), m 112,.....
and evaluate [R] y3 (t) v.,. v.)
all K attempts fail, then the instance x; is discarded, and we directly proceed to the next problem instance Ti+1-
Iteration and termination. The above process iterates for (. = 1,2,... with the updated history until one of the following conditions holds:
(Success) 0i V," and J. CORRECT,
4

---

Published as a conference paper at ICLR 2026
in which case we halt and record the finalized, verified trajectory {Pi, R1, P2, R2,.... P,. Pt. R
Ti
(Budget stop) mmax
in which case the instance is not recorded as a successful trajectory.
Supervised fine-tuning, Let {Tr)== denote the all set of successful trajectories, where each ( }. To stabilize subsequent perform supervised fine-tuning (SFT) of the model parameters 0 on the dataset {(r;, T:) H- Following empirical findings on iterative search training (Zhang et al., 2025). we adopt masked autoregressive negative log-likelihood (NLL) objective that excludes external observation tokens from the loss.
trajectory T represented as a token sequence = {yi ( y/2 ( Vr,
reinforcement learning and mitigate degenerate exploration,
Let O T denote the subsequence tokens in T corresponding to tool observations. The masked training objective is defined as
EE 11= Tw & O) log Pe (ye at Uet)
L(z, T;0)
E/ Vl Ve & O]
where [[- denotes the indicator function.
This masking restricts the loss to the agent's own outputs-Ssuut as internal reasoning steps and tool- calling decisions-whit excluding literal tool observations. In doing so, the objective encourages the model to faithfully reproduce the supervised reasoning process (e.g., when to retrieve and how to compose intermediate steps) while ignoring raw tool outputs, practice shown to improve both performance and robustness.
2.4 SELF-REFLECTIVE REINFORCEMENT LEARNING (SRRL)
Unlike other previous training frameworks, our reinforcement learning framework unifies SFT and RL under the self-reflection mechanism, which we call Self-Reflective Reinforcement Learning (SRRL). Specifically, SRRL as allows the model to submit answers multiple times within a single dialogue turn. This design enables the model to iteratively refine its reasoning based on external feedback, leading to more stable and effective exploration.
Formally, given input query and the ground-trute answer y", the LLM interacts with external tools to produce an evolving trajectory
T= {(11,0) 01), (u2,02), 02), (aT, oT)), where at step dg denotes the agent's action (e.g., tool call resulting observation. In particular, when the action corresponds to the special tool answer_subree the submitted answer g(t) is compared with y" and return
seaverrsubbit),.... and o; denotes the
(1) (i) x²) E [0.
The scalar feedback (()) is returned as text and appended to the dialogue context. If ( is below a predefined threshold, the environment allows the model to continue reasoning, enabling the model to revise its reasoning and potentially submit an improved answer at a later step. We employ hybrid optimization objective that integrates the advantage estimation from Group Relative Policy Optimization (GRPO) (Shao et al. (024)) with the asymmetric clipping mechanism from DAPO (Yu ct al. 2025). Specifically, for each query g. we sample group of G' outputs {o: LG from the old policy Tew The optimization objective is formalized as:
osl min (O),,))A,,, clip (rt,(0), Clow, 1 enigh Aee))
(()) B 1. -TR
7
(1)
where (0) computed using
no nolor.19.01.e denotes the probability ratio. Following GRPO, the advantage A,C
Taw 10.1010.
Åre R(o.) Pyrmup
where Hyroup
G
R(o;);
Ggroup
T Σ (R(o,) - Mgruppp (2)
Grrup
j=1
5

---

Published as a conference paper at ICLR 2026
Following DAPO, employ asymmetric clipping parameters elow and Ehigh to better accommodate the skewed distribution of reasoning rewards, preventing the policy from overfitting to noisy high- reward trajectories.
2.5 REWARD DESIGN
Because in addition to the correctness of the answer, also care about how many times the agent has tried, we define trajectory- reward. Let trajectory be denoted by T == {(o,, ae re) where o, is the observation (context), a, is the model output including potential tool invocations, and rs is the instantaneous reward. The total trajectory-wise reward then given by
R(T) Riormat (T) + Rearreat (T).
(3)
For format, let ly/ the output length, Lexpect the safe-zone threshold, and Lmax the hard limit. The reward is
0.
lvl S Lexpects
1yl Lexpoct
Riorma (T)
Lexpoct < |// S
Lmaxs
(4)
Limax
Lexpect
>
Liaxx-
Thus, outputs in the safe zone incur no penalty, those in the transition region are linearly penalized, and overly long ones receive the maximum penalty.
For correctness, let E [0, 1] be task-specific score (e.g., token-level F1 reward) and 7 the number of submission attempts. To discourage resubmissions, we apply an exponential discount (0, 1]:
Reoneel (T) r. r-aa".
(5)
EXPERIMENTS
3.1 EXPERIMENT SETUP
Datasets. We evaluate our model on a diverse suite of open-domain QA benchmarks, spanning both in-domain and out-of-domain settings. Following Zheng al. (2025). we adopt the same evaluation split: 512 examples sampled from the development sets of NQ ( Kwiatkowski et al. (019). TQ (Joshi
al.
HotpotQA (Yang et 2018), 2WikiMultiHopQA (Ho al.) 2020), MuSiQue (Trivedi
et al.
2022), PopQA (Mallen et al.
2023). FanouiQA (Zhu et al.) 2024). FRAMES (El Asri et al.
2017
and SimpleQA (Wei et al. 2024), along with 125 examples from Bamboogle.
Because valid answers in open-domain QA often admit multiple surface forms, rule-based string- matching metrics can lead to inaccurate performance estimates. To address this, we adopt LLM-as-a- Judge, following the methodology and prompt template of Zheng et al. (2025). During evaluation, all models are restricted to submitting a single answer. The full evaluation prompt and implementation details provided in Appendix A
Baselines. We compare our approach against several strong baselines that represent different paradigms for reasoning and retrieval in open-domain QA: (1) Closed-book (CoT); The model answers questions using only its internal parametric knowledge, without any external retrieval. following a chain-of -houuht prompting strategy. We choose Qwen2.5-7BBInsttune
(Qwen et al.
2025) the base model. (2) Local RAG: The model is allowed to access local vector-base retrieval system, where the knowledge source consists of the English Wikipedia page dump dated March 2022. We test and R R1-eearcher (Song et al.) 2025) in this setting. (3) Web Agents: The model is equipped with web-based tool access, including search engine querying and webpage parsing through Markdown conversion. We test DeepReserrcher (Zheng et al. 2025 in this setting.
(Qwen al., 2025), Search-rl (Jin et al. (025),,
Implementation Details
During inference, we use the Google Web Search API for real-time retrieval and the Jina API to bypass anti-crawling and extract clean, LLMM-friendly text.

---

Published as a conference paper at ICLR 2026
For training, we prioritize cost, stability, and consistency by restricting retrieval to Wikipedia via the Google Site Search API and fetching full pages through the official Wikipedia API. Training uses the verl framework (Sheng al. 2025). sampling 12 prompts per step, each with 8 candidate trajectories and up to 30 interaction turns. This controlled setup ensures stable, noise-reduced signals while still exposing the model realistic retrieval and comprehension tasks. We trained total of 100 steps, spending 480 A800 GPU hours.
Table 1: Main results on seven multi-hop question answering (MHQA) benchmarks. All the results
labelled with are taken from (Zheng et 2025
Inference
In Domain Hotpot
Out of Domain
Method
Environment
NQ
TQ
2Wiki
Avg
Musique
Bamb
PopQA
Avg
COTt
32.0
48.2
27.9
27.3
33.9
7.4
21.6
15.0
14.7
CoT+RAGE
Local RAG
59.6
75.8
43.8
24.8
51.0
10.0
27.2
48.8
28.7
Search-o11
Web Search
55.1
69.5
42.4
37.7
51.2
19.7
53.6
43.4
38.9
7B/8B Models
Qwen3-8B w/ Tools
Local RAG
67.0
76.4
50.8
33.0
56.8
18.4
43.2
44.0
35.2
Search-r1-base
Local RAG
60.0
76.2
63.0
47.9
61.8
27.5
$7.6
47.0
44.0
Bearch-r--insrueet
Local RAG
49.6
49.2
52.5
48.8
50.0
28.3
47.2
44.5
49.5
R RR-Secrcherr
Web Search
52.3
79.1
53.1
65.8
62.6
25.6
65.6
43.4
44.9
DeepResearcher
Web Search
61.9
85.0
64.3
66.6
69.5
29.3
72.8
52.7
51.6
14B Models
Qwen2,5-14B w/ Tools
Local RAG
72.1
83.8
62.9
70.9
72.4
29.7
72.0
46.1
49.3
Qwen2.5-14B w/ Tools
Web Search
72.5
87.9
67.9
80.3
77.2
26.6
73.6
$4.7
51.6
Qwen3-14B Tools
Local RAG
80.9
$4.9
22.7
63.2
46.7
Qwen3-14B Tools
Web Search
73.7
84.2
57.9
58.5
68.6
23.2
65.6
57.7
48.8
Search-r1
Local RAG
66.9
82.6
69.8
57.0
69.1
36,9
64.8
56.3
52.7
WebSeer
Local RAG
81.9
86.7
70.9
76.0
78.9
35.0
81.6
60.6
59.1
WebSeer
Web Search
82.8
91.0
72.3
84.2
82.6
35.2
80.0
58.0
57.7
3.2 MAIN RESULTS
Table 1 reports the performance of method against baselines on seven multi-hop QA bench- marks. Our approach consistently achieves the best results. substantially outperforming both closed-book and retrieval- vetrivaal-uummened baselines. On in-domain tasks, our model reaches an aver- age accuracy of 82.4%, exceeding the previous state-of-the-art method Search-rl by 12.5 points, The largest gains are observed on NQ and 2Wiki-
Table 2: Evaluation on three harder benchmarks. Qwen2.5-1444 and WebSeer use web search engine, while Search-rl relies on local RAG.
Model
FanoutQA
FRAMES
SimpleQA
Avg.
Qwen2.5-JJ 1-BBB
52.7
61.3
Search-r1-14B
12.6
29.5
36.4
26.2
WebSeer
55,4
56,1
90.0
65.3
MultiHopQA, with improvements of 15.9 and 27.2 points, respectively.
Beyond in-domain evaluation, our method also demonstrates strong generalization on out-of-distribution (OOD) datasets, indicat- ing that it does not merely overfit the retrieval distribution encountered during training. Instead, learns reasoning pat- terns and retrieval strategies that transfer effectively unseen question types, do- mains, and web sources. In this regime, WebSeer benefits from local RAG: on Bam- boogle, it achieves 81.6%, substantial 12.8-point improvement over the prior best, while on PopQA it reaches 60.6%. These results highlight the effectiveness of our re- inforcement learning framework and tool- augmented reasoning design in enabling robust cross-domain generalization.
Table 3: Accuracy and average tool call times for Qwen2.5 Models on HotporQA and SimpleQA
Model
HotpotQA
SimpleQA
Aec
Tool Call
Acc
Tool Call
Qwen2.5-3B
Instruct
44.73
4.31
41.02
4.17
SFT 41.21 (-3.52)
12.40
49.08 (+8.06)
11.46
Qwen2.5-7B
Instruct
51.95
2.95
51.56
3.24
SFT 46.09 (-5.86)
9,23
50.39 (-1.17)
11.09
Qwen2.5-14B
Instruct
62.89
3.57
65.43
3.76
SFT 68.75 (+5.86)
13.43
76.17 (+10.74)
10.82

---

Published as a conference paper at ICLR 2026
We further evaluate three challenging benchmarks: FanOutQA, Frames, and SimpleQA (Table 2) On FanOutQA, a fully OOD multi-documen QA benchmark, our model attains loose accuracy of 55.4, surpassing all baselines and nearly matching GPT-4o (55.8 It also achieves 56.1 % Frames and 90.0% on SimpleQA, while the RI-only Search-rl model performs poorly across these datasets. These results underscore the strong generalization of approach: despite being trained under site-restricted search, it performs even better when deployed in the open web, demonstrating robust and transferable retrieval-reasoningg policy that adapts to diverse domains.
3.3 QUANTITATIVE ANALYSIS
Model capacity matters for complex tool using. We find that sufficient model capacity is essential for multi-step reasoning in search agents, As shown in Table 3 SFT consistently increases tool usage across scales but its effect accuracy is uneven: the 3B model drops 3.52 points on HotpotQA yet gains 8.06 on SimpleQA, while the 7B model degrades on both. Only the 14B model achieves consistent improvements in both tool usage and accuracy, underscoring the role of scale.
Applying RL after SFT reinforces this pattern: the 14B model improves steadily, while smaller models (3B, 7B) show little benefit and suffer from instability, including repetitive text and malformed JSON that causes failed tool calls. Although rewards may rise initially, behavior often collapses. Overall, sufficient scale crucial for stable reasoning and reliable lool-augmented decision making,
Training progressively shapes tool-use be- havior from underuse to strategic deploy- ment. We examine tool invocation distribu- tions across three development stages: pre-SFT, Post-SFT, and post-RL. Using HotpotQA tra- jectories, we plot interaction counts per exam- ple (Figure B), revealing how supervision and reinforcement learning shape reasoning depth and tool ited, with most conversations involving around three calls- suggessug conservative strategy arising from insufficient mastery of tool behav- iors. After SFT, the distribution shifts markedly rightward, peaking at 10 calls and extending up to 50, indicating more active and flexi- ble tool engagement, often in lengthy multi- step interactions. Following RL, the distri- bution sharpens between 5 and 8 calls, with very high and very low counts becoming rare.
Postt
0.4
Proportion
Before SFT, tool usage is lim-
0.1
0.0
15 16 17 1920 Tool Calls per Example
Figure 3: Tool call distributions HotpotQA across three training stages: Pre-SFT, Post-SFT, and Post-RI.
Although we did not penalize underuse during RL, the model rarely produces trajectories with fewer than five calls. This implies that repeated tool use is implicitly reinforeed, it aids verifi- cation and validation. Overall, training progres- sion shows a shift from underuse, overuse, to strategic use. RL fine-tuning improves task performance and yields more stable. efficient behavior, encouraging sufficient- but not exces- too- invocation without hard-coded con- straints.
100
ratlo=1
-14
ratlo=1.5
90
ratio=2
12
10
Tool Toll Call Count
80
Accuracy
76.2
72.3
70
68.8
66.0
Data mixing ratio in SFT are also key to per- formance improvement.
HotpotQA
SimpleOA
We further examine the effect of data compo- sition during the SFT stage. As shown in Fig- ure 4 the ratio between single-pass correct tra- jectories- where the model produces the correct
Figure 4: Impact of SFT data composition
tool usage and accuracy. We vary the ratio be- tween single-pass correct trajectories and multi- refinement trajectories during SFT.
8

---

Published as a conference paper at ICLR 2026
answer in single step and multi-refinement trajectories- where where multiple reasoning or retrieval steps are required plays a pivotal role in shaping model behavior. Increasing the share of longer reasoning trajectories encourages more frequent tool usage, but does not necessarily translate into higher accuracy. Striking an appropriate balance between the two types of trajectories is therefore essential for effective SFT, and can even determine the success of subsequent RL fine-tuning.
3.4 ABLATION STUDIES
In this section, we present ablation experiments to assess the contribution of key components in our framework. We evaluate two variants:
HotporQA
SimpleQA
(1) restricting the model to produce only a sin- gle answer during reinforcement learning, and (2) training without cold-start initialization. Re- sults are summarized in Table 5
Method
Acc
Tool Call
Acc
Tool Call
SFT
68.75
13.43
76.17
10.82
w/ GRPO
67.27
7.38
75.98
6.15
We also in- clude additional ablations on reward design in Appendix G
w/ SRRI.
70.90
7.91
78.91
8.61
(WebSeer)
Our analysis yields three main observations: (1) Each component is critical to the success of training tranving-removig any of them consistently de- grades performance. (2) The cold-start strategy is especially important, as substantially im- proves the model's ability develop effective tool-use behaviors. For more challenging tasks, we additionally find that high-quality SFT data is indispensable for ensuring stable optimiza- tion. (3) The reward structure itself plays a cen- tral role: improper weighting can lead to reward hacking or premature termination, reinforcing the need for carefully balanced incentives.
HotpotQA
SimpleQA
Method
Acc
Tool Call
Acc
Tool Call
SRRL w/o SFT
0.00
N/A*
0.00
N/A*
SRRL. w/ SFT (WehSeer)
70.90
7.91
78.91
8.61
'The model generates malformed output, making valid tool calls impossible.
1.0
14b w/o sit
0.4
RELATED WORK
0.-
0.0
5
20
35
LLM With Tools A growing body of work fo- cuses on tugl-augeented LLMs designed to in- teract with external environments to perform complex tasks. Early approaches like Tool- LLaMA (Qin et al., 2023) applied structured decision-tree tree search to decompose multi-step in- structions, enhancing compositional reasoning through explicit planning. To further optimize tool interactions, recent studies have increasingly adopted Reinforcement Learning (RL). ReTool (Feng et al. (203) and SWiRL. (Goldie et al.) 2025) employs RL to enable strategic tool-use behaviors, while ToolPlanner (Wu et al. (0224) refines inter- action schemas for multi-granularity tasks. More recently, Tool-Star (Dong et al. (0255) introduced multi-tool self-critique framework with hierarchical reward design, which enhances the model's understanding of feedback in collaborative scenarios. However, these methods primarily treat tool use as forward planning problem optimized via hierarchical signals, often overlooking the dynamic nature of error correction. In contrast, WebSeer distinguishes itself by proposing unified framework that enables the model to master both explicit and implicit reflection patterns. This allows the agent to spontaneously backtrack and refine its search trajectory in open-ended web environments, capability largely absent in prior general tool-use frameworks.
Steps
Figure 5: (Top) Limiting the model to submit only answer results in decrease in the final model's performance. (Bottom) Without complex data for SFT, the 14B model collapses with decreasing re- wards.
Reasoning Agentic RAG Early RAG approaches are primarily linear or branching structures (Chen et al. 2024 Gao et al. 2024a b). They typically rely on manually crafted prompts or fixed execution workflows, which severely constrain the model's 's autonomy and flexibility. Recent work has begun to incorporate reinforcement learning into RAG (Jin et al.) 2025) (ong et al. (025). These methods adopt an outcome-driven RL framework that enables the model to explore how to invoke external

---

Published as a conference paper at ICLR 2026
retrieval systems during the reasoning process. DeepResearcher (Zheng al. 2025) extends this line of work to the web search setting, but their approach still depends on stronger models to act agents for webpage navigation. (Shi et al., 2025) propose Pangu DeepDiver, which combines carefully constructed dataset designed to foster information-sseeknng behavior in open-world internet environments with specialized DeepDiver framework to enhance search capabilities. However, despite these advances, the reasoning chains produced by current methods remain relatively shallow and are insufficient for solving more complex or open-ended tasks.
FUTURE WORK
While this work focuses multi-hop QA, the Self-Reflective Reinforcement Learning (SRRL) paradigm naturally extends to other complex reasoning domains, A particularly promising direction Code Generation. Current agents typically rely on a reactive "generate-xxecut--deiue which can be computationally expensive in large-scale or long-horizon tasks. In contrast, WebSeer's reflection mechanism can adapted for pre-execution perifcation-eenbliing the agent to statically analyze code logic and check for alignment with task goals before invoking the execution tool. By acting as a proactive filter for logical fallacies and bugs, this "think before you run" capability has the potential to significantly reduce the computational overhead associated with invalid trial-and-error. Similarly, in Mathematical Reasoning, the framework could verify the logical consistency of problem formulation steps before invoking calculation tools, ensuring rigorous process supervision.
loop,
CONCLUSION
In this work, introduced WebSeer, novel agent training paradigm tailored for real-world web-based retrieval environments. By synthesizing multi-refinement reasoning trajectories through rejection sampling and incorporating self-reflective reinforcement learning (SRRL). WebSeer learns to perform deeper, more robust reasoning that mimics human information-seeking behavior. Through extensive experiments across wide range of open-domain and out-of-domain question answering benchmarks, WebSeer consistently outperforms existing baselines achieving state-of-the-ar perfor- mance on datasets such as HotpotQA, Bamboogle, and SimpleQA. WebSeer lays a foundation for more general-pe general-purpose reasoning agents that can seamlessly interact with dynamic, heterogeneous web environments.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
This work is supported by the National Natural Science Foundation of China (Grant No. 52539001).
REFERENCES
Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi. Self-rag: Learning to retrieve, generate, and critique through self-reflection. In The Twelfth International Conference Learning Representations, 2023.
Jiawei Chen, Hongyu Lin, Xianpei Han, and Le Sun. Benchmarking large language models in retrieval-augmmented generation. Proceedings of the AAAI Conference on Artificial Intelligence, 38 (16::17554177622, Mar. 2024, doi: 10. ww..///aii.v881..222999 URL httts::///o/ aaai org/ hoddx..ppp//wA//rttiiee////////
DeepSeek-AI, Daya Guo, Dejian Yang. Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang. Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangbo Hao, Guanting Chen, Guowei Li, H. Zhang, Han Bao, Hanwei Xu, Haocheng Wang, Honghui Ding, Huajian Xin, Huazuo Gao, Hui Qu. Hui Li, Jianzhong Guo, Jiashi Li, Jiawei Wang. Jingchang Chen, Jingyang Yuan, Junjie Qiu, Junlong Li, J. L. Cai, Jiaqi Ni, Jian Liang, Jin Chen, Kai Dong, Kai Hu, Kaige Gao, Kang Guan, Kexin Huang, Kuai Yu, Lean Wang. Lecong Zhang, Liang Zhao, Litong Wang. Liyue Zhang, Lei Xu, Leyi Xia, Mingchuan Zhang, Minghua Zhang, Minghui Tang, Meng Li. Miaojun Wang, Mingming Li, Ning Tian, panpan Huang, Peng Zhang, Qiancheng Wang, Qinyu Chen, Qiushi Du Ruiqi Ge, Ruisong Zhang, Ruizhe Pan, Runji Wang, R. Chen, R. L. Jin, Ruyi Chen. Shanghao Lu, Shangyan Zhou. Shanhuang Chen, Shengfeng Ye. Shiyu Wang. Shuiping Yu, Shunfeng Zhou, Shuting Pan, S. S. Li, Shuang Zhou, Shaoqing Wu, Shengfeng Ye, Tao Yun, Tian Pei, Tianyu Sun, Wang, Wangding Zeng, Wanjia Zhao Wen Liu, Wenfeng Liang, Wenjun Gao, Wenqin Yu, Wentao Zhang, W. L. Xiao, Wei An, Xiaodong Liu, Xiaohan Wang, Xiaokang Chen, Xiaotao Nie, Xin Cheng, Xin Liu, Xin Xie, Xingchao Liu, Xinyu Yang, Xinyuan Li, Xuecheng Su, Xuheng Lin, X. Q. Li, Xiangyue Jin, Xiaojin Shen, Xiaosha Chen, Xiaowen Sun, Xiaoxiang Wang. Xinnan Song. Xinyi Zhou, Xianzu Wang, Xinxia Shan, Y. K. Li, Y. Q. Wang. Y. X. Wei, Yang Zhang, Yanhong Xu, Yao Li, Yao Zhao, Yaofeng Sun, Yaohui Wang, Yi Yu, Yichao Zhang, Yifan Shi, Yiliang Xiong, Ying He, Yishi Piao, Yisong Wang, Yixuan Tan, Yiyang Ma, Yiyuan Liu, Yongqiang Guo, Yuan Ou, Yuduan Wang, Yue Gong, Yuheng Zou, Yujia He, Yunfan Xiong, Yuxiang Luo, Yuxiang You, Yuxuan Liu, Yuyang Zhou, Y. X. Zhu, Yanhong Xu, Yanping Huang, Yaohui Li, Yi Zheng, Yuchen Zhu, Yunxian Ma. Ying Tang. Yukun Zha, Yuting Yan, Z. Z. Ren, Zehui Ren, Zhangli Sha, Zhe Fu, Zhean Xu, Zhenda Xie, Zhengyan Zhang, Zhewen Hao, Zhicheng Ma, Zhigang Yan, Zhiyu WU, Zihui Gu, Zijia Zhu, Zijun Liu, Zilin Li, Ziwei Xie, Ziyang Song, Zizheng Pan, Zhen Huang, Zhipeng Xu, Zhongyu Zhang, and Zhen Zhang. Deepseek-rl: Incentivizing reasoning capability in 1lms via reinforcement learning, 2025.
URL hhttps:/aarxiv.orr//bb////
12948
Guanting Dong, Yifei Chen, Xiaoxi Li, Jiajie Jin, Hongjin Qian, Yutao Zhu, Hangyu Mao, Guorui Zhou, Zhicheng Dou, and Ji-Rong Wen. Tool-star: Empowering IIm brained multi-tool reasoner via reinforcement learning, 2025. URL httpt:tttsssti...tppsp wttp:/iww.rgga/b////////a/ 2505. 16410
Layla El Asri, Hannes Schulz, Shikhar Sharma, Jeremie Zumer, Justin Harris, Emery Fine, Rahul Mehrotra, and Kaheer Suleman. Frames: corpus for adding memory to goal-oriented dialogue systems. In Kristiina Jokinen, Manfred Stede, David Vault, and Annie Louis (eds.), Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue, pp. 207-219, Saarbrücken, Germany, August 2017. Association for Computational Linguistics. doi: 10.18653/v1/W117-00 URL http::://aclaathooo///// http://p:://c.aaatooooyoooo/////
Jiazhan Feng, Shijue Huang, Xingwei Qu, Ge Zhang, Yujia Qin, Baoquan Zhong, Chengquan Jiang, Jinxin Chi, and Wanjun Zhong. Retool: Reinforcement learning for strategic tool use in llms, 2025. URL WtRLhhttts:/aaxiiii wttpii/.oog/b/////e/eee 11536

---

Published as a conference paper at ICLR 2026
Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, Meng Wang, and Haofen Wang. Retreeval- generation for large language models: A survey, 2024a. URL/Ittp:://axxx.oor///// 10997
Yunfan Gao, Yun Xiong, Meng Wang, and Haofen Wang. Modular rag: Transforming rag systems into lego-like reconfigurable frameworks, 2024b. hos.https///rrrvv.or/////////// 220059
Anna Goldie, Azalia Mirhoseini, Hao Zhou, Irene Cai, and Christopher D. Manning. Synthetic data generation multi-step rl for reasoning tool use, 2025. httphtss:///xii wttpaaawwii..ggb////////////
04736
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. Proceedings of the International Conference on Learning Representations (ICLR), 2021.
Xanh Ho, Anh-Khoa Duong Nguyen, Saku Sugawara, and Akiko Aizawa. Constructing multi-hop qa dataset for comprehensive evaluation of reasoning steps, 2020. URL htttts://axxii...o
abs/2011 2t////111.111 01060
Bowen Jin. Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan Arik, Dong Wang, Hamed Zamani, and Jiawei Han. Search-rl: Training Iims reason and leverage search engines with reinforcement learning, 2025. URL /tttps://arxiv..oo/ http://aaii.....rb////2//// 09516
Mandar Joshi, Eunsol Choi, Daniel S. Weld, and Luke Zettlemoyer. Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension, 2017. URL https:///axxii.....tmm/ orgg / 7705.03551
Shirley Kokane, Ming Zhu, Tulika Awalgaonkar, Jianguo Zhang, Thai Hoang, Akshara Prabhakar, Zuxin Liu, Tian Lan, Liangwei Yang, Juntao Tan, Rithesh Murthy, Weiran Yao. Zhiwei Liu, Juan Carlos Niebles, Huan Wang, Shelby Heinecke, Caiming Xiong, and Silivo Savarese. Toolscan: A benchmark for characterizing errors in tool-use Ilms, 2025. URL https:://rxiiv...t.tmm wrw./bs/////////////
2411.13547
Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wee Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoe Le, and Slav Petrov. Natural questions: A benchmark for question answering research. Transactions of the Association for Computational Linguistics, 7:452-466 2019. doi: 10.1162/tacl a 00276. URL https://acllanthology.yoo wttp:/hhoooooooooor111111110
Zhicheng Lee, Shulin Cao, Jinxin Liu, Jiajie Zhang, Weichuan Liu, Xiaoyin Che, Lei Hou, and Juanzi Li. Rearag: mnowledge-guided reasoning enhances factuality of large reasoning models with iterative retrieval augmented generation, 2025. URL https://arxiv org 255///b.//233211111
Patrick Lewis, Ethan Perez, Aleksandra Piktus. Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, al. searievl--augmente genera- tion for wowleege-intensive nip tasks. Advances in neural information processing systems, 33: 9459 9474, 2020.
Xiaoxi Li, Guanting Dong, Jiajie Jin, Yuyao Zhang, Yujia Zhou, Yutao Zhu, Peitian Zhang, and Zhicheng Dou. Search-ol: Agentic search-enhanced large reasoning models, 2025. URL https //arxiv.org/abs/2501 053666
Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi. and Hannaneh Hajishirzi. When not to trust language models: Investigating effectiveness of parametric and non non-parametric memories. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (eds.), Proceedings of the 61 Annual Meeting of the Association for Computational Linguistics ( (Volume I: Long Papers), pp. 9802-9822, Toronto, Canada, July 2023. Association for Computational Linguisties. doi: 10.18653/v 1/20233 aww.:23..cl-ooo.56666 URL httpps::/aclaatthlllgg/ www.//gg///2322 ac1-long. 546/
Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, Sihan Zhao, Lauren Hong. Runchu Tian, Ruobing Xie, Jie Zhou, Mark Gerstein, Dahai Li, Zhiyuan Liu, and Maosong Sun. Toolllm: Facilitating large language models to master 16000+ real-world apis, 2023. TtL.hhtss://arxtii http///wxww.org//////3////// 16789
12

---

Published as a conference paper at ICLR 2026
Qwen, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang. Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei 7hang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li, Tianyi Tang, Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang. Yu Wan, Yugiong Liu, Zeyu Cui, Zhenru Zhang, and Zihan Qiu. Qwen2.5 technical report, 2025. URL https://arxie orgorrgg///// abs/ 2412. 15115
David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang. Julien Dirani, Julian Michael, and Samuel R Bowman. Gpqa: graduate-level google-prost progle-proofoee q&a benchmark. In First Conference on Language Modeling, 2024.
Ashish Sardana. Real-time evaluation models for rag: Who detects hallucinations best? arXiv preprint arXiv:2503.2113: 2025.
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. Deepseekmath: Pushing the limits of mathemat- ical reasoning in open language models. arXiv preprint arXivv2402.03300, 2024.
Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng. Haibin Lin, and Chuan Wu. Hybridflow: A flexible and efficient rihf framework. In Proceedings of the Twentieth European Conference on Computer Systems, pp- 1279-1297, 2025.
Wenxuan Shi, Haochen Tan, Chuqiao Kuang, Xiaoguang Li, Xiaozhe Ren, Chen Zhang, Hanting Chen, Yasheng Wang, Lifeng Shang, Fisher Yu, and Yunhe Wang. Pangu deepdiver: Adaptive search intensity scaling via open-web reinforcement learning, 2025, /tL/https://rxxi.orrr orrg/ 2ess/2555.24332
Huatong Song, Jinhao Jiang, Yingqian Min, Jie Chen, Zhipeng Chen, Wayne Xin Zhao, Lei Fang, and Ji-Rong Wen. R1-searcher: Incentivizing the search capability in ilms via reinforcement learning, 2025. URL
2503. 05592
Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. musique: Multihop questions via single-hop question composition. Transactions of the Association for Computational Linguistics, 10:539-5545 2022.
Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. Interleaving retrieval with chain-of-thought reasoning for knowledge-intensive multi-step questions, 2023. httss/tps/
10509,
Jason Wei, Nguyen Karina, Hyung Won Chung, Yunxin Joy Jiao, Spencer Papay, Amelia Glaese, John Schulman, and William Fedus. Measuring short-form factuality in large language models, 2024. WRLhhtts:://axx http:///aawwwwwoaa///////// 2411. 04368
Qinzhuo Wu, Wei Liu, Jian Luan, and Bin Wang. Toolplanner: A tool augmented 11m for multi granularity instructions with path planning and feedback, 2024. URL http::s://// /bs/ 2409. 14826
Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W Cohen, Ruslan Salakhutdinov, and Christopher D Manning. Hotpotqa: A dataset for diverse, explainable multi-hop question 09600, 2018
answering. arXiv preprint
Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, Haibin Lin, Zhiqi Lin, Bole Ma, Guangming Sheng, Yuxuan Tong, Chi Zhang, Mofan Zhang, Wang Zhang. Hang Zhu, Jinhua Zhu, Jiaze Chen, Jiangjie Chen, Chengyi Wang, Hongli Yu, Yuxuan Song, Xiangpeng Wei, Hao Zhou, Jingjing Liu, Wei- Ma, Ya-Qin Zhang, Lin Yan, Mu Qiao, Yonghui Wu, and Mingxuan Wang. Dapo: An open-source 11m reinforcement learning system at scale, 2025. URL httpt:://ts:xii wwwpaaawwwww......aaaa httg/bb//2500 14476
Yue Yu, Wei Ping, Zihan Liu, Boxin Wang, Jiaxuan You, Chao Zhang, Mohammad Shoeybi, and Bryan Catanzaro. Rankrag: Unifying context ranking with eetrievallaugmeent generation in llms. Advances in Neural Information Processing Systems, 11112115661111 184, 2024.
13

---

Published as a conference paper at ICLR 2026
Dingchu Zhang, Yida Zhao, Jialong Wu, Baixuan Li, Wenbiao Yin, Liwen Zhang. Yong Jiang, Yufeng Li, Kewei Tu, Pengjun Xie, and Fei Huang. Evolvesearch: An iterative self-evolving search agent, 2025. BRLLhttps://ar//t
org/ abs/ 2505. 22501
Yuxiang Zheng. Dayuan Fu, Xiangkun Hu, Xiaojie Cai, Lyumanshan Ye, Pengrui Lu, and Pengfei Liu. Deepresearcher: Scaling deep research via reinforcement learning in real-world environments, 2025. HRLLhttps://arae http:////aa.ii.oooaaa/////////// 2504. 03160
Andrew Zhu, Alyssa Hwang. Liam Dugan, and Chris Callison-Burch FanOuiQA: A multi-hop, multi-document question answering benchmark for large language models. In Lun- Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeting the Association for Computational Linguistics ( (Volume 2: Short Papers), pp. 18-37, Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 10. 10.118653/11/2222ccl-s httpss//ecllatttoooooyyoo/////// -shhortt.
URL
A HYPERPARAMETERS AND PROMPTS
We employ Swwen-233BB-Al as the verfier model to generate and verify reasoning trajectories. We chose this model for its strong reasoning and tool-use capabilities. Due to we utilize the non- thinking mode of Bwen3-235B-A222 as our verifer, we follow the suggested decoding setting with Temperature=0.7. TopP=0.8, TopK=20, and MinP=O. For each query, we set the retry buget K 10.
Prompts for WebSeer
System:
You are a reasoning assistant with the ability to perform web searches and execute Python code help you process the content of the page and answer the user's question accurately. Do not use any knowledge you know; all facts in your thinking should be obtained from the information returned by the tools. You can repeat the search process multiple times necessary.
Once you have all the information you need, continue your reasoning. Please first make plan before calling tools. Please answer the following question. You should provide your final answer to the "submit answer" tool.
Tools:
You may call one or more functions to assist with the user query.
You are provided with function signatures within <tools></ools>> XML tags: <tools>
"type": "function". "function": "name":
"description": "Submit your final answer. You must use this tool to submit your answer before the dialog ends.". "parameters": "type": "object", "properties": "answer": "type": "string", "description": "Your final answer". "required": ["answer"]
"type": "function", "function": "name": "search", "description": "Call google to search for relevant information." "parameters": "type": "object", "properties": "query": "type": "string", "description": "Search keywords", "required": ["query"]
"type": "function", "function": "name": "query_on_ setrry_oo_pageeee oonppaae".____oo "description": "This tool will visit a specific page of url, and will answer the question based on the content of the page. The assistant has context information, please describe the question completely.". "parameters": "type": "object", "properties": "url": "type": "string". " "description": "The url of the page, must be page provided by the search tool.". "question": "type": "string", "description": "The question about the content of the page", "required": ["url", "question") </tools>
For each function call, return a json object with function name and arguments within <tool_ calo></ol>//ool call> XML tags: stool_calls
14

---

Published as a conference paper at ICLR 2026
"name": <function-name>, "arguments": <args-json-object> </tool_call
user: Question: Question
Prompts for verifier
System:
You are a reasoning assistant with the ability to perform web searches and execute Python code to help you process the content of the page and answer the question accurately. Do not use any knowledge you know; all facts in your thinking should be obtained from the information returned by the tools. You repeat the search process multiple times if necessary.
Once you have all the information you need, continue your reasoning. You should provide your final answer to the "submit, nswerit tool.
Tools:
You may call one or more functions assist with the user query. You are provided with function signatures within <tools></ools>> XML tags: <tools>
"type": "function", "function": "name": "submit_answer". "description": "Submit your final answer. You must use this tool submit your answer before the dialog ends.", "parameters": "type": "object", "properties": "answer": "type": "string", "description": "Your final answer", "required": ["answer"]
"type": "function", "function": "name": "search", "description": "Call google search for relevant information." "parameters": "type": "object", "properties": "query": "type": "string", "description": "Search keywords", "required": ["query"]
"type": "function", ", "function": "name": "query_on_page"" _a__onn________________ "description": "This tool will visit specific page of url, and will answer the question based the content the page. The assistant has no context information, please describe the question completely.", "parameters": "type": "object", "properties": "url": "type": "string". "description". "The url of the page, must be page provided by the search tool.", "question": "type": "string", "description": "The question about the content of the page". "required": ["url", "question"] </tools>
For each function call, return json object with function name and arguments within <tool_ call></tool callll_allllaaal XML tags:
"name": <function-namer "arguments": <args-json-objec>>
c/tool_calls
user:
Please verify if the answer of question 'question' is 'answer'. You can choose your answer from 'Correct,, 'Partly Correct' or 'Incorrect', You should provide your final answer to the 'submit,nswerr'tod
Prompts for Evaluation
You will be given question and its ground truth answer list where each item can be ground truth answer. Provided pred_and _aswerr you need to judge if the pred_answer correctly answers the question based on the ground truth answer list. You should first give your rationale for the judgement, and then give your judgement result (i.e., correct or incorrect).
Here is the criteria for the judgement: 1. The pred_ answer doesn't need to be exactly the same as any of the ground truth answers. but should be semantically same for the question.
15

---

Published as a conference paper at ICLR 2026
2. Each item in the ground truth answer list can be viewed ground truth answer for the question, and the pred answer should be semantically same to at least one of them.
question: [question] ground truth answers: (target)
pred_answer: answer: (predicted_ansee answer)
The output should in the following json format:
"rationale": "your rationale for the judgement, as a text", "judgement": "your judgement result, can only be 'correct' 'incorrectt"
Your output:
B CASE STUDY
Case study 1.1 is generated by the Qwen2.5- 10nn2.5-14B-intttiie model and exhibits a brittle reasoning process; it over-relies on information parsed from search snippets, leading to an incorrect inference about one player's batting hand, and prematurely abandons sub-task after single, poorly formulated query fails. This results in an incomplete and partially erroneous answer. In contrast, WebSeer employs more robust, iterative strategy. It consistently pairs web searches with targeted page queries for verification, avoiding the errors from snippet parsing. Crucially, when faced with an initial search failure for specific entity (player Jeff Austin), it demonstrates effective eπror recovery by refining its search query with disambiguating context ( ('1998 MLB draft'). This methodical approach of iterative refinement and verification, despite requiring
tool interactions, allows the second agent to successfully decompose the problem and provide a complete and correct solution.
Case study 2.1 involved more than dozen tool calls: multiple search calls to locate the "Panther tank" article and repeated query query_on_page checks to confirm it was officially the Panzerkampfwager Panther deployed both Eastern and Western fronts from mid-1943, followed by additional query on page calls verifying that the Jagdpanther tank destroyer used its armor and suspension; by repeatedly cross-validating the tank's name, deployment timeline, and derivative chassis. the process covered all key details and reinforced the conc lusion with multiple lines of evidence, perfectly demonstrating how to leverage tools systematically and efficiently support rigorous reasoning.
Case study 1.1 (Qwen2.5- 1---1------nn)))
System:
You are a reasoning assistant with the ability to perform web searches and execute Python code to help you process the content of the page and answer the user's question accurately. Do not any knowledge you know; all facts in your thinking should be obtained from the information returned by the tools. You can repeat the search process multiple times if necessary. Once you have you need, continue your reasoning. Please first make a plan before calling tools. Please answer the following question. You should provide your final answer to the "submit answer" tool.
information
User:
Question: What the batting hand of each of the first five picks in the 1998 ML.B draft?
Assistant:
To find the batting hand of each of the five picks the 1998 ML. draft, will follow these steps: 1. Search for the list of the first five picks in the 1998 MLB draft. 2. For each player, search for their batting hand (left-handed right-handed).
Let's start with step 1.
Tool Calls: suerch(query=fisst five picks 1998 MLB draft)
16

---

Published as a conference paper at ICLR 2026
Tool Response (search)):
1. Title: 1998 Major League Baseball draft Wikipedia http://1999_aaj__eeeeeeeee http://oo...oooccuuuuuuueeaaeaaa
URL: https://en.
Content Summary:
<b>First</b> round <b>selections</b> <b>5</b>, J. D. Drew & middot; St. Louis Cardinals 6, Ryan Mills, Minnesota Twins
7, Austin Kearns &middot; Cincinnati Reds 8, Felipe L middot; Toronto Blue&nbsp;...
2. Title: List of 1998 Seattle Mariners draft picks Wikipedia URL:
https://en. wikipedia.org/ org/wiki/ hww.::/i99...tttttt..eeiieereers
Content Summary:
In all, the Mariners selected 23 pitchers, 9 outfielders, 7 catchers, <b>5</b> shortstops, <b>first</b> basemen, 2 third basemen, and 1 second baseman.
3. Title: List of Major League Baseball first-round draft picks Wikipedia
URL: https://en. hwwppeddaa.rgg...... _ist_of_Major.._ Leaaeeeeeeeaaaalllllllitttttttte
Content Summary:
Contents &middot; Atlanta Braves; 1.1. &middot; 2 Miami Marlins;
1.1. &middot; New York Mets; 1.1. &middot; Philadelphia Phillies; 1.1. &middot; <b>5</b> Washington Nationals. space)
(content truncated to
Assistant: From the search results, the most relevant link appears to be the Wikipedia page titled "1998 Major League Baseball draft". This page contains the list of the first five picks in the 1998 MLB draft. Let's extract that information from the page.
Next, I'll visit the page and gather the names the first five picks.
Tool Calls: query_on, Serry_oo__aeeuuurreee page(url=https://er wikipedia. siipedia.org/wikiiii. org/wiki/19999 Las_MMMjo___e___ Lasbue_Baseballll___cee _dafft question=What What are the names of the first five picks in the 1998 MLBB draft?)
Tool Response (query page):
The first five picks in the 1998 MLB draft are as follows:
1. **Pat Burrell** Philadelphia Phillies
2. **Mark Mulder** Oakland Athletics
3. **Corey Pattersonx* Chicago Cubs
4. **ffff Austin** Kansas City Royals
5. **J. D. Drew** St. Louis Cardinals
Assistant:
Now have the of the first five picks in the 1998 MLBB draft. Next, I will search for the batting hand of each player.
I will start with Pat Burrell.
Tool Calls: searchl Burrell batting hand)
Tool Response (search):
1. Title: : Pat Burrell Wikipedia
URL: https://en. wikipedia. wttp:r///ww/aattturreeeee Content Summary:
<b>Patrick</b> </b>> Brian <b>Burrell </b> (born October 10, 1976)
nicknamed &quot;<b>Pat<<
the <b>Bat</b<&quot; is an American former prafessional <b>baseball</b> outfielder and current <b> hitting</bh </b> coach for&nbsp;...
2. Title: Brian Wilson (baseball) Wikipedia
URL: https://en.. wikipedia org/ wiki Briiarrinnn... Wilson_ (baseball) Content Summary:
17

---

Published as a conference paper at ICLR 2026
Brian <b>Patrick</b> Wilson (born March 16, 1982), nicknamed &quot the Beard&quot; and & BWeeeyyquut; is an American former professional <b>baseball </b> relief pitcher.
3. Title: List of Major League Baseball career assists leaders Wikipedia
URL:
https://en. wikipedia.orgie
Content Summary:
The second baseman often possesses quick <b>hands </b> and feet, needs the ability to <b>Pat Burrell</b>, 92, 0, 92. Hank Sauer, 92, 0, 92. 46, Joe Vosmik, 90, 89, 1. 47&nbsp;
(content truncated to save space)
Assistant:
From the search results, the Wikipedia page for Pat Burrell provides the most relevant information. Let's extract the batting hand of Pat Burrell from this page. now visit page and gather the required information. Tool Calls: query_or _nn___ooo______ pagc(url-htps://en.nn wikipediaa.or//iik/Paa. Burrell, question=What Pat Bur- rell's batting hand?)
Tool Response squerr_on__paeee
Pat Burrell batted right handed.
Assistant:
Pat Burrell batted right-handed. Now let's find out the batting hand of the next player, Mark Mulder. I will search for "Mark Mulder batting hand".
Tool Calls: search(query=Mark Mulder batting hand)
Tool Response (search):
1. Title: Mark Mulder Wikipedia
URL: https://en. wikipedia. wwtp:///////a/.k..aiie Content Summary:
<b>Mark</b> Alan <b>Mulder</b> (born August 5, 1977) is
American former professional <b>baseball </b> player. left
handed starting pitcher, <b>Mulder</b> </b> pitched in Major League& nbsp:...
2. Title: List of Major League Baseball annual wins leaders Wikipedia URL: https://en wikipedia.org// org/.../ Ltw.Maoor_Leaee_BBeeeeeeeee aaa.aaallaaaaaalaaaaaaaaaa Lam
Content Summary:
<b>Mark Mulder</b>, 21, Oakland Athletics &middot; Roger Clemens & middot; Jamie Moyer, 20. 2002 &middot; Barry Zito, 23, 0akland Athletics &middot; Derek Lowe, 21. 2003 &middot; Roy Halladay sup> </sup>, 22&nbsp;...
3. Title: 2806 St. Louis Cardinals season Wikipedia URL: https://en. crtipedia.org/ikkii. http://..s...ooosscdddaaaaaaaas. Content Summary:
Momentum would be slowed by injuries, starting pitcher <b>Mark Mulder</b> was <b>Batting</b> edit. Note: Games played AB At <b>bats</b>; (content truncated to save space)
Runs; H Hits; 28änbsp;...
Assistant:
From the search results, the Wikipedia page for Mark Mulder provides the most relevant information. According the summary, Mark Mulder is described as left-handed starting pitcher. Therefore, conclude that Mark Mulder batted lefi-handed. Next, I search batting hand Corey Patterson.
18

---

Published as a conference paper at ICLR 2026
I will search for "Corey Patterson batting hand".
Tool Calls: search(query=Corey Patterson batting hand)
Tool Response (search):
Corey Patterson Wikipedia URL: https://en. /ttp:p/di..rgg//i/////rreeeeeeee Content Summary:
1. Title:
Atlanta, Georgia, U.S.. <b>Batted</b>: Left. Threw: Right. MLB debut. September 18, 2000, for the Chicago Cubs.
2. Title: All-Star Final Vote Wikipedia URL: https://en. wiw.nedia.ppiooooom w Content Summary:
On Monday July 7, 2003, Kenny Lofton was added to the ballot to replace the Chicago Cubs&#39; <b>Corey Patterson </b> who was injured on the day before. <b>hand</b> and wearing&nbsp
3. Title: Cody Ross Wikipedia URL: https://en. wikipedia. org/ wttp:///w//oyyoooo/s.sss//s// Content Summary:
He is of the few Major League players to <b>bat</b> right- handed and throw left-handed. Cody Ross. Ross with the San Francisco Giants in 2011. Outfielder (content truncated to save space)
Assistant: From the search results, the Wikipedia page for Corey Patterson provides the most relevant information. According to the summary, Corcy Patterson batted left-handed.
Next, I will scarch for the batting hand of Jeff Austin. I will search for "Jeff Austin batting hand". However, since Jeff Austin might not
well-known as the other players, will also include his name and position in the search query to ensure accuracy.
I will search for "Jeff Austin, catcher batting hand".
Tool Calls: search(quer==Jeff Austin, catcher batting hand)
Tool Response (search):
1. Title: Austin Hedges Wikipedia
URL: https://en. wikipedia. wwtp:/.ki/wsstiin.ette.e.ee Content Summary:
<b>Austin</b> Charles Hedges (born August 18, 1992) is American professional <b>baseball catcher</b> for the Cleveland Guardians of Major League <b2Baseball</b> (MLB).
2. Title: Austin Nola Wikipedia URL: https://en. wikipedia.org htw.://arr///kkkustttt....o.. Content Summary:
<b>Austin</b> Kyle Nola (born December 28, 1989) is an American professional <b>baseball catcher<// </b> for the Colorado Rockies of Major League @b>Baseball</b> (MLB).
3. Title: Catcher Wikipedia
URL: https://en. wikipedia org wwtp/ii//aatterrr Content Summary:
<b>hitting</b>, defensive specialists Jerry Grote, <b> Jeff</b> Mathis <b>hand</b> in professional <b>baseball</b> </b> players with emphasis on digital ischemia in <b>catchers<</bb quot:
(content truncated to save space)
Assistant:
None of the search results directly mention Jeff Austin's batting hand. However, the Cincinnati Reds all -l-time roster mentions Jeff Austin, which suggests he played for Cincinnati Reds. get accurate
19

---

Published as a conference paper at ICLR 2026
information about Jeff Austin's 's batting hand, I will visit the Wikipedia page for Jeff Austin and check his details.
will now visit page and gather the required information.
Since there isn't a direct link to Jeff Austin's Wikipedia page, I will search for directly. I will search for "Jeff Austin Cincinnati Reds batting hand".
But since the search did yield direct result, I will assume that Jeff Austin's batting hand is widely documented online. Given the lack specific information, will proceed the next player, Drew, and then submit the answers have gathered so far. Let's move on to J.D. Drew.
I will search for 'J.D. Drew batting hand".
Tool Calls: search( Drew batting hand)
Tool Response (search):
1. Title: D. Drew Wikipedia
URL: https://en. wwwipediaa org/wiki/J..D.... Content Summary:
David Jonathan &quot;<b>J. D. </b>&quot; <b>Drew</b> (born November 20, 1975) an American former Major League <b>Baseball </b> right fielder. He began his major league career in 1998&nbsp
2. Title: Home
Wikipedia
URL: https://en. wikipedia..i... org/wwki// /Home_run Content Summary:
<b>J. D. Drew</b> has been part of two different sets of back-to- back-to-back-ae ... &#39; Ralph ,#33;; he said, &#39; the right-<b
>hand batters</b> hit the right field are driving&nbsp
3. Title: Adam Wainwright Wikipedia
URL: https://en. wwkipedia. orgeiii...gg/iiii wwrrg/wiki/// wttp:///ww/.....aaaaaaaaa// Adam_ Content Summary:
After the 2003 season, the Braves traded him to the Cardinals for <b>batting</b> practice on February 21. Three days later, the&nbsp;...
outfielder <b>J. D. Drew</b>.
(content truncated to save space)
Assistant:
From the search results, the Wikipedia page for J.D. Drew provides most relevant information. According to the summary, J.D. Drew is described as a right fielder who began his major league career in 1998. Additionally. the page states that he is right-handed batter.
Now have the batting hands for four out of the five players: Pat Burrell Right-handed 2. Mark Mulder seft-handed Corey Patterson Left-handed 4. J.D. Drew Right-handed
Since we couldn't 't find specific information about Jeff' Austin's batting hand, will assume that we don't have enough information to determine it. Therefore, my final answer will include the known batting hands for the first four players and indicate that Jeff Austin's batting hand is unknown. will now submit answer.
Final Answer:
Pat Burrell: Right-handed, Mark Mulder: Left-handed, Corey Patterson: Left-handed, Jeff Austin: Unknown. J.D. Drew: Right-handes handed
Case study 1.2 (WehSeer-141) 14B)
System:
You are a reasoning assistant with the ability to perform web searches and execute Python code to help you process the content of the page and answer the user's question accurately. Do not any knowledge you know; all facts in your thinking should be obtained from the information returned by the tools. You can repeat the search process multiple times if necessary. Once you have information
20

---

Published as a conference paper at ICLR 2026
you need, continue your reasoning. Please answer the following question. You should provide your final the "submit_anm" aarritt___werrree tool.
User: Question: What the batting hand each first picks in the 1998 MLB draft?
Assistant: Tool Calls: search(query=1998 ML.B draft first five picks)
Tool Response (search):
1. Title: 1998 Major League Baseball draft Wikipedia
URL: https://en. wikipedia. wrt.:iai.rg////i//////// /1998 / Legoo__eaaue__asssbllla Content Summary:
<b>First</b> round <b>selections </b> 2, Mark Mulder &middot; Oakland Athletics ; 3, Corey Patterson &middot; Chicago Cubs 4, Jeff Austin &middot; Kansas City Royals <b>5</b>, J. D. Drew &middot; St. Louisånbsp:...
2. Title: List of Major League Baseball first-round draft picks Wikipedia
URL: mttps://en. wwwipedia.org/....
Content Summary:
Contents &middot; Atlanta Braves; 1.1. &middot; 2 Miami Marlins;
1.1. &middot; New York Mets; 1.1. &middot; Philadelphia
Phillies; 1. &middot; <b>5</b> Washington Nationals.
3. Title: List 1998 Seattle Mariners draft picks Wikipedia URL: https://en. hrlppeda..rrgwwike List_off.998..aattiiaaaaeeeeeen
Content Summary:
In all, the Mariners selected 23 pitchers, outfielders, 7 catchers, <b>5</b> shortstops, <b>first</b> basemen, 2 third basemen, and 1 second baseman.
(content truncated to save space)
Assistant: Tool Calls: guery_on_paacleeee how__aggeurrstttsss/oooooooo http://ihhhppss/eo.iiiiieda...oo wrt//t//////////...r... L__MMMMjor_auue___ draft question=What the names of the first five picks in the 1998 MLB draft?)
Tool Response (query_on) _a___on___________________
The first five picks in the 1998 draft are as follows:
1. **Pat Burrell** Philadelphie Phillies
2. **Mark Mulder** Oakland Athletics
3. **rey Patterson** Chicago Cubs
4. Jeff Austin** Kansas City Royals
5. **J. D. Drew** St. Louis Cardinals
Assistant:
Tool Calls: search(query=Pat Burrell batting hand)
Tool Response (search):
1. Title: Pat Burrell Wikipedia
URL: https://en. wikipedia.org. http:////////ata.urreeeee Content Summary:
<b>Patrick</b> Brian <b>Burrell </b> (born October 10, 1976) nicknamed <quot;
the <b>Bat</b<&quot; is an American former prafessional <b>baseball</b> outfielder and current <b> hitting</b> </b> coach for&nbsp;...
2. Title: Brian Wilson (baseball) Wikipedia
URL: https://en.. wikipedia org/ wiki Brrr/rrinnn.. Wilson_ (baseball) Content Summary:
21

---

Published as a conference paper at ICLR 2026
Brian <b>Patrick</b> Wilson (born March 16, 1982), nicknamed &quot the Beard&quot; and & BWeeeyyquut; is an American former professional <b>baseball </b> relief pitcher.
3. Title: List of Major League Baseball career assists leaders Wikipedia
URL:
https://en. wikipedia.org//
Content Summary:
The second baseman often possesses quick <b>hands </b> and feet, needs the ability to <b>Pat Burrell</b>, 92, 0, 92. Hank Sauer, 92, 0, 92. 46, Joe Vosmik, 90, 89, 1. 47&nbsp;...
(content truncated to save space)
Assistant:
Tool Calls: query pegelurrhhnpsseeuuunkiiiiaaaaagg Burrell, question=What is Pat Bur- rell's batting hand?)
Tool Response (query _at___________ on_page)::
Pat Burrell batted right handed.
Assistant:
Tool Calls: search(query=Mark Mulder batting hand)
Tool Response (search):
1. Title:
Mark Mulder Wikipedia
URL: https://en. wikipedia. haa.iairrrrgiik/aaaa.u.u.dd Content Summary:
<b>Mark</b> Alan <b>Mulder</b> (born August 5, 1977) is an
American former professional <b>baseball </b> player. A left
handed starting pitcher, <b>Mulder</b> pitched in Major League& nbsp:...
2. Title: 2006 St. Louis Cardinals season Wikipedia URL: https://en.
Content Summary:
Momentum would be slowed by injuries, as starting pitcher <b>Mark Mulder</bb </b> was <b>Batting</b> edit. Note: Games played Runs; H Hits; 288nbsp;...
AB = At <b>bats</b>;
3. Title: Athletics (baseball) Wikipedia URL: https://en. wikipedia. wrg/wwki//thheeeeeeee (basebbal) Content Summary:
mark when player <b>bats</b>. Prior to the mid-290 2010s,, the A &#39; had long <b>Mark Mulder </b> &middot; P. 2008 200 (content truncated to save space)
Assistant:
Tool Calls: query_on suuery_on_pageuuulehttnss///// www.po/uuuttttt///aaaaaaaa/aaeen haadraggnnndddddeeeesssnntnnnan question=Whal Mark Mulder's batting hand?)
Tool Response (query_on_paaeee
Mark Mulder's batting hand is left.
Assistant:
Tool Calls: search(query=Conyy Patterson batting hand)
Tool Response (search):
1. Title: Corey Patterson Wikipedia
22

---

Published as a conference paper at ICLR 2026
URL: https://en. Content Summary: Atlanta, Georgia, U.S.. <b>Batted</b>: Left. Threw: Right. MLB debut. September 18, 2000, for the Chicago Cubs.
2. Title: All-Star Final Vote Wikipedia URL: https://en. wikipedia.ore wtt.a///////A---St...aaaaa. Content Summary:
On Monday July 7, 2003, Kenny Lofton added to the ballot to replace the Chicago Cubs&#39; <b>Corey Patterson </b> who was injured on the day before. <b>hand</b> and wearing&nbsp;
3. Title: Cody Ross Wikipedia
URL: https://en. wikipedia. org/ /ttp:/wii////..//.///// Content Summary:
He is one of the few Major League players to <b>bat</b> right- handed and throw left-handed. Cody Ross. Ross with the San Francisco Giants in 2011. Outfielder
(content truncated
save space)
query_on heterrrthhgreedaaannhhhaaaannnir Patterson, ques- tion=What Corey Patterson's 's batting hand?)
Assistant: Tool Calls:
Tool Response (query_om (eceer_ooo_aae___
Corey Patterson s batting hand is left.
Assistant:
Tool Calls: scarch(quer==Jeff Austin batting hand)
Tool Response (scarch):
1. Title:
Austin Hedges Wikipedia
URL: https://en. wikipedia...... wtt.://.oo/wwiiiisitii...eeee Content Summary:
Ab>Austin</b>> Charles Hedges (born August 18, 1992) is an American professional <b>baseball </b> catcher for the Cleveland Guardians of Major League <b>Baseball</b> (MLB).
2. Title: List of Major League Baseball batting champions Wikipedia https://en wikipedia. org/.../
URL:
Basebal bab.aaggglbatttig.chhh.
Content Summary:
In <b>baseball</b>, <b>batting<// </b> average (AVG) is measure of batter s success rate in achieving a hit during at <b> bat</b>. In Major League <b>Baseball</b> </b>>>>>>>>>> (MLB) &nbsp;...
3. Title: Austin Riley Wikipedia URL: https://en. wikipedia. /tt.e////i///iiiiiiiiiiiiei Content Summary:
Memphis, Tennessee U.S.. <b>Bats</b>: Right. Throws: Right. MLB debut. May 15, 2019, for the Atlanta Braves. (content truncated to save space)
Assistant:
Tool Calls: query_on wwtp://wwww..ooo.//////wwwww//wa question=What is Jeff Austin's batting hand?)
Tool Response (query pnge)):
The content provided not mention Jeff Austin 's batting hand. However, does identify Jeff Austin as a "Americaa Major
23

---

Published as a conference paper at ICLR 2026
League Baseball pitcher, which implies was baseball player Pitchers typically do not bat, so the question about his batting hand might not be applicable. If you are referring to specific Jeff Austin in the context of baseball, additional information would be needed determine his batting hand.
Assistant:
Tool Calls: search(query=J... Drew batting hand)
Tool Response (search):
1. Title: J. D. Drew Wikipedia URL: https://en.r
wikipedia httss://en.wikiiediaa..
DDe
Content Summary:
On August 9, what would have been sb>Drema#39;s<<b> first game Philadelphia, he sat out, citing a bruised right <b>hand< </b >. <b>Drew</b> had very good season, <b>batting</b> 284& nbsp:...
2. Title: Home run Wikipedia
URL: https://en. httppedii..ogg/iii///tttt..t witp//////HHo_ruu Content Summary:
<b>J. D. Drew</b> has been part of two different sets of back-to- back to-kk-ktobckooooooo ... &#39; Ralph, ,,#39; he said, &#39; the right-<b >hand batters</b>
to the right field are drivingånbsp
3. Title: Adam Wainwright Wikipedia URL: https://en. Content Summary:
After the 2003 season, the Braves traded him to the Cardinals for outfielder <b>J. D. Drew</b>, ... <b>batting</b> practice February 21. Three days later, the&nbsp:...
(content truncated to save space)
Assistant:
Tool Calls: query_on_ www.guuunaaaage//etttiititioiiin Drew, question=What is J. D. Drew's batting hand?)
Tool Response (query_on_paaeee
J. D. Drew's batting hand is left.
Assistant:
Tool Calls: search(query=Jeff Austin batting hand 1998 MLB draft)
Tool Response (search):
1. Title: 1998 Major League Baseball draft Wikipedia
URL: https://en. wikipedia. Content Summary:
The <b>1998</b2 Major League <b>Baseball draft</b>, was the choosing of high school and college bo>baseball</5 players, held June and 3, <b>1998</b>. total of 1445 players &nbsp;...
2. Title: List of Boston Red Sox first -roudd draft picks Wikipedia
URL: sttps://en wikipedia. org/ ... /tt. of.istt.o_.ostto_Ree_...om first- round_ draf drratt_pic....... t_pic...
24

---

Published as a conference paper at ICLR 2026
Content Summary:
The Boston Red Sox are a <b>Major League Baseball </b> (<b>MLB</b>) franchise based in Boston, Massachusetts. They play in the American League East division.
3. Title: List of Detroit Tigers first-round draft picks Wikipedia
URL: /ttps://en. www.pedia.orr.. ogg. round_draft_piee
Content Summary:
The Detroit Tigers are a <b>Major League Baseball<// </b>> (<b>MLB</b>) franchise based in Detroit, Michigan. They play in the
American League Central division.
(content truncated to save space)
Assistant:
Tool Calls: hot.ssoooooo.oggtttttttttptpop/o Jeff Austin's 's batting hand?)
wttpp//ttttteee/aaiiieee///////r huup:a/mnnuuaabblauouuuutoonnn-e question=What
Tool Response (query on page):
Jeff Austin's batting hand is **Righ***.
Assistant:
The information retrieved confirms that the batting hands of the first five picks in the 1998 MLB draft
are:
1. **Pat Burrel Right-handed 2. **Mark Mulder Left-handed 3. **Corey Patterson Left- handed 4. **Joff Austin**; Right-handed 5. **J. D. Drew**; Left-handed This matches the statement provided. Final answer: Correct.
So the answer batting hands the five picks the 1998 MLB draft are as follows:Pal Burrell: Right-handed, Mark Mulder: Left-handed, Corey Patterson: Left-handed, Jeff Austin: Right-handed, J.
D. Drew: Left-handed.
Final Answer:
The batting hands of the first five picks in the 1998 MLB draft follows:Pal Burrell: Right-handed, Mark Mulder: Left-hander Corey Patterson: Left-handed, Jeff Austin: Left-handed.
D. Drew:
Case study 2.1 (WebSeer-1ee 14B)
System:
You reasoning assistant with the ability perform web searches and execute Python code help you process the content the page and answer user's question accurately. Do not use any knowledge you know; all facts in your thinking should be obtained from the information returned by the tools. You can repeat the search process multiple times if necessary. Once you have all the information you need, continue your reasoning. Please first make a plan before calling tools. Please answer the following question. You should provide your final answer the "submit_answer" tool.
User:
Question: What German medium tank, sometimes called the "Mark V", that was deployed on the Eastern and Western fronts from mid-194 provided the armor and suspension for the chassis of the tank destroyer built by Nazi Germany during World War II?
Assistant:
Tool Calls: searchnqueyy=GGrman medium Mark WWII)
Tool Response (search):
1. Title: Panther tank Wikipedia
URL: https://en. wikipedia org / wiki Panther_tanm
Content Summary:
25

---

Published as a conference paper at ICLR 2026
The Panther tank, officially Panzerkampfwagen <b>V</b> Panther abbreviated Pz.Kpfw. <b>V</b>) with ordnance inventory designation: Sd. 171, is a <b>German medium tank <ank</b>bbbsp
2. Title: Mark tank Wikipedia
URL: https://en. haappedia.ogg/..k//Mar...... Content Summary:
There are eleven surviving <b>Mark V tanks</b>, The Mark VIII <b> tank</b> was an enlarged <b>Mark V</b> with greater power: only those with the Liberty engine saw post-war serviceanbsp;
3. Title: German tanks in World War II Wikipedia
URL: https://en. wikipedia. hrapaaao///wwwkeeemmaaaaa tanks ittankkiinoorrddWarr__... Content Summary:
List of <b>tanks</b> &middot; Grosstraktor &middot; Leichttraktor &middot; Panzer I &middot; Panzer II &middot; Panzer III & middot; Panzer <b>IV</b> &middot; Neubaufahrzeug &middot; Panther.
(content truncated
space)
Assistant:
Tool Calls: query_on_iee pwwpooppcccrttttpss/eennnnn http:///s//eeeeiiipiii....o wikkipedia.org/iiki/Paaeee _aantherrrrnn,,, question=Whar German medium tank, sometimes called the 'Mark V", was deployed on Eastern and Western fronts from mid-19433 (94-1943?)
Tool Response (query_on :pege)
The German medium um tank, sometimes called 'Mark V that was deployed on the Eastern and Western fronts from mid-1943 is the **Panther tank**, officially designated as the ** Panzerkampfwagen Panther**.
Assistant:
Tool Calls: query page(uul-htpps:/enn.no http:ttssseeeeiiiiiiiaaa../////e stroyer built by Nazi Germany during World War used the Panther tank's armor and suspension its chassis?)
tank, question=Which tank de-
Tool Response (query _n page):
The tank destroyer built by Nazi Germany during World War II that used the Panther tank armor suspension for its chassis was the ##Jagdaathhrr
The Jagdpanther
heavy Jagdpanzer (tank destroyer) that based on the Panther tank chassis, incorporating its armor suspension system. It was equipped with an 88 mm long-barrel gun, and was designed to provide superior armor protection and mobility compared to other German tank destroyers of the time.
Assistant:
The German medium tank, sometimes called the "Mark V." that was deployed the Eastern and Western fronts from mid- 1943 the Panther tank (officially Panzerkampfwagen V Panther). The tank destroyer built by Nazi Germany during World War that used the Panther tank's armor and suspension for its chassis is the Jagdpanther.
Final answer: Panther tank.
26

---

Published as a conference paper at ICLR 2026
So the answer Panther tankWait, I need verify the answer:
Tool Calls: search(queyy=Gemaan medium tank Mark V WWII)
Tool Response (search):
1. Title:
Panther tank Wikipedia
URL: https://en. wikipedia wtt.:o//www//aaaaa..ttctt Content Summary:
The Panther tank, officially Panzerkampfwagen <b>V</b> Panther ( abbreviated Kpfw. <b>V</b>) with ordnance inventory designation: Sd. 171, is <b>German medium tank bank</b>nbbsp
2. Title: Mark V tank Wikipedia URL: https://en.
http:://ee.wikipediaa... wikipedia. wttp//wii/aak.......
Content Summary:
There are eleven surviving <b>Mark V tanks </b>. The Mark VIII <b> Lank</br was an enlarged <b>Mark V</b> with greater power: only those with the Liberty engine saw post-war service&nbsp;
3. Title: German tanks World War
Wikipedia
URL: https://en. www.wwwiiiiid....ommm Content Summary:
List of <b>tanks</b> &middot; Grosstraktor &middot; Leichttraktor &middot; Panzer &middot; Panzer &middot; Panzer III middot; Panzer <b>IV</b> &middot; Neubaufahrzeug &middot; Panther.
(content truncated to save space)
Assistant:
Tool Calls: query_ _n_______________ oo__age(url-httss//aaa iikipedia.org/wikkiii taank question=Was the Panther tank deployed on the Eastern and Western fronts from mid-19437)
Tool Response (query on page):
Yes, the Panther tank was deployed on both the Eastern and Western fronts from mid-1943. On the Eastern Front, it was introduced during the Battle of Kursk in the summer of 1943, and it was used throughout the war. On the Western Front, it Panzer regiments
deployed
France, with the first panther-equipen arriving there in June 1944. The tank served in most European theatres of World War II from mid-1943 to the end of the war in May 1945.
Assistant:
Tool Calls: wwwppaaall-ttpps//e..iiiipppppep wwt.://wiii..og.owi////eeeeeee.e orgg/wiii//lgggpanherr question=Whar tank de- stroyer used the Panther tank's armor and suspension for its chassis?)
Tool Response (query_nd _al__oo_oo_________ page):
The tank destroyer that used the Panther tank 's armor and suspension for its chassis is the =*Jappaathhrr**
Assistant:
The Panther tank indeed deployed on both the Eastem and Westem fronts from mid-1943, and the lank destroyer that its armor and suspension for chassis the Jagdpanther.
27

---

Published as a conference paper at ICLR 2026
Final answer: Correct
So the answer Panther tank
Tool Calls: submit_ aanniinanaaeernnssseeaaannte tank)
Final Answer:
Panther tank
C SMALLER MODELS BEHAVIOR BEFORE SFT
Case study 3.1 (Qwen2.5 1.veen..5-3BB.. --Innruuut::
System:
You reasoning assistant with the ability perform web searches and execute Python code help you process the content the page and answer user's question accurately. Do not use any knowledge you know; all facts in your thinking should be obtained from the information returned by the tools. You can repeat the search process multiple times if necessary. Once you have all the information you need, continuc your reasoning. Please answer the following question. You should provide your final answer the "submit_answe aanseir"__"""""" tool.
User:
Question: What is the profession of Ansel Elgort's character in "Baby Driver?"
Assistant:
Tool Calls:
query on pagelaut-https//eenwiiiipppe wikipediaa.rg//iik/BBa.a (fimm), question=What is the profes- sion of Ansel Elgort's character in Baby Driver?)
Tool Response (query_on qquery_on_paee Baarr__oo___aae_ page):
Ansel Elgort's character in *Baby Driver* is **a getaway driver**.
He is skilled driver who uses his expertise to help group of robbers escape after heists, while also dealing with his personal challenges, including severe hearing condition that requires him to wear headphones constantly. His profession as driver is central to the film's plot, which revolves around high -speed chases and his role in the criminal underworld.
Assistant:
Tool Calls: submit.answordnswerr-nnnll Elgort's character in 'Baby Driver' is getaway driver.)
Final Answer:
Ansel Elgort's character in 'Baby Driver' is getaway driver.
Case study 3.2 (Qwen2.5-7B Instrucct: Smaller models typically perform short
System:
You are reasoning assistant with the ability to perform web searches execute Python code to help you process the content the page and answer the user's question accurately. Do not use any knowledge you know; all facts in your thinking should be obtained from the information returned by the tools. You can repeat the search process multiple times if necessary. Once you have all the information you need, continue your reasoning. Please answer the following question. You should provide your final answer to the Ssubmit_answer. tool.
User:
Question: What is the profession of Ansel Elgort's character in "Baby Driver?"
Assistant:
Let's search for information about the movie "Baby Driver" and the character played by Ansel Elgort. Tool Calls: search(queyy=Aneee Elgort character profession Baby Driver)
28

---

Published as a conference paper at ICLR 2026
Tool Response (search)):
1. Title: Baby Driver Wikipedia
URL: https://en. wikipedia wwtpdiii.rr//kkii/aa......ii.. Content Summary:
Baby Driver is a 2017 action crime film written and directed by getaway driver seeking
Edgar Wright. It stars Ansel Elgort freedom from a life of crime with his girlfriend Debora (Lily James). Kevin Spacey, Eiza Gonz lez, Jon Bernthal, Jon Hamm, and Jamie Foxx appear in supporting roles.
2. Title: Ansel Elgort Wikipedia URL: https://en.
Content Summary:
Musical career In 2017, Elgort played the title character in Edgar Wright 's action thriller Baby Driver, for which he received Golden Globe Award nomination
3. Title: Ansel Elgort IMDb
URL: https://www/ wwwww.wmm....................... com/name/ nm5052065/
Content Summary:
Ansel played title role Baby Driver (2017), director Edgar Wright action film, starring opposite Lily James and Kevin Spacey. Baby Driver was
(content truncated to save space)
Assistant:
From the search results, we can gather that Ansel Elgort plays the character Baby in the film "Baby Driver". The film deseribes Baby getaway driver who is coereed into working for crime boss. Therefore, Baby's profession in the movie is getaway driver. Now will use the query_on, page function to verify this information from the official IMDb page of the movie "Baby Driver".
Tool Calls: query_on _____o________ sion of Ansel Elgort's 's character in Baby Driver?)
question=Whal is the profes-
Tool Response
Failed get page content
Assistant:
seems that there was an issue retrieving the content from the IMDb page of "Baby Driver". However, based on the previous search results, we can reasonably conclude that Ansel Elgort's character, Baby, is getaway driver in the movic.
Let's proceed with submitting this answer.
Tool Calls: submit answer(answer==getaaay driver)
Final Answer:
getaway driver
D QUALITATIVE EXAMPLE IN 7B MODELS
Case studies 3.1 and 3.2 present examples where a 7B model answered correctly before SFT but incorrectly after SFT. Before SFT, the Qwen2.5-7B model performs a single, targeted search and directly finds the answer in the tool output. After SFT. however, the model imitates the long trajectories seen in the demonstrations and issues multiple tool calls, but many of these calls query repeated irrelevant information. The correct evidence (Marked with red background) becomes buried in the extended context and is effectively ignored by the model. In contrast, Qwen2.5-14BB-FFT
29

---

Published as a conference paper at ICLR 2026
handles the same long tool-call chain and successfully identifies the correct supporting evidence, demonstrating that larger capacity is necessary for robust long-contens reasoning.
Case study 4.1 (Qwen2.5-7B)
User:
Question:
Little Brown Stein is a rivalry trophy that imitates a mug made out of what item?
Assistant:
Let's search for information about the Little Brown Stein trophy find out what it is made of. search(query=Litle Brown Stein trophy material)
Tool Calls:
Tool Response (search):
1. Title: Little Brown Stein Wikipedia
URL: https://en. wikipedia. org wikk/LiittleBrronn Content Summary:
The trophy is. the name implies, large stein mug with the
results of the games between the two painted on.
2. Title: History of FCS Rivalry Trophies Reddit URL: https://www, wwt.ittwwwwwddd..eee////eett..t. wtt.c//c//////e//e// http::/wwww.rrdd...ooooo/o////// sestoryof_.cs_rivalryy..
Content Summary:
With this week being The Battle for the Little Brown Stein, am looking into the history of FCS rivalry trophies.
3. Title: Banner Society
college football rivalry trophies, kind of ranked
URL: https:/ /
www. //www.
trophies-collegeee -is -
Content Summary:
Little Brown Jug, Michigan-Minnen JUG, Melee weapon. Little Brown Stein, Idaho-Montane STEIN, Melee weapon. Mayor" Cup,
RMCe-SMU, Silver
(content truncated to save space)
Assistant:
From the search results, can gather the following information:
The Little Brown Stein trophy described large stein mug with the results of all the games hetween the teams painted
This description matches the trophy given the winner of the Idaho-Montana football rivalry game.
2. In the Wikipedia article, it mentions that the trophy made of wood, but does not specify the exact lype of wood used.
Given these points, it seems the Little Brown Stein trophy likely made of wood, mentioned in Wikipedia article. However, without more specific source confirming material, we cannot be absolutely certain.
Let's verify this information using the Wikipedia page directly. www.ooaaeuullhtttpss////o aikkppdia.org/wikiiliitttt Btwiik///LitleBBBrn..nno Btein__ttenn, question=What is
Tool Calls: qucry_on Bacly_oo___eeeeee the material of the Little Brown Stein trophy?)
Tool Response (query on page):
The **Little Brown Stein** trophy is **large stein mug**, The content explicitly describes it "large stein mug with the results the games between the two [teams] painted on. While the term "stein" traditionally refers to type of ceramic or stoneware drinking vessel, the provided text does not specify the exact material beyond the term "stein mug."
30

---

Published as a conference paper at ICLR 2026
Therefore, based on the given information, the trophy material described as **stein mug**.
**Anwerr:** The Little Brown Stein trophy is large stein mug.
Assistant:
Based the information provided by the Wikipedia page, the Little Brown Stein trophy is described large mug. While the term "stein" traditionally refers to type of ceramic stoneware drinking vessel, the exact material is not specified beyond the term "stein mug." Therefore, the most accurate answer to the question is that the Little Brown Stein trophy is made out of stein mug, which is typically made of ceramic or stoneware.
Let's submit this answer.
Tool Calls: submit answerransserr=hhe Little Brown Stein trophy is made out stein mug, which typically made of ceramic or stoneware.)
Final Answer:
Little Brown Stein trophy is made out stein mug, which typically made ceramic or stoneware.
Case study 4.2 (Qwen2.5-7B-B5)
User: Little Brown Stein is rivalry trophy that imitates mug made of what item?
Question:
Assistant:
To answer this question, need to determine the material used to make the Little Brown Stein rivalry trophy. This trophy is known to resemble a mug but is not made of actual ceramic. Let me perform search to find out the specific material used for its construction. Tool Calls: search(query=Litle Brown Stein rivalry trophy material)
Tool Response (search):
1. Title: Little Brown Stein Wikipedia
URL: https://en. wikipedia. otg./iiti/Liiiee.ero..sot Content Summary:
The trophy is, as the name implies, large stein mug with the results of all the games between the painted on.
2. Title: The 25 Best Rivalry Trophies in College Football : r/CFB Reddit URL: https://www reddit. oree:,5_les__iivaar__rrroheee..d
Content Summary:
trophy. The Little Brown Stein (very different story from the Little Brown Jug) and the Dakota Marker are awesome. Upvote 7. Downvote Award
3. Title: The 10 best rivalry trophies in college football | DraftKings Network
URL: https::/dkketwwot http///keettokk.aaaaakkkiig com /t////222311111 rivalry-trophies- cowpiiieeeiiioooeeeeee-e-eee-e foottgglllfoooll//ooooooobb Content Summary:
-10tbest-
The trophy is corn replica made of bronze, because of course it is. rivalry series, and the Grizzlies took home the Brown Stein. The
(content truncated to save space)
Assistant:
31

---

Published as a conference paper at ICLR 2026
Useful Information Summary: 1. The Little Brown Stein is rivalry trophy between Idaho and Montana universities. The trophy resembles large stein mug. The trophy was crafted by a local carpenter. costs $25 to make. The trophy has been awarded since 1938.
Thinking Process: Based on the search results, particularly the information from the page at URL "htips///en. wikippedia.or//wikkki wwgg/wii/Liitteeeee.. Brown Stein" and other related articles, it is clear that the Little Brown Stein trophy is made of wood. The fact that resembling large stein mug indicates that is wooden replica of a mug.
crafted by local carpenter and described as
Final Answer: Wooden.
So the answer is Wooden. Wait, need to verify the answer before submitting it. Tool Calls:
search(query=L.itte Brown Stein trophy material)
Tool Response (search):
Title: Little Brown Stein Wikipedia URL: https://en. wikipedia seieedia.rr/wkkiiiittleee
Content Summary:
The trophy is, as the name implies, large stein mug with the results of all the games between the painted on.
2. Title: History of FCS Rivalry Trophies Reddit URL: https://wwy reddit. of
Content Summary:
With this week being The Battle for the Little Brown Stein, am looking into the history of FCS rivalry trophies.
3. Title: 156 college football rivalry trophies, kind of ranked Banner Society
URL: https://www/
Content Summary:
Little Brown Jug. Michigan-Minnesot.. JUG, Melee weapon. Little Brown Stein, Idaho-Montamt STEIN, Melee weapon. Mayor Cup, Rice-SMU, Silver
(content truncated to save space)
Assistant:
The useful information from the search results includes:
1. The Little Brown Stein is rivalry trophy between Idaho and Montana universities. 2. It is described large stein mug with game results painted on it. 3. The trophy has been awarded since 1938. 4. It is associated with the rivalry between Idaho and Montana football teams.
Now, let's analyze the question and the proposed answer:
**Question:** Little Brown Stein is rivalry trophy that imitates a mug made out of what item? **Poposedd Answer: ** Wooden
From the information gathered, know that the Little Brown Stein large stein mug-like trophy. However, is direct mention
summaries provided about the material used to make trophy. The descriptions focus on its appearance and history rather than its composition.
Since the question specifically asks about the material used to make the trophy, and none of the summaries explicitly state the material, cannot confirm if the trophy is made of wood another substance. Therefore, the current To verify the accuracy the answer, would need conduct a more detailed search focusing on specific material used for the Little Brown Stein trophy. will perform an additional search to find this information.
"Wooden" is speculative and lacks concrete evidence.
Tool Calls: search(query=Little Brown Stein trophy material)
Tool Response (search):
1. Title: Little Brown Stein wikipedia
URL: kttps://en. wikipedia
Content Summary:
32

---

Published as a conference paper at ICLR 2026
The trophy is, as the name implies, large stein mug with the results of all the games between the two painted on.
2. Title: History of FCS Rivalry Trophies Reddit URL: https://www.. reddit
Content Summary: With this week being The Battle for the Little Brown Stein, I am looking into the history of FCS rivalry trophies.
3. Title: 156 college football rivalry trophies, kind of ranked Banner Society
URL: https://www trophies-collet -ost
Content Summary:
Little Brown Jug,
JUG, Melee weapon. Little Brown Stein, Idaho Montana, STEIN, Melee weapon. Mayor Cup,
Rice-SMU, Silver
(content truncated to save space)
Assistant:
None of the search results provide explicit information about the material used to make the Little Brown Stein trophy. The descriptions focus its appearance large stein mug and historical context, hut they do not mention the specific material.
To verify whether the trophy is made of wood, I need to look for additional details about the trophy's 's construction. I will conduct more focused search specifically targeting the material used for the Little Brown Stein trophy.
Tool Calls: search(query=Litlee Brown Stein trophy material made of)
Tool Response (search):
1. Title: Little Brown Stein - Wikipedia URL: https://en.. wikipedia. Content Summary:
Missing: material made
2. Title: Beer stein - Wikipedia
URL: https://e..wikpeddaa.oom brgg/wkii/Berr.c. /Beir Content Summary:
Such steins may made out stoneware, pewter, porcelain or even silver, wood or crystal glass; they may have open tops or hinged pewter lids with a thumb-
3. Title: History of FCS Rivalry Trophies Reddit URL: https://www reddit, cttcc/ocs////e/ee/eeeccc htwp/r/ffcsccmmme/////400000
Content Summary:
With this week being The Battle for the Little Brown Stein, am looking into the history of FCS rivalry trophies.
(content truncated to space)
Assistant:
None of the search results directly mention the material used make the Little Brown Stein trophy. While some pages discuss beer steins trophies general, none specifically address the composition of the Little Brown Stein.
33

---

Published as a conference paper at ICLR 2026
Given this lack of direct information, cannot confirm whether the trophy is made of wood or another material based solely the data retrieved far. provide accuratc answer, would need locate additional sources that specifically detail the construction materials of the Little Brown Stein trophy. Since the search results did not yield the required information, I cannot submit an answer at this time. I will need to perform further research consult authoritative sources about the trophy's 's construction to determine the correct material used for the Little Brown Stein.
I will attempt one more search focusing on the Little Brown Stein trophy itself see if any details about its material are available.
Tool Calls: search(query=LLing Brown Stein trophy made of)
Tool Response (search)):
1. Title:
Griz
battle Vandals for Little Brown Stein
Montana Athletics
URL:
https:// www/neww///eeetecceceee.w.. http////////2////1/// football -griz- ttettttleeetleee battle frandals-for-little--- -stein
Content Summary: Montana and Idaho first squared off 120 years ago in 1903 and meet on the gridiron for the 89th time this week, with the winner taking home "The Little Brown Stein," wooden traveling trophy The Stein was created by UM student newspaper writer John T.
2. Title: Little Brown Stein Wikipedia URL: https://en. wikipedia Content Summary:
The trophy is, as the name implies, large stein mug with the results of all the games between the painted
3. Title: History FCS Rivalry Trophies Reddit URL: https: //oo:://www reddit.
Content Summary:
With this week being The Battle for Little Brown Stein, am looking into the history of FCS rivalry trophies.
(content truncated to save space)
Assistant:
Useful Information Summary: 1. The Little Brown Stein is a wooden traveling trophy between the University of Idaho and the University of Montana football teams. 2. It is described as large stein mug with the results of their games painted writer John T. Campbell 1938. is unique trophy with dark stain, but the specific type of wood not mentioned the summarics.
The trophy created by UM student newspaper
Thinking Process: From the search results, we learn that the Little Brown Stein is a wooden trophy, specifically described
large stein mug with the results of the games painted on it. The trophy was crafted in 1938 by UM student newspaper writer named John Camphell. Although the exact type of wood not specified the summaries, the presence of term "wooden" in multiple contexts strongly suggests that the trophy indeed made of wood.
Based on this analysis, the answer "Wooden" appears to be correct.
Final Answer: Wooden.
So the answer Wooden
Tool Calls: uubmit.anwwer(nswvo=oWwdeen
Final Answer:
Wooden
34

---

Published as a conference paper at ICLR 2026
E FAILURE PATTERN ON SMALL MODELS
In Case study 5. the Qwen2.5-7B model did not properly close JSON tags when outputting tool calls. In Case study 5.2, the Qwen2.5-7B model kept repeatedly calling tools after search failure until it exceeded the context length.
Case study 5.1 (Qwen2.5-7B-RL)
System:
You are reasoning assistant with the ability to perform web searches and execute Python code to help you process the content page and answer the user's question accurately. Do not use any knowledge you know; all facts your thinking should be obtained from the information returned by the tools. You repeat the search process multiple times if necessary. Once you have all the information you need, continue your reasoning
Please answer the following question. You should provide your final answer to the "submit_aa Suubiit_answer" tool. /no_think For each function call, return json object with function and arguments within <tool_cal>><tool_ccal
tags:
<tool_call>>
("name": <function-nne ~nme>>, "arguments": joogs-json-objeee </tool_call>
Prompt:
Question:
What is the the "Billboard" 200 where "Vice Verses" debuted number 8?
Assistant:
To answer the question regarding the Billboard 200 chart debut of Vice Verses," I will first identify the song "Vice Verses" and determine its chart performance Then, will confirm if it debuted #8 on the Billboard 200. Let's proceed with this step -tep.
<tool_call>
("name": "search", "arguments": "query": "Billboard 200 chart debut of Vice Verses "}}
</tool_call>
Case study 5.2 (Qwen2.5-7B-RL.
User:
Assistant:
To answer this question, I will perform web search to find information about the Hoover Company 's sale in 2006 and the number of employees the holding company involved in the sale.
<tool_call>
("name": "search", "arguments": ("query": "The Hoover Company 2006 sale holding company number of employees"))
</tool_call>
tool response:
<tool_ret _responss>>
1. Title: Maytag Wikipedia
URL: https://en. wikipedia wrt/wwki/ /Maytag
Content: <b>Number of employees</b> </b>. 2, 5.000. <b>Parent</b> &middot;
Whirlpool <b>Corporation
>/ </b> (<b>2e)<</>>preseent) ... On April
35

---

Published as a conference paper at ICLR 2026
1, <b>2006</b>, Whirlpool completed acquisition Maytag <b>Coorporation//z
2. Title: Whirlpool Corporation - Wikipedia
URL: https://en. wikipedia. org wwwp//hirlpool_oorreeeroe Content:
In 2023, Fortune <b>company</b> had an annual revenue approximately $19 billion in <b>sales</b>, around 59,000 <b> employees</b>, and more than 55 manufacturing andånbsp;
3. Title: Ford Motor Company Wikipedia URL: https://en.. www.ppia.or///.tffoooooooooooo. Content:
F former British subsidiaries Jaguar and Land Rover, acquired in 1989 and 2000, respectively, were <b>sold</b> to the Indian automaker Tata Motors in March 2008.
4. Title: UnitedHealth Group Wikipedia
URL: https://en. hat.pp/aaa...oggkkkk///tttttttaa Content:
<b>Number of employees</b>.
480, ,00,000 (2024). Subsidiaries & In February <b>2006</b>, the <b>company</b> acquired John Deere Health Care.
middot; Optum
5. Title: The Hoover Company Wikipedia URL: https://en. httppeddi..gggkkkkeeee.o.ooeeer
Content:
<b>sales</b> <b>many</b> parts of Europe Today, <b>the Hoover</b> Europe brand The machine was so fondly remembered that it was reintroduced and <b>sold</b> from <b >2006</b> to 2009.
Please summarize the useful information in the returned results first, then explain your thinking. The above information will disappear soon. </ /ooo__sssposs
Assistant:
Based the information retrieved:
1. The Hoover Company 's sale details are not explicitly detailed on its Wikipedia page.
2. There's 's no direct confirmation of any holding company that purchased Hoover in 2006.
3. However, the sale of the Hoover Company (or its European brands) is referenced in the context of its sales in Europe and its re- introduction, but again, no specific buyer is named or linked to the sale event.
The most relevant link is the "Hoover Company Wikipedia page. does not explicitly mention sale 2006 with buyer and their employment number. Other pages like Whirlpool's mention acquisitions but not in connection to Hoover specifically. There 's an indirect mention that Whirlpool Corporation (acquiring another company
2006) was mentioned but unrelated to Hoover. Since no relevant data is found, and there 's confusion or lack of clarity, will perform more detailed search about the Hoover Company
<tool_call>
(: "search", "arguments": ( ("query"; acquisition details"})
"Hoover Company sale
</ c/loo__call>
F MULTI-SEED EVALUATION
We report the performance of SRRL and GRPO across multiple random seeds to assess the stability and variance of both methods. The detailed results are shown in Table
36

---

Published as a conference paper at ICLR 2026
Table Performance across different random seeds for SRRL and GRPO.
Seed
SRRL
GRPO
42
70.9
66.3
142
71.5
67.5
1142
69.6
68.1
Mean
70.7
67.3
Std
0.97
0.91
G ABLATION STUDIES ON REWARD DESIGN
To quantify the contribution of individual reward components, we conduct ablations targeting (i) the formatting penalty Riamat and (ii) the exponential discount coefficient in the correctness reward Reorees The results highlight the delicate interplay between structural constraints and incentives for iterative refinement.
G.1 EFFECT OF FORMATTING PENALTY RTORMAT
As shown in Figure 6 the formatting constraint is essential for maintaining stable training dynamics. Removing Rforat leads to a characteristic length-explosion failure mode: although early-stage performance (within the first 50 optimization steps) matches the full reward configuration, the policy quickly discovers a reward-hacking strategy. Without penalties on trajectory structure, the model aggressively accumulates tool calls and intermediate outputs to marginally improve the probability of producing a correct answer and thereby secure Reunect- Reinforcement learning amplifies this brittle behavior, creating positive feedback loop in which sequence length grows without bound. Once trajectories exceed the 32k-token context window, optimization collapses entirely. These findings demonstrate that structural constraints are not auxiliary but instead prevent pathological reward-seeking behaviors that undermine training stability.
Critic Score Mean Comparison
Response Le ngth Mean Comparison
16:02
Fxrma 2264 0.0)
Rasporaa Lerighh Moun Moan
C Hoan
7049
02.
Figure Accuracy and changes in output length before and after removing Riomat
G.2 EFFECT OF THE DISCOUNT COFFFICIENT a IN RECRRECT
We further study the sensitivity of the correctness reward to the exponential discount factor a. which modulates the influence of repeated attempts. As summarized in Table the choice of induces a clear trade-off between encouraging self-correction and avoiding inefficient exploration. When 1.0 (no discount), the agent over-relies on environment feedback and tends to spam submissions, resulting in reduced accuracy despite shorter trajectories. Conversely, a steep discount (a = 0.4) discourages refinement: after initial failure, the sharply diminished return of additional attempts causes the model to terminate prematurely. The intermediate value a 0.8 achieves the best balance, yielding the highest accuracy while maintaining reasonable trajectory lengths. Overall, these results indicate that carefully tuned discounting is crucial for enabling deliberate self reffrreflcttion while discouraging indefinite, low-confidence guessing strategies.
37

---

Published as a conference paper at ICLR 2026
Table 5: Performance across values of the discount coefficient a.
Metric
0.4
a
0.6
0.8
1.0
Accuracy
65.2
69.2
70.9
68.8
Length
4524
3807
3253
2301
H THE USAGE OF LLM
polished by the large language models (LLMs), which were used improve the clarity, grammar, and presentation of the text. The models were not used to generate research ideas, conduct experiments. analyze results; all conceptual contributions and empirical findings are the work of the authors. We carefully reviewed and edited all LLM-generated suggestions to ensure accuracy and alignment with the intended meaning.
Portions of this paper
THE USAGE OF LLM
Portions of this paper were polished by the large language models (LLMs), which were used to improve the clarity, grammar, and presentation of the text. The models were not used to generate research ideas, conduct experiments, or analyze results; all conceptual contributions and empirical
findings are the work of the authors. We carefully reviewed and edited all LLM-generated suggestions to ensure accuracy and alignment with the intended meaning.
38


Reviews:
[{"review_id": "AK0yCSEytm", "reviewer": "Reviewer_cCUs", "summary": "This paper presents WebSeer, a novel approach for training LLM search agents, i.e. agents able to perform iterative calls to Web search engines to answer a given query. The approach consists of two main stages: 1) supervised fine-tuning (SFT) on high quality multiturn search interactions,  and 2) a novel self-reflective reinforcement learning (SRRL) pipeline. The approach is used to finetuning Qwen 14B and achieves SOTA benchmark scores for it's size when given access to Web search.", "questions": "See weaknesses.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper studies a relevant problem: Tool calling (such as Web searches) is a very timely and challenging area of research, unveiling a lot of potential for LLMs. \n- The presented WebSteer approach is sound and, to the best of my knowledge, novel.\n- The experiments ablate two key choices made: 1) self-reflection and 2) the cold-start initialization via SFT, both proved to be beneficial.\n- Trained LLMs reach SOTA benchmark scores for their sizes.", "weaknesses": "- The analysis claims that the method's effectiveness is heavily dependent on model capacity. Only the 14B model showed consistent performance and stable behavior improvements, while smaller 3B and 7B models showed degraded performance after SFT and instability during RL. I find this quite puzzling and would like to know if the authors have further explanations on why this is the case. In particular, some qualitative example after SFT or during RL would further illustrate the cause. \n\n- The ablations focus primarily on the training methodology (SRRL, cold-start). It would be beneficial to see further ablations, e.g., on the impact of the reward design components ($R_{format}$ and $R_{correct}$).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "AmFgGg3yT8", "reviewer": "Reviewer_dqp4", "summary": "This paper proposes WebSeer, a tool-use agent trained with reinforcement learning (RL) that incorporates a self-reflection mechanism. Specifically, the agent is equipped with searching, webpage reading, and code execution tools. The training process contains a supervised fine-tuning (SFT) stage followed by an RL stage. This RL stage mostly follows GRPO, but is designed to encourage self-reflection. The resulting 14B model achieves state-of-the-art results on several multi-hop question answering datasets and demonstrates strong generalization to out-of-domain datasets.", "questions": "Questions regarding the RL algorithm:\n1. In SRRL, if a rollout achieves a low reward in an earlier step but a better reward in a later step (via self-reflection), how is the advantage calculated? Is it based only on the final reward, or does it consider every reward achieved throughout the trajectory?\n2. Alternatively, is every answer submission considered a separate rollout? (e.g., separate trajectories sharing initial steps but with different ending steps/submission times).\n3. Following 2., if not, does it mean the model will generate a trajectory containing multiple answer submissions? In that case, how do you determine when to end an trajecotry during inference?\n4. Could you elaborate on how the model's behavior differs between SRRL and regular GRPO? More analysis or examples here would be helpful.\n\nQuestions regarding experiments:\n1. What is the specific base model used for training (e.g., Qwen2.5-14B or Qwen3-14B)?\n2. It appears the \"Qwen3-14B w/ Web Search tools\" baseline is missing from Table 1. Can you add the results?\n3. Figure 5 is difficult to interpret. It might be clearer to present a direct numerical comparison between your model vs. the no-SFT baseline, perhaps in a format similar to Table 5.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper tackles the relevant and interesting problem of training tool-use agents. It presents a solid approach based on RL, with a notable mechanism for encouraging self-reflection (though some technical details remain unclear, as noted in the questions below).\n2. The empirical performance is strong. The model outperforms baselines on most target datasets and also demonstrates good OOD generalization.\n3. The analysis in Sec. 3.3 provides helpful insights on behaviors of tool-use agents trained with RL and empirical lessons on how to train them.", "weaknesses": "1. Limited discussion of related work: There is a significant body of work on using RL for general tool use, not limited to search [1,2,3]. The paper would be stronger if it more comprehensively discussed and positioned itself relative to this broader line of related work.\n2. Narrow experimental scope: The evaluation is focused on multi-hop question answering. While I think this is acceptable and does not undermine the paper's core contributions, showing the framework's effectiveness on other types of tasks would make the work even more impactful. If time and resources do not permit, it would be great if the authors can include some discussions.\n\n[1] ReTool: Reinforcement Learning for Strategic Tool Use in LLMs\n[2] Tool-Star: Empowering LLM-Brained Multi-Tool Reasoner via Reinforcement Learning\n[3] Synthetic Data Generation & Multi-Step RL for Reasoning & Tool Use", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Rd2v4lRsxI", "reviewer": "Reviewer_oPma", "summary": "The paper introduces WebSeer, a web search agent that learns to build deeper and more reliable tool chains for multi hop questions. It first collects long reflective trajectories by rejection sampling for supervised fine tuning, then runs a self reflective RL stage where the model can resubmit an answer, read feedback, and refine its plan. The agent uses a small toolset for search, page reading, and code execution, and it treats submitting the final answer as a tool. Training is site restricted on Wikipedia, while inference runs on the open web. A 14B model beats strong baselines on seven benchmarks and also does well on FanOutQA, FRAMES, SimpleQA, and Bamboogle. Taken together, the two stage recipe turns shallow tool use into steadier and longer chains and lets one model plan, act, verify, and decide when to stop. It also shows that learning in a clean, restricted setup can carry over to the open web with strong gains.", "questions": "- Restricted to open transfer\nDid you consider building a training set that mixes a small portion of open-web episodes with wiki episodes, then run SFT and also SRRL on the mixed set? If yes, how did accuracy, chain length, and number of submissions change as the share of open-web episodes grew? If not, what behavior would you expect and why?\n\n- SFT verifier details\nWhat model do you use for the verifier? What is the prompt, decoding setup, retry budget, and timeout? What rule accepts or rejects a candidate answer, and does the verifier use the same tools and constraints as the agent?\n\n- Verifier impact\nHow much does verifier stringency change the SFT distribution and later RL stability? If you lower or raise the verifier bar while keeping SFT size fixed, how do single-step share, tool-call patterns, and final accuracy move?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "- Clear and practical idea: learn long, reflective tool use in a clean Wiki-only setup, then run on the open web. The two-stage SFT → SRRL recipe is simple, and “submit answer” is treated as a tool, which makes stopping explicit.\n- Strong results across many benchmarks and tough OOD sets, with consistent gains over strong web-agent baselines.\n- Good behavior analysis and ablations that explain why it works: tool-call depth becomes more balanced, removing SRRL hurts, and the SFT data mix matters.\n- Interfaces and evaluation are well specified, with concrete tool APIs and LLM-as-a-judge prompts, plus transparent training and compute details.\n- Meaningful impact: the method makes web agents plan deeper and verify better, and the recipe is easy to adopt in other agent settings.", "weaknesses": "- The open-web transfer story is promising but the ablations are only partial. The paper already shows that limiting RL to a single submission hurts, analyzes tool-use distributions, and studies how the SFT data mix changes behavior. What’s missing are comparisons across training regimes (restricted vs mixed vs open), a sensitivity check for the page-reading/normalization stack, and systematic curves that relate accuracy to chain length under different tool budgets, search-depth caps, and submission-discount settings. A short discussion or small exploratory study on these fronts would make the restricted-to-open claim feel more complete.\n\n- The SFT verifier is under-specified and hard to reason about. We don’t know the model family/size, the prompt template and decoding settings, or the retry budget. It’s unclear how it accepts or rejects a candidate (string match vs semantic check), how it handles aliases and partial answers, or whether it uses the same tools and constraints as the agent. We also don’t see rates: how often it disagrees and forces a re-check, what fraction of trajectories get kept, how long those trajectories are, and how much of the SFT set ends up single-step vs multi-step. Without these basics, reproducibility is limited and it’s difficult to tell how much the verifier shapes the data distribution and later RL stability.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ChrUavuAP5", "reviewer": "Reviewer_KqpE", "summary": "This paper tackles the challenge of training open-domain, web-based question-answering agents capable of multi-hop reasoning and robust self-correction. Existing web agents often stop prematurely, fail to verify their reasoning, or overfit to closed-corpus retrieval. The proposed system, WebSeer, aims to build deeper and more reflective reasoning trajectories by combining:\n\n- Self-Reflective Supervised Fine-Tuning (SFT): multi-turn rejection-sampling trajectories verified by an independent verifier model, teaching the agent to backtrack and self-correct.\n\n- Self-Reflective Reinforcement Learning (SRRL): a reinforcement phase where the model can submit intermediate answers multiple times, receive F1-based feedback, and continue improving under a shaped reward.\n\nThe final agent interacts with real web tools (search, webpage reader, code executor) and is evaluated across seven multi-hop QA benchmarks, achieving state-of-the-art performance and strong generalization to out-of-distribution datasets.\n\nContributions:\n\nIntroduces a two-stage self-reflection training pipeline that operationalizes “reason-verify-refine” behaviors in web agents.\n\nProposes a self-reflective RL framework (SRRL) that allows iterative answer submission and feedback-guided reasoning.\n\nDesigns a trajectory-level reward that balances answer correctness, brevity, and depth of reasoning.\n\nDemonstrates substantial improvements over strong baselines (Search-r1, Qwen2.5-14B, etc.) on both in-domain and open-web QA datasets.", "questions": "How accurate is the independent verifier during trajectory construction? Did you measure verifier precision/recall or its effect on SFT data quality?\n\nHow sensitive is WebSeer’s performance to the number of verifier retries \n$K$ and the validity predicate threshold?\n\nIn SRRL, how did you balance the trade-off between early termination (short trajectories) and the benefit of deeper reflection?\n\nCould the approach generalize beyond QA (e.g., web-based decision-making or scientific reasoning tasks)?\n\nWhat is the relative wall-clock cost of SFT vs SRRL stages, and is RL training feasible for larger backbones (e.g., 70B models)?\n\nWould a joint training of verifier and reasoner be possible or beneficial compared to separate optimization?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Clear motivation and novelty: The idea of explicit self-reflection integrated into both SFT and RL phases is well justified and fills a gap between shallow tool use and deep verification-oriented reasoning.\n\nStrong empirical performance: WebSeer consistently outperforms baselines on seven benchmarks, with notable OOD generalization (e.g., +15.9 on NQ, +27.2 on 2Wiki, +4.0 avg over Qwen2.5-14B).\n\nWell-structured training pipeline: The paper carefully separates data construction, SFT masking (to avoid overfitting to observations), and SRRL fine-tuning, providing a replicable framework.\n\nInsightful analysis: The ablation studies show how self-reflective trajectories and multi-submission feedback both contribute to deeper tool usage and higher final accuracy.", "weaknesses": "Limited transparency of verifier reliability: The paper assumes the verifier’s correctness predicate $\\psi$\n yields high-quality reflective trajectories, but quantitative verifier accuracy or error propagation analysis is missing.\n\nReward design justification: The choice of exponential discount on multiple submissions is somewhat ad-hoc; an analysis of sensitivity to discount parameters or exploration–exploitation balance would strengthen claims.\n\nComputational cost: The reflective data collection (multi-turn verifier rejection sampling) seems expensive; runtime statistics or data efficiency analysis are lacking.\n\nAblation on real-web vs synthetic setting: While generalization results are good, it remains unclear whether the model trained with restricted search data directly scales to dynamic, noisy real-web environments.\n\nClarity and formalism: Some algorithmic details (e.g., GRPO vs DAPO implementation differences) could be more rigorously formalized.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hC5DffiDM4", "reviewer": "Reviewer_g4kW", "summary": "This paper introduces a search agent designed for multi-step question-answering on the web. The agent's core innovation is a two-stage training framework: 1) using multi-turn rejection sampling, where a reasoner model's proposed answers are validated by a separate verifier, to generate a high-quality dataset of self-correcting reasoning paths. 2) using Self-Reflective Reinforcement Learning,  where the agent can submit multiple answers and receive textual feedback on its performance, allowing it to iteratively refine its strategy.", "questions": "Please refer to weakness #1 and #2.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The research problem—improving the depth and self-correction capabilities of search agents—is interesting and important, as current agentic systems may struggle with error accumulation and shallow reasoning.\n\n\nThe proposed two-stage training framework, which combines supervised fine-tuning on curated reflective trajectories with reinforcement learning, is a logical approach to the problem.\n\n\nThe reward mechanism in the RL stage, which provides explicit feedback on answer quality and allows for multiple submission attempts, is a reasonable design choice.", "weaknesses": "Overall, the presentation of the methodology could be further improved.\n\nFor example, the paper's methodology relies on an independent \"verifier\" model (V) to generate the SFT dataset. The description of this verifier is insufficient (only mentioned between line 188 and line 197). It is unclear how the verifier is implemented. If the verifier itself is unreliable or follows an incorrect tool-use path, its judgments would be flawed, which could introduce noise or incorrect patterns into the SFT dataset and undermine the entire training pipeline. The paper needs to detail the verifier's implementation.\n\n\nIn addition, the term \"F1-based feedback\" is not standard in reinforcement learning literature. While the paper explains it as a feedback signal, it should be more formally defined as part of the observation space or reward function to avoid ambiguity. (It is possible that the authors refer to F1 score in classification. But according to line 253, it appears that F1 is calculated based on one predicted answer and one ground-truth answer, while the F1 score in classification should be calculated based on a set of predictions and GT answers.)\n\n\n(Minor issue) The paper requires careful proofreading. For instance, lines 182 and 183 contain nearly identical, repetitive sentences: \"To address this limitation, we propose a multi-turn rejection sampling method to collect reasoning paths that incorporate reflective patterns. To overcome this limitation, we propose a multi-turn rejection sampling method that collects reasoning paths enriched with reflective patterns.\"", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "haUMyXUPYj", "reviewer": "Reviewer_NZPh", "summary": "The paper proposes WebSeer, a search agent capable of performing multi-turn, real-world searches for QA tasks. The authors introduce a two-stage training pipeline that includes SFT and RL. After training, WebSeer demonstrates improvements in both the number of effective tool calls and accuracy.", "questions": "1. The paper emphasizes that WebSeer benefits from a self-reflection mechanism. But in Case study 1.2, the agent's outputs show no reasoning or reflection step. It directly generate tool calls in each round. Could the author clarify how self-reflection is implemented?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. Training an autonomous search agent capable of conducting deep, strategic, and reflective searches addresses an important and challenging research problem, and also has significant practical application value.\n\n2. WebSeer extends to real-world search scenarios and incorporates more reasonable tools, such as Webpage Reader, enhancing its applicability.\n\n3. The method achieves SOTA performance on HotPotQA and SimpleQA benchmarks.", "weaknesses": "1. In the experiments (Table 1), WebSeer (a 14B model) is compared against baselines using smaller 7B/8B models. This raises uncertainty about whether the observed improvements result from the proposed method itself or simply from scaling up model size.\n\n2. The SRRL method is a key contribution of the paper. However, Figure 5 (top) shows that SRRL only improves accuracy by 3.6% on HotPotQA and 2.9% on SimpleQA. Were the experiments conducted multiple times to assess variance? The relatively small gains make it unclear whether the improvement is statistically significant. \n\n3. A main feature of SRRL is that it allows the model to submit multiple answers within a single dialogue turn. However, there is no ablation study isolating the effect of this design choice. How would performance change under a conventional setting that allows only one submission during training?\n\n4. At Line 241, the paper claims that its RL framework unifies SFT and RL under a self-reflection mechanism. However, Sec 2.4 does not describe how this unification is achieved. Based on the content, SRRL appears to directly apply GRPO for policy optimization.\n\n5. (minor) Lines 182-183 contains repeated sentences.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)