WideSearch: Benchmarking Agentic Broad Info-Seeking
Authors: Ryan Wong
Jiawei Wang
Junjie zhao
Li Chen
Yan Gao
Zhanglong
Xuan Zhou
Zuo Wang
Kai Xiang
Ge Zhang
Wenhao Huang
Yang Wang
WANG KE
Venue: ICLR 2026 Poster (2026)
Keywords: LLM Evaluation
Info-Seeking Benchmark
Search Agent
Primary area: datasets and benchmarks

Abstract: From professional research to everyday planning, many tasks are bottlenecked by wide-scale information seeking, which is more repetitive than cognitively complex. With the rapid development of Large Language Models (LLMs), automated search agents powered by LLMs offer a promising solution to liberate humans from this tedious work. However, the capability of these agents to perform such "wide-context" collection reliably and completely remains largely unevaluated due to a lack of suitable benchmarks. To bridge this gap, we introduce WideSearch, a new benchmark engineered to evaluate agent reliability on these large-scale collection tasks. The benchmark features 200 manually curated questions (100 in English, 100 in Chinese) from over 15 diverse domains, grounded in real user queries. Each task requires agents to collect large-scale atomic information, which could be verified one by one objectively, and arrange it into a well-organized output. A rigorous five-stage quality control pipeline ensures the difficulty, completeness, and verifiability of the dataset. We benchmark over 10 state-of-the-art agentic search systems, including single-agent, multi-agent frameworks, and end-to-end commercial systems. Most systems achieve overall success rates near 0\%, with the best performer reaching just 7\%. However, given sufficient time, cross-validation by multiple human testers can achieve a near 100\% success rate. These results demonstrate that present search agents have critical deficiencies in large-scale information seeking, underscoring urgent areas for future research and development in agentic search.

Published as a conference paper at ICLR 2026
WIDESEARCH: INFO-SEEKING
BENCHMARKING AGENTIC BROAD
Ryan Z.Y. Wong,, Jiawei JannjJJnjii Zhao, Li Chen, Yan Gao, Long Zhang, Xuan Zhou, Zuo Wang, Kai Xiang, Ge Zhang, Wenhao Huang, Yang Wang', Ke Wang? Bytedance Seed https: http://wdeeeaeceseeeeeeeee github. io
ABSTRACT
From professional research to everyday planning, many tasks are bottlenecked by wide-scale information seeking, which is more repetitive than cognitively com- plex. With the rapid development of Large Language Models (L.LMs), auto- mated search agents powered by LLMs offer promising solution to liberate hu- mans from this tedious work. However, the capability of these agents to perform such "wide-contex(" collection reliably and completely remains largely uneval- uated due to lack suitable benchmarks. To bridge this gap, we introduce WideSearch,
benchmark engineered to evaluate agent reliability these large-scale collection tasks. The benchmark features 200 manually curated ques- tions (100 in English, 100 in Chinese) from over 15 diverse domains, grounded in real user queries. Each task requires agents to collect large-scale atomic in- formation, which could be verified one by one objectively, and arrange into a well-organized output. A rigorous five-stage quality control pipeline ensures the difficulty, completeness, and verifiability of the dataset. We benchmark over 10 state-of- sate-of-the-asd the-hhtrtt art agentic search systems, including single-agent, multi-agent frame- works, and end-to- and-to-end commercial systems. Most systems achieve overall success rates near 0%, with the best performer reaching just 7%. However, given sufficient time, cross-vall validation by multiple human testers can achieve a near 100% success rate. These results demonstrate that present search agents have critical deficiencies in large-scale information seeking, underscoring urgent areas for future research and development in agentic search.
!
Repetitive
WideSeerch Task
Human
Estimeted Time: hour Risk: Prone to Human Ervor
cm sods my iier tily publichy
Artorney posttone
whis the Deportmannt Jurricn USA2005, specriiolly these wink open date betwoue fenuary 2015
2025.
Tool Using
Process: Automated Autonomous Estinated Time: <5 minute
Sourced Multi-Ehily Structured Olopur
Output: Structured & Verifiable
1005
Insocuracy
Search Agent Overiong Context
Incomplatenees
Figure 1: A conceptual comparison of manual and agent-based approaches for WideSearch tasks. demonstrates the labor-intensive nature of the manual approach and the failure modes of agent approach, which calls for systematic evaluation to quantify agent performance and reliability.
"Co-first authors, 'Co-oorrepoondigg authors. Correspondence: syan.wong 0660@ma 1 com.

---

Published as a conference paper at ICLR 2026
Demenaice
DespSearch
DespResscrct
Wiadearch (Dural
Ove
Fixil sooil,, lakkea fatt inttiie
Exphece groerato conpunaenaiie
Sysconkinlly Exi
Nature Arower
gask begeessiee sosi intimaa
[Subjective Indights)
( Analysis Synthesii)
repor..
Esc. givm crtte
DeepResearch Bench
Ourput Formeot
oatitian
erretuud tolds of
feda.
farm sol.
Lo..
Evouurtion
Enat Match.
Sunjjrtiie Hight:
Ertlastion
Quanilatiion
Mettia
Deght,
(Narow))
(Bread) Scope
Flanee).
Soey Ligh.
(iigh so Very Hilh. laagstvv eming analytted
Machen
Horren Deffcuuty
l'ree eagritive
Smeehhtkreedd best rea-
Reasoning & Deep Mining)
Sodle & Reliability)
Cure Model Charllinge
Decy mouieis to stae
Tufertmtion sou lemin lov-
Balesmiieemin and 6
Browsedommp 4
WideSearch (Our Work) E
hennn
Productivtty Vaku
Sellw the "I cwn't Red at peokies.
Sieiee the "I cwn't witte
Selven the "I do Eae staur valuse ove-
(Objective Focts)
isf op watte
(8)
(b)
Figure 2: An overview and detailed comparison of DeepSearch, DeepResearch, and our WideSearch. (a) illustrates the high-level relationships and operational domains of the three
paradigms. (b) provides a detailed breakdown, contrasting them across key dimensions including
core tasks, evaluation methods, and primary value propositions.
INTRODUCTION
With the advent of advanced agentic frameworks such as OpenAl DeepResearch (OpenAl, 2024 and Manus (Manus 2025), the development of agent systems based Large Language Models (LLMs) is entering its second half, where the focus is rapidly shifting from demonstrating novel capabilities to achieving practical, real-world reliability. This transition is driven by fundamental recognition of the inherent limitations in standalone models: their finite parameters make it im- possible to store all knowledge, the prohibitive cost of retraining makes them lag behind real-time information, and they naturally struggle with long-tail tail or specialized facts. Consequently, in this evolving domain, the ability to effectively utilize search tools has become paramount. The most critical question in this race is no longer just what an agent can do, but how we can measure and improve its ability to leverage search in authentic user scenarios to deliver tangible value and drive meaningful product iteration.
Our in-depth analysis of real-world user queries reveals significant gap: common and critical class of information-seeking tasks is not adequately evaluated by existing agent benchmarks. We term this category WideScarch, which involves tasks that require an agent to thoroughly and accu- rately acquire all large-scale atomic information meeting a series of criteria, and then arrange it in well-organized output. For example, financial analyst may need to find all companies in a sector that meet specific revenue and growth criteria, or a job seeker may need to find all job vacancies that match their criteria for role, location, and experience level. For humans, executing such tasks is excruciatingly tedious; as depicted in Figure [ the transition from this laborious manual process to an automated agent workflow promises immense efficiency gains, but also introduces new failure modes that demand systematic evaluation. Consequently, WideSearch carves out distinct problem space. As illustrated in Figure [2] it diverges from DeepSearch, which targets the can't find it" problem of locating specific, hard-to-find facts, and DeepResearch, which addresses the "I can't write it well" problem of synthesizing complex narratives. Instead, WideSearch tackles tasks whose primary challenge is not cognitive difficulty but operational scale and fidelity- the "I could do it, but the sheer volume is overwhelming" problem- domain largely overlooked by current benchmarks.
To systematically evaluate this paradigm, we introduce WideSearch, the first benchmark specifically designed for this purpose, supported by sophisticated multi-stage data collection and verification framework, as well as a hybrid automated evaluation system that ensures objectivity. Benchmarking more than 10 state-of-the-art agent systems reveals a stark reality: current systems are profoundly challenged by the demands of comprehensiveness and fidelity at scale. The overall success rate is exceptionally low, with even the top- sop-erfforing mulli-agent framework achieving a mere 5.1%. 1,, and individual humans also struggling 20%. Our key insight, derived from a test-time scaling analysis, is that this failure does not stem from an inability to find individual facts item-level F1 scores can approach 80% with sufficient retries. Rather, the bottleneck is that must ensure the absolute
2

---

Published as a conference paper at ICLR 2026
completeness and accuracy of each atomic unit of information within large-scale search. Any single data omission or error, or the integration of extra data into the final result, results in total failure of the task execution. Our detailed eπror analysis traces this failure to fundamental deficiencies in advanced agentic capabilities, such as incomplete planning, a lack of reflection to iterate on failed searches, and the inability correctly use retrieved evidence. These findings provide clear roadmap, suggesting that future progress hinges on developing more sophisticated agent models and architectures, particularly multi-agent frameworks that enable parallel search and cross-validation mimicking the collaborative human processes required to tackle these complex, large-scale tasks.
WIDESEARCH BENCHMARK
The construction of the WideSearch benchmark a meticulous, human-centered process designed to ensure that each task is challenging, realistic, and aligned with our goal of evaluating wide-context information gathering. The entire workflow, from question design to final inclusion, is governed by strict set of criteria and multi-stage validation protocol.
2.1
TASK DEFINITION
The fundamental task in the WideSearch benchmark challenges an LLM agent act as a diligent information seeker. Given a complex natural language query and a predefined table schema, the agent's objective to populate the table by systematically gathering. synthesizing, and verifying information from the live web. This emulates real-world information-seeking scenarios that re- quire discovery and aggregation rather than simple fact retrieval. Formally, each task instance in WideSearch is defined by tuple (Q,5), where:
A Query (Q): A natural language question that implicitly specifies set of target entities and the information required about them. For example, Q could be: "I want to apply for Master's programs in civil engineering starting in 2026. Could you help me find the minimum GPA requirements for admission Ivy League institutions in the US and Group of Eight universities Australia?"
Table Schema (S): A predefined structure $ = [C1, C2,.... ...,, Cm}. where each Cj is a column header representing an attribute (e.g., 'Country', 'University', 'Alliance', 'Minimum GPA Require- ment'). ). The schema defines the exact structure of the required output for objective evaluation.
The agent's goal is to interact with a web environment, primarily via search tools, to produce final, populated table, Tugemt This objective decomposes into two primary challenges:
Entity Set Identification: The agent must first identify the complete and correct set of entities, E = {e1, ......, }. that satisfy the constraints of the query Q. In this example, the entities are the 8 Ivy League institutions and the Group of Fight universities. This tests the agent's ability to conduct a comprehensive search across different domains (US and Australian higher education).
Attribute Filling: For each identified entity e; E E, the agent must find corresponding values for each attribute (C1, C2,,CC..... C,...., Cm defined in the schema S, sourcing the information from web pages.
The final output, Tagent. is therefore table with each cell Tagont (i, j) contains the value of attribute Cj for entity e,, The quality this output is then measured against ground-truth table to assess its completeness and factual accuracy.
and columns (as defined by S), where
2.2 TASK CONSTRUCTION PRINCIPLES
The construction of tasks within the WideSearch benchmark is guided by a rigorous, principled methodology to ensure their quality, relevance, and alignment with the challenges of wide-context information seeking. Each task is manually curated by domain experts and must satisfy the following six fundamental principles:
High Search Volume and Breadth: Tasks are defined by their extensive information breadth, re- quiring the agent collate numerous distinct data points across multiple entities. This inherent breadth necessitates a high volume of search interactions and prolonged, multi-step procedural trajectory, distinguishing them from tasks that require only a singular, deep line of inquiry.
Temporal and Contextual Invariance:
The ground-trutt answers exhibit high stability. They static over time and are independent of geographical, ideological, or socio-cultural contexts, thereby guaranteeing the benchmark's 's long-term validity and global applicability.
3

---

Published as a conference paper at ICLR 2026
Predicted Table
Ground-truth Table
Large Fool of Real World User Queries
Humon onnotators screes, refine, and restructure quenes
Echoustive web search to create galch standard answer and collect metrics.
Refuse anbiguous defined
Volidetion?
Score:
Refuse ample
entwerable queries foode infereal knowledge
Stage
Shage
Nemmlizzation Aligrment
Pruning baste human netrica
question against non ougmented
Stage
Iterative Refinement & Validation
Hybrid Scoring
Previsooal Suncmork
Score Simplority 0.95
No
Evect Mascchig
Namerica Apprex
Data Metching
Matching
Judge
Homer
Revide Quastions Asnotations
iid! Findl Metrics
Aroi Whdetearch benchmark
Figure 3: An overview of our integrated data pipeline, detailing the five-stage data curation and validation pipeline (left), and the automated evaluation pipeline (right).
Objective Verifiability: Each task is associated with a deterministically verifiable set of facts. This allows for objective, consistent, and reproducible scoring against predefined gold standard.
The entire corpus of information required to formulate complete answer guaranteed be publicly accessible via standard web search engines, ensuring that tasks are solvable without privileged access to information.
Public Accessibility:
Tasks are explicitly designed exceed the bounds of LLM's parametric knowledge. Successful completion is therefore contingent upon the agent's ability to engage active, iterative, and effective web search, rather than relying on memorized information.
Reliance External Tools:
Scenario Diversity: The benchmark encompasses a heterogeneous collection scenarios. This cross-domain diversity ensures that evaluate generalizable agent capabilities, such planning and synthesis, rather than task-specific or domain-dependent knowledge.
2.3 DATA CURATION AND VALIDATION PROCESS
We implement multi-stage data curation and validation pipeline (shown in Figure 3). It transforms real-world information needs into standardized, high-quality tasks. A task is only accepted into the final benchmark after successfully passing all stages and the final iterative validation loop.
Sourcing and Refinement of Candidate Questions: We start with large pool of real user queries from diverse domains. Human annotators screen these raw queries, refining promising candidates into clear, unambiguous questions that align with our design principles.
For each question, an annotator conducts exhaustive web search create gold-standard answer. During this process, they record key metrics: time to completion, number of search queries, and the count of unique web pages consulted.
Gold Standard Annotation and Metric Collection:
Parametric Knowledge Filtering: To ensure tasks necessitate tool use, test each question against powerful LLMs. If any model can answer correctly from its internal knowledge, the question is discarded, guaranteeing our benchmark evaluaes genuine search and synthesis capabilities.
4

---

Published as a conference paper at ICLR 2026
To control the quality of the dataset, enforce minimum complexity threshold using the human-collected metrics. Tasks that an annotator completes in less than 10 minutes or by consulting fewer than 10 unique web pages are discarded.
Difficulty-Based Pruning:
Iterative Refinement and Validation: Tasks that pass the initial filters undergo a final validation loop to align automated scoring with human judgment. We generate response for each task using a commercial agentic system and have it rated by both our evaluation system (details in Section 2.4 and human experts. If the scores show a discrepancy (i.e., similarity is below 95% threshold), the task is revised. This cycle continues until our automated metries reliably predict human assessment.
This five-stage pipeline ensures every WideSearch task is complex, verifiable, grounded in real- world needs, and resistant to memorization. Most importantly, it calibrates our automated evalua- tion to be predictive of human judgment, creating a robust and reliable testbed for advanced search agents. An illustrative example of final task is provided in Appendix C Moreover, the bench- mark composition and statistics are shown in Appendix D
2.4 EVALUATION FRAMEWORK AND METRICS
To facilitate an accurate, scalable, and nuanced assessment of agent performance, develop com- prehensive evaluation framework centered around an automated scoring pipeline. This framework designed to handle the structured nature of our ground-truth data which is stored in tables and to address the inherent complexities of natural language responses.
2.4.1
AUTOMATED EVALUATION PIPELINE
To ensure robust and scalable evaluation, we designed an automated pipeline that formalizes scoring as table alignment and cell-wise verification. As depicted in Figure 3 our hybrid approach combines deterministic rule-based checks with semantic judgments from an LLM-as-a-judge We use GPT- 4.1-2025-04-14 14 as default judge model. The entire process is grounded in annotated data containing pre-defined primary keys for row alignment and column-specific methods for cell scoring.
Syntax Validation and Alignment: The pipeline begins with syntax validation. A response receives a score of zero if it is not a parsable Markdown table or if its column headers do not semantically match the ground truth in number and name; minor string differences are aligned using mapping prompt (Appendix H). Passing tables are then normalized. To enable alignment, we use mapping prompt to standardize entities the primary key columns. Finally, the predicted table is joined with the ground-truth table on these keys to identify matched rows, false positives, and false negatives.
Hybrid Item-level Scoring: For each pair of aligned rows, we iterate through the corresponding cells. The evaluation method for each cell dictated by its column's pre-annotated type, enabling nuanced and accurate scoring. Our framework supports comprehensive set of categories: (1)
Exact Match: For strings where absolute precision is paramount. (2) Numerical Approximation:
To validate numbers while allowing for minor, acceptable floating-poii or formatting variations. (3) Date Matching: To semantically compare dates that may appear in different but equivalent formats (e.g.. "July 4th, 2024" vs. "2024-07-04"). (4) URL Matching: To normalize and validate the correctness of web links. (5) LLM-aa---judge- Reserved for complex cases with high lexical variation (such translated names nuanced descriptions) that require semantic understanding for a fair assessment. The prompt of the LLM-as-a-judee is shown in Appendix H
2.4.2 EVALUATION METRICS
We assess agent-generater tables using three metrics of increasing granularity. The most stringent, Success Rate (SR), is a binary metric requiring the predicted table to be a perfect match to the ground-truth table in both content and structure. For more detailed analysis, the Row-level F1 Score evaluates performance by treating each row as a fundamental unit, comparing the set of pre- dicted rows (Prows) against the ground-truth rows (Grows) to calculate precision and recall. For the most fine-grained assessment, the Item-level F1 Score considers each individual cell as the basic unit, computing precision and recall based on the multiset of items in the predicted table (Pitem)) versus the ground-truth table (Gstema).
Furthermore, to provide a more comprehensive evaluation, we perform N independent runs for each task and report performance using three aggregation strategies: Avg@N: @N: This metric calculates the arithmetic mean of a given score (e.g., Success Rate, F1 Score) over N trials for a task. Pass@ N: This metric measures the percentage of tasks that were solved successfully at least once in N trials.
5

---

Published as a conference paper at ICLR 2026
It is used specifically for Success Rate. Max @N: For metries like the F1 Score, this takes the single highest score achieved across the N trials for each task, and then averages these maximum values over all tasks in the dataset.
3 EXPERIMENTS
3.1 EXPERIMENTAL SETUP
To evaluate agent capabilities on our WideSearch benchmark, our experiments include three aspects: Single Agent, Multi-Agent Framework and End-to-End systems. All agents use a search tool and webpage reader. We intentionally use naive agent architectures with minimal heuristic design to test the models' core capabilities. Appendix lists all model API identifiers. The details of the agents shown in Appendix G
Single Agent. We first benchmark the intrinsic problem-solving abilities of individual LLMss. In this baseline setup, a single LLM is responsible for the entire task. The evaluated models are: DeepSeek-
RI (uo et al. 2025). Doubao-Seed-1.6 (Thinking and Non-Thinking) (Seed et al. 2025). Claude
Sonnet 4 (Thinking) (Anthropic 2025). Gemini 2.5 Pro (Google Gemini (025b)). Kimi K2 TTam
et al.
2025). Qwen3-235B-A22 Thinking-2500 (Yang et al. 2025), OpenAl GPT-5 (OpenAl)
2025a), and OpenAl o3 (OpenAl 2025b).
Multi-Agent Framework. To test a "divide-and-conquee strategy, we evaluate the same foundation models within multi-agent framework. A main agent decomposes the query and aggregates results from multiple sub-agents that execute sub-tasks parallel. This allows a direct comparison of the architectural impact versus the single-agent paradigm.
End-to-End System. To contextualize performance, we benchmark against the integrated web- browsing modes of leading commercial systems: Gemini 2.5 Pro, Claude Sonnet (Thinking), and OpenAl o3. We manually get the results from the web interface.
Human Evaluation. Annotating the ground truth for WideSearch is arduous process. To gauge human performance, we had 10 annotators each solve two randomly selected questions (one in Chinese, one in English) from set of 20. Participants were given unlimited time and could use any tools, including Al assistants, to ensure their answers were complete.
3.2
MAIN RESULTS
We report the main experiment results in Table T and the complete results in Table We provide The conclusions are obtained as follows:
two typical example trajectories in Appendix K
Existing models still lack the advanced agentic abilities. Current advanced large language models show fundamental weaknesses when performing large-scale information-seekine tasks, with failures stemming from fundamental cognitive deficits beyond simple search inaccuracies. They exhibit poor planning by struggling break down complex questions into comprchensive sub-queries, which leads to incomplete information seeking. Furthermore, they lack reflection and fail to dynamically adjust their search strategy when initial attempts are unsuccessful, often giving up or answering with insufficient data instead of trying new methods. Even when they successfully find relevant information, they demonstrate faulty evidence use by misinterpreting or misattributing the content. These basic deficiencies in planning, dynamic adjustment, and reasoning are the primary reasons for their extremely low success rates on such complex tasks.
Multi-agent mode outperforms the single-agent mode on WideSearch. The multi-agent frame- work, using dlivideannd-oonuess strategy, consistently and significantly outperforms the single- agent mode on WideSearch tasks by more effectively addressing their inherent breadth. Although absolute success rates low for both, the multi-agent system shows distinet advantage in F1 scores, which measure partial correctness. This superior performance is due to its architecture, where planner decomposes a broad query into parallel sub-tasks assigned to different agents. This parallel search and division of labor not only improves the breadth and efficiency of information seeking but also mimics the specialized, collaborative process of human expert teams, making the framework better suited for complex, wide-ranging searches.
Current commercial AI assistants cannot yet seek information at a large scale. Although top commercial AT models have some information retrieval capabilities in their integrated web Browse

---

Published as a conference paper at ICLR 2026
Table 1: Main results on the WideSearch benchmark. We report Success Rate (SR), Row-level FI, and Item-level F1 for all evaluated systems. All scores are reported as percentages (%). We use boldface indicate the gighest-performing metric in each mode, and underlining mark the second-highest-perfonee metric.
Model System
Success Rate (%)
Row F1 Score (%)
Item F1 Score (%)
# Tool Calls
Avg@4
Puss@4
Avg@4
Max@4
Avg@4
Max@4
Search
Web Browse
Single Agent
GPT-5
13.5
37,9
52.2
54.0
68.2
25,64
14.27
OpenAl o3
9.0
34.0
44.1
52.6
62.3
13.26
5.75
Claude Sonnet 4 (Thinking)
2.3
5.0
31.7
41.9
$7.9
66.7
8.20
3.42
Gemini 2.5 Pro
1.5
5.0
30.0
41.4
51.0
63.6
7.48
1.58
Kimi
3.5
29.7
41.4
54.4
65.1
10.78
2.22
Qwen W-253 (Thinking)
0.5
1.5
20.6
30.8
44,9
55.5
3.01
2.61
DeepSeck-RI
0.4
1.5
20.7
31.7
41.3
55.1
2.91
1.40
Doubso- Suobao-Seed-L.fo (Thinking)
5.0
30.0
44.1
48.3
63.9
22,08
1.14
Doubao Doubaao-eed-1..
Seed-1.6 1.6 (Non-Thinking)
3.5
27.2
39.9
49.0
62.0
8.01
1.82
Midii-Agent Framework
GPT-5
7.3
12.0
42.5
54.2
59.7
74.5
37,50
22.00
OpenAl
9.5
37.8
50.5
57.3
68.9
16.29
Claude Sonner 4 (Thinking)
3.6
6.5
38.5
52.2
62.2
73.1
27.64
11.60
Gemini 2.5 Pro
2.0
6.5
33.5
44.6
57.4
66.3
20.73
4.72
Kimi
6.5
36,2
49.6
61.2
70.7
28.79
8.85
Qwen3-235B (Thinking)
0.6
2.0
25.8
37.4
51.9
62.6
13.72
14.52
DeepSeck-RI
0.8
3.0
22,9
36.6
44.3
60.3
11.81
7.02
Doubao Seed-1.6 (Thinking)
5.5
34.0
48.9
54.6
69.7
$2.34
6.44
Doubso-Soede Sced-1.6 (Non-Thinking
4.5
29.7
42.7
52.8
65.1
14.83
5.18
End-to- End System
OpenAl o3
3.0
5.5
23,9
36.0
45.5
56.5
Claude Sonnet (Thinking)
5.0
24.1
33.5
48.4
58.5
Gemini 2.5 Pro
8.0
36.6
45.4
59.1
67.2
Human
20.0
69.2
82.4
modes, the experiment results show that they still struggle with WideSearch. Several leading com- mercial models tested in the experiment, including Gemini 2.5 Pro, Claude Sonnet 4, and OpenAI o3, hover around a 5% table-level success rate. Furthermore, in the early stages of the experiment, found that some specialized DeepResearch systems tend generate lengthy reports rather than the single, well-formatted table required by the task. It demonstrates that the design of current main- stream Al assistants has not yet been optimized for large-scale information gathering. and they lack the stability and precision required to become reliable productivity tools.
Even humans cannot achieve high success rate in single-player mode. Experimental results show that even when given ample time and access any tools, the success for single individual completing the task independently is merely 20%. This outcome highlights the inherent difficulty of task itself. key characteristic of WideSearch tasks is the extreme density of data points, which means complete answer may contain thousands of individual facts. Under these circumstancess, any minor eиror (an extra, a missing, or an incorrect piece of data) results in the failure of the entire task. The creation of the ground truth is arduous process requiring multiple annotators and extensive gross-validation. This high standard confirms that WideSearch is challenging yet reasonable benchmark for evaluating the robustness and completeness of search agents.
For detailed domain-level performance analysis, please refer Appendix !
3.3 CONSISTENCY WITH HUMAN EVALUATION
To validate the stability and reliabil- ity of our proposed automated evalu- ation pipeline, we conduct consis- tency analysis against human assess- ment. For this analysis, we curate an evaluation set of 200 responses by randomly selecting one output from the pool of commercial agentic sys- tems for each task in WideSearch.
Table 2: Consistency between our evaluation pipeline using different judge models and human evaluation. (%)
Judge Model
Consistency Score
OpenAl o4-mini
98.3
Gemini 2.5 Pro GPT-4.1
98.1
98.0
Doubao-Seed- (Non-Thinkinl Thinking)
97.8

---

Published as a conference paper at ICLR 2026
These selected responses are then meticulously annotated by human experts to determine their item- level correctness against the ground truth.
Subsequently, we utilize our evaluation pipeline, employing different models as judges, to assess the same set of responses. The primary objective is to measure the degree of agreement between our automated pipeline's 's judgments and the human-annotated labels. The results of this compar- ison are presented in Table 2 As shown, the consistency between our pipeline's evaluation and human assessment is exceptionally high, exceeding 97.8% for all tested judge models, including both thinking and non-thinking variants. This high level of correlation underscores the effectiveness and reliability of our proposed evaluation methodology. Furthermore, it reinforces the objective na- ture of the WideSearch benchmark, demonstrating that performance can be assessed accurately and consistently without being subject to the variability of human evaluation.
The prompt of LLM-Judge is designed through multiple rounds of iteration. After manual inspec- tion, the consistency between the evaluations of LLM-Judge and human evaluations has reached a very high level. Unfortunately, there are still some inconsistent judgments between the LLM and human evaluators. For example, regarding "carlosslimhelu family" and "carlosslimheli" human annotators consider them consistent and believe the score should be 1. However, when the LLM conducted the evaluation, it output: "Standard answer: "carlossimmellfammll'' Answer provided: "carlosslimheld" The word 'family" missing, making it insufficient to determine factors such as children. Score: 0". In the current practice of LLM-Judge, such cases are usually unavoidable.
ANALYSIS
To gain a deeper understanding of the core challenges that current models face on the WideSearch benchmark, we conduct systematic analysis of the experimental data and failure cases. We catego- rize the primary failure modes into two main groups: 1) Challenges in Advanced Agentic Capabili- ties, which reflect fundamental deficiencies in complex cognitive skills such as planning, reasoning. and synthesis; and 2) Basic Failure Modes, which arise from the model's inability to reliably exe- cute explicit instructions or tool-use protocols. This classification not only highlights the technical bottlenecks of current search agents but also provides clear directions for future research.
4.1 CHALLENGES IN ADVANCED AGENTIC CAPABILITIESS
In large-scale information gathering scenarios (i.c., WideSearch). the balance between Precision and Recall remains core challenge, which consistent with the challenges faced in traditional infor- mation retrieval tasks. Experiments indicate that the model's performance is far from optimal, both at the row-level and item-level evaluations. A particularly prominent phenomenon is that Recall is significantly lower than Precision across all test subsets, as shown in Table 6 This finding reveals a critical deficiency in the current model's ability to capture comprehensive information, identify- ing that "inadequate recall" is the primary bottleneck. Through an in-depth review of the Agent's reasoning process, we have identified the following key failure patterns:
Incomplete Query Decomposition. Agents often fail to fully break down complex user intents into sufficient sub-queries. This incomplete decomposition causes them miss key information, highlighting a weakness in complex task planning. We provide an example in Figure [.I
Lack of Reflection and Iterative Refinement. Following a failed tool call, agents tend to give up rather than reflect and adjust their search strategy (e.g., by rewriting queries). They answer with incomplete data, showing a deficiency in adaptive planning. We provide an example in Figure
Failure in Evidence Utilization. Agents struggle to ground final answers in retrieved evidence. They may misinterpret, ignore, or misapply sources, revealing a critical gap between information retrieval and generation. We provide an example in Figure 1.3
Knowledge Hallucination and Factual Inconsistency. When search returns no results, agents sometimes turn to internal knowledge (which might be outdated or incorrect), leading to hallucina- tions and factual errors. It means the model may fabricate non-existent facts. This underscores the challenge of strictly grounding outputs in verifiable sources. We provide an example in Figure [.4
We manually examined the trajectories that ultimately succeeded in generating result tables but produced tables inconsistent with the ground truth tables. The results showed that nearly all unsuc-
8

---

Published as a conference paper at ICLR 2026
cessful trajectories exhibited deficiencies in the four aforementioned advanced agent capabilities, particularly in tasks involving large volumes of atomic information.
4.2 BASIC FAILURE. MODES
In addition to the sophisticated agentic deficiencies described above, we also catalog a series of more basic failures where the model failed to generate the desired output. These errors often lead directly to the termination of the task workflow. We have statistically analyzed the distribution of basic error patterns and presented it in the Table
This is one of the most common failures, typically caused by the model generating incorrect parameter formats, omitting necessary arguments, attempting call non- existent tool, leading to an API call failure.
Tool Invocation Error.
Some models fail to strictly adhere to the output format specified in the instructions, such as failing to generate a Markdown table or producing a malformed one. sometimes generates the final results in json format unstructured format.
Output Formatting Error.
Context Length Exceedance. The task is prematurely terminated because the model generated overly verbose intermediate steps or became trapped in ineffective loops, causing the total input to exceed the model's maximum context window. The core reason is overthinking.
Response Refusal. The model exhibits refusal behaviors for some queries. We have identified two primary patterns of refusal: The model perceives ambiguity in the user's question and conse- quently requests further clarification to narrow the scope of the inquiry. 2. The model deems the required information too extensive to presented in a single output, leading a direct refusal.
TEST-TIME SCALING
Allocating more compute resources during testing is a common method for probing the upper limits of a model's performance. In the experiments in this section, we use Kimi K2 as the foundation model, equipped with Search and Web Browse tools. Based on the single-agent mode, we attempt each question N times (where is expanded from to 128) and record the Success Rate (Pass @ N), Row-level F1 Score (Max @ N), and Item-level F1 Score (Max@N)
As shown Figure 4 the Item-level score shows significant improvement the num- ber of attempts increases. With the compute volume of 128 attempts, it reaches level close to 80 points. This fully indicates that single information-seeking action within WideSearch is not a particularly difficult task. Corresponding to real-world human scenarios, finding piece of basic information is not an exceedingly difficult task for person, provided enough time is spent. Quite the opposite, even with 128 attempts, the table-level score only reaches a level below 20 points. This strongly suggests that large-scale information retrieval on a fixed topic is extremely difficult task. It requires not only comprehensive but also accu- rate search results. For task with 5,000 atomic pieces of information, even you find 4,999
Test-time Performance Scaling
WideSeerch Performance
80
60
20
64
Test-time Compute
Figure 4: Test-time scaling experiments. We re- port the Pass@N for Success Rate, @ax@N for Row-level, and Item-level Fl score.
correct pieces, the entire task is considered failure if you retrieve one extra, one fewer, or in- correct piece of information. Even for humans, completing this task requires multiple annotators to perform several rounds of repeated revisions, which is how the ground truth table for each question was annotated).
Given the characteristics of the WideSearch, view the optimization of a multi-agent architecture as an important future research direction. Multiple agents can conduct parallel searches and perform mutual cross-vas voossvalliation, process that aligns highly with the cognitive process of human annotation.

---

Published as a conference paper at ICLR 2026
6
RELATED WORK
6.1 BENCHMARKS FOR SEARCH AGENTS
The evaluation of search agents has evolved from simple fact-retrieval (Joshi
al. 2017
Kwiatkowski et al. 2019) and multi-hop question answering (Yang et al. 2018 Ho et al.
2020
Trivedi et al. 2022) to more complex benchmarks that require deep. non- linear investigation of sin-
gle topics or comprehensive report generation (Lei al.) |025] Mialon et al. 2023 Chen et al. 2025
Wei al. 2025 Zhou al. 2025 Du al., (025). Existing benchmarks primarily test an agent's deep reasoning on query that seeks a hard-to-find information point. In contrast, our WideSearch
benchmark the first to evaluate the ability to gather broad information across multiple entities. We
provided more detailed comparison table for different search agent benchmarks in Table
6.2 SEARCH AGENTS
The development search agents includes powerful but closed-source proprietary systems (Ope- nAI 2024 Google Gemini 2025a x.ai 2025 Moonshot AI, 2025). In parallel, open-source efforts have focused on model-centric optimization (e.g., using Reinforcement Learning, simulated envi- ronments, and synthetic data) and workflow orchestration with multi-agent systems (Song et al.
2025a Jin et al. 2025 Zheng et al. 2025 Sun et al. 2025) Song et al., 2025b] Huang et al.) [2025] Wu et al.; 2025a Li et a 2025a bi Qiu et alll 2025). However, the performance of these agents on broad, multi-entity information-seekinn
tasks is underexplored. WideSearch is the first benchmark
specifically designed assess this capability.
7 CONCLUSION
We introduce WideSearch, a benchmark designed to evaluate LLM-Agents on "wide information seekin" tasks. Our evaluation of over 10 leading agent systems reveals significant shortcomings. Even the top performer achieved only 7% success rate on table-level tasks, with most scoring near 0%. In-depth analysis shows the primary cause of failure is not finding individual data points (item- level Fl scores can be high), but the inability to gather all atomic information comprehensively and accurately. We identify core deficiencies in agentic capabilities: poor problem decomposition into sub-queries, a lack reflection and iteration on failures, and improper use retrieved evidence. In summary, current search agents critically flawed for large-scale, high-fidelity information gathering. Our findings suggest a need for more sophisticated agent architectures, with multi-agent systems that simulate human collaboration- parallel search and cross- coos-validatton, represent- ing a promising future direction.
ETHICS STATEMENT
We confirm that this work adheres to the ICLR Code of Ethics. The WideSearch benchmark is derived from real-world user queries that were rigorously anonymized to protect privacy. Each task is constructed around publicly verifiable and objective facts. All human annotators involved in the data curation and validation process were fairly compensated for their contributions. The authors are fully responsible for the content and integrity of this research.
REPRODUCIBILITY STATEMENT
We are committed to ensuring the reproducibility of our research. To facilitate this, we provide a ZIP file the supplementary materials that contains all necessary components to reproduce our findings. This file includes: (1) the complete and anonymized WideSearch benchmark dataset, comprising all 200 tasks and their ground-truue truth tables; and (2) the full source code for our experimental framework and automated evaluation pipeline. The experimental setup is described in detail in Section 3.1. For precise replication, the specific model API identifiers used in our experiments are listed in Appendix
The exact system prompts and tool definitions for the agent frameworks are provided in Appendix and the prompts used within our evaluation pipeline, such as for the L-M--a-a-judgee are detailed Appendix H
REFERENCES
Anthropic. Introducing Claude 4.
2025.
10

---

Published as a conference paper at ICLR 2026
Nikos I Bosse, Jon Evans, Robert G Gambee, Daniel Hnyk, Peter Mühlbacher, Lawrence Phillips, Dan Schwarz, Jack Wildman, et al. Deep research bench: Evaluating ai web research agents. arXiv preprint wwx.i:250600628/ 2025.
Kaiyuan Chen, Yixin Ren, Yang Liu, Xiaobo Hu, Haotong Tian, Tianbao Xie, Fangfu Liu, Haoye Zhang, Hongzhang Liu, Yuan Gong, et al. xbench: Tracking agents productivity scaling with profession-aligned real-world evaluations. arXiv preprint arXiv:2506. 13651, 2025.
Mingxuan Du, Benfeng Xu, Chiwei Zhu, Xiaorui Wang, and Zhendong Mao. Deepresearch bench:
A comprehensive benchmark for deep research agents. arXiv preprint 2r6.::2506.11766, 2025.
Google
Gemini.
Deep
Research
is now available on Gemini 2.5
Pro Experimental
2025a.
Google
Gemini.
Gemini 2.5:
Our most
intelligent AI model.
Geminiimodel-thiee
2025b.
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in llms via reinforcement learning, arXiv preprint 2tt.::550..1294... 2025.
Xanh Ho, Anh-Khoa Duong Nguyen, Saku Sugawara, and Akiko Aizawa. Constructing multi-hop qa dataset for comprehensive evaluation of reasoning steps. In Proceedings of the 28th Interna- tional Conference on Computational Linguistics, 2020.
Ziyang Huang, Xiaowei Yuan, Yiming Ju, Jun Zhao and Kang Liu. Reinforced internal- external knowledge synergistic reasoning for efficient adaptive search agent. arXiv preprint arXiv:2505.07596, 2025.
Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan Arik, Dong Wang, Hamed Zamani, and Jiawei Han. Search-rl: Training lims to reason and leverage search engines with reinforcement learning, arXiv preprint urXiv:2503.09516 2025.
Mandar Joshi, Eunsol Choi, Daniel Weld, and Luke Zettlemoyer. Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension. In Proceedings of the 55th Annual Meet- ing of the Association for Computational Linguistics, 2017.
Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, et al. Natural questions: benchmark for question answering research. Transactions of the Association for Computational Lingnistics, 7:453-466, 2019.
Fangyu Lei. Jixuan Chen, Yuxiao Yc, Ruisheng Cao, Dongchan Shin, Hongjin SU, ZHAOQING SUO, Hongcheng Gao, Wenjing Hu, Pengcheng Yin, Victor Zhong, Caiming Xiong, Ruoxi Sun, Qian Liu, Sida Wang, and Tao Yu. Spider 2.0: Evaluating language models real-world enter- prise text-to-SQL workflows. In The Thirteenth International Conference on Learning Represen- tations, 2025. httlhtttts:::ppreeeeeee
net/ forum 1dd= 9CPS
Kuan Li, Zhongwang Zhang, Huifeng Yin, Liwen Zhang, Litu Ou, Jialong Wu, Wenbiao Yin, Baix- uan Li, Zhengwei Tao, Xinyu Wang, et al. Websailor: Navigating super-human reasoning for web agent. arXiv preprint arXiv:2507.00 025592, 2025a.
Xiaoxi Li, Jiajie Jin, Guanting Dong, Hongjin Qian, Yutao Zhu, Yongkang Wu, Ji-Rong Wen, and Zhicheng Dou. Webthinker: Empowering large reasoning models with deep research capability. arXiv preprint arXiv:2504.2177,, 2025b.
Manus. Leave it to Manus. https: ftttps://manu.... manus. im 2025.
Grégoire Mialon, Clémentine Fourrier, Thomas Wolf, Yann LeCun, and Thomas Scialom. Gaia: benchmark for general ai assistants. In The Tweifth International Conference on Learning Representations, 2023.
11

---

Published as a conference paper at ICLR 2026
Moonshot AL Researers::: End-to-End RL Training for Emerging Agentic Capabilities. https::/moonshota.l github............. /ee//Kiiiieeeeeerr///./. 2025.
OpenAl.
Deep
Research System Card.
http:://ppenai.comm/nnee
beep--reeaarccl--stttee---oooo/ 2024.
OpenAl. GPT-5 System Card.
2025a.
OpenAl. OpenAl o3 and 04-mini System Card. https://cdr.
openai
03 waw-aaddd-----sss--------------- cardd pdf 2025b.
Jiahao Qiu, Xuan Qi, Tongcheng Zhang, Xinzhe Juan, Jiacheng Guo, Yifu Lu, Yimin Wang, Zixin Yao, Qihan Ren, Xun Jiang, al. Alita: Generalist agent enabling scalable agentic reasoning with minimal predefinition and maximal self-evolution. arXiv preprint arXiv:2505.20286 2025.
ByteDance Seed, Jiaze Chen, Tiantian Fan, Xin Liu, Lingjun Liu, Zhiqi Lin, Mingxuan Wang, Chengyi Wang, Xiangpeng Wei, Wenyuan Xu, et al. Seed1. 5-thinking: Advancing superb rea- soning models with reinforcement learning. arXiv preprint
25ti:22004199144 2025.
Huatong Song, Jinhao Jiang, Yingqian Min, Jie Chen, Zhipeng Chen, Wayne Xin Zhao, Lei Fang, and Ji-Rong Wen. R1-searcher: Incentivizing the search capability in ilms via reinforcement learning. arXiv preprint arXivv2503.05592 2025a
Huatong Song, Jinhao Jiang, Wenqing Tian, Zhipeng Chen, Yuhuan Wu, Jiahao Zhao, Yingqian Min, Wayne Xin Zhao, Lei Fang, and Ji-Rong Wen. R1-searcher++: Incentivizing the dynamic knowl- edge acquisition of Iims via reinforcement learning. arXiv preprint arXiv:2505. 2025b.
Hao Sun, Zile Qiao, Jiayan Guo, Xuanbo Fan, Yingyan Hou, Yong Jiang, Pengjun Xie, Yan Zhang, Fei Huang, and Jingren Zhou. Zerosearch: Incentivize the search capability of Ilms without searching. arXiv preprint arXiv:2505.04588 2025.
Kimi Team, Yifan Bai, Yiping Bao, Guanduo Chen, Jiahao Chen, Ningxin Chen, Ruijue Chen, Yanru Chen, Yuankun Chen, Yutian Chen, al. Kimi k2: Open agentic intelligence. arXiv preprint S..iv:2557.2053::0 2025.
Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. Musique Multihop questions via single-hop question composition. Transactions of the Association for Computational Linguistics, 10:539-555 554, 2022.
Jason Wei, Zhiqing Sun, Spencer Papay, Scott McKinney, Jeffrey Han, Isa Fulford, Hyung Won Chung. Alex Tachard Passos, William Fedus, and Amelia Glaese. Browsecomp: simple yet challenging benchmark for browsing agents. arXiv preprint arXiv:2504.123 2025.
Jialong Wu, Baixuan Li, Runnan Fang, Wenbiao Yin, Liwen Zhang, Zhengwei Tao, Dingehu Zhang, Zekun Xi, Yong Jiang, Pengjun Xie, et al. Webdancer: Towards autonomous information seeking agency. arXiv preprint N.Niv:25055.26445 2025a.
Jialong WU, Wenbiao Yin, Yong Jiang, Zhenglin Wang, Zekun Xi, Runnan Fang, Linhai Zhang, Yulan He, Deyu Zhou, Pengjun Xie, et Webwalker: Benchmarking llms web traversal. arXiv preprini arXiv:2501.0733 207557 2025b.
x.ai. Grok 3 beta the age of reasoning agents.
2025.
An Yang. Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang. Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang. Jiaxi Yang. Jing Zhou, Jingren Zhou, Junyang Lin, Kai Dang, Keqin Bao, Kexin Yang, Le Yu, Lianghao Deng, Mei Li, Mingfeng Xue, Mingze Li, Pei Zhang, Peng Wang, Qin Zhu, Rui Men, Ruize Gao, Shixuan Liu, Shuang Luo, Tianhao Li, Tianyi Tang, Wenbiao Yin, Xingzhang Ren, Xinyu Wang, Xinyu Zhang, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yinger Zhang, Yu Wan, Yuqiong Liu, Zekun Wang, Zeyu Cui, Zhenru Zhang, Zhipeng Zhou, and Zihan Qiu. Qwen3 technical report, 2025. URL http::ttp./.......... /aaaxxivv org/ abs 2505. 09388
12

---

Published as a conference paper at ICLR 2026
Zhilin Yang. Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D Manning. Hotpotqa: A dataset for diverse, explainable multi-hop question answering. In Proceedings of the 2018 Conference Empirical Methods in Natural Language Processing, 2018.
Yuxiang Zheng, Dayuan Fu, Xiangkun Hu, Xiaojie Cai, Lyumanshan Ye, Pengrui Lu, and Pengfei Liu. Deepresearcher: Scaling deep research via reinforcement learning in real-wort world environ- ments. arXiv preprint arXiv:2504.03160, 2025.
Peilin Zhou, Bruce Leon, Xiang Ying, Can Zhang, Yifan Shao, Qichen Ye, Dading Chong, Zhiling Jin, Chenxuan Xie, Meng Cao, et al. Browsecomp-zhh Benchmarking web browsing ability of large language models in chinese. arXiv preprint 2543 2025.
13

---

Published as a conference paper at ICLR 2026
A LLM USAGE DISCLOSURE
We used large language models assist in the writing process, for purposes of checking grammar and improving language clarity. We did not use any large language models to generate research ideas, experimental code, or analyze results. Authors of this paper are responsible for the content of the paper.
B MODELS AND API LDENTIFIERS
The models used in this benchmark and their corresponding API identifiers (which often include version or date information) are listed in Table 3 'he "Benchmark Alias" is the shorthand name used refer the model within our paper.
It is worth noting that Deubao-Seed-1.6 (Thinking) and Doubao-Seed-11 (Non-nhinking) share the same API identifier but are configured with different generation parameters (with the "thinking" feature enabled and disabled, respectively). Similarly, C. aude Sonnet 4 (Thinking) represent specific configurations with the "thinking" feature enabled.
Table 3: Correspondence between Benchmark Aliases and API Identifiers
Benchmark Alias
API Identifier
Kimi K2
(Thinking) Deubao-Seed-1.6 (Non-Thinking)
Doubao-Seed-1e
Louba--seed-1-6
doubbo-seedd1166255666
DeepSeek-R1
deepseek-r1-0529
Claude Sonnet (Thinking)
claude-sonnet-
OpenAI o3
03-2025-04-16
Gemini 2.5 Pro GPT-4.1
geniaii-...5ppt--prrv/eee--..
spt-4.1-2025-04-14
OpenAI 04-mini
04-mini-2025-04-11
GPT-5
gpt-5-2025-08-07
Qwen3-235B (Thinking)
gven3-23Bb-020-thhhiinn------
C TASK EXAMPLE
We provide a visually enhanced example of a task from our benchmark for better illustration. The task is separated into styled Task Prompt box, a Ground-Truie Truth box, and Evaluation Criteria box.
Task Prompt
Could you list every single concert on Taylor Swift's tour from January 1, 2010, to May 1, 2025, including the specific date, the concert's English name, the country, the city, and the venue? Each show should on own line, chronological order from earliest latest. Please organize the results in one Markdown table with the following columns: Date, Concert's English Name, Host Country, Host City, Host Venue
Notes: Do not use date ranges for Date, list it in the format of "Day Month, Year", for example: 4th June. 2011
output format 'markdown /n date content datt__cnttet] \n
14

---

Published as a conference paper at ICLR 2026
Ground-Truth Truth
Date
Concert's English Name
Host Country
Host City
Host Venue
Brisbane
4th February, 2010
Fearless Tour
Australia
Brishane
Entertainment Centre
6th February, 2010
Fearless Tour
Australia
Sydney
Acer Arena
7th December, 2024
The Tour
Canada Canada
Vancouver
Place
Sth December, 2024
The Tour
Vancouver
Place
(Full table contains 533 entries and is truncated for clarity)
Evaluation Criteria
Unique Columns: ["Dato"]
Required Columns: ["Date", "Concert's English Name", "Host Country",
"Host City", "Host Venue"]
Evaluation Pipeline:
Date:
Pre-process: ""normstt""
Metric: (*exact (exxcttmaacch)
Concert's English Name:
PPre-process: ("norm_stz")
Metrrc: ("exact.match
Host Country:
Pre-process: ["norm_st=")
Metrri: ["cxact_match"]
Host City:
Pre-process: ["norm_str"]
Metric: ["1 wmlmm_j ("1mmjjudee]]
Criterion: It is sufficient if the semantics approximately the same as the reference answer
if they point the same entity. There need for word-for-wont correspondence.
Host Venue:
-Pre-process: "norm_stz"
Metric: ("1im_judge")
Criterion: It is sufficient if the semantics approximately the same the reference answer if they point the same entity. There need word-for-word correspondence.
D BENCHMARK COMPOSITION AND STATISTICS
The rigorous curation pipeline culminates in the final WideSearch benchmark, which comprises 200 high-quality tasks. For robust cross-lingual evaluation, these tasks distributed equally between English and Chinese (100 tasks per language). Furthermore, to ensure broad applicability and miti- gate dopain-specific biases, the tasks are methodically balanced across 18 diverse topics, as detailed in Figure
To quantitatively substantiate the complexity inherent in our benchmark, we conduct detailed hu- man annotation study with 30 participants. This evaluation is performed representative subset of 100 tasks, drawn equally from the Chinese and English pools (50 tasks each). Annotators are given ample time and instructed complete each task independently to achieve the highest possible accuracy. However, we acknowledge that due to the numerous data points required for each complex task, diligent human annotator may commit inadvertent errors in single session. To miti- gate the impact of such potential errors and establish a robust performance ceiling, we implement a aul-annotation protocol. Each task is independently completed by two annotators, and we exclu- sively utilize the data from the annotator who achieved higher accuracy. This rigorous methodology ensures that our complexity metrics are grounded in high-quality, successful task completions.
Our analysis focuses several key indicators. The first two metrics, derived from the human study, measure the procedural effort. As illustrated in Figure Joa we report the distribution of human completion times. It directly reflects the significant cognitive and temporal investment demanded
15

---

Published as a conference paper at ICLR 2026
Engiish Topic
Ala Caltar
Ara Celore
Govmment Pallo
Socskigg
Socilggy
Spara
Tachoogy
Ted
Acdderec
Ensisement
other
other
Doars
Trangprtaazs
Googupto
Beverage
Nurboer Cumsitons
Number Duedlor
Figure 5: Distribution of the 18 distinct topics across the 200 tasks in the WideSearch benchmark, ensuring broad domain coverage.
by each task, with an overall average completion time 2.33 hours. This is remarkably consistent across languages, with English tasks averaging 2.29 hours and Chinese tasks averaging 2.37 hours. This metric is comprehensive, encapsulating the entire workflow from initial query comprehension, through multi-step searching and information synthesis, to final result validation.
10
bumber of Questinn
Number of Questing
Projected Completion Time thours)
Number of Source Web Pages Required
(a) Distribution of task completion time.
(b) Distribution of source web pages consulted,
Figure 6: Statistical distributions of key complexity metrics from our human annotation study. Both charts compare performance Chinese and English tasks, showing (a) the time required for com- pletion and (b) the breadth of research needed.
Furthermore, to quantify the procedural depth, Figure 6b shows the number of unique source web pages that annotators consulted. The breadth of research required is extensive; on average, anno- tators need to consult 44. 10 unique web pages per task (48.74 for Chinese and 39.46 for English). Annotators are not limited in their choice of search tools. Crucially, this number represents not theoretical minimum, but the actual breadth of research performed, including the cross-verifiee verificatin of facts across multiple sources to ensure accuracy. therefore serves as a strong proxy for the non-trivial nature of the information-seekinn process.
Finally, characterize the informational scope across the entire benchmark, Table 4 presents the distribution answer data volume. This metric reflects the amount of factual information that must be synthesized and structured provide complete and correct solution.
16

---

Published as a conference paper at ICLR 2026
Table 4: Projected Distribution of Answer Data Volume. Data volume is defined as the number of discrete factual data points (e.g., rows multiplied by columns in result table) required for complete answer.
Volume Range
Chinese Questions
English Questions
0 100
100 1000
59
77
1000 2000
13
2000 3000
3000 4000
4000 5000
5500
Average Volume
2001.2
938.6
Table 5: Extended comparison with existing benchmarks, highlighting task dimensions. The Task Archetype, Output Format, and Core Model Challenge columns are derived from our def- initions in Table 2. Classifications for existing benchmarks inferred based their evaluation methods and task goals.
Benchmark
Horizon
of Tusles
Time- Vare-narynn
Task Archetype
Output Format
Evaluation
Core Model Challenge
WallWaleerAA
20255
Wu
Shoet
680
DeepSearch
Single/lew facts
Answer Masch
Deep
3023
DeeySeuchh
Single/few
Deep ressening
10 We Crmp
a 2025
Wel
Long
1,266
DeepSearch
Single/few facts
Answer Masch
Deep reasoning
Long
DeepSearch
Single/few
Deep reasoning
DeepRestarch
(Du ctal. 2025 2025
Bench
Long
100
DeppReserch
ferm-formmae report
LLM-as-- Julgee
logical organization Information synthssss
Information synthesis,
(Rosse
Kestirer Bench
Long
99
DequResearch
Few facts Long-
report
LLM-a--s-Jubee Julge
Match
logical organization
WideSenreh (Ours)
Long
200
WideSearch
A structured table of entiitiss
LLM-as-a- Judge
Answer Match
fidelity of Information
Exhaustiveness, and
E DETAILED COMPARISON WITH PREVIOUS WORK
F DETAILED EXPERIMENTS
We present detailed experimental data Table 6 including Success Rate, Row-level Precision, Row-level Recall, Row-level FI, Item-level Precision, Item-level Recall, and Item-level Fl on dif- ferent subsets of WideSearch.
17

---

Published as a conference paper at ICLR 2026
Table 6: Detailed experiments results on the WideSearch benchmark.
Model/ System
Succes Rade
Row Precision
Row Recall
Row Fl
Item Preciáon
Irem Recall
Item F1
Argl04
Passer
Argl04
Augeré
Argili
Macori
Asgllh
Max364
Augore
Maxer
Argie4
Max
Sinpl Agesr on Midedrarch
Owand-235B (Thiniigg
0.25
28.35
20.45
73.93
48.30
52.54
Usidde Sornet 4 (Chiking)
0,25
1.00
35.91
25,00
30.15
19.75
73.59
50.84
09.51
55.76
63.19
Grmm
1.00
3.0
15.9%
48.04
25 68
11
26.95
16.90
60.49
73:46
42.35
56.45
57.36
OpesAA o3 high
2,00
5.50
37.10
50.97
27.52
35.23
29.30
39.31
57.31
71.48
41.45
50.19
45.19
54,46
K2
0:28
1.00
18.83
40:35
25.80
15.50 68
22: 70
19:03
59.05
75.05
45.05
56.07
48.81
Despheck-R ---028
0:25
1.00
28 12
44.39
16.33
25.40
18.44
18:35
54.76
71.91
20.55
43.35
35.05
47.55
Deubao 1.6
1.75
4.00
37:20
54.25
27-13
39.02
29.25
42.08
58.29
77.78
40.00
$4.88
43.72
58.84
Deuboe-1 nce thinking
0.30
2,00
38.45
52.17
23.67
35.00
25. 50
37.41
59.97
70.31
39.14
52.01
42.87
55.79
Apest WideSnan
GPT-
9.00
38
$4.10
$1.14
34.93
52.56
73.06
71.27
Qwend 235B (Thiniiing
0.75
27.87
18,89
20.05
30.21
74.52
56.74
58.21
Ulande Sornet 4 (Thikingg
4,25
37101
31.97
13.15
48.05
79.35
08:33
62.02
10.17
Genini 2.5 Pro
2,50
7.50
37.66
52.63
31.37
43.74
35.03
45.82
65.57
81.10
53.67
67.36
56.35
69.97
OpesA ol-high K2
7,00
13000
4504
55 90
$7.48
41.33 26
19.70
48.84
68.78
79.10
5k.0
66.31
60.5
10/6
2.00
6.00
36.71
48.42
16.16
47.3 54
$1.54
45.65
67.08
73.51
$7.60
68 43
5991
70.53
DeepSeck RI 0028
0.50
2.00
28:05
44.39
21.10
32.53
22.88
25.03
60.25
75.06
44.99
16
48.55
02.36
3.50
43.99
16
81.63
65
68.58
Dauleas- fonono Blinking
1.50
35.84
51.98
50
39.83
42.31
69.29
82.92
51.23
64.86
55.05
68.17
Single Apentio MideShan
GPT-
6.58
13.50
44.00
36,28
$3.20
37.86
52.22
79.51
65.65
68.25
Querd (235l-235BB0 (Thinking)
0.50
1.50
27.91
43.46
18.63
28.20
20.56
30.76
61.91
74.37
40667
$1.52
4487
55.52
Clank: Sunaci (Thinking)
2.25
5,50
36.46
49.59
36.52
40.40
31.69
41.90
67.46
77.56
55.26
64.41
$7.99
66.75
Gemmi 2.5 Pm
1.50
5.00
35.82
56:35
28.52
39.40
30,00
41.39
63.05
77.32
48.01
60.2
50.9
6162
UpesA ol-high K2
4.50
9.000
40
53.1%
33.25
41.74
36.09
48.07
62.79
75.00 1M
49.76
59 25
5261
67.37
1.12
3.50
35.12
48.82
29.12
39.51
29.67
41.35
65.12
77.28
51.34
62:24
$4.35
53.8
DeepBock- -1-008
0:37
1.50
18.09
44.19
25.97 07
m5
52.31
73.98
41.25
5509
Damtaae-1
2.63
41.90
44.1 13
60.77
70.68
48.21
63366
Duubac 1.5 non thinking
35.15
52.07
25.29
27.21
79/52
58.44
51.8
Agan-Agent Franmwork on WarrSaurn-co
GPT:5
4,00
7,00
48.90
65.20
39.65
50.04
40.85
11.81
65.36
79.21
51.88
63.14
$4.04
65.10
Qued1-758B (Thinking!
0:00
51.46
56
13.65
77.47
46.71
Clande 4 (Thinkig)
2.75
41.27
59.42
36.85
79.50
67.62
57.13
09:53
Gemuni
42.21
73.96
51.79
00.87
2.75
41.11
59.03
50.17
33.83
59
K2
1.25
40.35
33.21
34.74
48.01
73.04
64.72
66.75
DeepSeck-M4 K1-1822
0.50
18.7
15.37
21.
15
53.66
71.11
49.55
37.65
53.15
Daubue 1.6 Douboe 16-0co thinking
2.25
6.00
39.22
57.11
36.94
45.21
32.83
47.49
60.34
79.62
45.51
61.32
48.79
64.43
0.50
1.00
36.75
56.15
24.76
37.75
26.93
40.30
64.13
79.27
42.54
56.05
46.52
59.63
Aget Prasewort on WhdeSeurchonn
GPPS
10 75
17.00
47
67.51
43.12
16
44.1 11
56.5 93
11.02
K1-56
03.96
24.00
65.25
75.56
Quend (Tinking!
30:55
22.95
35.20
24.57
37.21
78.19
51.64
63.01
66.09
4 (Thiking)
4,50
57.86
51.76
40.13
82.49
75.90
67.11
39
$4.87
71.35
OpenAl
7.50
13.00
40.11
41.75
53.20
81.54
74.00
K2
4,75
10000
40:31
54.00
37.71
51.20
10.91
99.55
63.83
13.79
74A8
DeepSeck-R 1--008
1,00
4.50
29.25
45.03
23.17
36.17
24.57
38.10
60.58
71.93
48.19
65.01
50.91
67.54
Douba-- 1.6
2.75
5.50
39.50 .9
$7.64
33,89
45.73
14
50.35
68.65
63.90
55.18
13.36
50.39
74,37
Daohve-1 Benco-thinking
9.78
8.00
36.51
50.54
$1.05
43.43
32.55
44.55 99
70.74
81.92
56.27
66.74
58.97
70.2
AgttiAgee Pranemort on WhirSorcc-al!
7.37
41.51
54.16
R1.39
68.61
59.65
70.48
Quent-235H (Thinicngg)
$1,85
Clande (Thinking
73.15
2,00
37.90
33.47
44.64
77.07
64.15
66.31
OpenAl
5.12
43.70
36.14
37.90
60.95
66.20
68.55
K2
3
6.50
40.35
55.32
35.06
48.36
36.23
49.00
69.15
70.28
58.78
69.26
61.15
70.72
DrepSeck R1-0528
0.75
3.00
28.97
46.06
21.27
34,00
22.87
36.59
57.82
75.02
40.9
57.20
40.25
60.34
Dnobae-1
2,50
3.30
19 35
57.38
33. 16
45. 97
35.00 33.95
48.95
69.49
81.26
51.85
67:35
59.59
Doubue 16-00 thinking
2.12
4.50
37.64
$4.34
27.91
40.58
29.65
42.65
67.43
80.59
49.41
62.15
52.75
65.13
Endkan-ad Sprinee Widebvandtogl
Clank:
0.00
0.00
27.91
38.94
19.29
27.50
20.84
28.92
$7.15
67.96
40.25
49.54
43.51
$2,14
Gemmi
1.50
4.50
37.25
$7.55
31.28
39.11
32.52
40.52
54.06
73.90
500n
58.17
52.22
50.46
OpesAf o3
3,00
3.00
3541
52.21
25.91
35.20
27.40
38.34
62.29
75.94
42.73
53.53
46.03
$6.31
Enciin-Fd Sprive Wideeranrdsan
5.00
10.00
30 49
48
36.75
27.30
38.07
74.22
63.83
64.11
OpssAl
3,00
20.42
67.22
53.73
56.47
Enchin-Ead Sprinee WideSvannfoall
Cland::
2.50
5.00
29.20
41.40
23.59
32.04
24.13
33.49
58.69
71.09
45.70
56.18
48.40
58.47
Gemmi
4.25
8:00
40.34
$1.37
35.51
46.13
36.63
45.41
67.72
77.16
36.09
05.1 18
59/15
67.17
OpenAt o3
3,00
3.50
29.90
46.43
23.64
34.08
23.91
36.03
59.91
71.58
42.99
53.63
45.52
56.49
G AGENT FRAMEWORK DETAILS
We provide the system prompt of the Single-Agent framework, the Multi Aulti-Agent framework, and the tools prompt for the agents follows. The "Create Sub-Agent" is only provided for the Multi-Agen framework.
SINGLE AGENT PROMPT
# Role
You are expert in online search. Your task is gathering relevant information using ad- vanced online search tools based on the user's query, and providing accurate answers ac- cording the search results.
18

---

Published as a conference paper at ICLR 2026
# Task Description
Upon receiving the user's query, you must thoroughly analyze and understand the user's requirements. order effectively address the user's query, you should make the best of the provided tools to acquire comprehensive and reliable information and data. Below are the principles you should adhere while performing this task:
Fully understand the user's needs: Analyze the user's query, if necessary, break it down into smaller components to ensure a clear understanding of the user's primary intent.
Flexibly use tools: After fully comprehending the user's needs, employ the provided tools to retrieve the necessary information.Ir the information retrieved previously deemed in- complete inaccurate and insufficient to answer the user's query, reassess what additional information required and invoke the tool again until all necessary data obtained.
MULTI AGENT PROMPT
Role
You are professional and meticulous expert in information collection and collation. You can fully understand users' needs, skillfully use search tools, and complete the tasks assigned by users with the highest efficiency. Task Description
After receiving users' questions, you need to fully understand their needs and think about and plan how to complete the tasks assigned by users efficiently and quickly. To help you complete tasks better and faster, I have provided you with three tools:
1. Search tool: You can use the search engine to retrieve information;
Link reading tool: link reading tool that can open links (which can be web pages, PDFs, etc.) and summarize all relevant information on the page according to the requirement description.
3. Sub Agent: The Sub Agent can complete various types of tasks according to the prompt you input. The Sub Agent itself can also use the search tool and the link reading tool. You can split your tasks into multiple sub-tasks according to your own needs, and then create one or more Agents help you complete these sub-tasks parallel.
TOOL DESCRIPTION
SEARCH TOOL
Description:
This is search tool. Enter search queries, and it will return a list of web pages along with their corresponding summary information. Search queries should be concise and clear; complex questions should be broken down into multiple steps and searched step step. no useful pages are found, you can adjust the question description (such as reducing qualifiers changing the search approach) and search again. The quality of search results related to the language: for Chinese resources, you can try entering Chinese queries; for non-Chinese resources, you can try using English the corresponding language. Parameters: query, count, summary.type use_english
TEXT BROWSER VIEW
Description:
This is a link reading tool that can open links (which can be web pages, PDFs, etc.) and summarize all relevant information on the page according to the requirement description. This tool can be called to obtain information for all valuable links. Valuable links include but are not limited to the following types: URLs explicitly provided in the task; 2. URLs with relevant summaries provided in search results; 3. URLs contained in the content returned by previous calls to TextBrowser View that are judged to potentially contain useful information. Please try to avoid constructing links out of thin air by yourself. Parameters: url, description
19

---

Published as a conference paper at ICLR 2026
CREATE SUB-AGENT
Description:
Creates sub-agents that can perform specific tasks based on the input prompt.
Parameters: sub_agents: ((prompty, indexø), (prompty, index,)]
H EVALUATION DETAILS
Sometimes, the column names in the markdown table generated by the model completely mantically consistent with those the ground truth, but the surface strings are not entirely the same. We believe that such cases should be considered the following mapping prompt to map column names with the same semantics to the same string. Similarly, the inner join on the primary key between the two tables also relies on string matching. We also use the mapping prompt to map entities with the same semantics under the corresponding primary keys in the two tables to the same string. We present an example of the mapping below.
correct predictions by the model,
we use
MAPPING PROMPT
Your task is to align two vocabularies. The inputs the vocabulary be aligned and the reference vocabulary, respectively. Note that you need to perform semantic alignment (not positional alignment). two strings are the same, they must correspond to each other. These Two strings are supposed to represent the same entity, with differences only in the expression forms and formats.
The vocabulary be aligned is as follows:
[response]
The reference vocabulary is follows:
freference)
The alignment rules as follows:
List the values in the vocabulary to be aligned one by one. If there is value in the reference vocabulary that has the same meaning this value, 'transform' should be represented as the value from the reference vocabulary; otherwise, 'transform' should be represented as the original value from the vocabulary to be aligned.
Note that 'origin' must be taken from the vocabulary to be aligned, keeping the original format, and 'transform' must be taken from the reference vocabulary. For example: Some words in the vocabulary to be aligned might be the words in the reference vocabulary with Markdown formatting added, keep the to be aligned format in 'origin' and the reference format in 'transform'.
For the 'origin', first find the 'transform' that is the closest in meaning, and then judge whether they correspond to each other. Those entities not correspond to each other cannot output.
Please output the alignment results in the following format: "'json
"origin_strll" "transform_ste ssrl",
"origin.str?" "transform-stre
MAPPING OUTPUT EXAMPLE
"'json
-gemini-2.5-pro (thinking))" "Gemini 2.5 Pro".
"gemini-2.5--f (thinking)":: "Gemini 2.5 tash-thinkingg..
"gemini-2..5ffanl (non-thinkingg):: "Gemini 2.5 Flash-non thinking",
"gemmni-2.55ffash-- (thinking)": "Gemini 2.5 Plash-Lite-thnkinng
-emmini-2.--llas-litte tnon-thinking)": "Gemini 2.5 Flash-Lite- thinking", "claude-3-7-sommet": "Claude 3.7 sonnet",
"claude-opus-4": "Claude Opus 4".
20

---

Published as a conference paper at ICLR 2026
"claude-sonnet-4". "Claude Sonnet 4",
"o3 (high)": "O3".
"o3-mini (high)": "03 mini".
o4-mini (high)": "o4-mini".
"doubao- doubaoo-.5-pro- -.-poot--iippr "Seed-Thinee Thinking v1.5",
doubaoo-..6 1.6" "Seed-Thinking v1.6",
deepseee-v3"" "DeepSeek-V3",
deepseek-r1-0528"" "DeepSeek-R1-0522"
For non non-primary key columns, some columns cannot be evaluated using rule-based methods and must be evaluated using the llm-as-judge approach. During the data annotation process, notated the LLM--s-judge criteria for these columns. To improve the efficiency of the judge, we perform evaluations column by column. The specific llm-as-judge prompt and evaluation examples are as follows:
LLM-AS-JUDGH PROMPT
You expert in grading answers. Your task is to the responses certain ques- tion. Below, you will be provided with set of standard answers, set of responses to be graded, and specific grading criteria.
Each answer and each response has index. Please score each pair of answers and re- sponses this set according to the following methods:
1. The scoring range is from 0 to 1. A score of indicates completely correct answer. For deduction items, please refer the specific grading criteria section.
2. After reading the standard answers, responses to be graded, and grading criteria, please first analyze and judge them item by item according to the grading criteria.
3. The score only be integer of 0 or 1.
4. After the analysis and judgment, please provide the final scoring results. Each pair should have score. Output in Markdown JSON format, as shown below: jsson
iddx xxxxxxx": score, "idx-yyy": score,
criterion-start
{criterion)
criterion-end
response-start
(response]
response-end
Now start scoring. Please make sure to analyze each item step by step before providing the final scoring results.
LLM-AS-JUDGE OUTPUT EXAMPLE
Let's analyze each answer and response according to the criterion: semantic equivalence is sufficient; answers do not have correspond word-for-word.
-idx_0: response = "Itookes'' target = 'Im'
""Immtkens"" clearly refers to =" million t" which semantically matches ""Im"" (1 million).
Score:
idx_1: response = "Imokkns'. target = 'Im'
Same as above.
Score:
21

---

Published as a conference paper at ICLR 2026
idx 2: response Ittokens', target 'Im"
Same as above.
Score:
idx 3: response = '128ktokens' target '200k'
"""128tokens""' (128,000 tokens) does not semantically match ""'200""" (200,000 tokens).
Scorr: 0
idx 4: response = "200ktokens', target '200k' """200ktokens""" corresponds directly to """200k""
Score:
idx 5: response '128ktoken'', target '200k' .""128ktokens"" does not correspond to """20k""".
Sccore:
-
idx_6: response = 'nan', target = '200k'
'""nan"" means there is answer, which does not match ""200k""".
SScor:
idx_7: response '32ktokens', target '32k'
"""32kookens refers to ""32,,00 tokens""" and matches ""2"""
Sccore:
idx_8: response = 'nan', target '256k' """nan""" no answer, does not match ""256k""".
SScore: 0
idx 9: response = 128ktokens', target = '64k' .""128ktokens" does not correspond to ""64k""
Scoor:
idx 10: response '128ktokens', target '64k'
-Same as above.
Score:
idx 11: response '128ktokens', target = '64k'
-aame as above.
-Score: 0
Here final scoring according to the criteria: "'json
"idx_0" 1, "'ixx_""" ""ixx_ "idx_3"" O, """; 1, "'idx_""" o,
"idx 6T': O, ""idx 7 1, "iidx 8 O, ""idx 9TT: 0, ""idx 10"h: 0. ""ixx 11"":
I ERROR ANALYSIS AND EXAMPLES
This section provides detailed examples for each of the failure modes identified in Section We the trajectories of Gemini-2.5-pr in single-agent mode as examples. Each case includes the user's s task, the agent's actions (e.g., search queries and retrieved evidence), and an analysis of the specific enror.
Case Incomplete Query Decomposition Figure I.I illustrates a case of incomplete query de- composition. The agent was tasked with compiling comprehensive table the top five universities across five subject areas from the QS 2025 rankings. The required information included not only the rankings but also logistical details such university websites, application deadlines. and fees. The agent successfully decomposed the initial part of the task, generating specific queries to identify the top universities in each subject area (c.g.. "Top universities in Arts and Humanities QS World Uni- versity Rankings by Subject 2025"). However, it failed to generate the necessary subsequent queries gather the additional required data points for each university. This omission demonstrates fail- ure in comprehensive task planning, as the agent did not create complete set of subtasks needed to fulfill all aspects of the user's request, resulting in a final output with significant information gaps.
Case 2: Lack of Reflection and Iterative Refinement Figure 1.2 provides example of the agent's 's failure to reflect and refine its strategy. The task was to find the annual GDP growth rates for all sub-Saharan African countries from 2022 to 2024, using data from the World Bank. The
22

---

Published as a conference paper at ICLR 2026
Error: Incomplete Query Decomposition
Task: Please compile Markdown table summarizing the top five universities in five subject areas according the 2025 rankings. The table should include following columns: subject, university, Q$ subject ranking 2025, QS overall ranking 2025, THE overall ranking 2025, university website, regular application deadline, and application fee.
Search Queries:
"QS World University Rankings by Subject 2025 broad subjects"
universines Arts and Humanities World University Kankings Sobyect 2025",
"Top 5 universities in Engineering and Technology QS World University Rankings by Subject 2025",
'Top
QS World University Rankings by Subject
"Top universities Natural Seiences QS World University Rankings by Subject 2025",
"Top
Social
Management
University Rankings by 20055
This query decomposition incomplete only identifies the top universities based the QS subject rankings It omnits the necessary subsequent queries to gather the other required data points, as overall runkings, university websites, application deadlines, for institution.
Anulysis:
Figure I.1: An illustration of Incomplete Query Decomposition error. The agent identifies the uni- versities but fails to generate subsequent queries for other required details like application deadlines and fees. Please note that the task description in the figure has been simplified for visualization purposes.
Error: Lack of Reflection and Iterative Refinement
compile Markdown table, by country name, summarizing
GDP
growth rates for sub- Sub-Salaran African countrics from 2022 to 2024, as published by the World
Bank. Indicate missing data with "NA." 'The table should include the following columas: Country,
2024 GDP Growth Rate (9)), 2023 GDP Growlh Ratc (%)), and 2022 GDP Growth Rate (%).
Search Queries:
"World bank africa GDP annual growth rare 2012 2023 2024"
Read Websites:
"tinge./ddii worldbank wwrldbank www.:rginnittttoNn...on.... NY..GKKKKPDD ZGllocation: ZF most, recest_yon dese true"
Final Output:
countries Sub-sahumm frica from 2022-20244 After several attenpes, was usable to retrieve the GDP anmal growth rase for all coantries in Sub saharan Africa from 2022 2024. The search results provide aggregated data for the region and mention some specific countries, but not comprehensive tuble."
sory
GDP growth
Analysis: This
shows lack of reflection and iterative refinement because the agent gave up after initial broad query returned only aggregated data. Instead of abandoning the task, should recognized the query's limitation iterated more specific searches, quarying for couniry-lvve data or for the World Bank's data portal directly.
Figure 1.2: An example of the Lack of Reflection and Iterative Refinement error. After initial broad query returns aggregated data, the agent gives instead of refining its search to find country- specific information. Please note that the task description in the figure has been simplified for visualization purposes.
agent initiated the search with very broad query: "World bank africa GDP annual growth rate 2022 2023 2024". The search results provided only aggregated data for the entire region, not the country-specific data required. Instead of recognizing this limitation and adapting its approach-for instance, by iterating through individual countries or searching for the specific World Bank data portal-thee the agent abandoned the task. It concluded that it was unable retrieve the information, demonstrating lack critical reflective mechanism to diagnose search failures and dynamically adjust its plan.
23

---

Published as a conference paper at ICLR 2026
Error: Failure in Evidenee Utilization
Task: compile Markdown socted by country. summarizing minimum GPA requirements for Master of Civil Engineering programs at Ivy League and Group of Eight universities. Use minimum GPA specified unstated requirements.. The table should include the following columas: Country, University, Alliance, and Minimum GPA Requirement.
Scarch Querles:
"Harvard Univensily Master's Civil Engineering GPA requiremem."
Retrieved Information:
Tiiler Civil Engincering Progrim Admission Requinementtt cive. Absumnt griee point averaage (apu) of at least 3,041.0 (b-average grade) un the last 60 hours attempted RUR. htt.R//ttps/////w.....o whh.eduisdmissiio hatrsssiinniiillennnerrnnggg
Final Output:
USA Tarvard University Ivy Lengue 3.044.0
Analysis: This exemplifies failure validation, The agent correctly extructs the GPA data point but fails to recognize that the evidence comes from an imelevant source (the University of Houston), thus critically misattributing the information Harvard University.
Figure 1.3: An illustration of the Failure in Evidence Utilization error. The agent correctly extracts a GPA requirement but fails to validate its source, misattributing information from the University of Houston Harvard University. Please note that the task description in the figure has been simplified for visualization purposes.
Case 3: Failure in Evidence Utilization Figure I.3 illustrates a critical failure in evidence utiliza- tion, specifically in the area of source validation. The agent was asked to find the minimum GPA requirement for Master of Civil Engineering Harvard University. issued a correct query and retrieved snippet of text stating GPA requirement of "3.0/4.0". However, the source of this infor- mation was cive.uh.edu, the website for the University of Houston, not Harvard. The agent failed to validate the context and relevance of the retrieved evidence, incorrectly attributing the GPA require- ment from the University of Houston to Harvard University in its final table. This error highlights fundamental gap between information retrieval and faithful generation, where the agent does not properly ground its answer in correctly attributed evidence.
Case 4: Knowledge Hallucination and Factual Inconsistency Figure I.4 presents clear of knowledge hallucination. The task was to find the entrance fee for U.S. National Parks for June 2025. The agent's search for this future-dated information correctly returned no results. However, when generating the final table for Everglades National Park, instead of indicating that the informa- tion was unavailable (e.g., with "NA"), the agent fabricated specific fee of "S15". This response was not based on any retrieved evidence but was instead generated from the model's internal pa- rameters, resulting in factual inconsistency. This case underscores the challenge of ensuring that LLMSs remain strictly grounded in external sources, especially when those sources do not contain the requested information.
24

---

Published as a conference paper at ICLR 2026
Error: Knowledge Hallucination and Factual Inconsistency
U.S. National Parks with following columns: Park, States, Price (per adult), and Mountain Scenery. The price should reflect the June 2025 lee for a single adult pedestrian, listed number 'Frce', For parks spanning multiple states, list all, separated by commas.. For the "Mountain Socnery" column, "Yes' or No'.
Please generate Markdown
Scarch Querles:
"List U.S. National Parks"
entrance per person"
Retrieved Information:
any
June 2025
Final Output:
Everglades Flocida 15 No
Analysis:
This is a clear instance of knowledge ballucination triggered by search failure. Unable Everglades National the agent fabricated specifie, incorrect value (515) instead of admitting lack information continuing search, leading to a direct factual inconsistency the ground
Figure L.:: An example of Knowledge Hallucination and Factual Inconsistency error. When search for a future entrance fee returns no results, the agent invents an incorrect value ($15) instead of stating the information is unavailable. Please note that the task description in the figure has been simplified for visualization purposes.
Table 7: Basic failure modes distribution.
Model
Tool Call Error
Context Length Exceeded
Output Format Error
Response Refusal
Single-Agent Mode Error Distribution
Deepweek
4.12%
0.12%
0.50%
3.00%
Noubao-Seed-1.6 (Thinking)
3.62%
12.62%
0.25%
1.62%
Doubao-Secd-1. (Non-Thinking
4.00%
3.88%
0.25%
1.12%
Claude Sonnet (Thinking)
0.62%
1.25%
0.00%
1.50%
Gemini 2.5 Pro
3.00%
0.00%
0.38%
7.25%
GPT-5
13.12%
6.75%
0.00%
10.75%
Kimi K2
0.50%
0.50%
0.38%
2.38%
Open o3
1.88%
0.38%
0.00%
16.00%
Qwen3-235B (Thinking)
1.50%
0,50%
2.00%
0.25%
Multi-Agent Mode Error Distribution
Deepseek-R
22. 75%
0.00%
0.00%
4.00%
Douhao Seed-1.6 (Thinking)
4.25%
0.12%
1.38%
Douhao Seedao-Seed-11 (Non Thinking)
9.88%
0.75%
0.25%
1.12%
Claude Sonnet (Thinking)
9.12%
0.25%
0.00%
0.88%
Gemini 2.5 Pro
6.75%
0.00%
0.00%
3.38%
GPT-5
23.75%
1.38%
0.00%
10.62%
Kimi K2
1.62%
0.00%
0.38%
0.38%
OpenAl o3
4.38%
0.62%
0.00%
10.12%
Qwen3 (Thinking)
0.75%
0.12%
J DOMAIN-SPECIHC PERFORMANCE ANALYSIS
To move beyond aggregate performance metrics, we conduct granular analysis to understand how different models and frameworks perform across variety of specific domains. This approach allows us to identify the strengths and weaknesses of each model and to assess the impact of the multi-agent framework tasks requiring topic-specifie knowledge. The evaluation performed on both English and Chinese datasets to capture any deepndden variations performance.
Figure J.1 presents detailed heatmap of the row-level Fl scores from this analysis. The figure is structured to facilitate two key comparisons: the performance of different models against each other, and the effectiveness of the single-agent framework (top half) versus the multi-agent framework (bottom half). The domains are grouped by language, with English results on the left and Chinese the right, separated by distinct vertical line. As the distribution of questions per domain is
25

---

Published as a conference paper at ICLR 2026
Chade dilge aprie
Tisks English
Chinese
Figure J.1: Model performance (Row-level Fl Score) across domains and languages. The heatmap compares Fl scores for various models under single-agent and multi-agent frameworks. Models evaluated on English and Chinese datasets across multiple domains using an automated pipeline. Darker blue indicates higher score.
uneven, the heatmap serves primarily to identify general patterns rather than provide precise statistical comparison.
From the figure, a consistent performance improvement across nearly all models and domains. This suggests that the collaborative approach is effective at enhancing the quality and relevance of the search results, regardless of the topic. Second, certain domains, such as "Academics" and "Transportation" prove be more challenging for all models, likely due the need for highly specialized and nuanced information. Finally, the results reveal model-specific aptitudes; for instance, some models show clear advantage in domains like "Healthcare," while others excel in topics like "Automotive" or "Law." This domain-level insight is crucial for selecting the optimal model and framework for spe- cific real-world applications.
draw several key observations. First, the multi-agent framework provides
26

---

Published as a conference paper at ICLR 2026
K EXAMPLE TRAJECTORIES
In this section, we show the agent trajectory examples. For Single-Agent, we show an complete example follows. For Multi-Agent, show the trajectory of the main agent and omit the ones of sub-agent. The trajectories sub-agent is like the trajectories of Single-Agent mode.
Case Trajectory for Single-Agent
My son is about start his university applications but he's still uncertain about both his major and which universities to apply to. Could you help me find the top five universities in each of the five broad subjects from the QS World University Rankings by Subject 2025, and also check their standings in the QS World University Rankings 2025 and the Times Higher Education World University Rankings 2025? And I need the home page of the university's s official website, standard application deadline for regular decison as well as the application fee without the fee wavier.Please organize the results in one Markdown table with the following columns:
Question:
Subject, University, QS World University Rankings by Subject 2025, QS World University Rankings 2025, Times Higher Education World University Rankings 2025, Home Page, Application Deadline, Application Fee
Please use the universities' full official names in English. Use only Arabic numerals in the ranking, for example: 1. The output format is m"marrdown {data content)
Step 1
Reasoning Content
The user's child is preparing for university applications and needs information about the top five universities in five major subjects based on the QS World University Rankings by Subject 2025. Additionally, they require each university's 's overall QS and THE rankings for 2025, official website URLs, regular decision application deadlines, and application fees (excluding fee waivers). All information must be organized into a Markdown table with specified columns.
First, the user's needs must be clarified. The key points include:
1. Five major subjects (but the user did specify which ones; this is a problem).
2. Top five universities per subject in QS Subject Rankings 2025.
Each university's 's 2025 QS overall ranking and THE overall ranking.
4. Official website homepage, regular application deadline, and application fee.
The critical issue here that the user mentioned "five broad subjects" but did not specify which ones. Common broad subjects might include Engineering, Medicine, Business & Management, Computer Science, Social Sciences, etc. However, since the user did not specify, need confirm. But since this is search task and I cannot ask the user for clarification, must infer or select typical subjects. For example, QS usually includes subjects like Engineering and Technology, Life Sciences and Medicine, Natural Sciences, Social Sciences and Management, Arts and Humanities. Alternatively, more specific subjects like Computer Science, Business, etc. However, this may affect the accuracy of the results. Since the user requires "broad subjects," perhaps choosing the five main subject areas from QS rankings would be appropriate. For example:
1. Engineering and Technology
Life Sciences and Medicine
3. Natural Sciences
Social Sciences and Management
5. Arts and Humanities
But need to verify whether QS's 2025 subject rankings use these broad categorics or more specific ones. For example, QS often has rankings for specific disciplines like Electrical Engineering, rather than broad "Engineering." This requires checking the structure of QS Subject Rankings 2025. Since the current time is April 2025, the 2025 rankings should already be released, but need to confirm the specific subject divisions.
Next steps:
27

---

Published as a conference paper at ICLR 2026
1. Determine the five broad subjects per QS 2025 rankings. This requires searching for
"QS World University Rankings by Subject 2025 broad subject categories" to identify the main subject areas.
2. For each of these five subjects, retrieve the top five universities.
For each university, collect:
QS overall ranking 2025
-THE overall ranking 2025
Official website homepage
Regular decision application deadline (for undergraduate programs, assuming)
Application fee (excluding waivers)
Challenges include:
Ensuring the 2025 rankings available. Since it's April 2025, they should be, but confirmation is needed.
Application deadlines and fees vary by program and country. The user likely refers to undergraduate deadlines, but consistency must be maintained (e.g., for universities in the US, UK, etc.).
Some universities may have different deadlines for different programs; the "regular decision" deadline for undergraduate studies required.
Application fees also vary by program and nationality; the user wants the standard fee without waivers, possibly for international students.
Now, step-by-step plamning:
**Step Confirm QS 2025 Subject Rankings' broad categories**
Search query: "QS World University Rankings by Subject 2025 broad subject areas" or check the official QS website to identify the five main subject groups. For example, QS typically divides into five broad areas: Arts & Humanities, Engineering & Technology, Life Sciences & Medicine, Natural Sciences, Social Sciences & Management. Assuming these five, proceed. but verification is needed via search.
**Step 2: For each of the five subjects, find top 5 universities For example, search "QS World University Rankings by Subject 2025 Engineering and Technology top 5" and repeat for each subject.
**Step For each university, collect overall QS and THE 2025 rankings**
Search "QS World University Rankings 2025" and "Times Higher Education World University Rankings 2025" to find each university's 's overall rank.
**Step 4: Collect official website. application deadline, and fee** For each university, visit their official admissions page to find the regular decision deadline and application fee for international undergraduate students. Note that deadlines may vary by program and country; the user likely wants the undergraduate regular deadline (e.g.. for US universities. usually January/February, while UK universities may have earlier deadlines like January 15 for UCAS). Application fees also vary (e.g., US universities often charge $75-$100, UK may be #20-E25 for UCAS, but individual universities may differ).
Potential issues:
Some universities may not be in the top of multiple subjects, but the user requires top subject, each subject will have five universities, possibly overlapping across subjects. For THE rankings, need to confirm if the 2025 edition is available. THE usually releases rankings in September, as of April 2025, the 2025 rankings should exist (e.g., THE 2025 released in September 2024).
Application deadlines and fees for 2025 entry: For example, US universities' 2025 fall entry deadlines may be late 2024 (e.g., January 2025), but the user is asking in April 2025, which may be after the deadline. However, the user may need deadlines for the intake (e.g., 2026 entry), but the query specifies "standard application deadline for regular decision" without year. This requires clarification, but since the user mentions "about to start his university applications," it's likely for 2025 entry, though deadlines may have passed. Alternatively, 2026 entry deadlines may not yet be published. This potential ambiguity, but the user may expect the most recent available deadlines (e.g., 2025 entry deadlines, even if passed) the upcoming 2026 deadlines. Need handle carefully during searches.
##mplemmnttttin **
28

---

Published as a conference paper at ICLR 2026
1. **Search for QS 2025 Subject Rankings' broad categories**
Query: "QS World University Rankings by Subject 2025 broad subject categories"
Aim: Confirm the five main subject areas. Assume they are Arts & Humanities, Engi- neering & Technology, Life Sciences & Medicine, Natural Sciences, Social Sciences & Management.
2. **For each subject, retrieve top 5 universities**
Example queries: "QS World University Rankings by Subject 2025 Engineering and
Technology top 5"
Repeat for each of the five subjects.
3. **Collect QS and THE 2025 overall rankings for each university
QQeer:: "QS World University Rankings 2025" and "Times Higher Education World University Rankings 2025"
For each university, find their overall ranks.
=*For each university, find official website, deadline, and fee**
Official website: Usually straightforward (e.g., "Harvard University official website"). Deadline and fee: Search "[University name] undergraduate regular decision application deadline 2025" and "[University name] undergraduate application fee international stu- dents".
**hallenges and Solutions:**
* *Sujeect selection uncertainty** If QS 2025 uses different broad categories, adjust accordingly. For example, if "Computer Science" is separate category instead of under Engineering, need to adapt.
**verlapping u The same university may appear in multiple subjects (e.g., MIT in Engineering and Natural Sciences), requiring duplicate entries in the table with different subjects.
**Missing THE rankings university isn't ranked by THE, note as "N/A" or check data available.
**Deadline/fee variability** Some universities have different deadlines for different programs: use the undergraduate regular decision deadline for the main campus. For fees, use the standard international undergraduate application fee without waivers. **Search Steps:*#
First, confirm the five broad subjects in QS 2025 Subject Rankings. ##Search QS 2025 Subject Rankings broad categories**
Query: "QS World University Rankings by Subject 2025 broad subject areas" Assume results confirm the five categories: Arts Humanitics, Engincering Technology, Life Sciences & Medicine, Natural Sciences, Social Sciences & Management. **xext, for each subject, search top 5 universities:**
**Subject 1: Engineering and Technology***
Query: "QS World University Rankings by Subject 2025 Engineering and Technology top 5"
Assume results: MIT (1), Stanford (2), Cambridge (3), Berkeley (4), Oxford (5) (hypotheti- cal: actual data needed via search).
**Subject 2: Life Sciences and Medicine**
Query: "QS World University Rankings by Subject 2025 Life Sciences and Medicine top 5" Example results: Harvard. Oxford, Cambridge. Stanford, MIT. **Subject Natural *ciences**
Query: "QS World University Rankings by Subject 2025 Natural Sciences top 5"
**Subject 4: Social Sciences and Management**
Query: "QS World University Rankings by Subject 2025 Social Sciences and Management top 5"
**Sbbject 5: Arts and #umanities**
Query: "QS World University Rankings by Subject 2025 Arts and Humanities top 5" Once the top 5 universities per subject are identified, collect their QS and THE overall rankings.
** *Search QS 2025 overall rankings:**
Query: "QS World University Rankings 2025 list"
**Search THE 2025 overall rankings:**
29

---

Published as a conference paper at ICLR 2026
Query: "Timms Higher Education World University Rankings 2025 list" For each university, extract their ranks.
Next, for each university, find the official website (usually the main domain, c.g., www.harvard.eduu, application deadline, and fee.
**Example for Harvard University:**
Official website: https//www.harvant edu/
Deadline: January (for 2025 entry, regular decision)
Application fee: $85 (international students)
** *ootential Issues During Search:**
** **ankings not released**; If QS 2025 subject rankings unavailable (though April
2025 should have them), use 2024 rankings and note, but the user specifically wants 2025.
** DDaddliee discrepancies **; Some universities may have rolling admissions or different
deadlines for different countries.
** Application fee waivers**, Ensure the fee the standard one without waivers.
Data Organization:**
The final table will have rows for each university per subject. For example:
Subject University - QS Subject Rank QS Overall - THE Overall Home
Page Deadline Fee
Engincering & Tech - MIT
2 https://iit.cd// - Jan 5 5- $75
*#Eecution Steps:**
1. Confirm QS 2025 broad subjects via search.
2. For each subject, retrieve top 5 universities.
3. For each university, get QS and THE 2025 overall ranks.
Collect website, deadline, fee for each university.
5. Compile into the required Markdown table.
Now, proceed simulate the search steps and data collection. Tool Call
"tool_name": "scarch_global",
"arguments"; "[\"query\" :\> "os World University Rankings by
4 Subject 2025 broad subject categories\", A \"connt":5,,
4 sunmary_type. ":""pee\"""sohrt(""""" "tool_call" Lal___alll______ idd"
"call : saalltgghnngggaa7ggyrrs5551
stool_name";: "search_globai*
"arguments": "('"query\":>> " " World University Rankings 2025\ \", A
4 "o100, V"sunnary tapemmaryyyyyey_e_ee
call id": "call plybhhwwjjkkbbb4922222
"search_global""
"arguments": "I'"query's ">>" Higher Education World
4 University Rankings 2025'", "count ):10, " "ssumarry_type 4 short\")", ittol_call_id"..
Thqri 38833bx5ccv*
Tool Response
[index] 0
[siteName| |
(search content omitted due to length limit)
30

---

Published as a conference paper at ICLR 2026
Step 2
Reasoning Content
I now need to continue analyzing the user's query. The user wants the top five universities in each of the five broad subjects from the QS World University Rankings by Subject 2025, along with their overall rankings in QS and THE 2025, official website homepage, regular decision application deadline, and application fee. These details need to be organized into a markdown table.
First, reviewing previous search results: The first search (ID 0) confirmed that the five broad subjects in QS 2025 are Arts and Humanities, Engineering and Technology. Life Sciences and Medicine, Natural Sciences, and Social Sciences and Management. However, specific rankings for each subject were provided. The second search (ID 1) retrieved QS over- all rankings but only mentioned the number of institutions and countries without specific university rankings. The third search (ID 2) for THE rankings only provided limited infor- mation, such as the University of Melbourne at 47th place and Johns Hopkins at 16th, but lacked comprehensive THE 2025 rankings.
Next, it is necessary to obtain the top five universities for each of the five subjects in QS by Subject 2025. Previous searches only identified the subject categories but not the specific rankings. Therefore, further searches for each subject's 's top five universities are required. For example, search queries like "QS World University Rankings by Subject 2025 Arts and Humanities top 5" for each subject separately.
Additionally, the overall QS and THE 2025 rankings for these universities are needed. Pre- vious QS overall rankings results were too vague, so the complete QS 2025 overall ranking list must be retrieved. Similarly, THE 2025 rankings require more detailed data, such as the top 100 universities or searchable database.
Furthermore, the user needs each university's official website homepage, application dead- line, and application fee. These details must be obtained from each university's 's official website reliable educational platforms. However, this information may vary by program and country, so it is necessary to confirm the general undergraduate regular decision dead- lines and fees for each university. Current missing information includes: 1. Top five universities for each of the five QS subjects.
2. QS 2025 overall rankings for these universities.
3. THE 2025 overall rankings for these universities.
4. Official website URLs, application deadlines, and fees for each university.
Next steps:
For each of the five subjects, search for the top five universities in QS by Subject 2025. For example, use queries like "QS 2025 Arts and Humanities top universities".
Search for the complete QS 2025 overall rankings to find the rankings of these universities.
Search for the complete THE 2025 overall rankings to find their respective positions.
For each identified university, search for their official website, application deadlines, and fees. This may require visiting each university's 's admissions page or using educational por- tals like Common App for deadline information.
Potential challenges:
QS and THE rankings might be published on official websites requiring access to detailed reports or tables.
snriversttysspecife application deadlines and fees could vary by program, the may need the general undergraduate deadline, which may require checking each university's ad- missions page.
Given the complexity, the process should be broken down into steps:
Retrieve top five universities for each of the five QS subjects.
2. Collect QS and THE overall rankings for these universities.
Gather official websites, deadlines, and fees for each university. Now, the first priority is to obtain the top five universities per subject. For example, search- ing for each subject individually:
Search queries could include:
"QS World University Rankings by Subject 2025 Arts and Humanities top 5"
QQ 2025 Engineering and Technology top universities"
31

---

Published as a conference paper at ICLR 2026
And similarly for the remaining three subjects.
Once these universities are identified, their overall rankings in QS and THE can be searched, followed by their specific application details.
Additionally, for QS and THE overall rankings, more precise searches are needed, such as "QS World University Rankings 2025 top 100" or accessing the official QS ranking table. Similarly for THE.
In summary, the next steps require multiple searches to collect detailed ranking informa- tion and university-specific application details. Given the volume of information, it may be necessary to search each subject's rankings separately and then verify each university's 's data individually.
Tool Call
"tool name": isearch_global
" "arguments": "('" "(""quer""": "QS 2025 and Humanities top
4 universitiess \"count\ 5 "ummaar_type'":"Mhhoo "tool_call_ion _d"; "call "call_hu hx1 7x77xx444499 4hh kbwt 7 xyxenn".
"tool _rool_name": #search_globall
"arguments"; "("" "query" "\> 2025 Engineering and Technclogy top 4 \\",\" \"count \oantt\"a :":5, \"summary type\": : \"short\")", "tool call id"; "call 4ykb syyb:yycmmeennSkbb222255
"tool "tool_name": "search global", "arguments": "I "I'"query":;"" 2025 Life Sciences and Medicine top + 5\" "ccunt )::,, "tool_call id"; "call_
:5 \ "summary ("rpe\""""ssor\\"""
"tool_name": "search_global"
"arguments": "(A"query\") ":\"08 2025 Natural Sciences top 5 4 universities ("count"" " "tool_call_ide "call 2
"tool_name": "search_global", " "arguments": "[A"query' "ompuryy"">*"" "08 2025 Social Sciences and Management
4 top 5 \"count "55, \"sumarry_type \=\"soort""""" ctool_calllioo id"; "call_tge4hhcrrrs2222220000
Tool Response
[index] [siteName] gs. com
(search content emitted due to length limit)
Step 3
Reasoning Content
Now I need to continue analyzing the user's question. The user wants the top five universities in each of the five major subjects from the QS World University Rankings by Subject 2025, along with their overall rankings in QS and THE 2025, official website homepage, regular application deadline, and application fee (without fee waiver). This information needs to be organized into Markdown table.
32

---

Published as a conference paper at ICLR 2026
First, previous searches have confirmed that the five major subjects in QS are: Arts and Hu- manities, Engincering and Technology, Life Sciences and Medicine, Natural Sciences, and Social Sciences and Management. The next step is to obtain the top five universities for each subject. Based on the results of the second round of searches (IDs 0-4), the information for each subject is as follows:
1. Arts and Humanities*$; From search results, ID 0 shows Harvard, Oxford, Cambridge, Stanford, but there inconsistency with (mentioning Stanford, Carnegie Mellon, etc.). The information is fragmented and lacks complete top five list.
** Engineering and Technology**; Search result ID mentions MIT, Oxford, Stanford, but the list is incomplete, with only partial rankings.
3. **Life Sciences and Medicine Search results are vague, mentioning Duke, University of Copenhagen, etc., but clear top five ranking.
4. **Natural Sciences**. ID 3 mentions MIT is first in 11 subjects, including Natural Sci- ences, but the top five list is unclear. ID shows Harvard, Oxford, Cambridge. Stanford, but is unclear which subject this belongs to.
**Social Sciences and Management* ID 4 mentions LSE ranked sixth globally, but lacks the top five list.
Clearly, the current subject rankings are incomplete or conflicting, requiring more precise searches. For example, searching for "QS 2025 Arts and Humanities top 5 universities" may yield more accurate results. Additionally, the overall QS and THE rankings, application deadlines, and fees for each university still missing.
Next steps:
For each subject, conduct separate search to confirm the top five universities "QS "QS
2025 Arts and Humanities top 5 universities").
Once the universities confirmed, search for their QS and THE 2025 overall rankings.
For each university, find official website URLs, regular application deadlines, and applica-
tion fees.
Since the user requires detailed information for each university, a significant amount of search work is needed. For example, after confirming Harvard University first in Arts and Humanities, further searches are required for its QS overall ranking, THE ranking, home- page URL, deadline, and fee. This process must repeated for each university across all five subjects, totaling 25 universities. However, considering potential overlaps (e.g.. Harvard may rank highly multiple subjects), efficiency can be improved by consolidating searches for overlapping universities.
Current search results are insufficient, so continued searches are necessary. First, prioritize completing the top five universities for each subject, then gather their overall rankings and application information.
Tool Call
"tool "tool_name": "search_global1
"arguments": (\"query\": \*os 2025 Arts and Humanities top 4 universities official list\", \"count\"; 3, \"summary_ 4 \ \"short\"1", "tool call id"; "call htthhhohhhhgggggeeeeeeeeesssss
"tool_namc": "scarch_global",
"arguments"; " (\"query\": \*os 2025 Engincering and Technology 4 top universities \"count 3, "ssummary_typel \"
4 short\")", "tool "tool_call_id"" twfl1_twf2734 1v555 1dddkkk
"tool "tool_name"; "search global", "search_global",
"arguments"; " (V"query\", \*os 2025 Life Sciences and Medicine + top 5 universities"", \"count\* 3, \"sunnary_type\ \"
+ short>"!",
33

---

Published as a conference paper at ICLR 2026
"tool_call_ ; "call wtwvddhheeee////////////aoagggg
@cool_name "search_globai""
"arguments": " (\"query\": " \*os 2025 Natural Sciences top
4 universities official\", >"ount\"; "ssumarry_type \" 4 short)")",
"tool call id"; "call (e
"tool name": "search global",
"arguments": ('query\"; \*0s 2025 Social Sciences and
4 Management top 5)", \"count \": !:: 3, "tymmary_type""" \"short
4 \")", "tool_calllon" "aal_scheggnnysSgggaaukkaaa
Tool Response
[index] 0
[siteName] es
(search content omitted due to length limit)
Step 4
Reasoning Content
Now need to continue analyzing the user's query. The user wants the top five universities in each of the five broad subjects from the QS World University Rankings by Subject 2025, along with their QS overall rankings, THE rankings, official website URLs, regular decision deadlines, and application fees. Previous search attempts have been made, but the results remain incomplete.
First, reviewing the earlier search results, the five broad subjects have been confirmed: Arts and Humanities, Engineering and Technology. Life Sciences and Medicine, Natural Sci- ences, and Social Sciences and Management. However, the top five universities for each subject are still unclear. For example, search results for Arts and Humanities mention Har- vard, Oxford, Cambridge, and Stanford, but the fifth university missing. Similarly, En- gineering and Technology mentions MIT and Georgia Tech but lacks the full top five. Life Sciences and Medicine, Natural Sciences, and Social Sciences and Management also have incomplete or conflicting information.
Next, regarding the QS overall rankings and THE rankings, the previous search results QS 2025 only provided basic information about the number of participating institutions and countries but did not list specific university rankings. THE rankings similarly lack detailed data, with only mentions of the University of Melbourne at 47th and Johns Hopkins at 16th, which is insufficient to cover all required universities.
Additionally, the user needs official website URLs, application deadlines, and fees. These details were not addressed in previous searches and will require separate research for each university.
Therefore, further searches are needed: For each of the five subjects, confirm the exact top five universities and their QS subject rankings.
Find the QS 2025 overall rankings and THE 2025 rankings for these universities. 3. Collect each university's 's official website URL, regular decision deadline, and application fee.
Since the user requires detailed information for each university, structured approach is essary. First, clarify the top five universities per subject, then gather their overall rankings, and finally collect application-related details. However, given the time constraints, priori- tizing the top five universities for each subject and their rankings may be necessary before proceeding application details.
34

---

Published as a conference paper at ICLR 2026
Current search results still lack clear top-five lists for each subject. For example, in the latest search for Arts and Humanities, topuniversities.com mentions Harvard, Stanford, Ed- inburgh, Princeton, Peking University, and Tokyo University, but it unclear which are the top five. More precise official rankings are needed, possibly by accessing the official QS website authoritative sources.
Similarly, Engineering and Technology search results mention MIT and Georgia Tech but lack the complete top five. Life Sciences and Medicine lists Harvard, Stanford, MIT, and Duke, but the fifth is missing. Natural Sciences mentions Tsinghua, Tokyo University, Toronto, Cornell, and Chicago, but their order unclear. Social Sciences and Manage- ment includes Toronto, Melbourne, Princeton, and Northwestern, with LSE ranked sixth, but the top five remain uncertain.
Thus, it is necessary to conduct more targeted searches for each subject, using queries like "QS 2025 [Subject] top universities official list" to obtain accurate rankings. Once the universities are confirmed, their QS and THE overall rankings can be searched individually. Finally, each university's is official website, deadline, and fee information must be collected, which may require visiting each university's s admissions page or reliable educational plat- forms.
In summary, current information insufficient, and further searches are required for each subject's top five universities, followed by their overall rankings and application details. Given the complexity, may be necessary to search each subject separately and then verify each university's information individually. Tool Call
"tool_namc": "scarch-global",
"a "arguments": " (\qquer\"; **os 2025 Arts and Humanitics top 5 4 universities official\", \"count\": 1, \"summary_type \" 4 short\")",
"tool laool_calll_oo_
"tool_name", "search_global"
"arguments": ('querr\" \*os 2025 Engineering and Technology 4 top 5 official\", \"count\": 1, ""suunrrr_tyypaaaaa \"short 4 \")", id" "call "aal:_shhhootteaa//33soooono
"t "tool_name": "scarch_global",
" "arguments": (\"query\": **os 2025 Life Sciences and Medicine 4 top 5 list\" \"count 1, "'suumary_typlll \"short\")", "tool_call_id"" id"; "eal__brrbhhocdhhh22e22222
"tool_ :ame="
"arguments"; [\'query\"; **os 2025 Natural Sciences tep 4 universities list\" \"count 1, \"summary_type"" \" 4 short>")",
"tool _tool_call_id"; "call waal1_gghyorrmmeessddssssssss
"tool_name" "search_global",
" "arguments"; " (''query ": **os 2025 Social Sciences and
4 Management top list\" \"count\": 1, sssumarry_tpely \"
+ short\"
it" "cal1 haal__Lm001011ddessssssssmmaa
35

---

Published as a conference paper at ICLR 2026
Tool Response
[index] [siteName| | ga.com
(search content omitted due to length limit)
Step
Reasoning Content
I now need continue analyzing the user's query and the existing search results determine the next steps. The user requires the top five universities in each of the five broad subject areas from the QS World University Rankings by Subject 2025, along with their overall QS and THE rankings, official website URLs, regular decision application deadlines, and appli- cation fees. The previous search attempts have yielded some information, but several critical points remain unclear.
First, the five broad subject areas identified from the initial search results are: Arts and Hu- manities, Engineering and Technology. Life Sciences and Medicine, Natural Sciences, and Social Sciences and Management. This information is clear and requires no further confir- mation.
Next, for each subject area's top five universities, previous searches have provided partial results, but inconsistencies exist. For example, in Arts and Humanities, search result 0 from the first round mentions Harvard, Oxford, Cambridge, Stanford, and MIT, while another re- sult lists Harvard, Stanford, Carnegie Mellon, etc. These discrepancies suggest unreliable data that may stem from incomplete or misaligned search results. Similarly, for Engineering and Technology, some results indicate MIT as first, while others mention Harvard, creating confusion. It appears the search results lack definitive official list for each subject area's top five universities.
Furthermore, even if the top five universities per subject are confirmed, their overall QS and THE rankings for 2025 remain unknown. For instance, while the user needs the QS World University Rankings 2025 and THE rankings, existing search results for QS overall rankings mention data like the number of institutions but not specific university rankings. Similarly, THE rankings only reference specific universities (e.g., University of Melbourne at 47th) without a comprehensive list. Thus, retrieving the overall rankings for these universities is essential.
Additionally, the user requires official website URLs, application deadlines, and applica- tion fees. None of the current search results include this information, necessitating separate searches for each university's details.
Given these gaps, the next steps should include:
1. **Confirm the top five universities for each of the five subject areas*#; Previous searches failed to yield clear, consistent official lists. More precise searches using queries like "QS 2025 [Subject] top universities official list" may help. potentially requiring direct access to QS's official website or authoritative sources.
2. **Retrieve overall QS and THE 2025 rankings for these universities Once the uni- versities are contirmed, search for their positions in the overall QS and THE rankings. For example, search "[University Name] QS World University Rankings 2025" and similarly for THE.
3. **Collect official website URLs, application deadlines, and fees**: For each confirmed university, search their official admissions pages for regular decision deadlines and applica- tion fees. This may require visiting each university's website individually using reliable educational platforms.
However, considering the limitations of search tools-such as potential failure to retrieve de- tailed ranking lists or university-specife five universities per subject is critical. Without this foundation, subsequent steps cannot proceed. For example, if Arts and Humanities rankings remain unconfirmed, subsequent searches for overall rankings and application details will be meaningless.
confirmation of the top
Another challenge is that search results for QS subject rankings are fragmented. For in- stance, multiple attempts to search for Engineering and Technology rankings have returned inconsistent results, possibly due to ambiguous query wording scattered official data. Us-
36

---

Published as a conference paper at ICLR 2026
ing more precise keywords like "QS World University Rankings by Subject 2025 [Subject] top 5" or accessing the official QS ranking page directly (via text lext.browser.view) might re- solve this.
For example, the first search result (name=0) includes link to qs.com's article on subject rankings. Using text_browser. view access this link could provide the complete top five list for each subject. Similarly, links from topuniversities.com (QS's official partner site) might contain detailed rankings.
Thus, the next logical step is to use the brr.brrwerr_veew tool to open authoritative links from previous searches (c.g.. the qs.com article link) and extract the official top five uni- versities for each subject. This would ensure accuracy and resolve inconsistencies in prior search results.
Once the subject rankings confirmed, searches for overall QS and THE rankings proceed, For example, if Harvard University tops Arts and Humanities, search for "Harvard University QS World University Rankings 2025" and "Harvard University THE World Uni- versity Rankings 2025" to obtain their overall positions.
Finally, for each university, search their official website for application deadlines and fees. For instance, querying "Harvard University undergraduate application deadline 2025" and "Harvard University application fee" would yield the required information.
In summary, the immediate priority is to confirm the top five universities per subject using text_ got.browser_vie wvew on authoritative links. This will provide reliable foundation for subse- quent searches. Without this step, data inconsistencies will persist, hindering the completion of the user's requested table. Tool Call
"tool_name"; "Lext_browser_viie
" "arguments"; "(V"url\":
4 "Nuutl".."httppp///ww... http:://www......oooo//oitttttas qs analysis/\", v'desariptiion
the top 5
4 universities for each of the five broad subject areas in os
4 World University Rankings by Subject 2025\")",
"tool
"call atall
Tool Response
Page url url: hel::htps://www/ qs.
http:///wwww..g.ccccco/gggiataaa -g-rorld-university-
Page
Summary:
(search content omitted due to length limit)
Step 6
Reasoning Content
Now need to continue analyzing the user's query and determine the next steps. The user requires the top five universities in each of the five broad subject areas of the QS World University Rankings by Subject 2025, along with their QS overall ranking, THE ranking, official website homepage, regular decision application deadline, and application fee. First, based on previous search results, I initially searched for the five broad subject cate- gories of QS 2025 and confirmed them Arts and Humanities, Engineering and Technol- ogy, Life Sciences and Medicine, Natural Sciences, and Social Sciences and Management. Then, attempted to search for the top five universities each subject area, but the initial results were incomplete inconsistent. Therefore, I used the text browser view tool to cess the official QS article link and successfully obtained the top five universities for each subject area:
Arts and Humanities: Harvard, Oxford, Cambridge, Stanford, UC Berkeley
Engineering and Technology: MIT, Oxford, Stanford, Cambridge, ETH Zurich
Life Sciences and Medicine: Harvard, Oxford, Johns Hopkins, Stanford, MIT
Natural Sciences: Harvard, MIT, Oxford, Cambridge, Stanford
Social Sciences and Management: Harvard, Oxford, Stanford, MIT, Cambridge
37

---

Published as a conference paper at ICLR 2026
Next, the remaining information collect includes:
1. QS World University Rankings 2025 (overall ranking) for each university
Times Higher Education World University Rankings 2025 for each university
3. Official website homepage URL
Regular decision application deadline
5. Application fee (without fee waiver)
Current status:
QS subject rankings are confirmed.
QS overall rankings and THE rankings have been searched but not yet obtained. Previous search results for QS overall rankings (name=l) only mentioned the number of universities and countries but not specific rankings. THE rankings (name=2) also lack complete data, with only some universities mentioned (e.g., University of Melbourne 47th, Johns Hop- kins at 16th).
Official website URLs, deadlines, and fees have not been searched yet. Next steps:
1. **Obtain QS 2025 overall rankings for each university**; Need search for the QS World University Rankings 2025 list, focusing on the top 50 universities to find the rankings of the 25 universities (5 subjects x universities each, but there may overlaps, such as Harvard appearing in multiple subjects).
2. **Obtain THE 2025 overall rankings**; Similarly, search for the THE World University Rankings 2025 list to find the rankings of these universities.
3. **Collect official website *RLs*#; For each university, search for their official home- page URL
**Collect application deadlines and fees**; For each university, search for their under- graduate regular decision deadline and application fee (excluding fee waivers).
Since these universities are mostly top global institutions, their overall rankings and THE rankings may be available in official ranking releases. However, previous searches for QS and THE rankings did not yield detailed lists, so more precise searches are needed, such as "QS World University Rankings 2025 top 50" or searching for each university individually (e.g.. "Harvard University QS ranking 2025").
Additionally, application deadlines and fees may vary by university and program, but the user specified "standard application deadline for regular decision" and "application fee with- out fee waiver," which should be obtainable from university admissions websites or official undergraduate admissions pages.
Given the large number of universities (approximately 25 unique institutions), it may be ef- ficient first compile list of unique universities, then search for their QS overall ranking, THE ranking, website, deadline, and fee one by one. Alternatively, search for QS and THE ranking lists first extract all required university rankings, then handle website, deadline, and fee information.
Possible steps:
**Step ** Extract unique universities from the five subject rankings to avoid redundant searches. For example, Harvard, Oxford, Stanford, MIT, Cambridge, etc., appear in multiple subjects.
**Step 2** Search for QS World University Rankings 2025 top 100 to obtain overall rankings for these universities.
**Step 3**; Search for THE World University Rankings 2025 top 100 to obtain their THE rankings.
**Step 4**; For each unique university, search for their official website, regular decision deadline, and application fee.
However, considering time constraints and search efficiency, using batch searches may be better. For example:
Search "QS World University Rankings 2025 top 50 list" to get list of universities and their rankings.
Similarly, search "THE World University Rankings 2025 top 50 list."
For website URLs, deadlines, and fees, since each university's 's information on their of- ficial site, it may be necessary to search individually for each university, such as "Harvard University undergraduate application deadline 2025" and "Harvard University application
38

---

Published as a conference paper at ICLR 2026
fee 2025."
But given the number of universities (approximately 15 unique institutions), this could be time-consuming. Alternatively, use parallel searches where possible, but the tool may limit the number of concurrent searches.
Current missing information is critical, so further searches are required. Next, prioritize ob- taining QS and THE overall rankings, followed by website URLs, deadlines, and fees. First, search for QS 2025 overall rankings. Previous attempts (name=l) used the query "QS World University Rankings 2025," but results only mentioned the number of institutions without specific rankings. Need to adjust the query to "QS World University Rankings 2025 top 50" or "QS World University Rankings 2025 list" to get detailed rankings.
Similarly, for THE rankings, previous searches (name=2) lacked complete data. Need to search "Times Higher Education World University Rankings 2025 top 50" similar to ob- tain the full list.
Once QS and THE rankings are obtained, proceed to collect website URLs, deadlines, and fees. For these, individual university searches may be necessary, such as "University of Ox- ford official website" or "MIT undergraduate application deadline 2025." Therefore, the next steps should be:
Search for QS 2025 overall ranking list extract rankings for the 25 universities. 2. Search for THE 2025 overall ranking list to extract their rankings.
For each university, search for official website, deadline, and fee. Since the user requires organizing the data into table, all information must be complete. Currently, QS and THE rankings are the most urgent missing pieces, followed by website URLs, deadlines, and fees.
Thus, the next search should focus on obtaining the QS 2025 and THE 2025 overall ranking lists.
Tool Call
"tool name": "search global",
"arguments": "I\ "[\"query'. "'S World University Rankings 2025 top 4 50 list\", \count ":5,, \ ""summary_ "sunmary_typel"""" ":pho":\""hoor""" "tool_call_id". "aal__Bsffvvbbgee2220ece
"tool "too"_name": "search global",
"arguments": "('"query\":>
"\"mmms "imms Higher Education World 4 University Rankings 2025 top 50)", \"count\";,, \" summary_type ":>\">ort>"""]>
"tool call id"; "call wtl.k_k2222o47732ppppop. 7d20z"
Tool Response
[index] [siteName] qs com
(search content omitted due to length limit)
Step 7
Reasoning Content
I now need to continue analyzing the user's question to determine the next steps. The user wants the top five universities in each of the five major subject areas from the QS World University Rankings by Subject 2025, along with their overall QS 2025 rankings, THE 2025 rankings, official website homepage, regular decision application deadline, and application fee. This information needs to be organized into a Markdown table.
First, based on previous searches, the five major subject areas have been confirmed: Arts and Humanities, Engineering and Technology. Life Sciences and Medicine, Natural Sci- ences, and Social Sciences and Management. Through the text_ browwbrowweeiicc vvew tool, the top five universities for each subject area have been obtained. For example, the top five in Arts
39

---

Published as a conference paper at ICLR 2026
and Humanities Harvard, Oxford, Cambridge, Stanford, and UC Berkeley. Next, the QS overall rankings and THE rankings for these universities are required. From search results, the top five QS overall rankings are MIT. Imperial College London, Oxford, Harvard, and Cambridge. However, the user needs the rankings of all 25 universities (5 sub- jects universities), which may extend beyond the top five. For instance, Stanford might rank sixth in QS overall, requiring confirmation. Similarly, THE rankings are not fully pro- vided current search results; only partial information like Harvard and MIT's positions is available, while other universities" THE rankings remain unclear.
Additionally, the official website homepage, application deadline, and application fee for each university must be collected. These details not covered in existing search results and require further searches for each university. For example, confirming Harvard's official website URL., regular decision deadline (typically January 1st), and application fee (around S85), but specific 2025 data may change and need verification. Current missing information includes:
1. QS overall rankings for all 25 universities.
THE overall rankings for all 25 universities.
3. Official website URLs for each university.
Regular decision application deadlines for 2025.
5. Application fees (excluding fee waivers) for 2025.
Next steps:
Search for the complete QS World University Rankings 2025 obtain overall rankings for each university.
Search for the complete THE World University Rankings 2025 for the same purpose.
For each university, search its official website to confirm the homepage URL.
Check each university's admissions page for 2025 regular decision deadlines and applica- tion fees.
Given the large volume of universities involved, parallel searches may be necessary. For ex- ample, first retrieve the full QS and THE rankings, then batch-search website URLs, dead- lines, and fees. However, due tool limitations, searches must be conducted step-by-step. First, prioritize obtaining complete QS and THE rankings. Previous searches showed in- complete QS top 50 results, so further searches the full QS 2025 rankings are needed. Similarly, THE rankings lack comprehensive data, requiring additional searches.
Once the overall rankings are confirmed, proceed to collect each university's 's official web- site, deadline, and fee. This may require searching each university individually, which is time-consuming but essential for accuracy.
In summary, the next steps involve:
1. Searching for the complete QS World University Rankings 2025 to get all 25 universities'
overall rankings.
2. Searching for the complete THE World University Rankings 2025 for the same.
For each university, searching official website URLs, application deadlines, and fees. Due to the complexity. multiple search steps will be required, potentially using parallel tool calls improve efficiency. For example, searching QS and THE rankings simultaneously, then processing university-specffic information in batches.
Tool Call
"t "tool_ name": "search_globaL"
"arguments"; (\"query'"' **os World University Bankings 2025 4 full list\" ("count 10, \ssumaryy_yype \"short\")", "tool_call_id", "all
"tool_name" "search_ggobba",
" "arguments": (\'quer\'": 'TTimes Higher Education World
4 University Rankings 2025 full list\" \"counti": 10, \" + summary summary_type typelyye"" " \"short\")",
"tall_call_dd" id"; caal_aacennndddkkllggggwwwww
40

---

Published as a conference paper at ICLR 2026
Tool Response
[index] [siteName| | ga.com
(search content omitted due to length limit)
Step 8
Reasoning Content
I now need continue analyzing the user's query and existing search results determine whether further searches are required. The user wants the top five universities in each of the five broad subjects from the QS World University Rankings by Subject 2025, along with their positions in the QS World University Rankings 2025 and Times Higher Education World University Rankings 2025, official website homepage. regular decision application deadline, and application fee (excluding fee waivers). The results should be organized into Markdown table.
First, the previous search has successfully confirmed the five broad subjects: Arts and Hu- manities, Engineering and Technology, Life Sciences and Medicine, Natural Sciences, and Social Sciences and Management. Through brx.bbroweer.vew, the top five universities for each subject were obtained:
Arts and Humanities: Harvard, Oxford, Cambridge, Stanford, UC Berkeley Engineering and Technology: MIT, Oxford, Stanford, Cambridge, ETH Zurich
Life Sciences and Medicine: Harvard, Oxford, Johns Hopkins, Stanford, MIT
Natural Sciences: Harvard, MIT, Oxford, Cambridge, Stanford
Social Sciences and Management: Harvard, Oxford, Stanford, MIT, Cambridge
Next, I need to find the QS World University Rankings 2025 and Times Higher Education (THE) 2025 rankings for these universities. The user's initial search for QS rankings found results indicating MIT is 1st, Imperial College London 2nd, Oxford 3rd, Harvard 4th, and Cambridge 5th. However, the list of top 50 QS rankings obtained later only partially men- tions some universities, and full rankings are unclear. For example, Stanford's 's QS ranking is unspecified, as are UC Berkeley, ETH Zurich, Johns Hopkins, etc. Therefore, more detailed QS 2025 rankings are needed to confirm each university's position.
Regarding THE rankings, previous searches show fragmented information, such as Harvard being 1st, MIT and Oxford tied for 2nd, and Cambridge ranked 4th. However, there is in- sufficient data for universities like Stanford, UC Berkeley, ETH Zurich, and Johns Hopkins. precise search for the complete 2025 rankings
The THE official website may require to obtain accurate positions for each university.
Additionally, the user needs official website URLs, regular decision deadlines, and applica- tion fees. This information is not covered in existing search results and must be obtained by searching each university's official admissions page. For example, Harvard's 's deadline is typically early January, with an application fee of $85, but confirmation is needed for 2025. Other universities like Oxford and MIT may have different deadlines and fees, requiring dividual verification.
Current missing information includes:
1. QS 2025 overall rankings for each university (e.g.. Stanford, UC Berkeley, ETH Zurich, Johns Hopkins)
2. THE 2025 overall rankings for each university
3. Official website URLs for each university
4. Regular decision application deadlines for 2025
5. Application fees (excluding fee waivers) for 2025
Next steps:
Search for the complete QS World University Rankings 2025 to confirm the overall rank-
ings of all 25 universities (5 subjects universities)
Search for the complete THE World University Rankings 2025 to confirm their overall rankings
For each university, search its official website to find the admissions page, record the home- page URL, regular decision deadline, and application fee Since each university's deadline and fee may differ and require individual verification, this
41

---

Published as a conference paper at ICLR 2026
could involve numerous searches. However, ensure accuracy, it may be necessary search each university separately. Alternatiely, seek aggregated data, but given the speci- ficity of the requirements, direct searches on official websites are more reliable.
Is further searching needed? Yes. Current information gaps include overall rankings for multiple universities, official website URLs, deadlines, and fees. These must be addressed through additional searches.
Tool Call
"tool_name"s "search_global
" "arguments"; (V"query>> "; **os World University Rankings 2025 4 full 1ist top 100 \"count "ssumaryy_tppe"" 4 short\")", stool_call_id";
"scarch_global",
"arguments"; (V"quer\\:: \"Times Higher Education World 4 University Rankings 2025 top 100\", \"counti": 5, \" 4 summary_type!". \"short\")", "tool_call_ idd;; m
"tool name::
"arguments"; (qquery\") \"Harvard University official website 4 \", >"count\:; 1, "'summary_typel ("short>")", "tool_call_ haallrrpouu(]ajjigddbbe.....
"tool "too!_name": "search global",
"arguments"; I ('"query>" \"Iniversity of Oxford official
4 website)", \"count ; 1, \"summary "s \"short\"j"
"tool_call
"tool_name"; "search_global"
"arguments"; (\"query\* ": \"Iniversity of Cambridge official
4 website\", \"count("; 1, ""sumary_type"": "short\"]" "tool_call_id", "call__"aaaa_aeee "eal__19222ttgrrrrsaaaaaaaa"""""
Tool Response
[index] 0
[siteName] topuniversities.cm com
(soarch content omittod due to length limit)
Step 9
Reasoning Content
Now I need to continue analyzing the user's question and existing search results determine the next steps. The user requires the top five universities in each of the five broad subjects from the QS World University Rankings by Subject 2025, along with their overall rankings in QS and THE 2025, official website, application deadlines, and fees. These need to be organized into Markdown table.
First, based previous searches, the five broad subjects have been confirmed: Arts and Hu- manities, Engineering and Technology, Life Sciences and Medicine, Natural Sciences, and Social Sciences and Management. Through wrw.brrwr.rveew the top five universities for
42

---

Published as a conference paper at ICLR 2026
each subject have been obtained, such Harvard University and MIT. This part relatively clear, but there may be a need to verify consistency across sources. Next, regarding the overall QS and THE rankings for these universities, existing search re- sults show partial information. For example, QS's top five include MIT, Imperial College London, Oxford, Harvard, and Cambridge. However, the user needs the rankings of each university in both QS and THE. For instance, where does Stanford rank in QS overall? The current search results not provide complete top 100 list, making impossible to accu- rately fill in the rankings for all 25 universities (5 subjects x 5 universities).
Additionally, official websites for some universities have been found, such as Oxford's ox.ac.uk uckukk and Cambridge's cam.ac.uk, but others like Stanford, MIT, and UC Berkeley still need verification. Application deadlines and fees are completely missing; these require ac- cessing each university's official website reliable admission pages obtain, which will be time-consuming.
Current missing information includes:
1. QS and THE 2025 overall rankings for each of the 25 universities. Official websites for all universities (some confirmed, others pending).
3. Regular decision application deadlines and fees for each university.
Next steps should prioritize obtaining the complete QS and THE 2025 rankings to fill in the overall ranking columns. If direct access to the full list is unavailable, searching for each university individually (c.g.. "Stanford University QS World University Rankings 2025") might be necessary. However, this could be inefficient. Alternatively, searching for "QS World University Rankings 2025 top 100 list" or similar keywords might yield a complete ranking table.
For application deadlines and fees, since each university 's official website must be consulted, this will require multiple searches direct access via text browser view. For example, searching for "Harvard University undergraduate application deadline 2025" and "applica- tion fee" for each institution. However, given the large number universities involved, this process will be time-intensive and may require parallel searches.
Additionally, verifying official websites is crucial avoid incorrect links. For example, while Harvard's official site is likely harvard.edu, confirmation through authoritative sources necessary.
Summary of next steps:
Continue searching for complete QS and THE 2025 top 100 rankings to fill in overall rankings for all 25 universities.
Collect official websites for all universities (some already found, others pending).
3. For each university, search for regular decision application deadlines and fees, preferably accessing official admissions pages directly.
Given the complexity, prioritizing the acquisition of overall rankings first advisable, as this is critical for table completion. Application deadlines and fees may require subsequent searches direct extraction from university websites via browser.viww.. Tool Call
"tool_name": "search_global"
"arguments" "1 (\"query\": \*os world University Rankings 2025 4 top 100 list "DF ("count\". 3, "ysumarry_type"" "sshort 4 \")", "tool_call id"; "call veal__oo22n2gggrvvaooortaaaatat
"tool_name": "search_globai"
"arguments": (\"query\": **Times Higher Education World 4 University Rankings 2025 top 100 1ist\", \"count\"; 3, 4 summary_type)"" \"shorti*)", call id": "call " htt___oooooonnttkkkkkokkooooeee
43

---

Published as a conference paper at ICLR 2026
name:: "search "search_global",
"arguments"; (\"query\" \*ttnford University official website 4 ,", >"ount\": 1, ("summary_type \"short\""", "tool_calllio" id"; "call_m788" wggg
"tool_name" "scarch_global", "a "arguments": "\ (\"query\" \"MIT official websitel", ("ccunt\"; 4 1, Vyuumary_typel \"short\")", "tool_call_id"" "call "cal1_w32999""
"tool_name": "search_global"
"a "arguments": (\"query\": **Cc Berkeley official website\", \"
4 count\": Sroumarr_tyyeyyl \"shor\\")"
"tool_call_id"" "call_ @eal_oookhhhacff@@@eeeeeu
Tool Response
[index]
[siteName] qs comcom
(search content omitted due to length limit)
Step 10
Reasoning Content
I now need to continue analyzing the user's query and existing search results to determine the next steps. The user requires the top five universities in each of the five broad sub- jects from the QS World University Rankings by Subject 2025, along with their rankings in the QS World University Rankings 2025 and Times Higher Education World University Rankings 2025, as well as official website links, regular decision application deadlines, and application fees. The results should organized into Markdown table.
First, reviewing previous search results, the five broad subjects have been confirmed: Arts and Humanities, Engineering and Technology, Life Sciences and Medicine, Natural Sci- ences, and Social Sciences and Management. Through the text browser view tool, the top five universities for each subject have been obtained. For example, Arts and Humanities includes Harvard University, University of Oxford, University of Cambridge, Stanford Uni- versity, and UC Berkeley.
Next, the QS World University Rankings 2025 and Times Higher Education rankings for these universities need to be determined. Previous searches for QS rankings found the top five: MIT, Imperial College London, University of Oxford, Harvard University, and Uni- versity Cambridge. However, other universities like Stanford and UC Berkeley are not included in this top five, so their QS rankings must be confirmed. Similarly, the Times Higher Education rankings lack complete top 100 data, making it impossible to determine the rankings for all required universities.
Regarding official websites, some universities' official sites have been found, such as Har- vard (harvard.com edu), University of Oxford (ox.ac.uk), University of Cambridge (cam.ac.u) ucku), Stanford (stanford.edu), and MIT (mit.edu). However, links for UC Berkeley and ETH Zurich may require verification.
Application deadlines and fees are entirely missing from current search results. The user specifically requires regular decision deadlines and fees without waivers, which must be searched for each university individually. Next steps:
1. **Confirm QS 2025 rankings for all 25 universities (5 subjects 5 universities)** Exist- ing results only cover top five QS rankings, so additional searches are needed for universities like Stanford, UC Berkeley, ETH Zurich, etc.
2. **Confirm Times Higher Education 2025 rankings for all 25 universities Current re- sults lack sufficient data, requiring more precise searches, such as using each university's
44

---

Published as a conference paper at ICLR 2026
keyword.
3. **Verify official website links** for all universities to ensure accuracy, particularly for institutions like UC Berkeley and ETH Zurich.
4. **Search for regular decision application deadlines and fees** for each university, this information entirely missing and critical for the user.
Potential challenges:
Incomplete QS and Times rankings may require multiple searches or accessing official ranking lists.
Application deadlines and fees may vary by program and could be scattered across univer- sity admissions pages, necessitating precise searches for each institution.
For example, to find Stanford's QS 2025 ranking, search "Stanford University QS World University Rankings 2025." Similarly, search "Stanford University Times Higher Education World University Rankings 2025" for Times rankings. For deadlines and fees, use queries like "Stanford University undergraduate regular decision deadline 2025" and "Stanford Uni- versity application fee undergraduate."
Since there are 25 universities, searching each individually would be time-consuming. How- ever, given the current lack of data, this step is unavoidable. To improve efficiency, batch searches could be attempted, such as querying multiple universities' QS rankings simulta- neously, but search engines may yield incomplete results. Individual searches per university remain more reliable.
Additionally, some universities' application deadlines and fees might not yet be updated for 2025, requiring reliance on 2024 data with notes indicating the year. However, the user specified 2025 rankings, efforts should focus finding the latest information.
Summary of next actions:
1. For each of the 25 universities, search their QS 2025 and Times Higher Education 2025 rankings.
Verify official website links for each university.
3. Search for regular decision application deadlines and fees for each university. Organize all collected data into the required Markdown table format.
Given the complexity, it may be necessary to prioritize completing rankings first, followed by deadlines and fees, as rankings are foundational to the table structure. some data cannot be found, note it as "Not Available" similar, but the user likely expects comprehensive information, so maximum efforts should be made to find required details. Tool Call
"tool_name": "search_global"
"arguments"; (\"query\", \"stanford University QS World
4 University Rankings 2025\", ("count\". 1, \"summary_typell 4 \"short\""", "tool_call id"; "eall_wwba2233eees6ffffmmm2???
"tmee_ "rearch_global"
"arguments": (\"query\" \oC Berkeley QS World University
4 Rankings 2025 \"counti": 1, \"summary_type\": V"short
4 {")
"tool call ia"; "call www.w//wwwaee///eteooeooooooooo
"tool_name": "search_globbl"
"arguments": (\"query\*) ": \EETH Zurich Qs world University
4 Rankings 2025\", \"counti": 1, \"summary_type"" \"short 4 {"}",
"tool_call id"; "call hww.e//ssssseeeeeeeeyyyyyyysssss
45

---

Published as a conference paper at ICLR 2026
"tool_name", "search globaai,
"arguments"; ( (>"query\": \*Johns Hopkins University @s world
4 University Rankings 2025 \"cunt\ 1, \"summary_type!" 4 \"short")"", "tool cooo__alllid"" "call_ful9nsodd 70ggy999w3771
"tool_ name__ "search_glaae
"arguments": " (V"quer\':: "HHavarrd University Times Higher 4 Education World University Rankings 2025\", \"oonn("; 1,
4 "'sumarry_yypo \"short\")*
"tool_call_oo" "aallpprv60iccc12ttt000hn
Tool Response
[index] 0
[siteName] com
(search content emitted due to length limit)
Step 11
Reasoning Content
I now need to continue analyzing the user's question determine whether further searches are necessary. The user requires the top five universities in each of the five broad subject areas from the QS World University Rankings by Subject 2025, along with their QS World University Rankings 2025 and Times Higher Education World University Rankings 2025 positions, official website homepage, regular decision application deadline, and application fee (excluding fee waivers). These should be organized into Markdown table.
First, based on previous searches, have obtained the top five universities for each of the five broad subjects from the QS by Subject 2025 rankings. For example:
Arts and Humanities: Harvard, Oxford, Cambridge, Stanford, UC Berkeley
Engineering and Technology: MIT, Oxford, Stanford, Cambridge, ETH Zurich
Life Sciences and Medicine: Harvard, Oxford, Johns Hopkins, Stanford, MIT
Natural Sciences: Harvard, MIT. Oxford, Cambridge, Stanford
Social Sciences and Management: Harvard, Oxford, Stanford. MIT. Cambridge
Next, need to confirm the QS World University Rankings 2025 and Times Higher Edu- cation (THE) 2025 rankings for each of these universities. Previous searches indicate that the top five QS World University Rankings 2025 are MIT (1), Imperial College London (2), Oxford (3), Harvard (4), and Cambridge (5). However, other universities like Stanford, UC Berkeley, ETH Zurich, and Johns Hopkins are not the top five, so their specific rankings need be found. For example, in the latest search for Stanford's QS ranking, the results did not directly provide the 2025 data, requiring further confirmation. Regarding THE rankings, previous search results mentioned in THE's reputation rankings but not its overall ranking. Other universities' THE rankings remain unclear, necessitating targeted searches for each institution's THE 2025 ranking.
fragmented. For instance, Harvard is
Additionally, the official websites of some universities have been confirmed:
Harvard: harvard.edu (from search results) Oxford: ac.ac.uk
Cambridge: cam.ac.uk
Stanford: stanford.edu
MIT: mit.edu
UC Berkeley: berkeley.edu
EETH Zurich: ethz.ch (to be confirmed)
-Johns Hopkins: jhu.edu (to be confirmed)
For application deadlines and fees, this information is not yet available and requires searches
for each university's undergraduate regular decision deadline and application fee for 2025. For example, Harvard's deadline is typically January 1st, with fee around $85, but 2025
46

---

Published as a conference paper at ICLR 2026
specifics need verification.
Current missing information includes:
1. QS 2025 overall rankings for Stanford, UC Berkeley, ETH Zurich, Johns Hopkins.
2. THE 2025 overall rankings for all universities.
Official websites for ETH Zurich and Johns Hopkins.
4. Regular decision application deadlines and fees for each university.
Further searches are needed for each missing piece of information. For example:
Search "[University Name] QS World University Rankings 2025" for each institution.
Search "[University Name] Times Higher Education World University Rankings 2025".
Confirm official websites for ETH Zurich and Johns Hopkins.
Search "[University Name] undergraduate regular decision deadline 2025" and "[Univer- sity Name] application fee 2025".
Given the extensive information required, parallel searches may be necessary to improve efficiency. For example, searching for QS and THE rankings for multiple universities simul- taneously, followed by application deadlines and fees.
Potential challenges include inconsistent ranking data across sources or delayed publication of 2025 THE rankings. If official data cannot be found, it may be necessary to use the latest available information or note that data pending release.
In summary, further searches are required for each university's 's QS and THE rankings, of- ficial websites (for remaining institutions). application deadlines, and fees. This step will involve multiple parallel searches to gather all necessary information for the table. Tool Call
"scarch_global"
" "arguments": (\qqurr\") Staanford University QS 2025 ranking 4 \", >"count":: 1, "ssumary_typel"" \"short\")", "tool_call_id", (eall_kk77tyyygssljjjp3...ss
"tool_name" "search_global"
"arguments"; (V"query\:: \*Uc Berkeley QS 2025 ranking)", \" 4 count\": 1, "ssumaary.type"" \"short\")", "tool_call id"; "call www.:/wwwwwwwwwwwwweee...... "
"tool_name" seearch_global"
"arguments"; (\"query\": \EETH Zurich Qs 2025 ranking\", \" 4 count 1, \ \"sumarry_type"" \"short\")", "tool_call__ id"; "call__unbaae (tt__uuboyydtaeeedeeeeeeggg
"tool_name"" "search_globai"
"arguments"; (V"query\ ": \"Johns Hopkins University Q8 2025
4 ranking"", \"count"; 1, ("summary_type""" \"short\")", "tool_call id"; "call wpw.eaatiiiieebbbeaaaiiiaaaaa
"tool_name"" name": "search_globb""
"arguments": P [q"uurr\"* ": HaHarvrrd University THE 2025 ranking 4 \", \"count)": 1, \"summary_tyee \"short\")>
"tool call id"; "eall wwwp.heeeeewurrrrrrrrrr....
Tool Response
[index]
47

---

Published as a conference paper at ICLR 2026
[siteName] es com
(search content cmitted due to length limit)
Step 12
Reasoning Content
I now need to continue analyzing the user's question and the existing search results to de- termine the next steps. The user wants the top five universities in each of the five broad subjects from the QS World University Rankings by Subject 2025, along with their QS and THE rankings, official websites, application deadlines, and fees. The results should be or- ganized into a Markdown table.
First, from previous searches, the five broad subjects have been confirmed: Arts and Hu- manities, Engineering and Technology, Life Sciences and Medicine, Natural Sciences, and Social Sciences and Management. The top five universities for each subject were extracted from the QS official website link using the text. browser view tool, which is reliable. For example, Arts and Humanities includes Harvard, Oxford, Cambridge, Stanford, and UC Berkeley.
Next, the QS World University Rankings 2025 and THE World University Rankings 2025 for these universities are required. Previous searches found the top five QS rankings: MIT (1), Imperial College London (2), Oxford (3), Harvard (4), and Cambridge (5). However, the top five universities in each subject may not all be within the top five overall. For instance, Stanford and UC Berkeley are not listed here, requiring their specific QS rankings. Simi- larly, THE rankings are incomplete; only Harvard, MIT, and Oxford's reputational rankings were found, but not their overall THE rankings.
Additionally, official websites for some universities have been identified, such as Harvard (harvard.edu)) Oxford (ox.ac.uk) Cambridge (cam.ac.uk) Stanford (stanford.edu), and UC Berkeley (berkeley.edu), However, websites for ETH Zurich, Johns Hopkins University, etc., still need confirmation.
Regarding application deadlines and fees, no information has been collected yet. These de- tails are typically found on university admissions pages, requiring further searches for each university's regular decision deadline and application fee (excluding fee waivers).
Current missing information includes:
1. QS 2025 overall rankings for universities beyond the top five (e.g., Stanford, UC Berke- ley, ETH Zurich, Johns Hopkins).
2. THE 2025 overall rankings for all listed universities.
Official websites for universities like ETH Zurich and Johns Hopkins.
4. Application deadlines and fees for each university. Next steps:
Search for QS 2025 rankings for Stanford, UC Berkeley, ETH Zurich, and Johns Hopkins.
Search for THE 2025 overall rankings for each university.
Confirm official websites for remaining universities.
Collect application deadlines and fees for each university via their official sites or reliable sources.
Since some universities' QS rankings remain unconfirmed (e.g., Stanford), further targeted searches are needed. For THE rankings, as previous searches yielded limited results, more precise queries (e.g.. "Harvard University THE World University Rankings 2025") may be required. Application deadlines and fees will likely necessitate visiting each university's admissions page, which could be time- time-consuming but critical for completing the table.
In summary, additional searches are required for QS and THE rankings, official websites, deadlines, and fees to ensure the table is comprehensive and accurate. Tool Call
" "tool_name" "scarch_global"
"arguments"; "(" "('"query's "; " ":\"Stanford University QS 2025 everall 4 ranking)"" ""cont \ : "tool_call_id" id"; "call_mOwy3 3
48

---

Published as a conference paper at ICLR 2026
"tool_name"" "search_globa1*
"arguments": "(" "'""uurr\":*>"0 Berkeley Qs 2025 overall ranking
4 \", ""ount)":ll wtt.://:../ssmay.ssn..oa. "a
"tool_call_id"; "call_ 1aaa.
"tool_namee: "search_g1obal",
"arguments": "(" "[""query\" ". "EETH Zurich THE 2025 ranking)", \ \"count
4 \":1, "summary__yye ""ccypp"""""soor""""""""""""""""
"tool_call id"; "call seall_#tttucckkkk111222c00a.aaa
"tool "tool_name": name": "search_ "search_global",
"arguments": "(\" ""A"query\"; "Joohss Hopkins University official
4 Beabsite\",""eee ""ounn\" \";1, "summary_type """"e"::"""""""""""""""""""""""" "tool_call id"; seall__ex3y/exx36cce
name:, "search_ "iorcch__gooba,,,
"arguments"; "I\"query"" " "Harvard University 2025 regular
"aarrar
4 decision deadl ine \", "ccunnt \ A "summary_type :
4 "")", "tool_calll_n "cal1_k494jes" hwt.5/imm33355555 mmh33355mmp223555
Tool Response
[index] 0
[siteName] gs.
(search content omitted due to length limit)
Step 13
Reasoning Content
now need to continue analyzing the user's question and determine the next steps. The user wants the top five universities in each of the five broad subjects from the QS World Uni- versity Rankings by Subject 2025, along with their QS and THE overall rankings, official website, regular decision deadline, and application fee, organized into table.
First, the previous searches have identified the five broad subjects: Arts and Humanities, Engineering and Technology. Life Sciences and Medicine, Natural Sciences, and Social Sciences and Management. The top five universities for each subject were obtained by ac- cessing the QS official website link (via text browser view). which is reliable. For example: Arts and Humanities: Harvard, Oxford, Cambridge, Stanford, UC Berkeley Engineering and Technology: MIT, Oxford, Stanford, Cambridge, ETH Zurich Life Sciences and Medicine: Harvard, Oxford, Johns Hopkins, Stanford, MIT Natural Sciences: Harvard, MIT, Oxford, Cambridge. Stanford
Social Sciences and Management: Harvard, Oxford, Stanford, MIT, Cambridge
Next, the QS and THE overall rankings for these universities are needed. Previous searches for the QS 2025 overall rankings only found the top five (MIT, Imperial College London, Oxford, Harvard, Cambridge), but most of the top five universities in the subject rankings (c.g., Stanford, UC Berkeley, ETH Zurich, Johns Hopkins) missing their QS overall rankings. Similarly, THE rankings lack data for most universities.
For example, Stanford University is top-five university in multiple subjects, but its QS 2025 overall ranking is unknown. Previous searches using "Stanford University QS 2025 ranking" yielded results. The same applies to UC Berkeley and ETH Zurich. Johns Hop- kins' QS ranking is also unclear. Additionally, THE rankings for Harvard, MIT, Oxford, etc., remain unspecified.
49

---

Published as a conference paper at ICLR 2026
Regarding official websites, universities have been confirmed:
Harvard: harvard.edu
Oxford: ax.ac.uk
Cambridge: cam.ac.uk
Stanford: stanford.edu
MIT: mit.edu
-C Berkeley: berkeley.edu
Johns Hopkins: jhu.edu (from search results)
ETH Zurich: ethz.ch (to be confirmed)
Application deadlines and fees mostly missing. Only Harvard's regular decision deadline (January 1) was found via admissionsight.com but this requires verification. Other univer- sities' deadlines and fees entirely unknown and need separate searches. Current missing information includes:
QS 2025 overall rankings for Stanford, UC Berkeley, ETH Zurich, Johns Hopkins, etc.
2. THE 2025 overall rankings for all 25 universities (five subjects five universities each), Official websites for ETH Zurich (needs confirmation).
4. Regular decision deadlines and application fees for all 25 universities.
Next steps:
Continue searching for QS 2025 overall rankings for each university (e.g.. "Stanford Uni-
versity QS World University Rankings 2025").
Search for THE 2025 overall rankings for each university (e.g.. "Harvard University Times Higher Education World University Rankings 2025"))
Confirm ETH Zurich's 's official website.
Search for each university's 's regular decision deadline and application fee (e.g., "Stanford University 2025 regular decision deadline" and "Stanford University application fee 2025"). Given the large number of universities, prioritizing key institutions or using batch searches may be necessary. However, due to tool limitations, individual searches are required. For example:
Search for Stanford's QS and THE rankings, deadline, and fee. Repeat for UC Berkeley, ETH Zurich, Johns Hopkins, etc.
Additionally, some universities might have similar deadlines (e.g., many U.S. universities use January 1), but verification is still needed. Application fees vary by institution and may require official website confirmation.
In summary, further searches required for rankings, deadlines, and fees for each univer- sity. This will involve multiple tool calls, potentially exceeding the current step limit, but given the user's 's request, it is necessary proceed gradually. Tool Call
"tool namee: "scarch_global"
"arguments"; (\"query\", \"stanford University QS 2025 everall 4 ranking)", ("count\"; 1, ""summary_ type))" \"short\";", @tool_call_id". "call_ejdq3i htw.geghheeggaaataayyllaoo
"tool _oool_name:: #search_globa)
"arguments"; (\"query\*) \oC Herkeley QS 2025 overall ranking 4 \", \"count"": 1, \"summary_type""" \"short>")", "tool call id"; "call zaustmssghhrrbbeeigaa..
"tool _ooo__name:: "search_globai"
"arguments"; [\"query\": \EETH Zurich THE 2025 ranking)", \" S count)"; 1, \"sunnary typely: \"short\"i", "tool call id"; "call totp:
50

---

Published as a conference paper at ICLR 2026
"tool_name"" "search_globai1
"arguments"; (V"query) ": \*Johns Hopkins University THE 2025
4 ranking\", \"count\"; 1, "ssumarry_typ""" ""short\"'"
"tool_call_id"; "aal_
"tool_ _ooo__nmme: "search global", "search_global",
"arguments": "A (V"quer\\",, : \"Stanford University 2025 regular 4 decision dcadline\", \"count\". 1, ""summary_type\ \"
4 short\")", "call_olwallllle www.hxpmmee 37113mmhlndde
Tool Response
[index] 0
[siteName] es.
(search content emitted due to length limit)
Step 14
Reasoning Content
I now need continue analyzing the user's query determine if further searches are re- quired to complete the response. The user needs the top five universities in each of the five major subject areas from the QS World University Rankings by Subject 2025, along with their QS overall ranking, THE ranking, official website, regular decision deadline, and ap- plication fee, all organized into a table.
First, reviewing previous search results: The five major subject areas have been confirmed as Arts and Humanities, Engineering and Technology, Life Sciences and Medicine, Natural Sciences, and Social Sciences and Management. Through multiple searches, the top five universities each subject area were obtained from the official QS article, including Har- vard, Oxford, Cambridge, Stanford, UC Berkeley, MIT, ETH Zurich, Johns Hopkins, etc. These names are clear.
Next, regarding the QS overall ranking and THE ranking: The QS 2025 overall ranking's top five include MIT, Imperial College London, Oxford, Harvard, and Cambridge. However, other universities like Stanford and UC Berkeley lack specific rankings. Previous attempts to search for their QS rankings yielded no direct results. Similarly, THE rankings for most universities remain unclear, with only Harvard mentioned as possibly No. and Johns Hop- kins ranked 16th. Other universities like Stanford, MIT, and Oxford lack THE ranking data. Regarding official websites: Some universities' official sites have been found, such as Harvard (harvard.edu), Oxford (ox.ac.uk), Cambridge (cam.ac.uk), Stanford (stanford. edu)) MIT (mit.edu), UC Berkeley (arkeleeyedu)) and Johns Hopkins (jhu.edu). However, ETH Zurich's official website still needs confirmation.
Application deadlines and fees: Only Harvard's regular decision deadline (January 1) and Stanford's 's deadline (around January have been partially identified. Other universities' deadlines and fee information are completely missing, including Oxford, Cambridge, MIT, UC Berkeley, ETH Zurich, Johns Hopkins, etc.
Current missing information includes:
QS overall rankings for Stanford, UC Berkeley, ETH Zurich, and Johns Hopkins.
2. THE rankings for all universities except Harvard and Johns Hopkins.
Official website for ETH Zurich.
4. Regular decision deadlines and application fees for all universities.
Next steps:
For QS rankings: Continue searching for Stanford, UC Berkeley, ETH Zurich, and Johns Hopkins' QS 2025 overall rankings. no results, consider whether the rankings were not released if search terms need adjustment.
- For THE rankings: Search individually for each university's THE 2025 ranking, e.g., "MIT THE 2025 ranking."
51

---

Published as a conference paper at ICLR 2026
ETH Zurich's official website: Directly search for "ETH Zurich official website."
Deadlines and fees: For each university, search for "(University Name) 2025 regular deci- sion deadline" and "(University Name) undergraduate application fee 2025."
Given the extensive missing information, particularly deadlines and fees requiring university-specific searches, multiple additional searches are needed. However, consider- ing previous repeated searches for QS and THE rankings yielded limited results, it may be necessary to use more precise keywords or accept potential data unavailability. For dead- lines and fees, since these are critical application details, further searches are essential even cone-con...
Tool Call
"tool_name" "search "search_global",
"arguments": ('"query'" \Sttanford University QS 2025 overall + ranking)", A \"count":: 1, "*uummary__ type\"; \"short\")", "tool_call id"; "call 9
"tool name": "search "search_global",
"arguments": " (\"quer\\:: \COC Berkeley QS 2025 overall ranking 4 \", ("coun\"" 1, ""summary_type"". \"short\"", ctool_callidde
"tool_ _ooo__nmmm:: saarrch_glbba,,
"arguments": (\"query\": *"ETH Zurich official website\", \" 4 count\": 1, \"summary_typel \ \"short\")", "tool_call id"; "call 1fdd7/cceecccsseep332222
"tool "tool_name": name":
"search_ "hearch_globa"", global",
"arguments"; (V"query\": *"MIT THE World University Rankings
4 2025 \"count\"; 1, "'sumaryy_type" \"short\")", "tool_call id"; "call wtw.ouuuuwkke0002/////yy2u2
"tool_name" "search "search_global",
"arguments"; (\'quer\"" ": \"University of Oxford 2025 regular 4 decision deadline\", \"count\": 1, "ssumary_typet \"
4 short\" "tool_call_id"" "aal__ffekllsppyyhooxxipppe
Tool Response
[index]
[siteName]
(search content omitted due to length limit)
Step 15
Reasoning Content
I now need to continue analyzing the user's query and determine the next steps. The user wants the top five universities in each of the five broad subject areas from the QS World University Rankings by Subject 2025, as well as their rankings in the QS and THE World University Rankings 2025, official website, regular decision deadline, and application fee, organized into table. First, have obtained the top five universities in each of the five broad subject from the
52

---

Published as a conference paper at ICLR 2026
QS website, which correct. Next, need confirm their overall rankings in QS and THE. However, previous searches for QS and THE rankings have yielded incomplete information. For example, the QS 2025 overall rankings only provided the top five, while THE rankings lack specific data for many universities.
Subsequent searches focused on individual universities' rankings, such as Stanford and UC Berkeley, but results remain unclear. For instance, Stanford's QS overall ranking is still unspecified, and THE rankings for ETH Zurich and MIT are missing. Additionally, infor- mation about application deadlines and fees for some universities, like UC Berkeley and ETH Zurich, has not been found. Next, I need to:
1. Confirm the QS 2025 overall rankings for Stanford and UC Berkeley.
2. Find THE 2025 rankings for ETH Zurich, MIT, and UC Berkeley.
3. Obtain official website links for all universities (some are already confirmed, but ETH Zurich's needs verification).
4. Collect regular decision deadlines and application fees for each university, such UC Berkeley, ETH Zurich, etc.
For example, the user has already confirmed Harvard's deadline as January 1, but Stanford's deadline mentioned as resuming on January 6, 2025, which may require clarification on the specific application deadline date. Similarly, Oxford's deadline is October 15, but this might be for undergraduate applications and needs confirmation.
In summary, further searches needed for:
QS overall rankings for Stanford and UC Berkeley.
THE rankings for ETH Zurich, MIT, UC Berkeley, etc.
Application deadlines and fees for UC Berkeley, ETH Zurich, Johns Hopkins, etc.
Official website links for ETH Zurich and Johns Hopkins (partially found but needing firmation).
Thus, additional searches should target these missing pieces of information ensure the table is complete and accurate.
Tool Call
Stool_name"" "search_globar"
"a "arguments": " (\"query\ ": \*Stanford University QS 2025 everall 4 ranking)", \"count \"; 1, \*uummary_ tyyeem"rr__y""".".. \"short\")", call id"; "call /wtp://gg/bbbby///11112222
"t "rool_name"s Esearch_global.
"arguments": " (\"query\": *ou Herkeley QS 2025 overall ranking 4 \", \"count)"; 1, ("summary_typel": \"short>";", "tool_call "call_ (oall__fooggfxxee22naaauuuooon
"tool name": "search global",
"arguments": ('querr\"; *ETHH Zurich THE 2025 ranking)", \" 4 count("; \"summar__tyee"" \"shert\")", "tool_calllid"" "fal
"tool_name": "search_global",
" "arguments"; (\"quer\\". *CCC Berkeley 2025 regular decision
4 deadline\", ("'cunti"; 1, Seunmary_typeli \"short\"]"
"tool_call_id" id";
"tool_name" "scarch_global"
53

---

Published as a conference paper at ICLR 2026
"arguments": " (Qquery\" "MIT 2025 application feel", \ \"count 4 :"; 1, Vasummary_type \"short\")", "tool_calllid"" _i"; "call_2revjk "aall_2neyjsshviirffeccccceee
Tool Response
[index] 0
[siteName] cs. com
(soarch content omitted due to length limit)
Step 16
Reasoning Content
I now need continue analyzing the user's query and determine the next steps. The user wants the top five universities in each of the five broad subjects from the QS World University Rankings by Subject 2025, along with their positions in the overall QS and THE rankings, official website, regular decision deadline, and application fee. The previous search results have covered some information, but several key points remain incomplete.
First, regarding the QS subject rankings, the top five universities in each subject were already obtained through the text_ browser. view tool, which should be accurate. Next, the overall QS and THE rankings for these universities are still missing for some institutions. For example, Stanford University's QS overall ranking and UC Berkeley's 's QS overall ranking have not been confirmed despite multiple searches. Similarly, ETH Zurich's 's THE ranking and MIT's THE ranking are unclear.
Second, regarding the official websites, most have been found, such Harvard, Oxford, Cambridge, Stanford, MIT, UC Berkeley, and Johns Hopkins, but ETH Zurich's official website was confirmed in the latest search result as www.ethz.ch.
Third, application deadlines and fees: Harvard's 's regular decision deadline is January 1, Stan- ford's deadline might in early January (but needs confirmation), UC Berkeley's ' deadline is December 2, 2024 (for Fall 2025), Oxford's ss deadline is October 15, and MIT's application fee was mentioned the search results as requiring further verification. Additionally, dead- lines and fees for other universities like ETH Zurich and Johns Hopkins are still missing. Current missing information includes:
1. QS overall rankings for Stanford University and UC Berkeley.
2. THE rankings for MIT, Stanford, UC Berkeley, ETH Zurich, and University of Cam- bridge.
3. Application deadlines for ETH Zurich, Johns Hopkins University, and University of Cam- bridge.
4. Application fees for all universities (only MIT's s was partially mentioned but not clear). Next steps should focus on filling these gaps. For example, search for Stanford's QS over- all ranking, UC Berkeley's 's QS ranking, MIT's s THE ranking, ETH Zurich's 's THE ranking, application deadlines for each university, and specific application fees. However, previous searches for Stanford and UC Berkeley's QS rankings have repeatedly yielded no results, possibly due to unavailable data or search limitations. In such cases, it might be necessary to use approximate values or indicate "Not Available," but the user requires accurate data, so further attempts with different keywords may be needed, such as "Stanford University QS World University Rankings 2025 position" or checking official QS reports.
For application fees, each university's official website should be checked, but since website access isn't possible, search queries like "Harvard University 2025 undergraduate appli- cation fee" would be needed. Similarly, deadlines for other universities need verification through searches.
Given the time-consuming nature and potential lack of results, prioritizing critical missing information is essential. For instance, confirm THE rankings for key universities, applica- tion deadlines, and fees. If searches continue to return no results, note the missing data and proceed with available information. Tool Call
"scarch_global"
54

---

Published as a conference paper at ICLR 2026
"arguments": I (qquey"""" \"Stanford University Qs 2025 overall 4 ranking official\", A"count \"; 1, Vauumary_type V"short 4 "")" "tool_call_ id"; "call "a
"tool_name":: "search_global",
"arguments"; (\"query\". " \oUC Berkeley Qs 2025 overall ranking + official\", \"count""; 1, : \"short\"", "tool_call id"; "call 4
"tool "tool_name" name": "search sloiccl_gobaa,,,_
"arguments": (V"query\", *MMIT Times Higher Education 2025 + ranking)", \"count\"; 1, suuumaayytypel""""" "short\")""
"tool_callid"" "eal1.gi
"tool name": "search globbaggll",,,
"a "arguments"; "\ (\"query\* "; **ETH Zurich 2025 undergraduate
4 application deadline \"count \"summary_type>"
4 short\")",
"tool "tool_call_id". "call Seell_dwwkgemm220ttttmmmm/
"tool_name"; "scarch_global"
"arguments": (\"query\". ""arvarrd University 2025 application 4 feel", \"ccunt\", 1, ("summary_type""" X"" \"short>")", "tool_calllide id";
Tool Response
[index] 0
[siteName] com
(search content emitted due to length limit)
Step 17
Reasoning Content
I now need to continue analyzing the user's question and determine if further searches are required to complete the response. The user needs the top five universities in each of the five broad subjects from the QS World University Rankings by Subject 2025, along with their positions in the overall QS and THE rankings, official website, regular decision deadline, and application fee (excluding fee waivers). The results should be organized into a Mark- down table.
First, based on previous searches, I have successfully identified the top five universities in each the five broad subjects (Arts and Humanities, Engineering and Technology. Life Sci- ences and Medicine, Natural Sciences, Social Sciences and Management). This information was obtained from the official QS website link, which accurate. For example, Arts and Humanities are led by Harvard, Oxford, Cambridge. Stanford, and UC Berkeley; Engineer- ing and Technology are led by MIT, Oxford, Stanford, Cambridge, and ETT Zurich, etc. Next, regarding the overall QS and THE rankings for these universities, some information has been collected, but several gaps remain. For instance, the QS overall rankings for Stan- ford and UC Berkeley have not been confirmed, and THE rankings for ETH Zurich and UC Berkeley are missing. Previous searches attempted to find this information, but the results inconsistent or incomplete. For example, Stanford's QS ranking searches yielded only
55

---

Published as a conference paper at ICLR 2026
score information without specific rankings, while ETH Zurich's THE ranking results only mentioned ranges like 251-300, which are not precise enough.
Regarding official websites, most have been confirmed: Harvard (harvard.edu), Oxford (ox.ac.uk), Cambridge (cam.ac.uk), Stanford (stanford.eduu MIT (mit.edu), UC Berkeley (berkeley.edu), ETH Zurich (ethz.ch), Johns Hopkins (jhu.edu), etc. These should be accu- rate.
For application deadlines and fees, some information available, such as Harvard's regular decision deadline of January and application fee of $85, Stanford's deadline around Jan- uary 6, UC Berkeley's 's deadline of December 2, etc. However, deadlines and fees for ETH Zurich, Johns Hopkins, and MIT are still unclear. For example, MIT's application fee was not found in search results, and ETH Zurich's undergraduate deadline is missing.
Current missing information includes:
1. QS overall rankings for Stanford and UC Berkeley.
THE rankings for Stanford, UC Berkeley, ETH Zurich, and MIT.
3. Application deadlines and fees for ETH Zurich, Johns Hopkins, MIT, and Oxford.
Confirmation of application fees for some universities (e.g.. Stanford, UC Berkeley).
Previous attempts to search for QS and THE rankings yielded limited results, possibly due to the specificity of keywords. For example, searching for "Stanford University QS 2025 overall ranking" might need more precise terms or official sources. Similarly, THE rankings may require direct access to official THE reports or reliable education platforms.
Regarding application deadlines and fees, some universities might only list graduate in- formation, requiring undergraduate-
-speccfc specific searches. For example, ETH Zurich's 's deadline might be found using keywords like "ETH Zurich undergraduate application deadline 2025." further searching necessary? Yes. Specific missing points must be addressed: Confirm QS overall rankings for Stanford and UC Berkeley.
Confirm THE rankings for key universities.
Find undergraduate deadlines and fees for ETH Zurich, Johns Hopkins, MIT, and Oxford. However, multiple previous searches for QS and THE rankings have failed to yield results, suggesting possible data unavailability or keyword issues. For example, THE's 2025 rank- ings might not be fully released, or official lists are not publicly accessible. In such cases, may be necessary to indicate "Not Available" in the table, but the user requires accurate data, so further attempts are warranted.
Alternative search strategies could include using different keywords, such as "QS World University Rankings 2025 top 20" or checking university official websites for their rankings. For example, MIT's official website might mention their THE ranking. For application fees and deadlines, searching university admissions pages directly might help. For example, MIT's undergraduate admissions page should list the application fee and deadline.
In summary, further searches are needed for the missing rankings, deadlines, and fees. If these searches continue to yield no results, the final table may need mark some fields as "Not Available," but every effort should be made to find accurate data first.
Tool Call
"tool_name?" "search_global*
"I "arguments"; (\"query\". ": **tanford University QS 2025 ranking 4 officiall", \"count \ 1, \"sumary__yyee \"short\")", "tool_calll id"; "call_ "call_Avg6cppe htt.://wwwpwwweeeeeeaaaa2222
"tool "tool_name" name": Syearch_globate
" "arguments": (\"query\": *CC Berkeley THE 2025 ranking)", 4 count("; 1, \"summary_type>> \"sher\"""" "tool_aall "aaloooleexxinnnwwepppp222.2
"tool_name" "scarch_global",
56

---

Published as a conference paper at ICLR 2026
"arguments": [Vquery\"" \"ETH Zurich undergraduate application 4 deadling ine 2025\", \"count\": 1, \"surmar__type"" \*short 4 )"), "tool_call_ id"; waall_1hhh7anhhhh55yrrr55
"tooo_name"" "search_global"
"arguments": "\ (\"query\": :: \MMIT undergraduate applicatien fee
+ 2025\", \"count"": 1, ""uumaryy_tyye type\": \"short\")",
"tool_call id"; "call pew..p/eekeooo//orrrk911111.
"tool "tool_name" name": "search gloccalglobaa,,.
"a "arguments": (qquery\".. \"University of Oxford undergraduate
+ application fee 2025\", \"count\": "oummary_type\? \"
4 short\")",
"tool stool_calllos "call "aal___fheeeeyxeee11100oo000x2
Tool Response
[index] [siteName] ] go.com
(search content omitted due to length limit)
Step 18
Reasoning Content
I now need to continue analyzing the user's question and determine the next steps. The user requires the top five universities each of the five broad subject areas from the QS World University Rankings by Subject 2025, along with their overall rankings in the QS and THE rankings, official website, regular decision deadline, and application fee. The previous search process has collected some information, but there still missing details that need to be supplemented.
First, reviewing the search history, the five broad subject areas have been confirmed Arts and Humanities, Engineering and Technology, Life Sciences and Medicine, Natural Sci- ences, and Social Sciences and Management. The top five universities for each subject have been obtained through the QS official website, such as Harvard, Oxford, and Cambridge for Arts and Humanities. However, some universities' overall QS and THE rankings remain unclear, such as Stanford and UC Berkeley's 's QS 2025 rankings, and ETH Zurich's THE ranking.
Next, regarding official websites, some universities' URLs have been confirmed, like Har- vard's harvard.edu, Oxford's ox.ac.uk, etc., but Johns Hopkins University's 's official website might still need verification. Application deadlines and fees missing for most universi- ties, such as ETH Zurich's 's deadline and UC Berkeley's application fee.
Additionally, some universities' THE rankings are still unconfirmed, such as MIT and UC Berkeley. The regular decision deadlines and application fees for most universities are also incomplete, requiring further searches.
Given the repeated failure to find Stanford and UC Berkeley's QS rankings in previous searches, it may be necessary to adjust search keywords, such as using "Stanford Univer- sity QS 2025 overall ranking" checking the official QS rankings page directly. Similarly, for THE rankings, trying more precise keywords like "ETH Zurich Times Higher Education 2025 ranking" might yield better results.
For application deadlines and fees, direct searches for each university 's undergraduate ad- missions page, such as "ETH Zurich undergraduate application deadline 2025" "UC Berkeley application fee 2025." could provide accurate information. Some previously found deadlines, like Harvard's January 1st and Stanford's January 6th resumption date, need con- firmation for correctness and completeness.
57

---

Published as a conference paper at ICLR 2026
In summary, the next steps should include:
1. Continuing to search for QS overall rankings for Stanford, UC Berkeley, ETH Zurich, and Johns Hopkins.
2. Searching for THE rankings for MIT, UC Berkeley, ETH Zurich, etc.
Confirming official websites for Johns Hopkins and ETH Zurich.
4. Collecting application deadlines and fees for all 25 universities (5 subjects x 5 universities each).
Since the user requires organizing results into table, each piece of information must be accurate. Given the current gaps, further searches are necessary to fill in missing data, par- ticularly rankings, deadlines, and fees. If certain data remains unobtainable after multiple attempts, may be noted as "Not Available," but efforts should be made to find official sources first.
Tool Call
"tool_name"" "search_globall
" "arguments"; (V"quer\"*. **os World University Rankings 2025 4 official full list\" \"count> 1, \"summary_type1 \" 4 short\")",
"tool lapol_call__dii "call_ oog
"tool_name": "search_global.
"arguments": (\"query\": \"Times Higher Education World 4 University Rankings 2025 official list\", \"count \": 1, \"
4 sunmary_type \"shorti")", "tool call id"; "call wttp:////g///////u////aaa//////
"tool name": "search "gobbal
"arguments": (Vqquery>> \: "*Johns Hopkins University official
+ website\", \"count \ 1, typee"" \"short\")",
"tool "tool_call_id", "call_jzaegi 9emebylvaiygh48e155
"tool name": "search global",
" "arguments"; (\"query\": \EETH Zurich undergraduate application
4 fee 2025\", \"counti": 1, \"summar__tyee"" \"short\""" "tool_call_d"" "aal_
"t "tool_name": "search_global"
"arguments"; ( (\"query\", : \CCc Berkeley undergraduate
4 application fee 2025\", \ \"count\": 1, \"summary_type"" \" 4 short>"!",
"tool_call_
Tool Response
[index] [siteName] gs.con
(search content omitted due to length limit)
Step 19
Reasoning Content
58

---

Published as a conference paper at ICLR 2026
I now need continue analyzing the user's query and the existing search results deter- mine whether further searches are required to complete the task. The user needs the top five universities each of the five broad subjects from the QS World University Rankings by Subject 2025, along with their QS overall rankings, THE rankings, official websites, regular decision deadlines, and application fees. These details should be organized into a Markdown table.
First, reviewing previous search results:
1. **QS Subject Rankings**; Through multiple searches and using the exx_browser_viee tool, the top five universities for each the five broad subjects have been confirmed:
Arts and Humanities: Harvard, Oxford, Cambridge, Stanford, UC Berkeley Engineering and Technology: MIT, Oxford, Stanford, Cambridge, ETH Zurich
Life Sciences and Medicine: Harvard, Oxford, Johns Hopkins, Stanford, MIT
Natural Sciences: Harvard, MIT, Oxford, Cambridge, Stanford
Social Sciences and Management: Harvard, Oxford, Stanford, MIT, Cambridge
2. **QS Overall Rankings**; The official QS website mentions MIT at 1st, Imperial Col- lege London 2nd, Oxford 3rd, Harvard 4th, and Cambridge 5th. However, rankings for other universities (e.g., Stanford, UC Berkeley, ETH Zurich, Johns Hopkins) remain un- clear. Multiple searches for individual universities' rankings yielded no results, possibly due to incomplete or unpublicized full rankings.
**THE Rankings**; Results are even more limited. Only Harvard's THE ranking was mentioned but not confirmed, while others like MIT, Stanford, and Oxford lack clear data. The THE website's search results provided partial information but no complete top 100 list. 4. **Official Websites**; Most universities' official websites have been found, including Harvard, Oxford, Cambridge, Stanford, MIT, UC Berkeley, ETH Zurich, and Johns Hop- kins (via graduate admissions links).
5. Deaddinnes and Application Fees**:
Deadlines: Harvard (January 1). Stanford (unspecified, but likely around January). UC Berkeley (December 2, 2024 for Fall 2025), Oxford (October 15), MIT (unspecified), ETH Zurich (varies by program, possibly January-April).
Fees: Harvard ($85 for law school, undergraduate fee unclear), MIT (S75), UC Berkeley ($100 for bootcamp, possibly different for undergrad), ETH Zurich (CHF 150 for architec- ture program, general undergrad fee unclear). **Missing Information*** ***
QS overall rankings for Stanford, UC Berkeley, ETH Zurich, Johns Hopkins.
-TE rankings for nearly all universities.
Undergraduate application deadlines and fees for some universities (e.g., ETH Zurich,
Johns Hopkins, Cambridge).
*#Chalengees*#:
QS and THE official websites do not publicly disclose complete rankings, making it diffi- cult obtain accurate data for specific universities.
Multiple searches for individual university rankings have failed, suggesting the information may not be publicly available requires accessing paid reports.
Application deadlines and fees vary by program and university, with some details only available on official admissions pages, which may require direct access each university's website.
**Next Steps
1. **QS and THE Rankings**; Since repeated searches for full rankings individual uni- versity rankings have failed, it may be necessary to abandon obtaining these data and note "Not Available" in the table.
2. Deadlinees and Fees**; For universities with unclear deadlines/fees (e.g.. ETH Zurich, more pre-
Johns Hopkins), attempt search their official admissions pages directly cise keywords (c.g.. "Johns Hopkins University undergraduate application deadline 2025"). **bebsite Verification Confirm that all university websites are official and correct, particularly for Johns Hopkins.
*Conclusion*** **: Given the inability to obtain QS and THE rankings for some universities despite multiple searches, and the potential unavailability of public data, it is reasonable to conclude that further searches will not yield additional information. The remaining dead-
59

---

Published as a conference paper at ICLR 2026
lines and fees can be supplemented through targeted searches, but if those also fail, they should be marked as "Not Available." Final Answer
marrkdown
Subject
University
Qs World University
Rankings by Subject 2025 QS World University Rankings 2025
Times Higher Education World Uni University Rankings 2025 Application
Bome Page
Deadline
Application Fee
$ c $ $ 4
Arts and Humanities
Harvard University
Not Available
httpp::/www.ttt
harvard $85
.edu
January
Arts and Humanities
University of Oxford 2
Not Available
betps://www ox. ac.
uk
October 15
Not
C 4 Available
Arts and Humanities
University of Cambridge
1 5
Not Available
setpp::/www... can. ac. Not
uk
October 15
Available
Arts and Humanities
Stanford university
4
+ I CCCCO
Not Available
Not Available
attps://wwv.
stanford.com cdu
January 5
| Not Available Arts and Humanities UCB)
University of California, Berkeley
Not
Available
I Net Available https://www.
berkeley.edi
December 2, 2024
$100
Engineering and Technologyl Massachusetts Institute of Technology
(MIT) |
4
Not Available
https::/webb mit edu
January 5
$75
Engineering and Technologyl University of Oxford
3
Available
htttps://ww..n.
60

---

Published as a conference paper at ICLR 2026
uk
October
Not
Available Engineering and Technologyl Stanferd University
c $ $ 100cce
Available
Available
https://wwt
stanford.edi
January 5
Not Available
Eng engineering and Technologyl University Cambridge
4
5
Not Available
https:///ww/ cam.ac......oooo ac.
October 15
Not
Available
Life Sciences and Medicinel Harvard University
$ I C c $
Net Available
attps://www.hht harvard
.edu
January
$85
Life seiences and Medicinel University Oxford
2
Nct Available
https://www wxw.xxxx.. ac.
October
Not
Available
Life Sciences and Medicinel Johns Hopkins University
3
Available
16
https://
www. jww.juuedu
January 3
Not Available
Life Sciences and Medicinel Stanford University
4
Not Available
Not Available
https://ww..
I
stanford.edi
January 5
| Not Available
Life Sciences and Medicinel Massachusetts Institute of Technology (MIT)
1
Available hhtpps://eeb mit.edu
4 1400
January 5
| $75
Natural Seiences
Harvard University
100023 $
4
Not Available https:: harvard
.edu
January
$85
Natural Sciences (MIT)
Massachusetts Institute of Technology
Available https:: /ttp://wbbb........ mit edu
January 5
| $75
61

---

Published as a conference paper at ICLR 2026
Natural Sciences
University Oxford
c $
3
3
Nct Available
https:///wwt ox. ac.
October
Not
Available
Natural Sciences
University of Cambridge
4
1 5
Net Available
htttp:://ww.co.
uk
October 15
| Not
Available
Natural Sciences
Stanford University
5
Not Available
Not Available
https://ww.
stanford.edi
January 5
4
| Not Available
Social Sciences and Management
| Harvard University
c $444500 $ $
4
Not Available
https://ww.harvard
harvard
.edu
January
$85
Social Sciences and Management | University of Oxford
2
$
Net Available
htttps///www ox. ac.
uk
October 15
Not
4 Available
Social Sciences and Management | Stanford University
3
Not Available
Not Available
https://www.
stanford.
January
| Not Availabe
Social Sciences and Management Massachusetts Institute of Technology (MIT)
1
Not Available
httpps:/ https::/wwbb... mit edu
January 5
| $75
Social sciences and Management University of Cambridge
5
5
Net Available
https://www. cam.ac.
uk
October 15
Not
Available
*Notes**:
Rankings marked "Not Available" indicate insufficient public 4 data from official QS/THE 2025 rankings.
62

---

Published as a conference paper at ICLR 2026
2. Application deadlines and fees are primarily sourced frem
4 university admi ssiission ons pages or verified third-party platforms
+ some may vary by program.
3. For the most accurate and up-to-date information, refer to the official university admissions websites.
Case Trajectory for Multi-Agent
My is about start his university applications but he's still uncertain about both his major and which universities to apply to. Could you help me find the top five universities in each of the five broad subjects from the QS World University Rankings by Subject 2025, and also check their standings in the QS World University Rankings 2025 and the Times Higher Education World University Rankings 2025? And need the home page of the university's 's official website, standard application deadline for regular decison well as the application fee without the fee wavier. Please organize the results in one Markdown table with the following columns:
Question:
Subject, University, QS World University Rankings by Subject 2025, QS World University Rankings 2025, Times Higher Education World University Rankings 2025, Home Page, Application Deadline, Application Fee
Please use the universities' full official names in English. Use only Arabic numerals in the ranking, for example: 1.
The output format is "* markdoww data content)
Step 1
Reasoning Content
Let me analyze this problem step by step. The user needs to help their child prepare for university applications by gathering information the top five universities across five broad subjects from the QS World University Rankings by Subject 2025. Additionally, they require these universities' overall rankings in the QS World University Rankings 2025 and Times Higher Education (THE) World University Rankings 2025, along with their official website homepages, regular decision application deadlines, and application fees (excluding fee waivers). The results should be organized into specific Markdown table. First, need to clarify the user's requirements:
1. **Vive broad subjects**: The user mentions "five broad subjects" but does not specify which ones. Common broad disciplines include Engineering & Technology, Natural Sci- ences, Social Sciences & Management, Arts & Humanities, and Life Sciences & Medicine. However, must confirm whether the QS Rankings by Subject 2025 uses these categories others. This step requires research to identify the exact five broad subject areas covered in the 2025 rankings.
2. **oop five universities per subject** For each of the five subjects, select the top five universities from the QS Subject Rankings 2025.
3. **Overall rankings**; For each selected university, obtain their positions in the QS World University Rankings 2025 and THE World University Rankings 2025. **Uiverrstty *+ Official website homepage URL, regular decision application deadline, and application fee (without waivers). Next, consider the challenges:
**Tmmelness The current time is April 2025, the 2025 rankings should already be released. However, need to verify whether the QS and THE 2025 rankings are publicly available. If not, the may need the latest available data, but the question explicitly specifics 2025, so must use that year's rankings.
** accuracy** University rankings, deadlines, and fees change annually. For example, application deadlines may vary by program or country (e.g.. US universities typically have deadlines in November-January, while UK universities may have different timelines). Fees also vary by country and program.
#Standardizztion Ensure university names in full official English, rankings use
63

---

Published as a conference paper at ICLR 2026
Arabic numerals, and deadlines/fees presented uniformly (e.g., deadline format: Month Day, Year; fees in USD or local currency? The user does not specify currency, so perhaps use the university's 's stated currency or convert USD? Need clarify, but the user may prefer the original currency as listed on the university's 's website). **Sep-by-SSer Plan**:
1. **Identify the five broad subjects in QS Subject Rankings 2025*#;
Search for the official QS World University Rankings by Subject 2025 to confirm classification of broad subject areas. For example, QS typically divides subjects into five broad categories: Arts & Humanities, Engincering & Technology, Life Sciences & Medicine, Natural Sciences, and Social Sciences & Management. However, this may change, so verification is essential.
2. ** R*etrieee top five universities for each subject**:
For each confirmed broad subject, extract the top five universities from the QS Subject Rankings 2025. Note that some universities may appear in multiple subjects (e.g.. MIT in Engineering and Natural Sciences), but they will be listed separately under each relevant subject.
3. **Collect overall rankings (QS and THE 2025)**:
For each university, search both the QS World University Rankings 2025 and THE World University Rankings 2025 to obtain their overall positions. Note that some universities may not be ranked in one of the lists, but this is unlikely for top institutions.
4. **Gather university-specift details*#;
**Official homepage**, Directly from the university's 's website; ensure URLs are correct and start with http/https.
** Regullrr decision " For undergraduate programs (assuming undergraduate applications, the mentions "university applications" generally). Deadlines vary by country and program; need to confirm the standard undergraduate regular decision deadline (e.g.. January for US universities. January 15 for others, etc.).
** "Appiccation fce**; Excluding fee waivers; note the currency (c.g., USD, GBP) as listed by the university.
Challenges and Solutions
** Unspecified subjects**; Since the user did not specify the five subjects, must first determine the exact broad subject categories from the QS 2025 rankings. This requires searching the QS official website or reliable sources for the 2025 subject classification.
**Ovellpping universities The same university may top multiple subjects (e.g., Harvard in Social Sciences). Each occurrence will need separate entries in the table under their respective subjects.
** Data availability*** If the 2025 rankings are not yet fully published (e.g., some released, others pending), need use the latest available data clarify with the user. However, the user explicitly requests 2025, so assume all rankings are published.
**Deadlines and fees**; These can vary by program (e-g., undergraduate vs. graduate). Since the user does not specify, focus on undergraduate regular decision deadlines and fees for the main undergraduate application portal (e.g., Common App. UCAS).
**oool Utilization
**Search *o Use 'search_global" to:
Confirm QS 2025 broad subject categories.
Retrieve top five universities per subject from QS Subject Rankings 2025.
Obtain QS and THE 2025 overall rankings for each university.
Find each university's application deadlines and fees.
L*Link reading tool*# After finding university websites via search, use 'text. browser.rwrrrrr..rr view" to extract deadlines and fees from official admissions pages (e.g.. check "Undergraduate Admissions" or "How Apply" sections).
**Sub-agents*#; Given the volume of tasks (five subjects five universities 25 universities, each requiring multiple data points), creating sub-agents parallelize work could improve efficiency. For example: Sub-agent Confirm QS 2025 broad subjects.
Sub-agent 2: For each subject, retrieve top five universities and their QS Subject rankings. Sub-agent Collect QS and THE overall rankings for each university.
64

---

to Published as a conference paper at ICLR 2026
Sub-agent 4: Gather website URLs, deadlines, and fees for each university.
Verificattio Steps*#:
1. Ensure all rankings for 2025.
2. Cross-check university names with official sources to avoid aliases (e.g., "MIT" vs.
"Massachusetts Institute of Technology").
3. Confirm deadlines are for regular decision (not early action/decision) and undergraduate programs.
4. Verify application fees exclude waivers and note the correct currency. **Potential Edge C
A university may not have undergraduate program (e.g., graduate-only institutions), but top-ranked universities typically offer undergrad programs.
Deadlines may differ by country (e.g., UK universities via UCAS have single deadline, while US universities vary).
Application fees may be waived for certain regions, but the user explicitly requests fees without waivers.
#*xecution Plan**
1. **Step I: Identify QS 2025 broad subject categories**
Use 'search global' with query: "QS World University Rankings by Subject 2025 broad subject categories" to confirm the five categories. Assume results return Arts & Humanities, Engineering Technology, Life Sciences & Medicine, Natural Sciences, Social Sciences & Management.
**Step 2: Retrieve top five universities per subject** **
For each subject, search "QS Subject Rankings 2025 [Subject Name] top 5 universities". For example, "QS Subject Rankings 2025 Engineering & Technology top universities". Record university names and their subject rankings.
3. **Step 3: Collect overall rankings**
For each university, search "QS World University Rankings 2025 [University Name]" and "THE World University Rankings 2025 [University Name]" to get their overall positions.
**Step 4: Gather university d
For each university:
Search official homepage URL (e.g., "Harvard University official website").
Use "text_browse view* the admissions page to find deadlines and fees. For example, visit
confirm the fee.
**Exmpple Table Structure**;
Subject University QS Subject 2025 QS Overall 2025 THE Overall 2025
Home Page Deadline Application Fee
Engineering Tech Massachusetts Institute...
https//www.mit.edu
Jan 5, 2025 $75
**Now, determine the initial tool calls**;
First, confirm the five broad subjects via 'search global'.
Thee, for each subject, retrieve top five universities.
For each university, collect overall rankings, website, deadline, fee.
Given the complexity, creating sub-agents to handle each subject's data collection parallel
would be efficient. For example:
Sub-agent Handle Arts Humanities
Sub-agent 1: Handle Engineering & Technology
Sub-agent 2: Handle Life Sciences & Medicine
Sub-agent Handle Natural Sciences
Sub-agent 4: Handle Social Sciences & Management Each sub-agent would be tasked with:
1. Retrieving top five universities in their subject from QS 2025. For each university, get QS overall and THE 2025 rankings.
3. Collect website, deadline, and fee.
Thus, the first step is to create these five sub-agents using 'create_sub_agent'', each with
65

---

Published as a conference paper at ICLR 2026
prompt tailored their subject. Then, each sub-agent independently gather the required data, which can later be aggregated into the final table.
**Tool Call**;
Use "create.sue agents" to generate five sub-agents, each assigned to one of the five broad subjects (after confirming the subjects via initial search). However, since the subjects are not yet confirmed, the initial step must first search for the QS 2025 broad subject categories. **Revised *lan**:
1. **Initial search*#; Confirm the five broad subjects in QS Subject Rankings 2025.
Tool: search.global with query "QS World University Rankings by Subject 2025 broad
subject categories".
Purpose: Obtain the exact five subject areas to ensure accuracy.
2. * Create sub-agents per subject** Once subjects confirmed, create sub-agent for each to collect data.
Thus, the first tool call is 'search.global" identify the five subjects. Once subjects known, create sub-agents for each.
**Final Tool Call**:
First, execute a search to confirm the five broad subjects. Then proceed with sub-agents. Therefore, the initial action is:
< C--FnctioCCalllBginnn
(""name"""search_ghll
World University Rankings by Subject 2025 broad egories"."count"" 10ountt":00""""" 110,"summarr" sareyy-ty""""""""" "spo"t""hort"""" "hhort",usee "useet"rueeeengliss"" english":false]]
subject cat-
<-FfunciioCallEEnd
This search will return the necessary subject categories, enabling the creation of targeted sub-agents in subsequent steps.
Tool Call
"tool_name": "search_global",
"arguments": "(\"query\": " World University Rankings by
4 Subject 2025 broad subject categories)", " \"count>::1,, \" rummary_typell """"""""""""""""" "uss__nnglish f:llsse "tool_call_ id"; feall__fooyy22emmiddaeezz
Tool Response
[index] [siteName] qs com
(search content omitted due to length limit)
Step 2
Reasoning Content
Now need to continue analyzing the user's question. The user wants the top five univer- sities in each of the five broad subjects from the QS World University Rankings by Subject 2025, along with their positions in the QS and THE rankings, official website homepage, ap- plication deadlines for regular decision, and application fees. The results must be organized into a Markdown table with specific columns.
First, the previous search results have confirmed the five broad subject categories in the QS 2025 Subject Rankings: Arts and Humanities, Engineering and Technology. Life Sciences and Medicine, Natural Sciences, and Social Sciences and Management. This addresses the subject classification issue.
Next, need to obtain the top five universities for each subject. The current search results mention that the QS rankings cover these five broad subjects but do not provide specific university names or rankings. For example, the first result from qs.com states the five broad subjects but lacks detailed rankings. Other results either mention specific disciplines (e.g., Accounting & Finance, Pharmacy Pharmacology) or discuss ranking expansions without providing the required top five list for each broad subject.
Therefore, the next step is to search for the top five universities in each of these five broad subjects. Since each subject requires separate searching. I should split this into five sub-
66

---

Published as a conference paper at ICLR 2026
tasks. For instance, search for "QS World University Rankings by Subject 2025 Arts and Humanities top 5" and repeat similarly for the other four subjects. However, given the lim- itations of search tools, might need to use multiple search queries or consider creating sub-agents to handle parallel searches.
Additionally, the user requires the universities' positions in the overall QS World University Rankings 2025 and THE World University Rankings 2025. The current search results do not include these overall rankings, so separate searches will needed for both QS and THE's 2025 overall rankings. This adds two more search tasks.
Furthermore, for each university, the official website homepage, regular decision application deadline, and application fee (without fee waivers) must be collected. These details are spe- cific to each university and may require visiting their official websites or trusted educational platforms. However, given the large number of universities (5 subjects x 5 universities 25 total), this could be time-consuming. Using sub-agents to handle different subjects or universities in parallel might improve efficiency.
Now, considering the tool options: the create greate_sub_agee wwt.esuuaaeaeetteeeeeeea bab_aneets function allows creating sub-agents to perform specific tasks. Since the user needs information across multiple subjects and rankings, creating sub-agents for each broad subject to collect top five universities, overall rankings, and application details would be appropriate. Alternatively, split the tasks into:
1. Retrieve top five universities for each of the five QS broad subjects.
2. Retrieve QS 2025 overall rankings.
3. Retrieve THE 2025 overall rankings.
4. For each university, collect official website, deadline, and application fee.
But given the complexity, creating sub-agents for each broad subject might be more efficient. For example, create five sub-agents, each responsible for one subject, tasked with finding the top five universities, their QS subject ranking, overall QS and THE rankings, official web- site, deadline, and fee. However, sub-agents can use search and link-reading tools, so each sub-agent could independently complete its assigned subject's tasks.
Alternatively, first obtain the top five universities for each subject, then collect their overall rankings and application details. The first step to confirm the top five universities per sub- ject. Since the current search results lack this information, further searches are necessary. Possible search strategies: For each subject, use precise keywords like "QS World Univer- sity Rankings by Subject 2025 [Subject Name] top 5 universities". For example:
QS World University Rankings by Subject 2025 Arts and Humanities top 5
QS World University Rankings by Subject 2025 Engineering and Technology top 5 and so on for the remaining three subjects.
Since each subject requires separate search, using parallel tool calls search for all five subjects simultaneously would save time. However, the current search_ global tool allows only one query per call, so multiple search-globe sub-agents for each subject to perform searches in parallel.
calls are needed. Alternatively, create
Given that the user needs results for five subjects, creating five sub-agents (each assigned to one subject) with prompts instructing them to find the top five universities, their QS subject ranking, overall QS and THE rankings, official website, deadline, and fee would be efficient. Each sub-agent can independently search and compile data for its subject, then aggregate the results into a table.
However, the create_sub waa.uuuuubburro agents tool requires specifying prompts and indexes for each sub- agent. For example:
Sub-agent O: Prompt- "Find the top 5 universities in QS World University Rankings by Sub- ject 2025 for Arts and Humanities, including their QS subject ranking, QS overall ranking 2025, THE overall ranking 2025, official website homepage, regular decision application deadline, and application fee without waiver."
Similarly for sub-agents to 4, corresponding the other four subjects. But creating five sub-agents might be manageable. Each sub-agent would handle one sub- ject, search for the required information, and return the results. Then, the main agent aggre- gates all data into the table.
However, there's a potential issue: Sub-agents might encounter difficulties retrieving all required information (e.g., application deadlines and fees may not be easily available in search results and might require accessing university websites). Thus, sub-agents may need
67

---

Published as a conference paper at ICLR 2026
use the cext_browser_vke tool to extract details from official sites.
Another consideration: The QS and THE overall rankings for 2025 might not yet be pub- licly available. The current search results include THE article from January 2025 and QS-related articles from March 2025, suggesting the rankings are released. However, spe- cific university positions need verification.
Additionally, application deadlines and fees vary by program and country. The user spec- ifies "standard application deadline for regular decision" and "application fee without fee waiver," which may require checking each university's ss undergraduate admissions page. This could be timecconsuming, but sub-agents can handle this by searching for each university's admissions information.
In summary, the next steps should be:
1. Create five sub-agents, each responsible for one of the five broad subjects.
2. Each sub-agent searches for the top five universities in its assigned subject, retrieves their QS subject ranking, QS and THE overall rankings, official website, deadline, and fee.
3. Aggregate all sub-agent results into the required table.
Thus, the next tool call should use create preate_sub.ac www.ssuuuuau.... aagents to generate these five sub- sub-agents. Each sub-agent's prompt must clearly specify the task: retrieve the top five universities for the sub- ject, along with the required six data points (QS subject ranking, QS overall, THE overall, homepage, deadline, fee), ensuring the university's full official English name and numeric rankings.
Now, constructing the create sub agents parameters. The sub agents array should contain five objects, each with prompt and index (0 to 4). The prompts must be detailed enough for sub-agents understand the requirements.
For example, the prompt for index (Arts and Humanities) could be: "Task: Retrieve the top 5 universities in the 'Arts and Humanities' category of the QS World University Rankings by Subject 2025. For each university, collect: 1. Full official English name; 2. QS World University Rankings by Subject 2025 rank (numeric): 3. QS World University Rankings 2025 overall rank (numeric); 4. Times Higher Education World University Rankings 2025 overall rank (numeric): 5. Official website homepage URL: 6. Standard regular decision application deadline (format: Month Day, Year, e.g., January 15, 2025); 7. Application fee (without fee waiver, specify currency, e.g., USD 80). Ensure rankings are Arabic numer- als. If any data is unavailable, mark as 'N/A'. Return results in a list of dictionaries with keys matching the required fields."
Similarly, adjust the subject name for the other four sub-agents (Engincering and Technol- ogy, Life Sciences and Medicine, Natural Sciences, Social Sciences and Management). This approach ensures each sub-agent knows its task and data requirements. Using sub- agents allows parallel processing, improving efficiency.
Therefore, the next step is to call create sub aab_ag with these five sub-agents. Tool Call
"tool "tool_name": "create _uu sub_agents",
"arguments": "s\"sub "s""subaagents"" aagentss" (nnn "ooo\/prrnpt/"" "Retrieve the top 5 4 universities in the 'Arts and Humanities' category of the
+ QS World University Rankings by Subject 2025. For each
university, collect: 1. Full official English name; 2. Q8
World University Rankings by Subject 2025 rank (numeric);
4 3. QS World University Rankings 2025 overall rank (numeric)
4 4. Times Higher Education World University bankings 2025
S overall rank (numeric); 5. Official website homepage URL;
6. Stancard regular decision application deadline (fornat:
Month Day, Year, e.g., January 15, 2025); 7. Application
fee (without fee waiver, specify currency, e.g., USD 80)
+ Ensure all rankings are Arabic numerals. If any data is 'N/A', Return results in 1ist of dictionaries with keys: university_name, qs_subject_ran qs ge_overall_rank, the_overall paaeek, homepage, deadline,
unavailable, mark
+ application_fedt "",,Aeenexi" \
4 the top 5 universities in the 'Engineering and Technology'
68

---

Published as a conference paper at ICLR 2026
category of the QS Worid University Rankings by Subject 2025. For each university, collect: 1, Full official 4 English name; 2. QS world University Rankings by Subject
2025 rank (numeric); 3. QS world University Rankings 2025
overall rank (numeric); 4. Times Higher Education World
University Rankings 2025 overall rank (numerid); 5.
Official website homepage URL; 6. Standard regular decision application deadline (format: Menth Day, Year, e.g.,
January 15, 2025); 7. Applicaticn fee (without fee waiver, specify currency, e.g., USD 80). Ensure all rankings are Arabic numcrals. If any data is unavailable, mark as 'N/A" Return results in a list of dicticnaries with keys:
4 uni,,, rs_subject_rank, gs_overall_rank,
the_overall_rank, homepage, deadline, application_fed \", the top 5 universities + in the 'Life Sciences and Medicine' category of the QS
index)":l), "ipppp/""""""pppp"""""""""""""""
+ World University Rankings by Subject 2025. For each
4 university, collect: 1. Full official English name; 2. Q8 World University Rankings by subject 2025 rank (numeric);
+ 3. Qs World University Rankings 2025 overall rank (numeric)
4 4. Times Higher Education World University Rankings 2025
4 overall rank (numeric); 5. Official website homepage URL;
6. Standard regular decision application deadline (format:
Month Day, Year, e.g., January 15, 2025); 7. Application
fee (without fee waiver, specify currency, e.g., USD 80)
Ensure all rankings are Arabic numerals. If data is unavailable, mark 'N/A". Return results in list of distionaries with keys: university_name, gs_subject_rank, q8_overail_rank, che_overall_rants homepage, deadl ine, 4 applicaticn_feet seattiion_eeeevve \"i'denxxxx) /ptp:/ptotptt::
+ the top 5 universities in the 'Natural Sciences' category of the QS World University Rankings by Subject 2025. For each university, collect: 1. Full efficial English name; 2. QS World University Rankings by Subject 2025 rank (numeric
+ ); 3. Qs World University Rankings 2025 overall rank
S numeric); 4. Times Higher Education World University
Rankings 2025 overall rank (numeric); 5. Official website
homepage URL; 6. Standard regular decision application
deadline (format: Month Day, Year, e.g., January 15, 2025);
7. Application fee (without fee waiver, specity currency,
e.g., USD 80). Ensure all rankings are Arabic numerals. any data is unavailable, mark as "N/A". Return results in list of dictionaries with keys: university_name, ms_subject_rank, q=_overall_rank, the_overall_rank
homepage, deadline, application_ \"indxx"""x \n/\"
prompti "R'"ttriieeeee the top 5 universities in the 'Social Sciences and Management' category of the os World University Rankings by Subject 2025. For each university, collect: 1. Full official English name; 2. Qs World
University Rankings by Subject 2025 rank (numeric); 3. Qs
World University Rankings 2025 everall rank (numeric);
Times Higher Education World University Rankings 2025
4 overall rank (numeric); 5. Official website homepage URL,
4 6. Standard regular decision application deadline (format:
Month Day, Year, e.g., January 15, 2025); V. Application
fee (without fee waiver, specity currency, e.g., USD 80).
Ensure all rankings are Arabic numerals. If any data is
+ unavailable, mark as 'N/A' Return results in a list of
+ dictionaries with keys: university_name, ss_subject_rank, ga_overall_rank, che_overall_rank, homepage, deadline, 4 application_fee. fee. (",ndxiexx)":44)aa
"tool_call_id", "eall_fgiikkbvridbburrroooi
69

---

Published as a conference paper at ICLR 2026
Tool Response
H " 0, "prompt": "Retrieve the top universities the
Arts and Humanities' category of the cs World University
Rankings by Subject 2025. For each university, collect: 1.
Full official English name; 2. world University Rankings by Subject 2025 rank (numeric); ; 3. QS world University Rankings 2025 overall rank (numeric); 4. Times Higher Education World University Rankings 2025 overall rank ( numeric); 5. Official website homepage URL; 6. Standard regular decision application deadline (format: Month Day, Year, e.g., January 15, 2025); Application fee (without fee waiver, specify currency, e.g., USD 80). Ensure all rankings are Arabic numerals. If any data is unavailable, mark 'N/A". Return results in list of dictionaries with keys: university_name, rs_subject_rank, gs_overall_rank, the_overall 1_rak,, homepage, deadline, application_fee." "universityy_aaee"
response": "I'n
I'n subject_rankk". rank)"rnk)" 1,
\"Harvard
University\",
"qs_subbeect
qs gs_overall rank)"; An "homepage \"https:///t /ttpp:://www...// http://wwwaw.d.....o eewrarrdeduu,,.
\"the_ overall rank)":
deadline\"; \"January 1, 20251", An \": "UUD 85 in (In
\"application ("university_name
("University of Oxford)", An
rank: 2, An
ove__ovaall__rnn 3, \"homepage\"; "https::/www.ox. /wtp///x......k/ in
\"the_overall_ rank
in
\"Gaddine\\ :"aine""""'""Cttto""" \"October 15, 2024\",\n
ioplication_fee".. \"GEP 28. 20 university
in AAn
\"
"UIniversity of Cambridge An rs_swbject_rankl 3, An
1"
\"qs_ ranki"; 5, Ia \"homepage ' \"
\ \"the_averall rankii" 5, In
https://www.crm http://wwwcamm.c.. w...kaamaaa..........
\"deadlinei"; '"October 15, ("application_fer feel": AGGBP 60) }n "university name\":
20241*, \n
Inn
\"Stanford University
'' se__subjectraakeee
4,
"ys_ooveral_rank \"homepage\"; \"dcadlinel":
6, An \'https:/ sww.sstdfford.u"".
Sche__verrallraae rank)": An
cooplicatioo_feelie *USD 90>"\n California, Berkeley (UCB))",
January 5, 2025\", L'university_name"""" \"University of "qs_subjectt_aan1
in (nn
\
5, An the_overall_rank!"" 8,
\"as_overall_rank 12, Barkeley.edu)",K.
\"homepage\". "httpss: /www. \"deadline)" \"November 30, ""pppiccation_tee" \"USD 80\"in ["index": 1, "proupt": "Retrieve the top 5 universities in the Engineering and Technology' category of the Q8 world University Rankings by Subject 2025. For each university, collect: 1. Full official English name; 2. QS World University Rankings by Subject 2025 rank (numeric); 3. os
2024\",
Anj"),
World University Rankings 2025 overall rank (numeric); 4.
Times Higher Education World University Rankings 2025 overall rank (numeric); 5. Official website homepage URL; 6.
Standard regular decision applicaticn deadline (format: Month Day, Year, c.g., January 15, 2025); 7. Application fee without fee waiver, specify currency, e.g., USD 80) Ensure all rankings are Arabic numerals. If any data is unavailable 1ist of dictionaries with keys: university name, qs_ us__suject__ee rank, qs overrll rank, the overall rank, homepage, deadline, application fee.", name\": \" Massachusetts Institute of Technology (MIT) \", rs_subject_rank"" 1, An
mark as 'N/A". Return results
response"; "I'n I'n
\"
\"us_overall 1, \"the_overall_ennk"" 2, An \"homepage\" \hhttp
70

---

Published as a conference paper at ICLR 2026
w//web.mit.eaiee An
\ddaddine\\; \"January 6,
2025\", ""uiiversity_namel" \University of oxford)", + ('n
f \"USD 7n b In
"es_subject_ranki" \n
""g__overall_rant 'hhomppage \"application_fes" feel=: \"G3?
\": 3,
\"che_overall_rank An
\": /'https://ww/ wttp://wwwxxxx.uk////o u
\"deadline\":
4 October 15, 2024\",
a 28.501"
in I'n
"university_nann \" "qe_subject_rank" An
stanford University\",
"re__ovcral_raan"" 6,
\"the_overail
6, An
\"homepage\";
An
\ "deadline\": \'January 5, 2025\",
\"
opplication_fee""" \"USD 90 J, {nn
university_name!"" ("University of Cambridge)", qs us__ubject__anki in
\"
\ ""q__overal" overall ranki": 5, Ia
\"the_ cene_overall rank(": 5, In
\"homepage""; \"
https: petps::/www.ca.... htt.//w..cacc.....mmm// "ooc.uk/",,V
"deadline \ "CCttber Vuniversity_namer
15, 2024\*,
("application_feel \GBB 7 b
fin ["ss_sbbecttraadee
\"ETH Zurichi", In o"r__overall_rank""
5, An
"thh_oveeral__rak11 11 O ("https:// // www. /ttp//ttbbb.ch//////
7,
'hhmepagee)" \ "March 31, ("application_fed feel": \"CHF 150\"in An }"), iincex": 2, "prompt"; "Retrieve the top universities in the 'Lffe Sciences and Medicine' category of the QS World University Bankings by Subject 2025. For each university, collect: 1. Full official Knglis ish name; 2. QS World
\"deadline\":
2025\",
University Rankings by Subject 2025 rank (numeric); 3. Qs world University Rankings 2025 overall rank (numeric); 4. Times Higher Education World University Rankings 2025 overall rank (numeric); 5. Official website homepage URL; 6.
Standard regular decision application deadline (format: Menth Application fee without fee waiver, specify currency, e.g., USD 80) Σnsure all rankings are Arabic numerals. If any data is unavallable, mark as 'N/A". Return results in list of dictionaries with keys: university_nane qs_subject_rank gs_overall_rank, the_overall_rant
Day, Year, e.g., January 15, 2025);
homepage, deadline, application_fes, fee.", "'nniversity_name""" \"Harvard
response": "['n
I'n
University\", \n
\'gs su_jsctt__aaak' in
\"
qs ge_overali_rank] in '"homppgge"" \"https:/ hhhttps:/www.harree
\"the_ \"the_overall rank)": 1,
www.harvard.edu.
\"
deadline)": \' 1, 20251", An \": \"USD 85\"\n An (In
\"applicat ionliication_fes
("University of Oxford ", An
\'qs "qs__sbbject__
subject subject_ranki"? ": 2, \n
\ ""ss overvveealllaa.. rank(": 3,
\"the_overail) rankal
1, \"daddline\"
\"homepage\"; \"https:/
http://wwwwoo.....kk..
\ \"GBP 28.50) A"NA::""""AVV"""""""" \"N/A\", In in {in
Vapplication _em__ffee"
"universiity_amee"" \"
Johns Hopkins University\",
"u__eebeet_raank 3,
n
"ve__overall__ani"" 32, In \"homepage)": /"https: /tttps::/www...
"thh__verral__end
\": 16,
/ww. www.juu.edu.. An
(ddeadline \'January 2, 2025\" ", An appli canlicattoneeeeee \"USD 70
1, {An
siaversityynameve \"stanford University An qs subjject_rankii in
{"
"qs_overall rank 6,
overall ranki": 6, An https://wwe. /tw..sttaffrr.eedu""
("homepage\"; \" \"deadline": \*January An
5, 2025\", i'n
("application_feel\ A \"USD 90
""aniversity_name"" \"Massachusetts Institute
of Technology (MIT) \", ",, An \"qs_subjeat_raat ranctt 5, \n
Scg__overall_rank 1, \"hneppge\\"
["the_overall_rane
2, An
71

---

Published as a conference paper at ICLR 2026
\aapplication_ \"; \"UsD 75\"An ) ("index": 3, "prompt": "Retrieve 4 the top 5 universities in the 'Natural Sciences' category of the Qs world University Rankings by Subject 2025. For each university, collect: 1. Full official English nane; 2. Qs World University Rankings by Subject 2025 rank (numeric); 3.
\ ("deadiin\\; \ '"January 6, 2025\", ,
os World University Rankings 2025 overall rank (numeric); 4. Times Higher Education World University Rankings 2025 overall rank (numeric); 5, Official website homepage URL; 6.
Standard regular decision application deadline (format: Month Day, Year, c.g., January 15, 2025); 7. Application fee without fee waiver, specify currency, e.g., USD 80) Ensure all rankings are Arabic numerals. If any data is unavailable, 1ist of dictionaries with keys: university name, qs si__sujjec__aaae qs rs__overall_re the__overall__
mark as 'N/A". Return results
homepage, deadline, application fee.", response"; "I'n In Mumiverrityy_eaeee
\"Harvard
University\",W. \"gs_subjectrrank": 1, \"
rs_overall_rank(" 4, An \"the_overall_rank::: 3, \"deadline\";
homepage\"; \"https: /ttp///w.a........ www.harrvarddduu......
\"January 1, 2025 in "apppiication_fee""" ""DDD B5
), An (In ("university_nam _arrsitt__mmmm: \ \"Massachusetts Institute
of Technology (MIT) re_overall_rank!" 1, homepage\":
\=os_subjectrrank"" 2,
""the_overall_rank""" 2,
\ \"deadline \"
January 6, 2025\", An "applicaticn_feel": \"USD 75
,, An {1n Sonnieessitynamevee : \"University of Oxford An
\"es_subject_rankt"" 3, \=ss_vverall_rank 3, \n
\"the_overall_rank
i, >"horepage>"; "https::/ www.
ox.ac.uk)", \"deadline\" \"October 15, 2024\", in
"eeliccaiionfee!!. \"GEP 28.50)"|a I, Inn
university.name)) (Unniversity of Cambridge \", In
gs_subject_rank(". An org__overal_raakt " 5, the_overall_rank"" in \"homppage\" \hhttps: / www.cam can.
as. ac.uk\", "ddealinei"; \"Octeber 15, 20241", An
application_ffee!" \"GBP 60 An (In \"
university_name!"" ("Stanford University\", qs_subject_rankk": An "ag_oveeall__ane _aankli 6,
\"
\"
the_overall_ranki in "homepage\"; \"https:/ htttpp:://ww.... www.
stanfordd.eul), \"deadline\": \"January 5, 2025\", An ""appiication_iee!""" \"USD "0 Anj"), ["index":
prompt": "Retrieve the top 5 universities in the rsocial Sciences and Management' category of the Qs world University Rankings by subject 2025. For each university, collect: 1. Full official English name; 2. Qs World University Rankings by Subject 2025 rank (numeric); 3. World University
Rankings 2025 overall rank (numeric); ; 4. Times Higher Education World University Rankings 2025 overall rank ( numeric); 5. Official website homepage URL; 6. Standard regular decisien application deadline (format: Month Day, Year, c.g., January 15, 2025);
Application fee (without fee waiver, specify currency, e.g., USD 80). Ensure all rankings are Arabic numerals. If any data is unavailable, mark 'N/A". Return results in list of diationaries with keys: university -amee,,sttyymme,,,,oteee qs qs__ubject_rra qs_overall rank, the sel_oveeall_raate homepage, deadline, application fee.", response": "Inn I'n \"university_nanee ("a \"Harvard University\",.nn \y 1, 1,
\"
gs_overall_rank1 An \"the_overall_rnnk:: 3n \"
homepage\";
\"deadline)";
("January 1, 2025
\"application_feel feel:; ""UD 85)"(n
7
An {In ""niiversity namel \"University of Oxford An tis__sujjectraakk" An "gg__overall_rakii 3, An
72

---

Published as a conference paper at ICLR 2026
oteheooeeral ox.ac.uk\",>n \n
rank 1, An \ "hcneppgee"" \"https:/ //wwv. /www. ("deadline\* \"otcbbr 15, 20251", In
F application_feet fee_ti"
\"GBP 75 Jn (An \" \"
university_namelV
\"Stanford University\",
qs_subject_rank
An
"=g__overall_rank 6,
\"
the che_overaii__ann""" 6, in >hhmmppge\\; \"https:/ /Nww.
stanford.edul", \"US 90 In I'n \"
\"deadline\" \"January 5, 2025\", An "epplicctiionfee\""
MIT) \", \n \ \"Massachusetts Institute of Technology 4, ""gs_overal_ 2,
university_name\"
\": in
\"homepage"" : \"
\'deadline\; A'January 6, 2025\", ("application_feel "UUSD 75\"An in (An
""university_name/" \"University of Cambridge)" qs
\"
rankk thhe_overall__ \n "qs_overall rank ": 5, \n \"
5, in \"homepage\": \"https:/ / /www. can.
ac. sc.uki", \n "ddeadliee";; \*Octeber 15, 2025\", An application_feel""" \GBBP 50\"\n Ann"]]
Step 3
Reasoning Content
Got it, let's tackle this user's request. They need a Markdown table with the top five universities in each of five broad subjects from QS Subject Rankings 2025, along with their overall QS and THE rankings, homepage, application deadline, and fee. First, I confirmed the five broad subjects from the initial search: Arts and Humanities, En- gineering and Technology. Life Sciences and Medicine. Natural Sciences, Social Sciences and Management. Then created sub-agents for each subject to gather the required data. Now, the sub-agents have returned their responses. need to check each one for complete- ness. Let's go through each subject's data:
1. ** Arts and Humanities Top 5 are Harvard, Oxford, Cambridge, Stanford, UCB. All fields seem filled except maybe deadlines and fees. Harvard's deadline Jan 1, 2025, fee USD 85. Oxford's deadline is Oct 15, 2024, fee GBP 28. 50. Cambridge Oct 15, 2024, GBP 60. Stanford Jan 5, 2025, USD 90. UCB Nov 30, 2024, USD 80. Rankings are numeric. Looks good.
2. **Engineering and Technology**;; MIT, Oxford, Stanford, Cambridge, ETH Zurich. MIT's deadline Jan 6, 2025, USD 75. Oxford same before. Stanford same. Cambridge deadline Oct 15, 2024, fee GBP 75 (wait, carlier in Arts and Humanities, Cambridge's 's fee check consistency. Maybe different programs? The user asked for
was GBP 60 standard regular decision, so perhaps it's correct if different subjects have different fees? Or maybe mistake. But since the sub-agent provided I'll note it but proceed unless conflicting.) ETH Zurich deadline March 31, 2025, CHF 150. Rankings all numeric.
**Life Sciences and Medicine**; Harvard, Oxford, Johns Hopkins, Stanford, MIT. Harvard same. Oxford's deadline is N/A N/A-sub-agent marked it as N/A, so we keep that. Johns Hopkins deadline Jan 2, 2025, USD 70. Stanford same. MIT same. Rankings okay. 4. **Natural Sciences* Harvard, MIT, Oxford, Cambridge, Stanford. All data filled, deadlines and fees consistent with previous entries.
5. **Social Sciences and Management* Harvard, Oxford, Stanford, MIT, Cambridge. Oxford's deadline here Oct 15, 2025 (typo? Should be 2024?), Cambridge also Oct 15, 2025. That might be an error. The user needs accurate deadlines. Since the other subjects for Oxford and Cambridge have 2024, 2025 might be a mistake. But the sub-agent provided that, so maybe it's correct for this subject? Or perhaps a typo. Since can't verify without further search, but the user wants the info as per the data, I'll include it but note that if there's conflict, but since the sub-agent is supposed to retrieve accurate info, I'll proceed with what's given.
Now, compiling all into a table. The columns needed are: Subject, University, QS Subject Rank 2025, QS Overall 2025, THE Overall 2025, Home Page, Application Deadline, Application Fee.
Each subject will have 5 rows. Need ensure university names full official English
73

---

Published as a conference paper at ICLR 2026
names. Check for duplicates in universities across subjects, but that's okay as they can top multiple subjects.
Check for any missing data marked as N/A (only Oxford in Life Sciences has N/A for deadline). All rankings are Arabic numerals. Currencies are specified (USD, GBP, CHF). URLs are complete.
Now, structure the Markdown table. Start with the header, then each row per university per subject. Make sure Subject column repeats the broad subject for each of the universities in that subject.
Let me draft the table:
Subject University QS World University Rankings by Subject 2025 - QS World University Rankings 2025 - Times Higher Education World University Rankings 2025
Home Page Application Deadline Application Fee
Arts and Humanities Harvard University 4 https//www.hhrraard.cm
January 1, 2025 USD 85
Arts and Humanities University Oxford - https//www..x..cc.uk
October 15, 2024 and so on for each subject's 5 universities.
GBP 28.50
Need to fill in all 25 rows (5 subjects universities). Check each entry against the sub-agent responses to ensure accuracy.
For example, in Social Sciences and Management, Oxford's deadline is Oct 15, 2025, which differs from other subjects. Maybe it's correct for that program, so include as is. Cambridge's 's application fee here is GBP 50, whereas in Arts and Humanities it GBP 60 a0aan,in, possibly different programs, so acceptable.
Once all rows are filled, wrap in "'markdown and ensure the table properly formatted with pipes and alignment. No markdown syntax errors.
I think that's all. The data from sub-agents seems complete, and the table can be constructed now.
Final Answer
"markdown
Subject University QS World University Rankings by Subject
2025 QS World University Rankings 2025 Times Higher
Education World University Rankings 2025 Home Page
Application Deadline Application Fee
Arts and Humanities Harvaré University 4 www.harvardeea January 1, 2025 | USD 85
|
https://
Arts and Humanities University of Oxford 2 3
1 https://
S www.ox. ac. October 15, 2024 CBP 28.50
Arts and Humanities University of Cambridge 1 ://www. cam. Arts and Humanities Stanford University |
5 https
October 15, 2024
60
6
https://
< www. www.stantee www.stad..ddedi
January 5, 2025 USD 90
Arts and Humanities University of California, Berkeley (UCB)
5 12
https:://////
November 30, 2024
4
Engineering and Technology | Massachusetts Institute of 1 2 httt://webbmiie edu/ January
+ Technology (MIT) 1
6, 2025 USD 75
Kngineering and Technology University 0xford
3
4 https://www. www.oxxxccuul. uk/ October 15, 2024 | GBP 28.50
Engineering and Technology I Stanford University I + http::/wwww.sttttte
January 5, 2025 USD 90
Engineering and Technology University of Cambridge
https:tt//// www. cam, ac, uk/ October 15, 2024
74

---

Published as a conference paper at ICLR 2026
Engineering and Technology ETH Zurich
| https://
4 www. www.ethz.ch/ March 31, 2025 | CHF 150
Life Sciences and Medicine | Harvard University 1
https://www https::/wwwhhaaaadd...a. hww.ararvaa..ee edu January 2025 | USD 85
Life sciences and Medicine University aww.ccx.c.uuk.
Oxford
4
N/A | GBP 28.50
Life Sciences and Medicine | Johns Hopkins University | 3 32
+ 16 https:/ //wp:///wwwwww//// jdu edu January 2, 2025 USD 70
Life Sciences and Medicine Stanford University
4 https://www stanford.edu
January 5, 2025 USD 90
Life sciences and Medicine | Massachusetts Institute of
4 Technology (MIT) 5
https://ww..mite
January
2025 | USD 75
Natural Sciences Harvard University 1
https://wwv:
4 harvard.eau January 1, 2025 I USD 85
Natural Sciences Massachusetts Institute of Technology (MII) hetps::/web.mit..
January 6, 2025 I USD 75 Natural Sciences University of Oxford
+ 2
3
https://ww/
4 ao..ac.kk October 15, 2024 GB2 28.50
Natural Sciences University of Cambridge 1
5
https://
4 www.cam.ac..o
October 15, 2024 GBP 60
|
Natural Sciences Stanford University 1 stanford.edu January 2025 USD 90
6 https: //www
Social Seiences and Management Harvard University
4
January 1, 2025 USD 85
Social Sciences and Management University of Oxford 2 3
1
https://www.c ac.uk
October 15, 2025 GBP 75
Social Sciences Management
Stanford University |
4 | https://www.sttafford..
January 5, 2025 USD 90
Social Sciences and Management
Massachusetts Institute of
+ Technology (MIT) 4
2 http:://eb.mitt.ee
January
2025 USD 75
Social Sciences Management
University of Cambridge I | October 15, 2025
+ | https: /www.wwwommmmoom///.// cam, ac.uk
GB? 50
75


Reviews:
[{"review_id": "TIoNTP5RPZ", "reviewer": "Reviewer_kg32", "summary": "This paper introduces WideSearch, a benchmark designed to evaluate the reliability and comprehensiveness of LLM-based search agents in large-scale information-gathering tasks. WideSearch assesses agents on tasks requiring exhaustive and structured collection of numerous atomic facts across multiple entities. The benchmark consists of 200 manually curated queries (100 English, 100 Chinese) spanning 18 diverse domains, each paired with a well-defined schema and gold-standard table. \n\nA five-stage human-in-the-loop curation pipeline ensures that all tasks are complex, objectively verifiable, and reliant on real web tool usage. The evaluation framework integrates rule-based scoring with LLM-as-a-judge scoring, achieving over 97.8% agreement with human judgment.\n\nThe authors further perform an extensive benchmarking study across state-of-the-art agentic systems, revealing severe limitations in current models—particularly in maintaining completeness and fidelity at scale—highlighting the urgent need for more advanced planning, reflection, and evidence-grounding capabilities in next-generation agents.", "questions": "(1) The paper reports 97.8% agreement between LLM-as-judge and human evaluation, which is strong. Could you provide more details on the specific disagreement cases? It would be great if the authors could provide more insights into this. \n\n(2) The benchmark relies on live web data. How do you plan to ensure its stability over time? Is there a plan for versioning or periodic re-validation as the web content evolves?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "(1) The paper tackles an underexplored yet practically crucial dimension of agent evaluation—broad, high-fidelity information gathering—which complements existing reasoning- and synthesis-oriented benchmarks. The conceptual framing of WideSearch as the “breadth” counterpart to DeepSearch and DeepResearch is clear, coherent, and well-motivated.\n\n(2) The five-stage human-in-the-loop curation pipeline is rigorous and systematic, ensuring that all tasks are complex, verifiable, and genuinely dependent on real web search. The inclusion of both English and Chinese queries further enhances the benchmark’s generality and cross-lingual coverage.\n\n(3) The evaluation and analysis are comprehensive and insightful. The experiments benchmark over ten single-agent, multi-agent, and end-to-end commercial systems using multiple aggregation metrics (Avg@N, Pass@N, Max@N), providing a well-rounded assessment of agent reliability at scale. The error taxonomy—separating advanced agentic failures (planning, reflection, evidence use) from basic operational errors—is particularly insightful. The test-time scaling analysis further isolates completeness, rather than retrieval accuracy, as the key bottleneck, offering a concrete and data-driven direction for future work.", "weaknesses": "The benchmark construction and evaluation are solid in general. It would be great if the authors could further conduct quantitative analysis regarding the major challenges and failure patterns mentioned in sections 4.1 and 4.2.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ZCRZm6r8Jq", "reviewer": "Reviewer_dPH5", "summary": "The paper introduces WideSearch, a benchmark designed to evaluate search agents on broad information-seeking tasks, focusing on collecting atomic factual data across many entities that are objectively verifiable. The benchmark comprises 200 manually curated tasks (100 English, 100 Chinese). It also evalutes a broad set of LLMs and the LMs under agent frameworks.", "questions": "- It's suggested to include a table of comparison to the recent deep research benchmarks on key dimensions.\n- The paper would be benefited by including more evaluated agents, and at least OpenAI Deep Research, which is usually the leading agent across recent benchmarks.", "limitations": "", "rating": 4, "confidence": 5, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The benchmark is unique from previous ones, which extend the commonly used browsecomp-style simple-answer questions to broad information gathering.", "weaknesses": "- Most importantly, even though the paper's motivation is very clear and reasonable, it remains a straightforward extension to the existing browsecomp-style benchmarks (from simple fact to a list of fact). Despite its potential usefulness (given the success of browsecomp), it's hard to admire from a research perspective. Also, it's still constrained to this very specific type of tasks, for the convenience of evaluation.\n\n- Regarding the experiments: \n  - 1) the tasks appear to be too challenging for simple LLMs (with searches) because they usually only execute very limited search steps. This makes related analysis trivial to know. And it's also not necessary to call them as \"single agent\" \n    2) The tasks appear to be more suitable for Deep Research agents such as Deep Research from OpenAI/Gemini/Grok/Qwen/Kimi/.... But according to the content in the appendix, the evaluted systems are more or less from self-implemented agent scaffoldings, while leaving those important frontier deep research systems omitted? The paper could be much more benefited by including experiments on more frontier agents and their analysis.\n\n- Writing and experiments: might as well include more details in the main content.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Fod8ezDweB", "reviewer": "Reviewer_29pm", "summary": "The paper introduces WideSearch, a benchmark targeting “agentic broad information-seeking” tasks where an LLM agent must find a complete set of entities and fill a structured table from the live web (e.g., “all DOJ attorney openings in a date window”). The dataset contains 200 manually curated questions (100 EN / 100 ZH) spanning 15 domains, built with a five-stage curation pipeline to ensure breadth, verifiability, and temporal stability, and scored with a hybrid, table-alignment evaluation (rule checks + LLM-as-judge). Benchmarking 10 systems shows very low table-level success (most ≈0%; best single/multi-agent ≈7%), while humans achieve ~20% in single-annotator mode; the gap stems from strict completeness requirements rather than difficulty of single facts (item-level F1 can be high with retries). Multi-agent frameworks improve F1 via divide-and-conquer but still rarely achieve perfect tables, highlighting deficiencies in planning, reflection, and evidence use.", "questions": "1. What exactly do the source queries come from? Are they purely curated by human annotators or modified from existing data sources (like NaturalQuestions)? Are there any time-sensitive queries (for example, the top-selling electrical toothbrush in the past week)?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Overall, it's a very good work that extends the frontiers of evaluating newly emerged deep research systems and tools.\n\n1. This paper proposes a novel dataset aimed at breadth and completeness across many entities. Rigorous five-stage human-centered curation ensures realistic and verifiable tasks.\n\n2. Results expose a fundamental limitation of current agents—completeness at scale—and show multi-agent setups help but don’t solve it, offering a concrete target for future research.", "weaknesses": "1. The evaluation protocol might not be robust enough. This paper uses markdown format as a protocol, applies several fuzzy or exact matches to number/date/urls, and finally applies LLM-as-a-judge to evaluate complex answers. This paradigm replies on pre-crafted ground truths and would be sensitive to those queries whose answer might change with time (for example, the top-selling electrical toothbrush in the past week).\n\n2. The current success criterion may be too brittle, which requires perfect table equality; small, arguably negligible deviations lead to failure, which can conflate usability with strict exactness.\n\n3. Tasks are claimed “temporally invariant”, yet the benchmark relies on the live web; concrete procedures for refreshing ground truths and handling site changes are under-specified.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FjbJnziJZ8", "reviewer": "Reviewer_tbLU", "summary": "This paper introduces WideSearch, a benchmark for broad, multi-entity, verifiable information-seeking by LLM agents. Each task pairs a query with a predefined table schema.\nThe agent is required to 1) identify the complete entity set and 2) fill attributes for each entity using web tools. \nA five-stage, human-in-the-loop curation pipeline ensures difficulty, verifiability, and temporal stability.\nLLM-as-judge is used as an automated evaluator for semantically variable cells. \nExperiments on a few models show very low table-level success, while item-level F1 can be much higher with test-time scaling.", "questions": "1. Fonts are too small for Fig.1 and Fig. 2\n2. In line 476, note that the left quote is wrong.\n3. Too many numbers in Table 1. Could you bold important numbers?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Clearly scoped evaluation setting: The paper frames `wide information seeking` as distinct from general deep research style tasks, emphasizing breadth, completeness, and structured outputs rather than deep synthesis of a single topic.\n2. Rigorous curation: The five-stage pipeline is thoughtful and practical for benchmark validity.\n3. Hybrid automatic evaluation that mixes exact or approximate rules with LLM-as-judge, including column types and primary keys for deterministic alignment. This is a careful table-centric evaluation.", "weaknesses": "1. Although 18 topics are covered, the distribution still shows heavier representation in Business/Finance/Arts & Culture. A sensitivity analysis showing robustness of conclusions across domains/languages would be useful.\n2. To the best of my knowledge, Manus is the only product claiming wide research. Is there any result from that system?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)